Bucket synchronization is configured using S3 PUT bucket replication API requests that define rules. Each bucket can have up to 1,000 rules, but all rules must be sync-to or sync-from rules. Each rule defines the following:
- External bucket settings
- A set of one or more prefixes; an object with one of the prefixes is mirrored
- A set of one or more tags; an object with all, or any, of the tags is mirrored
- For sync-from, external queue settings
Because you can configure multiple rules with multiple tags, you have flexibility in selecting objects to mirror. For example:
- To mirror all objects that contain Tag1 and Tag2, you can configure one rule that includes both tags.
- To mirror all objects that contain Tag1 or Tag2, you can configure two rules, one for each tag.
For information on PUT bucket replication see Configure bucket synchronization (PUT bucket replication).
Visibility of new buckets and objects
After they are created, buckets and objects are immediately visible.
Rule collisions
HCP for cloud scale can apply multiple bucket synchronization rules to each new object, so long as the destination bucket has identical rules.
A rule collision is when two or more rules that apply to an object have the same destination (that is, the same external host, port, and bucket). HCP for cloud scale does not allow rule collisions, so PUT bucket replication requests are rejected if they contain rule collisions. To avoid rule collisions, you can define as many tags in a rule as necessary, so that multiple rules with the same destination are not needed.
Effect of configuration changes
After an object operation is performed, the policy engine asynchronously checks if that object needs to be copied according to the sync-to rules. When bucket synchronization rules are created, updated, or deleted, the changes only apply to new objects, object operations, and to objects that have not been yet processed by the policy engine. Objects that existed before the rules were configured are not synchronized. If an object exists in the PENDING state when a rule is created, updated, or deleted, the rule change might not be applied.
Synchronizing to the same source and destination
You cannot set up bucket synchronization with the same bucket as both the source and the destination.