Synchronization from an external bucket: high-level tasks

Content Platform for Cloud Scale S3 Console Guide
Version
2.4.x
Audience
anonymous
Part Number
MK-HCPCS009-04

Synchronization from an external bucket involves assigning roles and permissions to users, creating and synchronizing buckets, and then reading from and writing to the buckets.

This description of high-level tasks assumes three classes of user:

  1. An HCP for cloud scale system administrator to create roles and assign them to users using an IdP
  2. An HCP for cloud scale bucket administrator, who could be a tenant administrator, to create and configure an HCP for cloud scale bucket
  3. An AWS user, who could be a customer, to create a remote bucket using AWS S3, create an AWS SQS queue, and then configure S3 notifications to SQS
Note: The default HCP for cloud scale account has full permissions and can perform the tasks assigned to the first two user classes.
  1. The system administrator assigns permissions to the bucket administrator to configure bucket synchronization.
    1. In the System Management application, create a role with the permission group bucket_sync.
    2. In the IdP server, set up two groups: bucket administrators and bucket users.
    3. In the IdP server, register users in these groups.
    4. In the System Management application, assign the role to the bucket administrator group.
  2. The bucket administrator creates local and remote buckets.
    1. In the S3 User Credentials application, generate S3 credentials.
      Tip: Use the base64 utility to encode S3 credentials.
    2. Using the S3 credentials, use an S3 API to create an HCP for cloud scale (local) bucket.
    3. Use an AWS S3 API to create an S3 (remote) bucket.
  3. The AWS user creates a standard queue in SQS.
    1. Using an AWS account, create a queue of the type Standard Queue.
    2. Create a policy document.
  4. The AWS user configures the remote bucket to send S3 notifications to the AWS SQS queue.
    1. Add a notification for all object creation events to the remote bucket.
  5. The bucket administrator configures bucket synchronization between the S3 bucket and the HCP for cloud scale bucket using an S3 PUT Bucket Replication method, replacing the bucket ARN with configuration settings. By using multiple rules and filters, the bucket administrator can specify what objects are synchronized to the local bucket.
  6. The bucket administrator sets access control lists to let the bucket user read data from the HCP for cloud scale bucket.
    1. Using a management API, get the user ID of the bucket user.
    2. Using an S3 API, assign write permission to the bucket user for the HCP for cloud scale bucket.
  7. The AWS user is now free to read objects from the HCP for cloud scale bucket, which is now synchronized with the remote bucket.