Support for the Amazon S3 API

Content Platform for Cloud Scale Administration Guide

Version
2.6.x
File Size
1950 KB
Audience
anonymous
Part Number
MK-HCPCS008-11

HCP for cloud scale is compatible with the Amazon Simple Storage Service (Amazon S3) REST API, which lets clients store objects in buckets. A bucket is a container of objects that has its own settings, such as ownership and lifecycle. Using HCP for cloud scale, users can perform common operations on objects and buckets and manage ACL settings through the client access data service.

For information about using Amazon S3, see the Amazon S3 API documentation.

For information about obtaining S3 user credentials, see the S3 Console Guide.

The following tables list the level of support for each of the HCP for cloud scale S3 API methods compared with the Amazon S3 API methods and describes any implementation differences in the HCP for cloud scale S3 APIs.

Buckets

Amazon S3 API Support level Implementation differences
CreateBucket Fully supported None
DeleteBucket Supported with differences

To support legacy S3 buckets, HCP for cloud scale supports bucket names of less than three characters.

When anonymous requests to create or remove a bucket use a bucket name that isn't valid, Amazon S3 verifies access first and returns 403. HCP for cloud scale returns 400 if the bucket name validation fails.

DeleteBucketAnalytics​Configuration Not supported Not supported
DeleteBucketCors Not supported Not supported
DeleteBucketEncryption Fully supported None
DeleteBucketIntelligentTiering​Configuration Not supported Not supported
DeleteBucketInventory​Configuration Not supported Not supported
DeleteBucketLifecycle Supported with differences

HCP for cloud scale supports the latest API for bucket lifecycle management. Old and deprecated V1.0 methods are not supported.

HCP for cloud scale does not support Object Transition actions. Including these actions causes a Malformed XML exception.

DeleteBucketMetrics​Configuration Not supported Not supported
DeleteBucketOwnership​Controls Not supported Not supported
DeleteBucketPolicy Not supported Not supported
DeleteBucketReplication Fully supported None
DeleteBucketTagging Not supported Not supported
DeleteBucketWebsite Not supported Not supported
DeletePublicAccessBlock Not supported Not supported
GetBucketAccelerate​Configuration Not supported Not supported
GetBucketAcl Supported with differences In Amazon S3 each grantee is specified as a type-value pair, where the type is one of the following:
  • emailAddress if the value specified is the email address of an AWS account
  • id if the value specified is the canonical user ID of an AWS account
  • uri if granting permission to a predefined group
HCP for cloud scale does not support emailAddress. HCP for cloud scale fully supports id. HCP for cloud scale supports uri for the predefined groups Authenticated Users and All Users.

HCP for cloud scale does not support the Amazon S3 predefined grant ("canned ACL") aws-exec-read.

HCP for cloud scale supports the canned ACL authenticated-read-write.

HCP for cloud scale does not mirror or mirror back ACLs or policies.

GetBucketAnalytics​Configuration Not supported Not supported
GetBucketCors Not supported Not supported
GetBucketEncryption Fully supported None
GetBucketIntelligentTiering​Configuration Not supported Not supported
GetBucketInventory​Configuration Not supported None
GetBucketLifecycle Supported with differences

HCP for cloud scale supports the latest API for bucket lifecycle management. Old and deprecated V1.0 methods are not supported.

HCP for cloud scale does not support Object Transition actions. Including these actions causes a Malformed XML exception.

GetBucketLifecycle​Configuration Not supported Not supported
GetBucketLocation Supported with differences The caller must be the bucket owner.
GetBucketLogging Not supported Not supported
GetBucketMetricsConfiguration Not supported Not supported
GetBucketNotification Fully supported None
GetBucketNotification​Configuration Fully supported None
GetBucketOwnershipControls Not supported Not supported
GetBucketPolicy Not supported Not supported
GetBucketPolicyStatus Not supported Not supported
GetBucketReplication Supported with differences HCP for cloud scale supports only V1.0 methods. Advanced filtering is not supported.
GetBucketRequestPayment Not supported Not supported
GetBucketTagging Not supported Not supported
GetBucketVersioning Fully supported Returns the bucket versioning configuration and status (always on).
GetBucketWebsite Not supported Not supported
GetPublicAccessBlock Not supported Not supported
HeadBucket Supported with differences

To support legacy S3 buckets, HCP for cloud scale supports bucket names of less than three characters.

When anonymous requests to create or remove a bucket use a bucket name that isn't valid, Amazon S3 verifies access first and returns 403. HCP for cloud scale returns 400 if the bucket name validation fails.

ListBucketAnalytics​Configurations Not supported Not supported
ListBucketIntelligentTiering​Configurations Not supported Not supported
ListBucketInventory​Configurations Not supported Not supported
ListBucketMetrics​Configurations Not supported Not supported
ListBuckets Fully supported None
PutBucketAccelerate​Configuration Not supported Not supported
PutBucketAcl Supported with differences In Amazon S3 each grantee is specified as a type-value pair, where the type is one of the following:
  • emailAddress if the value specified is the email address of an AWS account
  • id if the value specified is the canonical user ID of an AWS account
  • uri if granting permission to a predefined group
HCP for cloud scale does not support emailAddress. HCP for cloud scale fully supports id. HCP for cloud scale supports uri for the predefined groups Authenticated Users and All Users.

HCP for cloud scale does not support the Amazon S3 predefined grant ("canned ACL") aws-exec-read.

HCP for cloud scale supports the canned ACL authenticated-read-write.

HCP for cloud scale does not mirror or mirror back ACLs or policies.

PutBucketAnalytics​Configuration Not supported Not supported
PutBucketCors Not supported Not supported
PutBucketEncryption Fully supported None
PutBucketIntelligentTiering​Configuration Not supported Not supported
PutBucketInventory​Configuration Not supported Not supported
PutBucketLifecycle Supported with differences

HCP for cloud scale supports the latest API for bucket lifecycle management. Old and deprecated V1.0 methods are not supported.

HCP for cloud scale does not support Object Transition actions. Including these actions causes a Malformed XML exception.

PutBucketLifecycle​Configuration Supported with differences HCP for cloud scale supports only V1.0 methods. Advanced filtering is not supported.
PutBucketLogging Not supported Not supported
PutBucketMetricsConfiguration Not supported Not supported
PutBucketNotification Fully supported None
PutBucketNotification​Configuration Supported with differences A configuration can have to up 100 rules.

Amazon S3 considers that two rules overlap if both apply to the same object and share at least one event type. HCP for cloud scale supports notification from the same object to multiple targets. However, rules are blocked if they send a message for the same event to the same target.

All notification message fields are returned except Region and Glacier Storage. The field awsRegion is returned but left empty.

HCP for cloud scale does not support the x-amz-skip-destination-validation HTTP request header.

PutBucketOwnershipControls Not supported Not supported
PutBucketPolicy Not supported Not supported
PutBucketReplication Supported with differences

HCP for cloud scale supports replication to only one destination (1:1). All rules must share a single, common destination bucket. If more than one destination appears in the collection of rules, the entire policy will be rejected with a 400 status code.

Sending encrypted data to a remote bucket is not supported.

PutBucketRequestPayment Not supported Not supported
PutBucketTagging Not supported Not supported
PutBucketVersioning Not supported With HCP for cloud scale versioning is always enabled .
PutBucketWebsite Not supported Not supported
PutPublicAccessBlock Not supported Not supported

Objects

Amazon S3 API Support level Implementation differences
AbortMultipartUpload Fully supported None
CompleteMultipartUpload Fully supported None
CopyObject Supported with differences

The copy object source and destination must have the same encryption state. For example, encrypted to encrypted or unencrypted to unencrypted.

HCP for cloud scale supports using the x-amz-server-side-encryption header if encrypting only a single object. The header is not needed if the encryption policy is set at the bucket level.

CreateMultipartUpload Supported with differences

HCP for cloud scale supports using the x-amz-server-side-encryption header if encrypting only a single object. The header is not needed if the encryption policy is set at the bucket level.

DeleteObject Supported with differences

Bucket synchronization or removal of an object or a specific version of an object is not supported.

To improve performance, if the current version of an object is a delete marker, HCP for cloud scale does not create another delete marker.

DeleteObjects Supported with differences

Bucket synchronization is not supported.

DeleteObjectTagging Fully supported None
GetObject Supported with differences If a lifecycle policy is configured for a bucket, HCP for cloud scale displays the expiration date of an object (in the x-amz-expiration header) fetched using the subresource ?versionId.

Legal hold is fully implemented.

Object retention is fully implemented.

Object names cannot contain NUL or backslash (\) characters. GET methods on objects so named fail with a 400 error.

HCP for cloud scale supports using the x-amz-server-side-encryption header if encrypting only a single object. The header is not needed if the encryption policy is set at the bucket level.

The partNumber parameter is not supported.

GetObjectAcl Supported with differences

In Amazon S3, each grantee is specified as a type-value pair, where the type is one of the following:

  • emailAddress if the value specified is the email address of an AWS account
  • id if the value specified is the canonical user ID of an AWS account
  • uri if granting permission to a predefined group

HCP for cloud scale does not support emailAddress. HCP for cloud scale fully supports id. HCP for cloud scale supports uri for the predefined groups Authenticated Users and All Users.

HCP for cloud scale does not support the aws-exec-read canned ACL.

GetObjectAttributes Not supported Not supported
GetObjectLegalHold Fully supported None
GetObjectLockConfiguration Fully supported None
GetObjectRetention Fully supported None
GetObjectTagging Fully supported None
GetObjectTorrent Not supported Not supported
HeadObject Supported with differences

If a lifecycle policy is configured for a bucket, HCP for cloud scale displays the expiration date of an object (in the x-amz-expiration header) fetched using the subresource ?versionId.

HCP for cloud scale supports using the x-amz-server-side-encryption header if encrypting only a single object. The header is not needed if the encryption policy is set at the bucket level.

The partNumber parameter is not supported.

ListMultipartUploads Fully supported None
ListObjects Fully supported In Amazon S3, the NextMarker element is returned only if you have a delimiter request parameter specified. HCP for cloud scale always returns NextMarker when the response is truncated.
ListObjectsV2 Fully supported None
ListObjectVersions Fully supported None
ListParts Fully supported None
PutObject Supported with differences

HCP for cloud scale adds additional content-type validation.

Bucket synchronization is supported.

Legal hold is fully implemented. AWS object lock permissions are not supported; that is, a bucket owner can set a legal hold without restriction.

Object retention is implemented, but not governance mode; that is, once a retain-until date is set, it can be extended but not removed. AWS object lock permissions are not supported; that is, a bucket owner can set object retention without restriction.

Object locking can be applied to a bucket even after it's created. To enable object locking, in the S3 API PUT Bucket ObjectLockConfiguration, include the URI request parameter x-amz-bucket-object-lock-token (with any string).

Object names cannot contain NUL or backslash (\) characters. PUT methods on objects so named fail with a 400 error.

HCP for cloud scale supports using the x-amz-server-side-encryption header if encrypting only a single object. The header is not needed if the encryption policy is set at the bucket level.

PutObjectAcl Supported with differences

Bucket synchronization is not supported.

In Amazon S3, each grantee is specified as a type-value pair, where the type is one of the following:

  • emailAddress if the value specified is the email address of an AWS account
  • id if the value specified is the canonical user ID of an AWS account
  • uri if granting permission to a predefined group

HCP for cloud scale does not support emailAddress. HCP for cloud scale fully supports id. HCP for cloud scale supports uri for the predefined groups Authenticated Users and All Users.

HCP for cloud scale does not support the aws-exec-read canned ACL.

PutObjectLegalHold Fully supported None
PutObjectLockConfiguration Fully supported None
PutObjectRetention Fully supported None
PutObjectTagging Fully supported None
RestoreObject Not supported Not supported
SelectObjectContent Supported with differences

Scan range is supported.

HCP for cloud scale supports the use of * by itself with no alias reference. For example, this SQL query is supported:

select *, first_name from s3object s where s.salary > 100000 limit 10

HCP for cloud scale supports a wider range of date-time formats than AWS. The full list is available at https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html.

HCP for cloud scale supports nested aggregate functions. For example, this expression is supported: count(sum(s.salary))

HCP for cloud scale SQL queries on columns are case sensitive, while AWS SQL queries are case insensitive. For example, given an object s with the columns ID, iD, and id, an SQL query to select s.id will return column id in HCP for cloud scale but column ID in AWS.

Parquet compression is managed automatically, so the CompressionType argument is not needed, and if specified returns an error.

AWS calculates the size of a record returned in an S3 Select query as the total size of the record, including any delimiters. HCP for cloud scale calculates the size as the total data of each column returned. These calculations can sometimes differ slightly.

UploadPart Supported with differences HCP for cloud scale supports using the x-amz-server-side-encryption header if encrypting only a single object. The header is not needed if the encryption policy is set at the bucket level.
UploadPartCopy Supported with differences

HCP for cloud scale supports using the x-amz-server-side-encryption header if encrypting only a single object. The header is not needed if the encryption policy is set at the bucket level.

The upload part copy source and destination must have the same encryption state. For example, encrypted to encrypted or unencrypted to unencrypted.

WriteGetObjectResponse Not supported Not supported

Unsupported HTTP request headers

HCP for cloud scale does not support the following HTTP request headers in APIs it otherwise supports. If supplied as part of the request, these headers will be ignored.

  • x-amz-expected-bucket-owner
  • x-amz-sdk-checksum-algorithm
  • x-amz-request-payer
  • x-amz-storage-class
  • x-amz-website-redirect-location
  • x-amz-bypass-governance-retention
  • x-amz-request-mfa
  • x-amz-security-token

HCP for cloud scale does not support the following conditional HTTP request headers or equivalent x-amz extensions for putCopy:

  • If-Match
  • If-Modified-Since
  • If-None-Match
  • If-Unmodified-Since
  • x-amz-copy-source-if-match
  • x-amz-copy-source-if-none-match
  • x-amz-copy-source-if-unmodified-since
  • x-amz-copy-source-if-modified-since