Uploading a part of a multipart object by copying

Content Platform Tenant Management Help

Version
9.7.x
File Size
4269 KB
Audience
anonymous
Part Number
MK-95HCPH002-19

You can upload a part of a multipart upload by copying all or part of the data for an existing object. To do this, you use the HTTP PUT method with the uploadId and partNumber query parameters and the x-amz-copy-source header. The bucket where you upload the part must be the same bucket as the one where the multipart upload was initiated.

You use the x-amz-copy-source header to specify the object you're copying (the source object) and the bucket that contains that object (the source bucket). The source bucket can be the same bucket as the one to which you're uploading the part. Alternatively, the source bucket can be a different bucket that's owned by the same tenant.

To upload a part by copying, you must be an authenticated user. You need to use the same user account as the account used in the request to initiate the multipart upload. Additionally, you need write permission for the bucket where you're uploading the part and read permission for the source bucket or for the source object itself.

You use the uploadId query parameter to specify the upload ID of the multipart upload for which you're uploading the part. If the upload ID you specify doesn't match the upload ID of an in-progress multipart upload or isn't associated with the object specified in the request, HCP returns a 404 (Not Found) status code and does not upload the part.

When you upload a part by copying, you use the partNumber query parameter to specify the part number of the data you're uploading. The part number must be an integer in the range 1 (one) through 10,000.

The parts of the data for an object are ordered, but the part numbers do not need to start at one and do not need to be consecutive numbers. For example, if you're uploading the data for an object in three parts, you could number the parts 1, 2, and 3, but you could also number them 2, 7, and 19.

You can upload the parts of a multipart upload in any order. However, in the request to complete the multipart upload, you need to list the parts in ascending numeric order.

You can upload multiple parts of a multipart upload concurrently.

If a request to upload a part by copying includes any custom metadata (x-amz-meta-) headers, HCP returns a 400 (Bad Request) status code and does not upload the part.

HCP ignores ACL headers in requests to upload a part by copying.

Request line

Depending on whether the bucket name is included in the hostname in the S3 compatible request, for a request to upload a part of a multipart upload by copying has either of these formats:

  • With the bucket name included in the hostname:
    PUT /object-name?uploadId=upload-id&partNumber=part-number HTTP/1.1
  • With the bucket name following the hostname:
    PUT /bucket-name/object-name?uploadId=upload-id&partNumber=part-number HTTP/1.1

The uploadId and partNumber query parameters are not case sensitive.

Required headers

The table below describes the headers you can use in a request to upload a part of a multipart upload by copying.

Authorization
Specifies user credentials or requests anonymous access.
Date
Specifies the date and time when the request is being made according to the requester. Normally, this is the current date and time.
The date and time must always be specified using Greenwich Mean Time (GMT).
To specify the date and time, use this format:
DDD, dd MMM yyyy HH:mm:ss (+0000|GMT)
In this format:
DDD
The three-letter abbreviation for the day of the week, with an uppercase first letter (for example, Mon).
dd
The two-digit day of the month.
MMM
The three-letter abbreviation for the month, with an uppercase first letter (for example, Feb).
yyyy
The four-digit year.
HH
The hour on a 24-hour clock.
mm
The number of minutes.
ss
The number of seconds.
For example:
Thu, 23 Mar 2017 14:27:05 +0000
All S3 compatible requests must include either a Date header or an x-amz-date header. If a request includes both headers, HCP uses the date and time in the x-amz-date header.
Host
Specifies the hostname for the request. The host name identifies either a tenant or a bucket.
For a tenant, use this format:
tenant-name.hcp-domain-name
For a bucket, use this format:
bucket-name.tenant-name.hcp-domain-name
x-amz-copy-source
Specifies the source bucket and object or object version, in this format:
/bucket-name/source-object-name[?versionId=source-object-version-id]
The initial forward slash (/) is required.
Valid values for source-object-version-id are the IDs of versions of the source object specified in the request.
The versionId query parameter is not case sensitive.
If you include the versionId query parameter in the x-amz-copy-source header with an invalid value while versioning is enabled, HCP returns a 404 (Not Found) status code and does not perform the upload part copy operation.
If you include the versionId query parameter in the x-amz-copy-source header while versioning is disabled, the parameter is ignored, and the current version of the specified object is used as the source for the upload part copy operation.
x-amz-date
Specifies the date and time at which the request is being made according to the requester. Normally, this is the current date and time.
For the valid values for this header, see the description of the Date header above.

Optional headers

Content-Type
Specifies the Internet media type of the data being copied. Valid values are Internet media types (for example, text/plain, application/xml, or image/jpeg).
x-amz-copy-source-if-match
Specifies one or more values for comparison with the ETag of the specified source object or object version. If the ETag matches one of the specified values, HCP continues processing the request. If the ETag doesn’t match any of the specified values, HCP returns a 412 (Precondition Failed) status code and does not upload the part.
To specify the values for this header, use this format:
"value"[, "value"]...
In this format, each value can be any string of one or more characters and must be enclosed in double quotation marks (").
Alternatively, you can specify a single asterisk (*) as the value for the x-amz-copy-source-if-match header. All ETags match an asterisk in an x-amz-copy-source-if-match header.
x-amz-copy-source-if-modified-since
Specifies a date and time, in Greenwich Mean Time (GMT), for comparison with the date and time the specified source object or object version was last modified. If the modification date and time is later than the specified date and time, HCP continues processing the request. If the modification date and time is equal to or earlier than the specified date and time, HCP returns a 412 (Precondition Failed) status code and does not upload the part.
To specify the date and time for this header, use one of these formats:
  • DDD,ddMMMyyyyHH:mm:ss(+0000|GMT)

    For example: Tue, 07 Feb 2017 14:27:05 +0000

  • DDDD,dd-MMM-yyyyHH:mm:ss(+0000|GMT)

    For example: Tuesday, 07-Feb-17 14:27:05 +0000

  • DDDMMMdHH:mm:ssyyyy

    For example: Tue Feb 7 14:27:05 2017

x-amz-copy-source-if-none-match
Specifies one or more values for comparison with the ETag of the specified source object or object version. If the ETag doesn’t match any of the specified values, HCP continues processing the request. If the ETag matches any of the specified values, HCP returns a 412 (Precondition Failed) status code and does not upload the part.
To specify the values for this header, use this format:
"value"[, "value"]...
In this format, each value can be any string of one or more characters and must be enclosed in double quotation marks (").
Alternatively, you can specify a single asterisk (*) as the value for the x-amz-copy-source-if-match header. No ETags match an asterisk in an x-amz-copy-source-if-match header.
x-amz-copy-source-if-unmodified-since
Specifies a date and time, in Greenwich Mean Time (GMT), for comparison with the date and time the specified source object or object version was last modified. If the modification date and time is equal to or earlier than the specified date and time, HCP continues processing the request. If the modification date and time is later than the specified date and time, HCP returns a 412 (Precondition Failed) status code and does not upload the part.
For valid values for this header, see the description of the x-amz-copy-source-if-modified-since header above.
x-amz-copy-source-range
To upload a part by copying only part of the data for the source object, you use the x-amz-copy-source-range request header. The value of the x-amz-copy-source-range header is the range of bytes you want to copy. The first byte of the data for an object is in position 0 (zero), so a range of one to five specifies the second through sixth bytes, not the first through fifth.
To specify a byte range in an x-amz-copy-source-range header, use this format:
bytes=start-position-end-position
These considerations apply to x-amz-copy-source-range header values:
  • If you specify a valid range in which the start position is less than the size of the source object data, HCP uploads the part and returns a 200 (OK) status code.
  • If you specify a valid range in which the start position is greater than or equal to the size of the source object data, HCP returns a 400 (Bad Request) status code and does not upload a part.
  • If you specify an invalid range (for example, a range in which the start position is greater than the end position, HCP returns a 400 (Bad Request) status code and does not upload a part.

Response headers

The list below describes the headers returned in response to a successful request to upload a part of a multipart upload by copying.

Content-Type
Specifies the Internet media type of the response body. For a request to upload a part by copying, the value of this header is always application/xml;charset=UTF-8.
Date
The date and time when HCP responded to the request, in Greenwich Mean Time (GMT). The date and time are returned in this format:
DDD dd MMM yyyy HH:mm:ss GMT
For example:
Fri, 18 Sep 2020 14:27:05 GMT
ETag
Specifies the ETag for the object.
ETags are useful for making object-level operations conditional based on the object content. Operations that can be made conditional are checking the existence of an object, copying an object, and retrieving an object.
Transfer-Encoding
Indicates that HCP could not determine the size of the response body before formulating the response. For a request to list the buckets you own, the value of this header is always chunked.
x-amz-copy-source-version-id
Specifies the version ID of the source object. This header is returned only while versioning is enabled for the source bucket.
x-amz-server-side-encryption
Specifies whether objects stored in HCP are encrypted. Possible values are:
  • If objects are encrypted, AES256
  • If objects are not encrypted, None
This header is returned only if the request headers include x-amz-server-side-encryption.
x-amz-version-id
Specifies the version ID of the object. This header is returned only while versioning is enabled for the bucket.

Response body

HCP returns information about the part of a multipart upload that results from a successful upload part copy request in an XML response body.

The list below describes the XML elements in the response body returned in response to a request to upload a part by copying. The elements are listed in alphabetical order.

CopyPartResult
Root element.
ETag
Child of the CopyPartResult element.
The ETag element specifies the ETag for the uploaded part.
LastModified
Child of the CopyPartResult element.
The LastModified element specifies the date and time when the uploaded part was last modified, in Greenwich Mean Time (GMT). The date and time are expressed in this format:
yyyy-MM-ddTHH:mm:ss.SSSZ
For example:
2020-02-18T19:46:03.856Z

Status codes

The table below describes HTTP status codes that can be returned in response to a request to upload a part of a multipart upload by copying.

Code Meaning Description
200 OK HCP successfully uploaded the part.
400 Bad Request

Possible reasons include:

  • The specified part number is not an integer in the range 1 (one) through 10,000.
  • The request specifies a range, and one of these is true:
    • The start position for the range is greater than or equal to the size of the source object data.
    • The start position for the range is greater than the end position.
  • The request includes conflicting conditional headers (for example, x-amz-copy-source-if-match and x-amz-copy-source-if-none-match).
  • The request includes a custom metadata (x-amz-meta-) header.
403 Forbidden

Possible reasons include:

  • The request specifies anonymous access. Only an authenticated user can upload parts for a multipart upload.
  • The credentials provided with the request are invalid.
  • The credentials provided with the request do not match the credentials used to initiate the multipart upload.
  • You do not have permission to upload parts in the specified bucket.
  • The S3 compatible API is currently disabled for the specified bucket.
  • The source object exists, but the HCP system does not have the source object data. Try the request again later.
  • The source object is in the process of being deleted.
404 Not Found

One of these:

  • The specified multipart upload does not exist. Either the upload ID is invalid, or the multipart upload was aborted or completed.

    In the case of completion, if the part being uploaded is a replacement for an existing part, the existing part is used for completing the multipart upload.

  • The specified bucket does not exist.
409 Conflict The specified multipart upload was completed or aborted while the upload part copy operation was in progress.
412 Precondition Failed

One of these:

  • The request included an x-amz-copy-source-if-match header, and the ETag for the specified source object or object version does not match any of the values specified by the header.
  • The request included an x-amz-copy-source-if-none-match header, and the ETag for the specified source object or object version matched a value specified by the header.
  • The request included an x-amz-copy-source-if-modified-since header, and the specified source object or object version was not modified after the date and time specified by the header.
  • The request included an x-amz-copy-source-if-unmodified-since header, and the specified source object or object version was modified after the date and time specified by the header.
500 Internal Server Error

An internal error occurred.

If this error persists, contact your tenant administrator.

503 Service Unavailable

Possible reasons include:

  • The source object exists but cannot be read.
  • HCP is temporarily unable to handle the request, probably due to system overload, maintenance, or upgrade.

Try the request again, gradually increasing the delay between each successive attempt.

If this error persists, contact your tenant administrator.

Example: Conditionally creating a part from an object

Here’s a sample PUT request that conditionally uploads a part of the multipart upload for an object named acctg/RulesAndRegulations.pdf in the finance bucket by copying the data from an existing object named acctg/AcctgRR-Summary in the same bucket.

For this example, assume that part 6 of the multipart upload has already been uploaded and has an ETag of 7914d874df2c1d55cfab4fa82088ff56. The request directs HCP to create a new part 6 from the copied data only if the ETag of the copied data is different from the ETag of the data previously uploaded for part 6.

The example shows the response headers HCP returns while versioning is enabled for the bucket.

Request with s3curl command line

./s3curl.pl --id=lgreen --copysrc=finance/acctg/AcctgRR-Summary -- -k
     "https://finance.europe.hcp.example.com/acctg/RulesAndRegulations.pdf
             ?uploadId=94837746087105&partNumber=6"
     -H "x-amz-copy-source-if-none-match:7914d874df2c1d55cfab4fa82088ff56"
     -H "x-hcp-pretty-print: true"

Request headers

PUT /acctg/RulesAndRegulations.pdf?uploadId=94837746087105&partNumber=6 HTTP/1.1
Host: finance.europe.hcp.example.com
Date: Fri, 07 February 2020 17:19:26 +0000
Authorization: AWS bGdyZWVu:PBZDJM2WbzEIX4jFBO4Crah4GMQ=
x-amz-copy-source: finance/acctg/AcctgRR-Summary
x-amz-copy-source-if-none-match:7914d874df2c1d55cfab4fa82088ff56
x-hcp-pretty-print: true

Response headers

HTTP/1.1 200 OK
Date: Fri, 07 February 2020 17:19:26 GMT
ETag: "f79eac0151d6b62226986e721c89a2f1"
x-amz-version-id: 94870864774145
x-amz-copy-source-version-id: 94842948673409
Content-Type: application/xml;charset=UTF-8
x-amz-server-side-encryption: None
Transfer-Encoding: chunked

Response body

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CopyPartResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
     <LastModified>2017-02-23T17:19:26.000Z</LastModified>
     <ETag>"f79eac0151d6b62226986e721c89a2f1"</ETag>
</CopyPartResult>

Example: Creating a part from part of an object

Here’s a sample PUT request that uploads a part of the multipart upload for an object named acctg/RulesAndRegulations.pdf in the finance bucket by copying part of the data for an existing object named AcctgBestPractices.doc in the same bucket.

The example shows the response headers HCP returns while versioning is enabled for the bucket.

Request with s3curl command line

./s3curl.pl --id=lgreen --copysrc=finance/AcctgBestPractices.doc -- -k
     "https://finance.europe.hcp.example.com/acctg/RulesAndRegulations.pdf
             ?uploadId=94837746087105&partNumber=5"
     -H "x-amz-copy-source-range: bytes=2800000-9499999"
     -H "x-hcp-pretty-print: true"

Request headers

PUT /acctg/RulesAndRegulations.pdf?uploadId=94837746087105&partNumber=5 HTTP/1.1
Host: finance.europe.hcp.example.com
Date: Fri, 07 February 2020 17:19:26 +0000
Authorization: AWS bGdyZWVu:a0wC/DuV+w2Agq6FagnIe6cShaU=
x-amz-copy-source: finance/AcctgBestPractices.doc
x-amz-copy-source-range: bytes=2800000-9499999
x-hcp-pretty-print: true

Response headers

HTTP/1.1 200 OK
Date: Fri, 07 February 2020 17:19:26 GMT
ETag: "bbe438b2a0376f08dc37867a82ea87e6"
x-amz-version-id: 94871138333377
x-amz-copy-source-version-id: 94860356828929
Content-Type: application/xml;charset=UTF-8
x-amz-server-side-encryption: None
Transfer-Encoding: chunked

Response body

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CopyPartResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
     <LastModified>2017-02-23T17:19:26.000Z</LastModified>
     <ETag>"bbe438b2a0376f08dc37867a82ea87e6"</ETag>
</CopyPartResult>