Listing bucket contents (version 2)

Content Platform Tenant Management

Version

9.7.x

File Size

4269 KB

Audience

anonymous

Part Number

MK-95HCPH002-19

You use the HTTP GET method to list the contents of a bucket. To list the contents of a bucket, you need browse permission for the bucket. You do not need to be an authenticated user.

For the purpose of a bucket listing, the bucket contents consist not only of the objects you stored in the bucket but also of the folders that you created in the bucket or that HCP created automatically from the object names. For example, by default, if a bucket contains an object named quarterly_rpts/Q4_2019, a list of the bucket contents includes these two items:

quarterly_rpts/
quarterly_rpts/Q4_2019

By default, a bucket listing includes only the current (or only) versions of objects.

A listing that includes versions of objects may include delete markers. A delete marker indicates that a version of the object was deleted.

A bucket listing is returned in an XML response body. In the response body, items in the bucket listing occur in ascending alphanumeric order by item name. If the listing includes multiple versions of an object, those versions are listed in ascending chronological order by last-modified time.

Request line

Depending on whether the bucket name is included in the hostname in the S3 compatible request, a request to list the contents of a bucket of a bucket has either of these formats:

With the bucket name included in the hostname:
```
GET /[?[query-parameters]] HTTP/1.1
```

With the bucket name following the hostname:

GET /bucket-name[?[query-parameters]] HTTP/1.1

query-parameters can be one or more of:

continuation-token: By default, a bucket listing can include at most one thousand items. If your request exceeds this or exceeds the value set for the max-keys query parameter, the response is truncated and you can use the continuation-token query parameter to list the next set of objects. Also, when the IsTruncated response element value is set to true, the response includes the NextContinuationToken element. The value of this element can be used in your next request as the continuation token to list the next set of objects.; The continuation-token is a value that only the S3 compatible API understands, and objects are listed in UTF-8 character encoding in lexicographical order. If you specify an invalid continuation-token query parameter in a request, a 400 (Bad Request) error is returned, including a message that specifies the invalid continuation-token query parameter.; The continuation-token parameter name is case sensitive.
delimiter: You use the delimiter query parameter to request a bucket listing that includes a list of common prefixes, where a prefix is the name of an item up through the first occurrence of the character string specified by the delimiter parameter. Each common prefix is listed only once regardless of the number of items with matching names. The items with matching names are not included elsewhere in the listing.; The returned listing also contains items with names that do not include the character string specified by the delimiter parameter. In the listing, all the named items are listed first, followed by all the common prefixes.; The items and common prefixes included in a listing are subject to any other criteria specified in the request.; Both the items and the common prefixes count toward the maximum number of items that can be included in the bucket listing.; The list of items does not include in-progress multipart uploads. The list of common prefixes does not include common prefixes that are exclusively for multipart uploads.; The character string you specify for the delimiter query parameter can contain any valid UTF-8 characters. including white space. Percent-encode non-ASCII characters and reserved special characters such as ampersands (&), commas (,) and equal signs (=). If the character string contains spaces, enclose the entire string in quotation marks.; The delimiter parameter name and the character string you specify are both case sensitive.
encoding-type: You use the encoding-type query parameter to request the S3 compatible API to encode the response. You can also use this query parameter to specify the encoding method to use.; The key for an object can contain any Unicode character. Some characters, such as those with an ASCII value from 0 to 10, cannot be parsed by XML 1.0 parsers. For these characters, you can add the encoding-type query parameter to request the S3 compatible API to encode the keys in the response.; The encoding-type parameter name is case sensitive.
fetch-owner: You use the fetch-owner query parameter to retrieve owner information. When you specify the fetch-owner query parameter to true, the S3 compatible API returns the owner information in the response.; The S3 compatible API no longer returns owner information by default in the response. To include owner information in the response, you need to set the fetch-owner query parameter to true.; The fetch-owner parameter name is case sensitive.
list-type: You use the list-type query parameter in order to use version 2 of the S3 compatible API. The value for this query parameter must always be set to 2 if you want to use version 2 of S3 compatible API requests.; If any value other than 2 is specified for this query parameter, version 1 of the S3 compatible API is used by default.; The list-type parameter name is case sensitive.
max-keys: By default, a bucket listing can include at most one thousand items. However, you can use the max-keys query parameter in a bucket listing request to specify a smaller maximum number of items. If more than the maximum number of items satisfy the criteria for a request, you can use query parameters to retrieve the parts in groups.; Valid values for the max-keys query parameter are integers in the range 0 (zero) through 2,147,483,647. If you specify an integer greater than one thousand, the returned bucket listing includes only the first one thousand items that satisfy the request criteria (or fewer if fewer than one thousand items satisfy the criteria).; The max-keys parameter name is case sensitive.
prefix: You use the prefix query parameter to request a bucket listing that contains only items with names that begin with a specified character string (the prefix) and, if applicable, common prefixes that begin that prefix.; The list of items does not include in-progress multipart uploads or common prefixes that are exclusively for multipart uploads.; The character string you specify for the prefix query parameter can contain any valid UTF-8 characters, including white space. Percent-encode non-ASCII characters and reserved special characters such as ampersands (&), commas (,) and equal signs (=). If the character string contains spaces, enclose the entire string in quotation marks.; The prefix parameter name and the character string you specify are both case sensitive.
start-after: You use the start-after query parameter to specify that you want the S3 compatible API to return key names after a specific object key. The start-after query parameter that you specify is always returned in the response, regardless of whether the specified value is valid.; The start-after query parameter is valid only in the first request. If the response is truncated, you can use the continuation-token query parameter to list the next set of objects. The S3 compatible API lists objects in UTF-8 character encoding in lexicographical order.; The start-after parameter name and the character string you specify are both case sensitive.

Required headers

The list below describes the headers you can use in a request to list the contents of a bucket.

Authorization

Specifies user credentials or requests anonymous access.

Date

Specifies the date and time when the request is being made according to the requester. Normally, this is the current date and time.

The date and time must always be specified using Greenwich Mean Time (GMT).

To specify the date and time, use this format:

DDD, dd MMM yyyy HH:mm:ss (+0000|GMT)

In this format:

DDD: The three-letter abbreviation for the day of the week, with an uppercase first letter (for example, Mon).
dd: The two-digit day of the month.
MMM: The three-letter abbreviation for the month, with an uppercase first letter (for example, Feb).
yyyy: The four-digit year.
HH: The hour on a 24-hour clock.
mm: The number of minutes.
ss: The number of seconds.

For example:

Thu, 23 Mar 2017 14:27:05 +0000

All S3 compatible requests must include either a Date header or an x-amz-date header. If a request includes both headers, HCP uses the date and time in the x-amz-date header.

Host

Specifies the hostname for the request. The host name identifies either a tenant or a bucket.

For a tenant, use this format:

tenant-name.hcp-domain-name

For a bucket, use this format:

bucket-name.tenant-name.hcp-domain-name

Optional headers

x-hcp-pretty-print

Optionally, requests that the XML response body be formatted for readability. Valid values are:

true: Format the XML response body for readability.
false: Do not apply any special formatting to the XML response body.

The default is false.

The values true and false are not case sensitive.

Response headers

The list below describes the headers returned in response to a successful request to list the contents of a bucket.

Connection

Specifies if the connection to the server is open or closed.

Content-Length

The size, in bytes, of the response body if HCP can determine the size before formulating the response.

If the response does not include a response body, the value of the Content-Length header is 0 (zero).

Content-Type

Specifies the Internet media type of the response body. For a request to list the contents of a bucket, the value of this header is always application/xml;charset=UTF-8.

Date

The date and time when HCP responded to the request, in Greenwich Mean Time (GMT). The date and time are returned in this format:

DDD dd MMM yyyy HH:mm:ss GMT

For example:

Fri, 18 Sep 2020 14:27:05 GMT

Server

The version of the HCP system that responded to the request. The value of this header is always HCP followed by the HCP version number (for example, HCP V9.0.0.2).

Transfer-Encoding

Indicates that HCP could not determine the size of the response body before formulating the response. For a request to list the buckets you own, the value of this header is always chunked.

Response body

The list below describes the XML elements in the response body that is returned in response to a request to list the contents of a bucket. The elements are listed in alphabetical order.

CommonPrefixes

Child of the ListBucketResult element and container for the Prefix element.

The response body contains one CommonPrefixes element for each common prefix in the bucket listing.

Contents

Child of the ListBucketResult element and container for the elements that describe a current item in the bucket.

The response body contains one Contents element for each item that satisfies the request criteria.

ContinuationToken

Child of the ListBucketResult element.

If sent with the request, the ContinuationToken element specifies the continuation token for listing the next set of objects.

Delimiter

Child of the ListBucketResult element.

The Delimiter element specifies the value of the delimiter query parameter included in the request. If the request did not include the delimiter query parameter, the response body does not include the Delimiter element.

DisplayName

Child of the Owner element.

If the item owner is identified by an HCP user account, the value of the DisplayName element is the user name for that account. If the item owner is identified by an AD user account, the value of the DisplayName element is the user name of that account followed by an at sign (@) and the AD domain name.

If the item has no owner, the value of the DisplayName element is nobody.

For folders, the value of the DisplayName element is always nobody.

Encoding-Type

Child of the ListBucketResult element.

The Encoding-Type element specifies the type of encoding used by the S3 compatible API to encode the key names of objects.

ETag

Child of the Contents element.

The ETag element specifies the ETag of the applicable item.

The response body includes ETag elements for the listed folders. However, because they have no content, all folders have the same value for this element.

ID

Child of the Owner element.

If the item owner is identified by an HCP user account, the value of the ID element is the user ID for that account. If the item owner is identified by an AD user account, the value of the ID element is the SID for that account.

If the item has no owner, the value of the ID element is nobody.

For folders, the value of the ID element is always nobody.

IsTruncated

Child of the ListBucketResult element.

The IsTruncated element indicates whether the returned bucket listing includes the last item that satisfies the request criteria. Possible values are:

true: The returned listing includes the last item that satisfies the request criteria.
false: The returned listing does not include the last item that satisfies the request criteria.

Key

Child of the Contents element.

The Key element specifies the name of an item in the bucket listing.

KeyCount

Child of the ListBucketResult element.

The KeyCount element specifies the number of keys in the response. The value of the KeyCount element is always less than or equal to the value of the MaxKeys element.

LastModified

Child of the Contents element.

The LastModified element specifies the date and time when the applicable item was last modified, in Greenwich Mean Time (GMT).

The date and time are expressed in this format:

yyyy-MM-ddTHH:mm:ss.SSSZ

For example:

2020-02-18T19:46:03.856Z

Modifying an object means modifying its metadata. You cannot modify the content of an object.

ListBucketResult

Root element for a listing of current items.

MaxKeys

Child of the ListBucketResult element.

The MaxKeys element specifies the value of the max-keys query parameter included in the request. If the request did not include the max-keys query parameter, the value of the MaxKeys element is 1000.

Name

Child of the ListBucketResult element.

The Name element specifies the name of the applicable bucket.

NextContinuationToken

Child of the ListBucketResult element.

If the response is truncated, the NextContinuationToken element is included in the response. The value of this element can be used in your next request as the continuation token to list the next set of objects.

Owner

Child of the Contents element and container for the DisplayName and ID elements.

The Owner element identifies the owner of the applicable item.

Prefix

One of these:

Child of the ListBucketResult element.
In this case, the Prefix element specifies the value of the prefix query parameter included in the request. If the request did not include the prefix query parameter, the response body includes Prefix as an empty element.
Child of the CommonPrefixes element.
In this case, the Prefix element specifies a common prefix.

Size

Child of the Contents element.

The Size element specifies the size, in bytes, of the content of the item.

The response body includes Size elements for the listed folders. However, because folders have no content, the value of this element for a folder is always 0 (zero).

StartAfter

Child of the ListBucketResult element.

If sent with the request, the StartAfter element includes the specific object key after which the S3 compatible API returns key names.

StorageClass

Child of the Contents element.

The value of the StorageClass element is always STANDARD.

Status codes

The table below describes HTTP status codes that can be returned in response to a request to list the contents of a bucket.

Code	Meaning	Description
200	OK	HCP successfully retrieved the requested bucket listing.
400	Bad Request	Possible reasons include that a query parameter has an invalid value.
403	Forbidden	Possible reasons include: The credentials provided with the request are invalid. You do not have permission to list the contents of the specified bucket. The S3 compatible API is currently disabled for the specified bucket.
404	Not Found	The specified bucket does not exist.
500	Internal Server Error	An internal error occurred. If this error persists, contact your tenant administrator.
503	Service Unavailable	HCP is temporarily unable to handle the request, probably due to system overload, maintenance, or upgrade. Try the request again, gradually increasing the delay between each successive attempt. If this error persists, contact your tenant administrator.

Example: Listing the items in a folder

Here’s a sample GET request for a bucket listing that lists the objects that are in the mktg folder and, separately, the subfolders that are in the mktg folder. The request uses these query parameters:

list-type=2: Uses version 2 of the S3 compatible API
max-keys=1000: Lists no more than 1000 items in the response
prefix=mktg/: Lists only items that start with mktg/
start-after=mktg/: Starts the listing with the item that follows mktg/ by itself
delimiter=/: Treats items that have a forward slash (/) after mktg/ as having a common prefix

Request with s3curl command line

./s3curl.pl --id=lgreen -- -k "https://finance.europe.hcp.example.com?list-type=2&max-keys=1000
     &prefix=mktg/&start-after=mktg/&delimiter=/" -H "x-hcp-pretty-print: true"

Request headers

GET /?list-type=2&max-keys=1000&prefix=mktg/&delimiter=/&start-after=mktg/ HTTP/1.1
Host: finance.europe.hcp.example.com
Date: Fri, 07 February 2020 17:19:26 +0000
Authorization: AWS bGdyZWVu:ysiVYJ/cOr78z69BuDy6fy07ryo=
x-hcp-pretty-print: true

Response headers

HTTP/1.1 200 OK
Date: Fri, 07 February 2020 17:19:26 GMT
Content-Type: application/xml;charset=UTF-8
Content-Length: 157
Connection: close
Server: HCP V9.0.0.2
Transfer-Encoding: chunked

Response body

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
     <Name>finance</Name>
     <Prefix>mktg/</Prefix>
     <StartAfter>mktg/</StartAfter>
     <NextContinuationToken>1eGcxW1r/XYCxHnhbY24cK3/Jd36Hy4vZkbM=</NextContinuationToken>
     <KeyCount>1000</KeyCount>
     <MaxKeys>1000</MaxKeys>
     <Delimiter>/</Delimiter>
     <IsTruncated>true</IsTruncated>
     <Contents>
          <Key>mktg/campaign_GoGetEm_expenses.xls</Key>
          <LastModified>2020-02-13T17:44:53.000Z</LastModified>
          <ETag>"6ed7faad1e0661c03ad65a4317d4a94c"</ETag>
          <Size>94328</Size>
          <Owner>
               <ID>b9d39144-a081-4762-b0e8-b8fb51e10192</ID>
               <DisplayName>lgreen</DisplayName>
          </Owner>
          <StorageClass>STANDARD</StorageClass>
     </Contents>
     <Contents>
          <Key>mktg/campaign_LiveIt_expenses.xls</Key>
          <LastModified>2019-12-05T14:32:29.110Z</LastModified>
          <ETag>"7ad452af1e2f61b33a865c4362be5921"</ETag>
          <Size>81578</Size>
          <Owner>
               <ID>b9d39144-a081-4762-b0e8-b8fb51e10192</ID>
               <DisplayName>lgreen</DisplayName>
          </Owner>
          <StorageClass>STANDARD</StorageClass>
     </Contents>
       .
       .
       .
     <CommonPrefixes>
          <Prefix>mktg/budget_proposals/</Prefix>
            .
            .
            .
     </CommonPrefixes>
</ListBucketResult>