Listing bucket contents (version 2)

You use the HTTP GET method to list the contents of a bucket. To list the contents of a bucket, you need browse permission for the bucket. You do not need to be an authenticated user.

For the purpose of a bucket listing, the bucket contents consist not only of the objects you stored in the bucket but also of the folders that you created in the bucket or that HCP created automatically from the object names. For example, by default, if a bucket contains an object named quarterly_rpts/Q4_2019, a list of the bucket contents includes these two items:

  • quarterly_rpts/
  • quarterly_rpts/Q4_2019

By default, a bucket listing includes only the current (or only) versions of objects.

A listing that includes versions of objects may include delete markers. A delete marker indicates that a version of the object was deleted.

A bucket listing is returned in an XML response body. In the response body, items in the bucket listing occur in ascending alphanumeric order by item name. If the listing includes multiple versions of an object, those versions are listed in ascending chronological order by last-modified time.

Request line

Depending on whether the bucket name is included in the hostname in the S3 compatible request, a request to list the contents of a bucket of a bucket has either of these formats:

  • With the bucket name included in the hostname:
    GET /[?[query-parameters]] HTTP/1.1
  • With the bucket name following the hostname:
    GET /bucket-name[?[query-parameters]] HTTP/1.1

query-parameters can be one or more of:

  • continuation-token

    By default, a bucket listing can include at most one thousand items. If your request exceeds this or exceeds the value set for the max-keys query parameter, the response is truncated and you can use the continuation-token query parameter to list the next set of objects. Also, when the IsTruncated response element value is set to true, the response includes the NextContinuationToken element. The value of this element can be used in your next request as the continuation token to list the next set of objects.

    The continuation-token is a value that only the S3 compatible API understands, and objects are listed in UTF-8 character encoding in lexicographical order. If you specify an invalid continuation-token query parameter in a request, a 400 (Bad Request) error is returned, including a message that specifies the invalid continuation-token query parameter.

    The continuation-token parameter name is case sensitive.

  • delimiter

    You use the delimiter query parameter to request a bucket listing that includes a list of common prefixes, where a prefix is the name of an item up through the first occurrence of the character string specified by the delimiter parameter. Each common prefix is listed only once regardless of the number of items with matching names. The items with matching names are not included elsewhere in the listing.

    The returned listing also contains items with names that do not include the character string specified by the delimiter parameter. In the listing, all the named items are listed first, followed by all the common prefixes.

    The items and common prefixes included in a listing are subject to any other criteria specified in the request.

    Both the items and the common prefixes count toward the maximum number of items that can be included in the bucket listing.

    The list of items does not include in-progress multipart uploads. The list of common prefixes does not include common prefixes that are exclusively for multipart uploads.

    The character string you specify for the delimiter query parameter can contain any valid UTF-8 characters. including white space. Percent-encode non-ASCII characters and reserved special characters such as ampersands (&), commas (,) and equal signs (=). If the character string contains spaces, enclose the entire string in quotation marks.

    The delimiter parameter name and the character string you specify are both case sensitive.

  • encoding-type

    You use the encoding-type query parameter to request the S3 compatible API to encode the response. You can also use this query parameter to specify the encoding method to use.

    The key for an object can contain any Unicode character. Some characters, such as those with an ASCII value from 0 to 10, cannot be parsed by XML 1.0 parsers. For these characters, you can add the encoding-type query parameter to request the S3 compatible API to encode the keys in the response.

    The encoding-type parameter name is case sensitive.

  • fetch-owner

    You use the fetch-owner query parameter to retrieve owner information. When you specify the fetch-owner query parameter to true, the S3 compatible API returns the owner information in the response.

    The S3 compatible API no longer returns owner information by default in the response. To include owner information in the response, you need to set the fetch-owner query parameter to true.

    The fetch-owner parameter name is case sensitive.

  • list-type

    You use the list-type query parameter in order to use version 2 of the S3 compatible API. The value for this query parameter must always be set to 2 if you want to use version 2 of S3 compatible API requests.

    If any value other than 2 is specified for this query parameter, version 1 of the S3 compatible API is used by default.

    The list-type parameter name is case sensitive.

  • max-keys

    By default, a bucket listing can include at most one thousand items. However, you can use the max-keys query parameter in a bucket listing request to specify a smaller maximum number of items. If more than the maximum number of items satisfy the criteria for a request, you can use query parameters to retrieve the parts in groups.

    Valid values for the max-keys query parameter are integers in the range 0 (zero) through 2,147,483,647. If you specify an integer greater than one thousand, the returned bucket listing includes only the first one thousand items that satisfy the request criteria (or fewer if fewer than one thousand items satisfy the criteria).

    The max-keys parameter name is case sensitive.

  • prefix

    You use the prefix query parameter to request a bucket listing that contains only items with names that begin with a specified character string (the prefix) and, if applicable, common prefixes that begin that prefix.

    The list of items does not include in-progress multipart uploads or common prefixes that are exclusively for multipart uploads.

    The character string you specify for the prefix query parameter can contain any valid UTF-8 characters, including white space. Percent-encode non-ASCII characters and reserved special characters such as ampersands (&), commas (,) and equal signs (=). If the character string contains spaces, enclose the entire string in quotation marks.

    The prefix parameter name and the character string you specify are both case sensitive.

  • start-after

    You use the start-after query parameter to specify that you want the S3 compatible API to return key names after a specific object key. The start-after query parameter that you specify is always returned in the response, regardless of whether the specified value is valid.

    The start-after query parameter is valid only in the first request. If the response is truncated, you can use the continuation-token query parameter to list the next set of objects. The S3 compatible API lists objects in UTF-8 character encoding in lexicographical order.

    The start-after parameter name and the character string you specify are both case sensitive.

Required headers

The list below describes the headers you can use in a request to list the contents of a bucket.

  • Authorization

    Specifies user credentials or requests anonymous access.

  • Date

    Specifies the date and time when the request is being made according to the requester. Normally, this is the current date and time.

    The date and time must always be specified using Greenwich Mean Time (GMT).

    To specify the date and time, use this format:

    DDD, dd MMM yyyy HH:mm:ss (+0000|GMT)

    In this format:

    • DDD

      The three-letter abbreviation for the day of the week, with an uppercase first letter (for example, Mon).

    • dd

      The two-digit day of the month.

    • MMM

      The three-letter abbreviation for the month, with an uppercase first letter (for example, Feb).

    • yyyy

      The four-digit year.

    • HH

      The hour on a 24-hour clock.

    • mm

      The number of minutes.

    • ss

      The number of seconds.

    For example:

    Thu, 23 Mar 2017 14:27:05 +0000

    All S3 compatible requests must include either a Date header or an x-amz-date header. If a request includes both headers, HCP uses the date and time in the x-amz-date header.

  • Host

    Specifies the hostname for the request. The host name identifies either a tenant or a bucket.

    For a tenant, use this format:

    tenant-name.hcp-domain-name

    For a bucket, use this format:

    bucket-name.tenant-name.hcp-domain-name

Optional headers

  • x-hcp-pretty-print

    Optionally, requests that the XML response body be formatted for readability. Valid values are:

    • true

      Format the XML response body for readability.

    • false

      Do not apply any special formatting to the XML response body.

    The default is false.

    The values true and false are not case sensitive.

Response headers

The list below describes the headers returned in response to a successful request to list the contents of a bucket.

  • Connection

    Specifies if the connection to the server is open or closed.

  • Content-Length

    The size, in bytes, of the response body if HCP can determine the size before formulating the response.

    If the response does not include a response body, the value of the Content-Length header is 0 (zero).

  • Content-Type

    Specifies the Internet media type of the response body. For a request to list the contents of a bucket, the value of this header is always application/xml;charset=UTF-8.

  • Date

    The date and time when HCP responded to the request, in Greenwich Mean Time (GMT). The date and time are returned in this format:

    DDD dd MMM yyyy HH:mm:ss GMT

    For example:

    Fri, 18 Sep 2020 14:27:05 GMT

  • Server

    The version of the HCP system that responded to the request. The value of this header is always HCP followed by the HCP version number (for example, HCP V9.0.0.2).

  • Transfer-Encoding

    Indicates that HCP could not determine the size of the response body before formulating the response. For a request to list the buckets you own, the value of this header is always chunked.

Response body

The list below describes the XML elements in the response body that is returned in response to a request to list the contents of a bucket. The elements are listed in alphabetical order.

  • CommonPrefixes

    Child of the ListBucketResult element and container for the Prefix element.

    The response body contains one CommonPrefixes element for each common prefix in the bucket listing.

  • Contents

    Child of the ListBucketResult element and container for the elements that describe a current item in the bucket.

    The response body contains one Contents element for each item that satisfies the request criteria.

  • ContinuationToken

    Child of the ListBucketResult element.

    If sent with the request, the ContinuationToken element specifies the continuation token for listing the next set of objects.

  • Delimiter

    Child of the ListBucketResult element.

    The Delimiter element specifies the value of the delimiter query parameter included in the request. If the request did not include the delimiter query parameter, the response body does not include the Delimiter element.

  • DisplayName

    Child of the Owner element.

    If the item owner is identified by an HCP user account, the value of the DisplayName element is the user name for that account. If the item owner is identified by an AD user account, the value of the DisplayName element is the user name of that account followed by an at sign (@) and the AD domain name.

    If the item has no owner, the value of the DisplayName element is nobody.

    For folders, the value of the DisplayName element is always nobody.

  • Encoding-Type

    Child of the ListBucketResult element.

    The Encoding-Type element specifies the type of encoding used by the S3 compatible API to encode the key names of objects.

  • ETag

    Child of the Contents element.

    The ETag element specifies the ETag of the applicable item.

    The response body includes ETag elements for the listed folders. However, because they have no content, all folders have the same value for this element.

  • ID

    Child of the Owner element.

    If the item owner is identified by an HCP user account, the value of the ID element is the user ID for that account. If the item owner is identified by an AD user account, the value of the ID element is the SID for that account.

    If the item has no owner, the value of the ID element is nobody.

    For folders, the value of the ID element is always nobody.

  • IsTruncated

    Child of the ListBucketResult element.

    The IsTruncated element indicates whether the returned bucket listing includes the last item that satisfies the request criteria. Possible values are:

    • true

      The returned listing includes the last item that satisfies the request criteria.

    • false

      The returned listing does not include the last item that satisfies the request criteria.

  • Key

    Child of the Contents element.

    The Key element specifies the name of an item in the bucket listing.

  • KeyCount

    Child of the ListBucketResult element.

    The KeyCount element specifies the number of keys in the response. The value of the KeyCount element is always less than or equal to the value of the MaxKeys element.

  • LastModified

    Child of the Contents element.

    The LastModified element specifies the date and time when the applicable item was last modified, in Greenwich Mean Time (GMT).

    The date and time are expressed in this format:

    yyyy-MM-ddTHH:mm:ss.SSSZ

    For example:

    2020-02-18T19:46:03.856Z

    Modifying an object means modifying its metadata. You cannot modify the content of an object.

  • ListBucketResult

    Root element for a listing of current items.

  • MaxKeys

    Child of the ListBucketResult element.

    The MaxKeys element specifies the value of the max-keys query parameter included in the request. If the request did not include the max-keys query parameter, the value of the MaxKeys element is 1000.

  • Name

    Child of the ListBucketResult element.

    The Name element specifies the name of the applicable bucket.

  • NextContinuationToken

    Child of the ListBucketResult element.

    If the response is truncated, the NextContinuationToken element is included in the response. The value of this element can be used in your next request as the continuation token to list the next set of objects.

  • Owner

    Child of the Contents element and container for the DisplayName and ID elements.

    The Owner element identifies the owner of the applicable item.

  • Prefix

    One of these:

    • Child of the ListBucketResult element.

      In this case, the Prefix element specifies the value of the prefix query parameter included in the request. If the request did not include the prefix query parameter, the response body includes Prefix as an empty element.

    • Child of the CommonPrefixes element.

      In this case, the Prefix element specifies a common prefix.

  • Size

    Child of the Contents element.

    The Size element specifies the size, in bytes, of the content of the item.

    The response body includes Size elements for the listed folders. However, because folders have no content, the value of this element for a folder is always 0 (zero).

  • StartAfter

    Child of the ListBucketResult element.

    If sent with the request, the StartAfter element includes the specific object key after which the S3 compatible API returns key names.

  • StorageClass

    Child of the Contents element.

    The value of the StorageClass element is always STANDARD.

Return codes

The table below describes HTTP status codes that can be returned in response to a request to list the contents of a bucket.

CodeMeaningDescription
200OKHCP successfully retrieved the requested bucket listing.
400Bad RequestPossible reasons include that a query parameter has an invalid value.
403Forbidden

Possible reasons include:

  • The credentials provided with the request are invalid.
  • You do not have permission to list the contents of the specified bucket.
  • The S3 compatible API is currently disabled for the specified bucket.
404Not FoundThe specified bucket does not exist.
500Internal Server Error

An internal error occurred.

If this error persists, contact your tenant administrator.

503Service Unavailable

HCP is temporarily unable to handle the request, probably due to system overload, maintenance, or upgrade. Try the request again, gradually increasing the delay between each successive attempt.

If this error persists, contact your tenant administrator.

Example: Listing the items in a folder

Here’s a sample GET request for a bucket listing that lists the objects that are in the mktg folder and, separately, the subfolders that are in the mktg folder. The request uses these query parameters:

  • list-type=2

    Uses version 2 of the S3 compatible API

  • max-keys=1000

    Lists no more than 1000 items in the response

  • prefix=mktg/

    Lists only items that start with mktg/

  • start-after=mktg/

    Starts the listing with the item that follows mktg/ by itself

  • delimiter=/

    Treats items that have a forward slash (/) after mktg/ as having a common prefix

Request with s3curl command line

./s3curl.pl --id=lgreen -- -k "https://finance.europe.hcp.example.com?list-type=2&max-keys=1000
     &prefix=mktg/&start-after=mktg/&delimiter=/" -H "x-hcp-pretty-print: true"

Request headers

GET /?list-type=2&max-keys=1000&prefix=mktg/&delimiter=/&start-after=mktg/ HTTP/1.1
Host: finance.europe.hcp.example.com
Date: Fri, 07 February 2020 17:19:26 +0000
Authorization: AWS bGdyZWVu:ysiVYJ/cOr78z69BuDy6fy07ryo=
x-hcp-pretty-print: true

Response headers

HTTP/1.1 200 OK
Date: Fri, 07 February 2020 17:19:26 GMT
Content-Type: application/xml;charset=UTF-8
Content-Length: 157
Connection: close
Server: HCP V9.0.0.2
Transfer-Encoding: chunked

Response body

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
     <Name>finance</Name>
     <Prefix>mktg/</Prefix>
     <StartAfter>mktg/</StartAfter>
     <NextContinuationToken>1eGcxW1r/XYCxHnhbY24cK3/Jd36Hy4vZkbM=</NextContinuationToken>
     <KeyCount>1000</KeyCount>
     <MaxKeys>1000</MaxKeys>
     <Delimiter>/</Delimiter>
     <IsTruncated>true</IsTruncated>
     <Contents>
          <Key>mktg/campaign_GoGetEm_expenses.xls</Key>
          <LastModified>2020-02-13T17:44:53.000Z</LastModified>
          <ETag>"6ed7faad1e0661c03ad65a4317d4a94c"</ETag>
          <Size>94328</Size>
          <Owner>
               <ID>b9d39144-a081-4762-b0e8-b8fb51e10192</ID>
               <DisplayName>lgreen</DisplayName>
          </Owner>
          <StorageClass>STANDARD</StorageClass>
     </Contents>
     <Contents>
          <Key>mktg/campaign_LiveIt_expenses.xls</Key>
          <LastModified>2019-12-05T14:32:29.110Z</LastModified>
          <ETag>"7ad452af1e2f61b33a865c4362be5921"</ETag>
          <Size>81578</Size>
          <Owner>
               <ID>b9d39144-a081-4762-b0e8-b8fb51e10192</ID>
               <DisplayName>lgreen</DisplayName>
          </Owner>
          <StorageClass>STANDARD</StorageClass>
     </Contents>
       .
       .
       .
     <CommonPrefixes>
          <Prefix>mktg/budget_proposals/</Prefix>
            .
            .
            .
     </CommonPrefixes>
</ListBucketResult>