Listing bucket contents (version 2)
You use the HTTP GET method to list the contents of a bucket. To list the contents of a bucket, you need browse permission for the bucket. You do not need to be an authenticated user.
For the purpose of a bucket listing, the bucket contents consist not only of the objects you stored in the bucket but also of the folders that you created in the bucket or that HCP created automatically from the object names. For example, by default, if a bucket contains an object named quarterly_rpts/Q4_2019, a list of the bucket contents includes these two items:
- quarterly_rpts/
- quarterly_rpts/Q4_2019
By default, a bucket listing includes only the current (or only) versions of objects.
A listing that includes versions of objects may include delete markers. A delete marker indicates that a version of the object was deleted.
A bucket listing is returned in an XML response body. In the response body, items in the bucket listing occur in ascending alphanumeric order by item name. If the listing includes multiple versions of an object, those versions are listed in ascending chronological order by last-modified time.
Request line
Depending on whether the bucket name is included in the hostname in the S3 compatible request, a request to list the contents of a bucket of a bucket has either of these formats:
- With the bucket name included in the hostname:
GET /[?[query-parameters]] HTTP/1.1
- With the bucket name following the hostname:
GET /bucket-name[?[query-parameters]] HTTP/1.1
query-parameters can be one or more of:
continuation-token
By default, a bucket listing can include at most one thousand items. If your request exceeds this or exceeds the value set for the max-keys query parameter, the response is truncated and you can use the continuation-token query parameter to list the next set of objects. Also, when the
IsTruncated
response element value is set totrue
, the response includes theNextContinuationToken
element. The value of this element can be used in your next request as the continuation token to list the next set of objects.The continuation-token is a value that only the S3 compatible API understands, and objects are listed in UTF-8 character encoding in lexicographical order. If you specify an invalid continuation-token query parameter in a request, a 400 (Bad Request) error is returned, including a message that specifies the invalid continuation-token query parameter.
The continuation-token parameter name is case sensitive.
delimiter
You use the delimiter query parameter to request a bucket listing that includes a list of common prefixes, where a prefix is the name of an item up through the first occurrence of the character string specified by the delimiter parameter. Each common prefix is listed only once regardless of the number of items with matching names. The items with matching names are not included elsewhere in the listing.
The returned listing also contains items with names that do not include the character string specified by the delimiter parameter. In the listing, all the named items are listed first, followed by all the common prefixes.
The items and common prefixes included in a listing are subject to any other criteria specified in the request.
Both the items and the common prefixes count toward the maximum number of items that can be included in the bucket listing.
The list of items does not include in-progress multipart uploads. The list of common prefixes does not include common prefixes that are exclusively for multipart uploads.
The character string you specify for the delimiter query parameter can contain any valid UTF-8 characters. including white space. Percent-encode non-ASCII characters and reserved special characters such as ampersands (&), commas (,) and equal signs (=). If the character string contains spaces, enclose the entire string in quotation marks.
The delimiter parameter name and the character string you specify are both case sensitive.
encoding-type
You use the encoding-type query parameter to request the S3 compatible API to encode the response. You can also use this query parameter to specify the encoding method to use.
The key for an object can contain any Unicode character. Some characters, such as those with an ASCII value from 0 to 10, cannot be parsed by XML 1.0 parsers. For these characters, you can add the encoding-type query parameter to request the S3 compatible API to encode the keys in the response.
The encoding-type parameter name is case sensitive.
fetch-owner
You use the fetch-owner query parameter to retrieve owner information. When you specify the fetch-owner query parameter to true, the S3 compatible API returns the owner information in the response.
The S3 compatible API no longer returns owner information by default in the response. To include owner information in the response, you need to set the fetch-owner query parameter to
true
.The fetch-owner parameter name is case sensitive.
list-type
You use the list-type query parameter in order to use version 2 of the S3 compatible API. The value for this query parameter must always be set to
2
if you want to use version 2 of S3 compatible API requests.If any value other than
2
is specified for this query parameter, version 1 of the S3 compatible API is used by default.The list-type parameter name is case sensitive.
max-keys
By default, a bucket listing can include at most one thousand items. However, you can use the max-keys query parameter in a bucket listing request to specify a smaller maximum number of items. If more than the maximum number of items satisfy the criteria for a request, you can use query parameters to retrieve the parts in groups.
Valid values for the max-keys query parameter are integers in the range 0 (zero) through 2,147,483,647. If you specify an integer greater than one thousand, the returned bucket listing includes only the first one thousand items that satisfy the request criteria (or fewer if fewer than one thousand items satisfy the criteria).
The max-keys parameter name is case sensitive.
prefix
You use the prefix query parameter to request a bucket listing that contains only items with names that begin with a specified character string (the prefix) and, if applicable, common prefixes that begin that prefix.
The list of items does not include in-progress multipart uploads or common prefixes that are exclusively for multipart uploads.
The character string you specify for the prefix query parameter can contain any valid UTF-8 characters, including white space. Percent-encode non-ASCII characters and reserved special characters such as ampersands (&), commas (,) and equal signs (=). If the character string contains spaces, enclose the entire string in quotation marks.
The prefix parameter name and the character string you specify are both case sensitive.
start-after
You use the start-after query parameter to specify that you want the S3 compatible API to return key names after a specific object key. The start-after query parameter that you specify is always returned in the response, regardless of whether the specified value is valid.
The start-after query parameter is valid only in the first request. If the response is truncated, you can use the continuation-token query parameter to list the next set of objects. The S3 compatible API lists objects in UTF-8 character encoding in lexicographical order.
The start-after parameter name and the character string you specify are both case sensitive.
Required headers
The list below describes the headers you can use in a request to list the contents of a bucket.
Authorization
Specifies user credentials or requests anonymous access.
Date
Specifies the date and time when the request is being made according to the requester. Normally, this is the current date and time.
The date and time must always be specified using Greenwich Mean Time (GMT).
To specify the date and time, use this format:
DDD, dd MMM yyyy HH:mm:ss (+0000|GMT)
In this format:
DDD
The three-letter abbreviation for the day of the week, with an uppercase first letter (for example, Mon).
dd
The two-digit day of the month.
MMM
The three-letter abbreviation for the month, with an uppercase first letter (for example, Feb).
yyyy
The four-digit year.
HH
The hour on a 24-hour clock.
mm
The number of minutes.
ss
The number of seconds.
For example:
Thu, 23 Mar 2017 14:27:05 +0000
All S3 compatible requests must include either a
Date
header or anx-amz-date
header. If a request includes both headers, HCP uses the date and time in thex-amz-date
header.Host
Specifies the hostname for the request. The host name identifies either a tenant or a bucket.
For a tenant, use this format:
tenant-name.hcp-domain-name
For a bucket, use this format:
bucket-name.tenant-name.hcp-domain-name
Optional headers
x-hcp-pretty-print
Optionally, requests that the XML response body be formatted for readability. Valid values are:
true
Format the XML response body for readability.
false
Do not apply any special formatting to the XML response body.
The default is false.
The values
true
andfalse
are not case sensitive.
Response headers
The list below describes the headers returned in response to a successful request to list the contents of a bucket.
Connection
Specifies if the connection to the server is open or closed.
Content-Length
The size, in bytes, of the response body if HCP can determine the size before formulating the response.
If the response does not include a response body, the value of the
Content-Length
header is0
(zero).Content-Type
Specifies the Internet media type of the response body. For a request to list the contents of a bucket, the value of this header is always
application/xml;charset=UTF-8
.Date
The date and time when HCP responded to the request, in Greenwich Mean Time (GMT). The date and time are returned in this format:
DDD dd MMM yyyy HH:mm:ss GMT
For example:
Fri, 18 Sep 2020 14:27:05 GMT
Server
The version of the HCP system that responded to the request. The value of this header is always HCP followed by the HCP version number (for example, HCP V9.0.0.2).
Transfer-Encoding
Indicates that HCP could not determine the size of the response body before formulating the response. For a request to list the buckets you own, the value of this header is always
chunked
.
Response body
The list below describes the XML elements in the response body that is returned in response to a request to list the contents of a bucket. The elements are listed in alphabetical order.
CommonPrefixes
Child of the
ListBucketResult
element and container for thePrefix
element.The response body contains one
CommonPrefixes
element for each common prefix in the bucket listing.Contents
Child of the
ListBucketResult
element and container for the elements that describe a current item in the bucket.The response body contains one
Contents
element for each item that satisfies the request criteria.ContinuationToken
Child of the
ListBucketResult
element.If sent with the request, the
ContinuationToken
element specifies the continuation token for listing the next set of objects.Delimiter
Child of the
ListBucketResult
element.The
Delimiter
element specifies the value of the delimiter query parameter included in the request. If the request did not include the delimiter query parameter, the response body does not include theDelimiter
element.DisplayName
Child of the
Owner
element.If the item owner is identified by an HCP user account, the value of the
DisplayName
element is the user name for that account. If the item owner is identified by an AD user account, the value of theDisplayName
element is the user name of that account followed by an at sign (@) and the AD domain name.If the item has no owner, the value of the
DisplayName
element isnobody
.For folders, the value of the
DisplayName
element is alwaysnobody
.Encoding-Type
Child of the
ListBucketResult
element.The
Encoding-Type
element specifies the type of encoding used by the S3 compatible API to encode the key names of objects.ETag
Child of the
Contents
element.The
ETag
element specifies the ETag of the applicable item.The response body includes
ETag
elements for the listed folders. However, because they have no content, all folders have the same value for this element.ID
Child of the
Owner
element.If the item owner is identified by an HCP user account, the value of the
ID
element is the user ID for that account. If the item owner is identified by an AD user account, the value of theID
element is the SID for that account.If the item has no owner, the value of the
ID
element isnobody
.For folders, the value of the
ID
element is alwaysnobody
.IsTruncated
Child of the
ListBucketResult
element.The
IsTruncated
element indicates whether the returned bucket listing includes the last item that satisfies the request criteria. Possible values are:true
The returned listing includes the last item that satisfies the request criteria.
false
The returned listing does not include the last item that satisfies the request criteria.
Key
Child of the
Contents
element.The
Key
element specifies the name of an item in the bucket listing.KeyCount
Child of the
ListBucketResult
element.The
KeyCount
element specifies the number of keys in the response. The value of theKeyCount
element is always less than or equal to the value of theMaxKeys
element.LastModified
Child of the
Contents
element.The
LastModified
element specifies the date and time when the applicable item was last modified, in Greenwich Mean Time (GMT).The date and time are expressed in this format:
yyyy-MM-ddTHH:mm:ss.SSSZ
For example:
2020-02-18T19:46:03.856Z
Modifying an object means modifying its metadata. You cannot modify the content of an object.
ListBucketResult
Root element for a listing of current items.
MaxKeys
Child of the
ListBucketResult
element.The
MaxKeys
element specifies the value of the max-keys query parameter included in the request. If the request did not include the max-keys query parameter, the value of theMaxKeys
element is 1000.Name
Child of the
ListBucketResult
element.The
Name
element specifies the name of the applicable bucket.NextContinuationToken
Child of the
ListBucketResult
element.If the response is truncated, the
NextContinuationToken
element is included in the response. The value of this element can be used in your next request as the continuation token to list the next set of objects.Owner
Child of the
Contents
element and container for theDisplayName
andID
elements.The
Owner
element identifies the owner of the applicable item.Prefix
One of these:
- Child of the
ListBucketResult
element.In this case, the
Prefix
element specifies the value of the prefix query parameter included in the request. If the request did not include the prefix query parameter, the response body includesPrefix
as an empty element. - Child of the
CommonPrefixes
element.In this case, the
Prefix
element specifies a common prefix.
- Child of the
Size
Child of the
Contents
element.The
Size
element specifies the size, in bytes, of the content of the item.The response body includes
Size
elements for the listed folders. However, because folders have no content, the value of this element for a folder is always0
(zero).StartAfter
Child of the
ListBucketResult
element.If sent with the request, the
StartAfter
element includes the specific object key after which the S3 compatible API returns key names.StorageClass
Child of the
Contents
element.The value of the
StorageClass
element is always STANDARD.
Return codes
The table below describes HTTP status codes that can be returned in response to a request to list the contents of a bucket.
Code | Meaning | Description |
200 | OK | HCP successfully retrieved the requested bucket listing. |
400 | Bad Request | Possible reasons include that a query parameter has an invalid value. |
403 | Forbidden |
Possible reasons include:
|
404 | Not Found | The specified bucket does not exist. |
500 | Internal Server Error |
An internal error occurred. If this error persists, contact your tenant administrator. |
503 | Service Unavailable |
HCP is temporarily unable to handle the request, probably due to system overload, maintenance, or upgrade. Try the request again, gradually increasing the delay between each successive attempt. If this error persists, contact your tenant administrator. |
Example: Listing the items in a folder
Here’s a sample GET request for a bucket listing that lists the objects that are in the mktg folder and, separately, the subfolders that are in the mktg folder. The request uses these query parameters:
list-type=2
Uses version 2 of the S3 compatible API
max-keys=1000
Lists no more than 1000 items in the response
prefix=mktg/
Lists only items that start with mktg/
start-after=mktg/
Starts the listing with the item that follows mktg/ by itself
delimiter=/
Treats items that have a forward slash (/) after mktg/ as having a common prefix
Request with s3curl command line
./s3curl.pl --id=lgreen -- -k "https://finance.europe.hcp.example.com?list-type=2&max-keys=1000 &prefix=mktg/&start-after=mktg/&delimiter=/" -H "x-hcp-pretty-print: true"
Request headers
GET /?list-type=2&max-keys=1000&prefix=mktg/&delimiter=/&start-after=mktg/ HTTP/1.1 Host: finance.europe.hcp.example.com Date: Fri, 07 February 2020 17:19:26 +0000 Authorization: AWS bGdyZWVu:ysiVYJ/cOr78z69BuDy6fy07ryo= x-hcp-pretty-print: true
Response headers
HTTP/1.1 200 OK Date: Fri, 07 February 2020 17:19:26 GMT Content-Type: application/xml;charset=UTF-8 Content-Length: 157 Connection: close Server: HCP V9.0.0.2 Transfer-Encoding: chunked
Response body
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <Name>finance</Name> <Prefix>mktg/</Prefix> <StartAfter>mktg/</StartAfter> <NextContinuationToken>1eGcxW1r/XYCxHnhbY24cK3/Jd36Hy4vZkbM=</NextContinuationToken> <KeyCount>1000</KeyCount> <MaxKeys>1000</MaxKeys> <Delimiter>/</Delimiter> <IsTruncated>true</IsTruncated> <Contents> <Key>mktg/campaign_GoGetEm_expenses.xls</Key> <LastModified>2020-02-13T17:44:53.000Z</LastModified> <ETag>"6ed7faad1e0661c03ad65a4317d4a94c"</ETag> <Size>94328</Size> <Owner> <ID>b9d39144-a081-4762-b0e8-b8fb51e10192</ID> <DisplayName>lgreen</DisplayName> </Owner> <StorageClass>STANDARD</StorageClass> </Contents> <Contents> <Key>mktg/campaign_LiveIt_expenses.xls</Key> <LastModified>2019-12-05T14:32:29.110Z</LastModified> <ETag>"7ad452af1e2f61b33a865c4362be5921"</ETag> <Size>81578</Size> <Owner> <ID>b9d39144-a081-4762-b0e8-b8fb51e10192</ID> <DisplayName>lgreen</DisplayName> </Owner> <StorageClass>STANDARD</StorageClass> </Contents> . . . <CommonPrefixes> <Prefix>mktg/budget_proposals/</Prefix> . . . </CommonPrefixes> </ListBucketResult>