The following considerations apply to working with multipart uploads.
Support for multipart uploads
Whether a bucket supports multipart uploads depends on a bucket configuration setting. The tenant administrator determines the default for this setting for new buckets. This setting is not visible through the S3 compatible API.
If a bucket is empty, the tenant administrator can change it from supporting multipart upload to not supporting multipart upload. Whether the tenant administrator can change a bucket from not supporting multipart upload to supporting multipart upload depends on which other access protocols are enabled for the bucket.
To learn whether the bucket you're working with supports multipart upload, contact your tenant administrator.
Part size and count
The parts you upload for a multipart upload can be any size up to and including five gigabytes. However, the minimum size for the parts you include in a complete multipart upload request, except the last part, is one megabyte. The last part can be smaller than one megabyte.
A multipart upload can have at most ten thousand parts. Therefore, the maximum size for the object resulting from a multipart upload is five terabytes (ten thousand times five gigabytes).
Automatic abort of multipart uploads
If a multipart upload is never completed or aborted, the disjointed parts can remain in HCP indefinitely. Even though the multipart upload is incomplete, these parts count toward the storage used by the bucket where they were uploaded.
To prevent parts of multipart uploads from remaining in HCP indefinitely, the tenant administrator can set the maximum amount of time for which a multipart upload can remain incomplete before the multipart upload is automatically aborted. This time is counted from the time the multipart upload was initiated.
Each bucket has its own automatic abort time for multipart uploads. This time is not visible through the S3 compatible API. By default, the automatic abort time for new buckets is 30 days.
The tenant administrator can change the automatic abort time for a bucket at any time. When the tenant administrator changes the time, the new time applies to all subsequent multipart uploads as well as to multipart uploads that are already in progress.
For example, if you initiate a multipart upload on September 9th while the automatic abort time is set to 30 days, that multipart upload will be automatically aborted on October 9th. If the tenant administrator then changes the automatic abort time to 25 days, the date on which the multipart upload will be automatically aborted changes to October 4th.
Creation date and time for multipart objects
The creation date and time for the object resulting from a multipart upload is the time when the multipart upload was completed, regardless of the amount of time that has elapsed since the multipart upload was initiated.
Retention setting for multipart objects
The retention setting for the object resulting from a multipart upload is calculated based on the time when the multipart upload was completed, regardless of the amount of time that has elapsed since the multipart upload was initiated.
For example, suppose the bucket default retention setting for new objects is 10 days. If a multipart upload in the bucket is completed on September 9th, the resulting object will be under retention until September 19th, regardless of when the multipart upload was initiated.
Retention is not set for the individual parts of an in-progress multipart upload. Replaced parts and parts not used in completed multipart uploads are automatically deleted regardless of the retention setting for the completed multipart object.
For more information about retention, see Retention.
Multipart uploads and versioning
If versioning is enabled for a bucket, you can use multipart uploads to store new versions of objects in the bucket. However, HCP does not check whether versioning is enabled until you submit the request to complete a multipart upload. If versioning is disabled and an object with the same name already exists when you submit the complete request, the request fails with a 409 (Conflict) status code.
If versioning is disabled for a bucket and you are concerned about object conflicts, before you initiate a multipart upload for an object, you can check the bucket for an existing object with the same name. For information about checking for existing objects, see Checking the existence of an object or folder.
![]() |
Tip: If a request to complete a multipart upload fails because versioning is disabled for the bucket and an object with the same name already exists, you can delete the existing object and then submit the complete request again with the same request body. A failed request to complete a multipart upload has no effect on the uploaded parts, so you don't need to upload them again. |
Multiple multipart uploads for objects with the same name
Regardless of the bucket versioning status, you can initiate multiple concurrent multipart uploads with the same object name in a single bucket. These multipart uploads are distinguished by upload ID. Because you specify both the upload ID and the object name in each request to upload a part, each part you upload applies to only one of the multipart uploads.
If versioning is enabled for the bucket, each multipart upload with the same object name, when completed, creates a separate version of the object. The order of the versions is determined by the order in which the multipart uploads are completed. The oldest version is the one resulting from the multipart upload that was completed first. The most recent version is the one resulting from the multipart upload that was completed last.
If versioning is disabled for the bucket, the object stored is the one resulting from the multipart upload that was completed first. Each subsequent completion of a multipart upload for the same object results in a 409 (Conflict) status code, and the resulting object is not stored. The parts of a multipart upload that cannot be completed remain in HCP until the multipart upload is aborted.
For more information about versioning, see Versioning.
Multipart uploads for objects under retention or on hold
You cannot store a new version of an object that's under retention or on hold. However, HCP does not check the retention or hold status of the existing object until you submit the request to complete the multipart upload. If the object is under retention or on hold when you submit the complete request, the request fails with a 403 (Forbidden) status code.
Before you initiate a multipart upload to store a new version of an object, you should use the HTTP data access protocol to check the retention and hold status of the existing object. Because this status can change, you may also want to check the retention and hold status of the existing object periodically while the multipart upload is in progress. If the object is put under retention or on hold while the multipart upload is in progress, you can abort the multipart upload.
For more information about retention and hold, see Retention. For information about using the HTTP data access protocol to check retention status, see
Multipart uploads and replicated buckets
All requests related to a given multipart upload must be sent to the same HCP system. That is, even if the bucket on which you initiate the multipart upload is replicated on other HCP systems, requests to upload parts for, list parts for, complete, or abort that multipart upload must be sent to the system on which you initiated the multipart upload.
For more information about replication, see Replication.
© 2015, 2020 Hitachi Vantara LLC. All rights reserved.