Skip to main content

CompleteMultipartUpload

CompleteMultipartUpload is the last step of a multipart upload started with CreateMultipartUpload. The request lists the parts - each part's number and the ETag that UploadPart or UploadPartCopy returned for it - and STRG joins them, in part-number order, into a new version of the object. From then on the object is there to read like any other, and the upload ID has done its job and can no longer be used.

tip

The high-level commands of the AWS tools - aws s3 cp, boto3's upload_file, and Upload from @aws-sdk/lib-storage - run CreateMultipartUpload, UploadPart and CompleteMultipartUpload for you. Calling this operation directly is for code that manages the parts itself.

Good to know on STRG
  • If-Match and If-None-Match get a clear answer straight away: STRG responds 501 NotImplemented, so a client knows before anything is stored.
  • An empty part list is answered 400 InvalidRequest (Amazon S3 says 400 MalformedXML).
  • Every part but the last needs at least 5,000,000 bytes (Amazon S3: 5 MiB).
  • <Location> is the object's URL on the endpoint you called, in the addressing style you used, so it can be used as it is.
  • x-amz-expected-bucket-owner is welcome and has nothing to check: a key only ever reaches buckets of its own account.

When you'd use it​

  • A nightly backup of a large archive. A backup job splits a multi-gigabyte database dump into parts, uploads them - several at once, if it likes - and finishes with CompleteMultipartUpload. A single PutObject takes up to 5 GiB, so anything larger is uploaded this way.
  • An upload that survives a dropped connection. A desktop app sending a long video can lose its network halfway through. Because each part is stored as it arrives, the app reads back what is already there with ListParts, sends the rest, and completes the upload.
  • A data pipeline or an agent writing a large result. A job producing a big export streams it out part by part as it is generated, and completes the upload once the last part is written.

Request​

POST /photos/2024/cat.jpg?uploadId=UploadId HTTP/1.1
Host: my-bucket.s3.eu-central-1.strg.com
Authorization: AWS4-HMAC-SHA256 Credential=...
x-amz-date: 20260926T120000Z
x-amz-content-sha256: <SHA-256 of the body>
Content-Type: application/xml

<CompleteMultipartUpload xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Part>
<PartNumber>1</PartNumber>
<ETag>"85f30635602dc09bd85957a6e82a2c21"</ETag>
</Part>
<Part>
<PartNumber>2</PartNumber>
<ETag>"2dc09bd8595785f3063560a6e82a2c21"</ETag>
</Part>
</CompleteMultipartUpload>

Path-style, the same request is POST /my-bucket/photos/2024/cat.jpg?uploadId=UploadId on s3.eu-central-1.strg.com. The S3 API overview describes both addressing styles; a bucket whose name contains a dot is reached path-style.

Permissions​

The access key needs Write objects (objects:write). For a key limited to certain buckets or to a key prefix, the bucket and the object key need to be within them. Access keys has the details.

URI parameters​

NameTypeRequiredDescription
BucketStringYesThe bucket the upload was started in.
KeyStringYesThe object key the upload was started for.
uploadIdStringYesThe upload ID that CreateMultipartUpload returned.

Request headers​

Every header the S3 reference lists for this operation, and what STRG does with it:

NameTypeRequiredDescription
Content-MD5StringNoBase64 MD5 of the body. When present and it does not match, the answer is 400 BadDigest.
x-amz-checksum-crc32StringNoBase64 CRC32 checksum of the whole object. Accepted.
x-amz-checksum-crc32cStringNoBase64 CRC32C checksum of the whole object. Accepted.
x-amz-checksum-crc64nvmeStringNoBase64 CRC64NVME checksum of the whole object. Accepted.
x-amz-checksum-sha1StringNoBase64 SHA-1 digest of the whole object. Accepted.
x-amz-checksum-sha256StringNoBase64 SHA-256 digest of the whole object. Accepted.
x-amz-checksum-sha512StringNoBase64 SHA-512 digest of the whole object. Accepted.
x-amz-checksum-md5StringNoBase64 MD5 digest of the whole object. Accepted.
x-amz-checksum-xxhash64StringNoBase64 XXHASH64 checksum of the whole object. Accepted.
x-amz-checksum-xxhash3StringNoBase64 XXHASH3 checksum of the whole object. Accepted.
x-amz-checksum-xxhash128StringNoBase64 XXHASH128 checksum of the whole object. Accepted.
x-amz-checksum-typeStringNoCOMPOSITE or FULL_OBJECT: how the part checksums combine into the object's. Accepted.
x-amz-mp-object-sizeLongNoThe total object size you expect. Accepted.
If-MatchStringNoSTRG answers 501 NotImplemented.
If-None-MatchStringNoSTRG answers 501 NotImplemented.
x-amz-server-side-encryption-customer-algorithmStringNoAES256, for an upload started with a customer-provided encryption key (SSE-C). Accepted.
x-amz-server-side-encryption-customer-keyStringNoThe Base64 SSE-C key. Accepted.
x-amz-server-side-encryption-customer-key-MD5StringNoBase64 MD5 of the SSE-C key. Accepted.
x-amz-expected-bucket-ownerStringNoWelcome, with nothing to check: an access key only ever reaches its own account's buckets.
x-amz-request-payerStringNoNot needed: STRG has no Requester Pays buckets.

The body is signed with a SHA-256 payload hash or UNSIGNED-PAYLOAD. For a streaming (STREAMING-*) payload signature STRG answers 501 NotImplemented; the AWS SDKs never send this operation that way, so it concerns hand-written clients alone.

Request body​

A CompleteMultipartUpload XML document, up to 2 MiB. STRG checks that the body is well-formed XML with a CompleteMultipartUpload root element before passing it on, and answers 400 MalformedXML when it is not.

ElementTypeRequiredDescription
CompleteMultipartUploadContainerYesThe root element.
PartContainerYes, one or moreOne per part to include, in ascending PartNumber order. Parts you uploaded but leave out are discarded.
Part/PartNumberIntegerYesThe part number, 1-10,000.
Part/ETagStringYesThe ETag returned when the part was uploaded, quotes included.
Part/ChecksumCRC32StringNoThe part's CRC32 checksum, as UploadPart returned it. Belongs in the list when the part was uploaded with one.
Part/ChecksumCRC32CStringNoThe part's CRC32C checksum.
Part/ChecksumCRC64NVMEStringNoThe part's CRC64NVME checksum.
Part/ChecksumSHA1StringNoThe part's SHA-1 digest.
Part/ChecksumSHA256StringNoThe part's SHA-256 digest.
Part/ChecksumSHA512StringNoThe part's SHA-512 digest.
Part/ChecksumMD5StringNoThe part's MD5 digest.
Part/ChecksumXXHASH64StringNoThe part's XXHASH64 checksum.
Part/ChecksumXXHASH3StringNoThe part's XXHASH3 checksum.
Part/ChecksumXXHASH128StringNoThe part's XXHASH128 checksum.

Part sizes: every part but the last is at least 5,000,000 bytes, and a part can be up to 5 GiB.

Response​

200 OK with an XML body.

A 200 can carry an error

As on Amazon S3, a CompleteMultipartUpload that fails after the response has started still answers 200 OK, with an Error document as its body. The AWS SDKs detect this and treat it as an error, retrying where appropriate. A client calling the API without an SDK needs to read the body before treating the upload as complete. Errors lists the codes such a response can carry.

Response headers​

The headers a successful completion carries:

NameDescription
x-amz-version-idThe version ID of the new object. Buckets are always versioned, so every completed upload creates a new version.
x-amz-server-side-encryptionAES256 when the object is encrypted with STRG-managed keys.
x-amz-server-side-encryption-aws-kms-key-idNever returned: STRG encrypts with its own managed keys or yours (SSE-C), and has no SSE-KMS.
x-amz-server-side-encryption-bucket-key-enabledNever returned, for the same reason.
x-amz-expirationThe expiry date and rule ID of a lifecycle rule that applies to the object. STRG does not promise this header; the bucket's lifecycle configuration is the dependable record of its rules.
x-amz-request-idThe request ID, handy for support requests.

x-amz-request-charged is never returned.

Response body​

ElementDescription
LocationThe object's URL on the endpoint you called: https://<bucket>.s3.<region>.strg.com/<key> virtual-hosted, https://s3.<region>.strg.com/<bucket>/<key> path-style.
BucketThe bucket name.
KeyThe object key.
ETagThe object's entity tag. For a multipart object it is not the MD5 of the data: it has the form "<hex>-<number of parts>".
ChecksumCRC32, ChecksumCRC32C, ChecksumCRC64NVME, ChecksumSHA1, ChecksumSHA256The object's checksum, present when the parts were uploaded with one. A composite checksum is calculated from the part checksums, not from the whole object.
ChecksumSHA512, ChecksumMD5, ChecksumXXHASH64, ChecksumXXHASH3, ChecksumXXHASH128As above, for the corresponding algorithm.
ChecksumTypeCOMPOSITE or FULL_OBJECT, present with a checksum.
<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Location>https://my-bucket.s3.eu-central-1.strg.com/photos/2024/cat.jpg</Location>
<Bucket>my-bucket</Bucket>
<Key>photos/2024/cat.jpg</Key>
<ETag>"3729bf569924c845dfe90bfe6281a9ff-2"</ETag>
</CompleteMultipartUploadResult>

Errors​

The errors this operation can answer, and the situation behind each:

CodeHTTP statusWhen
InvalidPart400A listed part was not uploaded, or its ETag does not match the uploaded part's.
InvalidPartOrder400The parts are not in ascending PartNumber order.
EntityTooSmall400A part other than the last is smaller than 5,000,000 bytes.
InvalidRequest400The part list is empty.
MalformedXML400The body is not well-formed XML, or its root element is not CompleteMultipartUpload.
MaxMessageLengthExceeded400The body is larger than 2 MiB.
BadDigest400Content-MD5 does not match the body.
XAmzContentSHA256Mismatch400x-amz-content-sha256 does not match the body.
NoSuchUpload404The upload ID does not exist, or the upload was already completed or aborted.
NoSuchBucket404The bucket does not exist, it is in a region other than the endpoint's, or your account stores nothing in this endpoint's region.
NotImplemented501The request carries If-Match, If-None-Match, or a streaming payload signature.
AccessDenied403The access key lacks Write objects, or the bucket or key is outside its scope or prefix.
InvalidAccessKeyId403The access key does not exist, has expired or been revoked, or is not yet usable in this endpoint's region - for example a key created moments ago.
SignatureDoesNotMatch403The request signature is wrong.
RequestTimeTooSkewed403The request time is more than 15 minutes from the server's.
AuthorizationHeaderMalformed400The signature's credential scope names a region other than the endpoint's.
SlowDown503Too many requests at once. The response carries a Retry-After interval, and the AWS SDKs retry on their own.
ServiceUnavailable503A temporary failure. The request is safe to retry, and the AWS SDKs do so on their own.
InternalError500An internal failure. The request is safe to retry, and the AWS SDKs do so on their own.

An error returned inside a 200 OK (see Response) carries one of InvalidPart, InvalidPartOrder, EntityTooSmall, NoSuchUpload, SlowDown or InternalError, and no x-amz-version-id or checksum headers - no object was created.

Differences from Amazon S3​

Where STRG answers differently from Amazon S3, and what that means for your code:

AspectAmazon S3STRG
Minimum part size5 MiB (5,242,880 bytes) for every part but the last5,000,000 bytes for every part but the last - so a part size chosen for S3 works unchanged
If-Match, If-None-MatchConditional completion501 NotImplemented, answered before anything is stored
Empty part list400 MalformedXML400 InvalidRequest
LocationThe object's S3 URLThe object's URL on the STRG endpoint and addressing style you used
VersioningDepends on the bucket's versioning settingAlways on: each completed upload is a new version, and every version is billed as stored data until it is deleted or a lifecycle rule removes it
Bucket in another region301 redirect with x-amz-bucket-region404 NoSuchBucket. Each bucket answers on its own region's endpoint, which the dashboard shows beside it
x-amz-expected-bucket-ownerChecked against the bucket ownerNothing to check: a key only ever reaches buckets of its own account
SSE-KMSAvailableSTRG-managed keys (AES256) or your own (SSE-C) instead

Examples​

Completing an upload for photos/2024/cat.jpg whose parts are already uploaded, with the part list read back through ListParts:

aws s3api list-parts \
--bucket my-bucket \
--key photos/2024/cat.jpg \
--upload-id "$UPLOAD_ID" \
--endpoint-url https://s3.eu-central-1.strg.com \
--region eu-central-1 \
--query '{Parts: Parts[].{PartNumber: PartNumber, ETag: ETag}}' \
--output json > parts.json

aws s3api complete-multipart-upload \
--bucket my-bucket \
--key photos/2024/cat.jpg \
--upload-id "$UPLOAD_ID" \
--multipart-upload file://parts.json \
--endpoint-url https://s3.eu-central-1.strg.com \
--region eu-central-1

A whole upload from start to finish: a local file sent in 64 MiB parts, with the upload aborted if anything fails, so no parts are left behind:

set -euo pipefail

ENDPOINT=https://s3.eu-central-1.strg.com
REGION=eu-central-1
BUCKET=my-bucket
KEY=backups/archive.tar

UPLOAD_ID=$(aws s3api create-multipart-upload \
--bucket "$BUCKET" --key "$KEY" \
--endpoint-url "$ENDPOINT" --region "$REGION" \
--query UploadId --output text)
trap 'aws s3api abort-multipart-upload --bucket "$BUCKET" --key "$KEY" --upload-id "$UPLOAD_ID" --endpoint-url "$ENDPOINT" --region "$REGION"' ERR

WORKDIR=$(mktemp -d)
split -b 64M -d -a 5 archive.tar "$WORKDIR/part-"

number=1
for file in "$WORKDIR"/part-*; do
aws s3api upload-part \
--bucket "$BUCKET" --key "$KEY" --upload-id "$UPLOAD_ID" \
--part-number "$number" --body "$file" \
--endpoint-url "$ENDPOINT" --region "$REGION" > /dev/null
number=$((number + 1))
done

aws s3api list-parts \
--bucket "$BUCKET" --key "$KEY" --upload-id "$UPLOAD_ID" \
--endpoint-url "$ENDPOINT" --region "$REGION" \
--query '{Parts: Parts[].{PartNumber: PartNumber, ETag: ETag}}' \
--output json > "$WORKDIR/parts.json"

aws s3api complete-multipart-upload \
--bucket "$BUCKET" --key "$KEY" --upload-id "$UPLOAD_ID" \
--multipart-upload "file://$WORKDIR/parts.json" \
--endpoint-url "$ENDPOINT" --region "$REGION"

rm -r "$WORKDIR"
Unfinished uploads are billed

The parts of an upload that is never completed or aborted are billed as stored data. Every upload that is started wants to end in CompleteMultipartUpload or AbortMultipartUpload; a lifecycle rule that aborts incomplete multipart uploads automatically is a good safety net - Lifecycle policies shows one.

View as Markdown