Details
Closing the socket allows the handler to return IncompleteBody and release the pages. Keeping multiple sockets open retains multiple reservations. A sufficiently large declared length can terminate the process before a response is returned.
The final automated validation performed this sequence through the real HTTP listener rather than calling waitForTurn or Reserve directly:
- Started an anonymous
serve s3server on loopback with a local streaming-capable backend. - Set
MultipartStreamingBufferLimitto 1 MiB. - Created a multipart upload with an HTTP
POSTand parsed its returned upload ID. - Recorded
pool.Global().InUse()after upload creation. - Opened a raw TCP connection and sent an
UploadPartrequest declaringContent-Length: 16777216. - Sent no request-body bytes.
- Observed the in-use count increase by at least sixteen 1 MiB pages while the connection remained open.
- Closed the connection, producing the expected short-body
unexpected EOFlog rather than completing an upload.
The central assertion was:
const declared = int64(16 << 20)
baseline := pool.Global().InUse()
connection, err := net.DialTimeout("tcp", server.Addr().String(), 5*time.Second)
require.NoError(t, err)
_, err = fmt.Fprintf(connection,
"PUT %s HTTP/1.1\r\nHost: %s\r\nContent-Length: %d\r\nConnection: close\r\n\r\n",
partPath, server.Addr().String(), declared)
require.NoError(t, err)
wantPages := int(declared / int64(pool.BufferSize))
require.Eventually(t, func() bool {
return pool.Global().InUse()-baseline >= wantPages
}, 5*time.Second, 10*time.Millisecond)
It passed on Windows/amd64 with Go 1.26.2:
=== RUN TestSecurityValidationS3MultipartHeaderAllocatesBeforeBody
NOTICE: serve s3: No auth provided so allowing anonymous access
ERROR : serve s3: unexpected EOF
--- PASS: TestSecurityValidationS3MultipartHeaderAllocatesBeforeBody (0.06s)
This demonstrates network reachability, header-only amplification, admission beyond the configured 1 MiB reorder limit, and actual allocation. The test deliberately capped the reservation at 16 MiB; it did not attempt to exhaust the validation host. The same code path reserves pages linearly as the declared length increases.
Impact Assessment
A network client can force the S3 server to reserve memory proportional to an unverified request header before paying the bandwidth cost of sending the declared body. One large part can exceed the out-of-order buffering limit; concurrent uploads multiply memory consumption.
The directly observed primitive is memory reservation proportional to an unverified header before any body bytes arrive. At a sufficiently large declared length, or across concurrent requests, this can exhaust process or host memory, terminate the process, or permanently block request goroutines when the global memory semaphore cannot satisfy an oversized acquisition. Those outcomes cause loss of S3 service availability; no confidentiality or integrity impact is required.
Unauthenticated exploitation applies when the operator uses the documented anonymous S3 mode. With auth_key, the attacker must possess an accepted key. Binding to loopback or a trusted management network removes untrusted network reachability but does not correct the resource-accounting defect.
Remediation Guidance
Do not reserve the declared content length before reading verified bytes. Separate the current in-order part from out-of-order buffering:
- For
partNumber == nextPart, stream the request body directly into the upload pipe while computing MD5. This path does not need a full-part memory buffer merely to return the ETag after the body has streamed. - For out-of-order parts, allocate incrementally as body bytes arrive and charge each page against the per-upload budget before allocation. Apply backpressure, spool to a bounded temporary file, or reject the request when the budget is exhausted.
- Remove
Reserve(contentLength)from the untrusted HTTP path. If preallocation remains as an optimization, cap it to a small trusted amount and grow only after bytes are received and accounted. - Enforce an explicit maximum part size before allocation, including both
Content-LengthandX-Amz-Decoded-Content-Length. Match the intended S3 compatibility limit and return an S3-compatible error such asEntityTooLargeorInvalidRequest. - Reject negative, overflowing, or platform-
int-unrepresentable page counts before arithmetic or conversion. - Apply a total server-wide budget across uploads in addition to the per-upload reorder budget. The budget acquisition must use the request context and must fail immediately when a single request exceeds capacity; do not wait forever on
context.Background()for an impossible weight. - Limit concurrent multipart uploads and idle request-body time so a client cannot retain reservations indefinitely.
- Reconcile the option documentation with the actual guarantee. If one part can exceed the reorder limit for compatibility, state that explicitly, but still enforce an independent safe maximum or incremental allocation.