Files
seaweedfs/weed/s3api
Chris Lu d65c568cbb fix(s3api): validate SSE-S3 chunk IV length; add multipart direct reader tests (#9218)
* fix(s3api): validate SSE-S3 chunk IV length; add multipart direct reader tests

DeserializeSSES3Metadata does not require an IV, and a corrupted or
legacy chunk without one would have flowed into cipher.NewCTR and
panicked. Validate that each per-chunk IV is exactly AESBlockSize bytes
before decryption, closing the current and any already-appended chunk
readers on error.

Factor the per-chunk decryption loop out of
createMultipartSSES3DecryptedReaderDirect into buildMultipartSSES3Reader
so it can be driven with a mock chunk fetcher, and add tests covering:
the happy path with two parts (distinct per-chunk DEKs/IVs, out-of-order
chunks) to lock in the fix from #9211; missing-IV and short-IV metadata
rejection without panic; and reader cleanup when a later chunk fails.

* address review: sort chunks copy; close encryptedStream on error

- buildMultipartSSES3Reader now sorts a copy of the chunks slice so
  callers do not observe entry.Chunks reordered (other code paths,
  e.g. ETag computation, can rely on the original order).
- createMultipartSSES3DecryptedReaderDirect now closes encryptedStream
  on the error path from buildMultipartSSES3Reader. All current
  callers pass nil, but this keeps cleanup symmetric with the
  success path.
- Extend TestBuildMultipartSSES3Reader_PerChunkKeys to assert the
  input slice is not mutated.

* address review: defer single close; extend chunk-copy + IV-guard pattern

- createMultipartSSES3DecryptedReaderDirect: collapse the duplicated
  encryptedStream.Close() calls into a single nil-guarded defer so the
  error and success paths share cleanup.
- createMultipartSSECDecryptedReaderDirect,
  createMultipartSSEKMSDecryptedReaderDirect: sort a copy of entry.Chunks
  instead of mutating the caller's slice, matching the SSE-S3 helper.
- createMultipartSSECDecryptedReaderDirect: validate per-chunk IV length
  before handing it to cipher.NewCTR; a base64-decoded empty or short
  IV from malformed/corrupt metadata would otherwise panic.
- SSE-KMS needs no IV guard: CreateSSEKMSDecryptedReader already calls
  ValidateIV before cipher.NewCTR. Note recorded in the sort comment.

* address review: close appended readers on SSE-C/SSE-KMS error paths

createMultipartSSECDecryptedReaderDirect and
createMultipartSSEKMSDecryptedReaderDirect only closed the current chunk
reader on error and leaked any chunk readers already appended to the
local readers slice, mirroring the leak previously fixed in the SSE-S3
helper. Add the same closeAppendedReaders() closure pattern to both
functions and invoke it on every error return inside the loop so failed
requests do not leak volume-server HTTP connections.

* address review: defer encryptedStream close in SSE-C/SSE-KMS; drop chunks reassignment

- Move encryptedStream.Close() to a nil-guarded defer at the top of
  createMultipartSSECDecryptedReaderDirect and
  createMultipartSSEKMSDecryptedReaderDirect so the stream is closed on
  every return path (including error returns from inside the per-chunk
  loop), mirroring the SSE-S3 helper.
- In buildMultipartSSES3Reader, iterate sortedChunks directly instead of
  reassigning chunks = sortedChunks.
2026-04-24 13:59:23 -07:00
..
2026-01-28 14:34:07 -08:00
2024-07-04 11:00:41 -07:00
2026-02-20 18:40:47 -08:00
2024-07-04 11:00:41 -07:00
2025-08-22 01:15:42 -07:00
2025-07-28 02:49:43 -07:00

see https://blog.aqwari.net/xml-schema-go/

1. go get aqwari.net/xml/cmd/xsdgen
2. Add EncodingType element for ListBucketResult in AmazonS3.xsd
3. xsdgen -o s3api_xsd_generated.go -pkg s3api AmazonS3.xsd
4. Remove empty Grantee struct in s3api_xsd_generated.go
5. Remove xmlns: sed s'/http:\/\/s3.amazonaws.com\/doc\/2006-03-01\/\ //' s3api_xsd_generated.go