We were getting errors such as:
2025/02/07 19:24:28 Internal Error, write object data: write exceeds content length 87
whenever the chunk encoding did not have the correct chunk
signatures. The issue was that the chunk encoding reader
was reading from the underlying reader and then passing the full
buffer read back to the caller if the underlying reader returned
an error. This meant that we were not processing the chunk
headers within the buffer due to the higher level error, and
would possibly hand back the longer unprocessed chunk encoded
stream to the caller that was in turn trying to write to the
object file exceeding the content length limit.
Fixes#1056
An invalid chunk encoding, or parse errors leading to parsing
invalid data can lead to a server panic if the chunk header
remaining is determined to be larger than the max buffer size.
This was previously seen when the chunk trailer checksums were
used by the client without the support from the server side
for this encoding. Example panic:
panic: runtime error: slice bounds out of range [4088:1024]
goroutine 5 [running]:
github.com/versity/versitygw/s3api/utils.(*ChunkReader).parseChunkHeaderBytes(0xc0003c4280, {0xc0000e6000?, 0x3000?, 0x423525?})
/home/tester/s3api/utils/chunk-reader.go:242 +0x492
github.com/versity/versitygw/s3api/utils.(*ChunkReader).parseAndRemoveChunkInfo(0xc0003c4280, {0xc0000e6000, 0x3000, 0x8000})
/home/tester/s3api/utils/chunk-reader.go:170 +0x20b
github.com/versity/versitygw/s3api/utils.(*ChunkReader).Read(0xc0003c4280, {0xc0000e6000, 0xc0000b41e0?, 0x8000})
/home/tester/s3api/utils/chunk-reader.go:91 +0x11e
This fix will validate the data length before copying into the
temporary buffer to prevent a panic and instead just return
an error.
