Closes#1159Fixes#1161
Implements signed chunk encoding with trailers in the gateway.
The signed encoding (both with and without trailers) is now handled by the `ChunkReader`.
Fixes the `ChunkReader` implementation to validate encoding headers byte by byte.
The chunk encoding with trailers follows the general signed chunk encoding pattern, but the final chunk includes the trailing signature (`x-amz-trailing-signature`) and the checksum header (`x-amz-checksum-x`, where `x` can be `crc32`, `crc32c`, `sha1`, `sha256`, or `crc64nvme`).
Adds validation for the `X-Amz-Trailer` header.
Make the code scanners happy with a bounds check before we do the
integer conversion from int64 to int, since this can overflow on
32 bit platforms.
Best error to return here is a signature error since this is a
client problem and the chunk headers are considered part of the
request signature.
Fixes#512. For chunked uploads, we parse the chunk headers in place
and then move the data payload up on the buffer to overwrite the
chunk headers for the real data stream.
For the special case where the chunk header was truncated in the
current read buffer, the partial header is stashed in a temporary
byte slice. The following read will contain the remainder of the
header that we can put together and parse.
We were correctly parsing this, but we forgot that the data offset
is calculated based on the start of the header. But the special
case where part of the header was stashed means we were incorrectly
calculating the data offset into the read buffer.
Easy fix to just remove the stash size from the data offset return
value.
As described in
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
this adds support for reading from a chunked upload encoded request
body. The chunked reader modifies the data stream to remove the
chunk encoding while validating the chunk signatures in line. This
allows the upper layers to get just the object data stream.
