Make the code scanners happy with a bounds check before we do the
integer conversion from int64 to int, since this can overflow on
32 bit platforms.
Best error to return here is a signature error since this is a
client problem and the chunk headers are considered part of the
request signature.
Fixes#512. For chunked uploads, we parse the chunk headers in place
and then move the data payload up on the buffer to overwrite the
chunk headers for the real data stream.
For the special case where the chunk header was truncated in the
current read buffer, the partial header is stashed in a temporary
byte slice. The following read will contain the remainder of the
header that we can put together and parse.
We were correctly parsing this, but we forgot that the data offset
is calculated based on the start of the header. But the special
case where part of the header was stashed means we were incorrectly
calculating the data offset into the read buffer.
Easy fix to just remove the stash size from the data offset return
value.
Fixes#457. There are some buggy clients that include headers not
actually set on the request in the signed headers list. For these
we need to include them in the signature canoncal string with
empty values.
This fixes the case where clients can include the content-length
header in the signed headers for a 0 length file (like s3cmd).
Since we had to hoist the aws code into versitygw, we can also
remove the hack for the "User-Agent" header in the hard coded
excludes list and just remove it from the excludes list.
This is a hack to replace the ignored headers in the aws-sdk-go-v2
internal/v4 package. The headers in the default ignore list include
User-Agent, but this is included is signed headers from some clients.
fixes#396
As described in
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
this adds support for reading from a chunked upload encoded request
body. The chunked reader modifies the data stream to remove the
chunk encoding while validating the chunk signatures in line. This
allows the upper layers to get just the object data stream.
This change removes all spaces after the algorithm to have
standard parsing for the following key/value pairs. This fixes
some clients that were using a slightly different format than
the example AWS request strings.
This builds on the previous work that sets up the body streaming
for the put object and put part requests. This adds the auth and
checksum readers to postpone the v4auth checks and the content
checksum until the end of the body stream.
This means that the backend with start reading the data from the
body stream before the request is fully validated and signatures
checked. So the backend must check the error returned from the
body reader for the final auth and content checks. The backend
is expected to discard the data upon error.
This should increase performance and reduce memory utilization
to no longer require caching the entire request body in memory
for put object and put part.
* feat: Added admin CLI, created api endpoint for creating new user, created action for admin CLI to create a new user, changed the authentication middleware to verify the users from db
* feat: Added both single and multi user support, added caching layer for getting IAM users
* fix: Added all the files
* fix: Fixed error cases of primitive values
* feat: Added unit test for: DeleteBucket, DeleteObjects, DeleteActions, HeadBucket, HeadObject, CreateActions controllers
* feat: Added unit tests for GetUserMetaData, CreateHttpRequestFromCtx, MarshalStructToXML utility functions
* fix: fixed CreateHttpRequestFromCtx unit test case
