25 Commits

Author SHA1 Message Date
Ben McClelland
1366408baa feat: add chunked upload support
As described in
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
this adds support for reading from a chunked upload encoded request
body. The chunked reader modifies the data stream to remove the
chunk encoding while validating the chunk signatures in line. This
allows the upper layers to get just the object data stream.
2024-01-22 11:35:01 -08:00
Ben McClelland
b5894dd714 fix: allow spaces in Authorization string
This change removes all spaces after the algorithm to have
standard parsing for the following key/value pairs. This fixes
some clients that were using a slightly different format than
the example AWS request strings.
2024-01-17 10:45:57 -08:00
Ben McClelland
ba501e482d feat: steaming requests for put object and put part
This builds on the previous work that sets up the body streaming
for the put object and put part requests. This adds the auth and
checksum readers to postpone the v4auth checks and the content
checksum until the end of the body stream.

This means that the backend with start reading the data from the
body stream before the request is fully validated and signatures
checked. So the backend must check the error returned from the
body reader for the final auth and content checks. The backend
is expected to discard the data upon error.

This should increase performance and reduce memory utilization
to no longer require caching the entire request body in memory
for put object and put part.
2023-12-14 19:19:46 -08:00
jonaustin09
27eb43d089 feat: Closes #290, implemented request body stream reading for PutObject and UploadPart actions. 2023-12-14 18:27:51 -08:00
jonaustin09
9d813def54 fix: Fixes 275, Changed the gateway request URL encoding, to accept some more special characters 2023-10-06 15:51:38 -04:00
jonaustin09
856d79d385 fix: Fixes #274, Fixed putting and getting object metadata case normalization issue 2023-10-05 15:33:03 -04:00
jonaustin09
a67a2e5c8f fix: Fixes #249, Changed ListObjects default max-keys from -1 to 1000 2023-09-23 21:03:54 -07:00
jonaustin09
cb2f6a87aa fix: Fixes #207, Fixes #198: added lexicographical order by object key and uploadId for ListMultipartUploads response, Added FE support to pass the necessary arguments to BE for ListMultipartUploads 2023-09-01 15:33:58 -04:00
jonaustin09
dde13ddc9a fix: Fixes #183. Added a validation for max-keys for ListObjects/ListObjectsV2 2023-08-03 20:47:34 +04:00
Jon Austin
77080328c1 Issue 156 (#167)
* fix: Fixes #156, Added bucket name validation on bucket creation
2023-07-27 11:04:50 -07:00
jonaustin09
9db01362a0 feat: increased unit testing coverage in controllers, utility functions and server functions. Fixed bucket owner bug in putbucketacl. 2 more minor changes in controllers 2023-06-30 22:04:46 +04:00
jonaustin09
c01d3ed542 feat: control over logging in debug mode and control logging for specific actions 2023-06-20 19:39:58 +04:00
jonaustin09
127b79e148 feat: Logging system set up 2023-06-19 23:18:16 +04:00
Jon Austin
160a99cbbd feat: Added admin CLI, created api endpoint for creating new user, cr… (#68)
* feat: Added admin CLI, created api endpoint for creating new user, created action for admin CLI to create a new user, changed the authentication middleware to verify the users from db

* feat: Added both single and multi user support, added caching layer for getting IAM users

* fix: Added all the files
2023-06-09 10:30:20 -07:00
Ben McClelland
002c427e7d fix signature check when content length not included 2023-06-07 08:37:14 -07:00
Ben McClelland
1717d45664 fix: v4 auth signature to only use specified signed headers 2023-06-06 13:28:17 -07:00
Jon Austin
83b494a91f feat: Head object response serialization (#58) 2023-06-06 08:41:47 -07:00
Jon Austin
3cfee3a032 Utils unit tests (#54)
* fix: Fixed error cases of primitive values

* feat: Added unit test for: DeleteBucket, DeleteObjects, DeleteActions, HeadBucket, HeadObject, CreateActions controllers

* feat: Added unit tests for GetUserMetaData, CreateHttpRequestFromCtx, MarshalStructToXML utility functions

* fix: fixed CreateHttpRequestFromCtx unit test case
2023-06-06 08:38:12 -07:00
Ben McClelland
b6f3ea3350 fix upload from aws cli 2023-06-05 11:38:52 -07:00
jonaustin09
04bbe61826 fix: Removed root user flags 2023-06-01 00:16:01 +04:00
jonaustin09
f174308e3f fix: Merge conflicts resolved 2023-05-31 22:41:52 +04:00
jonaustin09
ecd28bc2f7 feat: Completed SigV4 authentication for the root user 2023-05-31 22:20:58 +04:00
jonaustin09
510cf6ed57 feat: Added root user flags on application start 2023-05-31 15:26:19 +04:00
Ben McClelland
5cbcf0c900 add copyright headers to source files 2023-05-28 14:38:45 -07:00
jonaustin09
09d42c92fd feat: Changed PutObject argument list, added used defined metadata and content length 2023-05-24 15:18:37 +04:00