65 Commits

Author SHA1 Message Date
Ben McClelland
c1587e4c1c fix: make s3 PutBucketAcl have receiver pointer like others for consistency 2023-12-26 10:31:56 -08:00
Ben McClelland
6e37096b35 fix: s3 backend performance increase with client reuse
The previous way of initializing the s3 client in each call was
adding a lot of overhead and would tank performance beyond about
20 simultaneous requests.

Since the backend access is through a single account, we can init
and store this client for use from each api call.
2023-12-22 15:09:44 -08:00
Ben McClelland
2fb5ecfbc4 chore: clarify s3 backend struct with better name 2023-12-16 09:50:26 -08:00
Ben McClelland
ba501e482d feat: steaming requests for put object and put part
This builds on the previous work that sets up the body streaming
for the put object and put part requests. This adds the auth and
checksum readers to postpone the v4auth checks and the content
checksum until the end of the body stream.

This means that the backend with start reading the data from the
body stream before the request is fully validated and signatures
checked. So the backend must check the error returned from the
body reader for the final auth and content checks. The backend
is expected to discard the data upon error.

This should increase performance and reduce memory utilization
to no longer require caching the entire request body in memory
for put object and put part.
2023-12-14 19:19:46 -08:00
Jon Austin
90bb43f7c9 S3 Proxy PutBucketAcl reference bug (#341)
* feat: implemented the logic to run integration tests separately

* fix: Fixes #336, fixed s3 proxy PutBucketAcl action's referance bug
2023-12-14 09:43:02 -08:00
Jon Austin
206231f27b S3 Proxy DeleteObjects empty input (#340)
* feat: implemented the logic to run integration tests separately

* fix: Fixes #335, fixed s3 proxy DeleteObject empty input bug
2023-12-14 09:42:22 -08:00
jonaustin09
ef5a94420c feat: Created admin CLI actions in s3 proxy, Created iam proxy for proxy server 2023-12-11 11:05:08 -08:00
jonaustin09
c5007a68aa eat: Closes #317, Changed s3 proxy behavior to run in single user mode with CLI provided credentials 2023-12-04 10:06:59 -05:00
Ben McClelland
c48f126557 Merge pull request #318 from versity/fix/issue-300-proxy-listbuckets-panic
S3 proxy ListBucket panic
2023-11-29 09:51:17 -08:00
Jon Austin
315cb0ea81 fix: Fixes #307, Fixed ListMultipartUploads and ListParts parts timestamp in s3 proxy (#316) 2023-11-29 08:54:12 -08:00
jonaustin09
c4b019f6ee fix: Fixes #300, Removed Displayname field from ListBuckets output as it's not supported in the gateway 2023-11-29 11:49:27 -05:00
jonaustin09
2b22509a90 fix: Fixes #302, #308. Added error handling function in s3 proxy implementation, which converts sdk error to API Errors 2023-11-29 08:40:06 -08:00
Ben McClelland
cd8ad7d482 fix: breaking changes with aws sdk updates 2023-11-28 13:51:32 -08:00
jonaustin09
3a60dcd88f feat: Added proxy backend actions: DeleteObjectTagging, GetObjectTagging, PutObjectTagging, PutBucketAcl, GetBucketAcl 2023-10-11 09:00:02 -07:00
Ben McClelland
f58646b58d feat: s3proxy backend
This backend redirects incoming requests to another s3 service.
This will use the incoming credentials to setup the client
requests to the external s3 service. So the IAM accounts (or
root account) must match what the external s3 service expects.
2023-10-11 09:00:02 -07:00