Commit Graph

49 Commits

Author SHA1 Message Date
jonaustin09
600aca8bdc fix: Fixed the request uri path escape to support object key special characters 2024-09-17 13:28:30 -04:00
jonaustin09
7545e6236c feat: Implement bucket ownership controls
Bucket ACLs are now disabled by default the same as AWS.
By default the object ownership is BucketOwnerEnforced
which means that bucket ACLs are disabled. If one attempts
to set bucket ACL the following error is returned both in
the gateway and on AWS:
	ErrAclNotSupported: {
		Code:           "AccessControlListNotSupported",
		Description:    "The bucket does not allow ACLs",
		HTTPStatusCode: http.StatusBadRequest,
	},

ACls can be enabled with PutBucketOwnershipControls

Changed bucket canned ACL translation

New backend interface methods:
PutBucketOwnershipControls
GetBucketOwnershipControls
DeleteBucketOwnershipControls

Added these to metrics
2024-06-28 21:03:09 -07:00
jonaustin09
be6f9a86cd feat: Integrated metric manager into the gateway 2024-05-28 12:50:50 -07:00
jonaustin09
2a2f9c827c feat: Closes #484. Added support to run the gateway on read only mode 2024-05-06 16:41:39 -04:00
jonaustin09
fbaba0b944 feat: Added object WORM protection by object-lock feature from AWS with the following actions support: PutObjectLockConfiguration, GetObjectLockConfiguration, PutObjectRetention, GetObjectRetention, PutObjectLegalHold, GetObjectLegalHold 2024-04-22 13:13:40 -07:00
jonaustin09
e6203c5765 feat: Closes #441, Added access control integration tests, fixed some bugs in bucket policy and acl access checking flow 2024-03-28 14:52:56 -04:00
jonaustin09
d4f17bf32f feat: Added bucket policy actions implementation in FE 2024-03-06 13:56:29 -05:00
jonaustin09
9b989970d0 fix: Fixes #420, Fixed bucket creation bug, which contains closing / in the url 2024-02-29 16:06:13 -05:00
jonaustin09
e6852b3a99 feat: Closes 417, Added the following versioning related actions: PutBucketVersioning, GetBucketVersioning, ListObjectVersions. Added versionId support in FE for the following actions: GetObject, DeleteObject 2024-02-28 09:48:05 -05:00
jonaustin09
fa54dfeb9f feat: Added userplus role in IAM, who has the same opportunities as the user, but may also create a bucket 2024-02-20 14:24:26 -05:00
jonaustin09
e21e514997 feat: Added 20 integration tests for v4 authentication with query params. Fixed few bugs in v4 query params authentication 2024-02-12 16:31:01 -05:00
jonaustin09
be17b3fd33 feat: Closes #355. Added support for presigned URLs, particularly v4 authentication with query params 2024-02-07 09:17:35 -05:00
jonaustin09
d70ea61830 feat: Added the following actions support in posix backend: PutBucketTagging, GetBucketTagging, DeleteBucketTagging 2024-01-31 10:09:48 -08:00
Ben McClelland
1366408baa feat: add chunked upload support
As described in
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
this adds support for reading from a chunked upload encoded request
body. The chunked reader modifies the data stream to remove the
chunk encoding while validating the chunk signatures in line. This
allows the upper layers to get just the object data stream.
2024-01-22 11:35:01 -08:00
Ben McClelland
ba501e482d feat: steaming requests for put object and put part
This builds on the previous work that sets up the body streaming
for the put object and put part requests. This adds the auth and
checksum readers to postpone the v4auth checks and the content
checksum until the end of the body stream.

This means that the backend with start reading the data from the
body stream before the request is fully validated and signatures
checked. So the backend must check the error returned from the
body reader for the final auth and content checks. The backend
is expected to discard the data upon error.

This should increase performance and reduce memory utilization
to no longer require caching the entire request body in memory
for put object and put part.
2023-12-14 19:19:46 -08:00
jonaustin09
27eb43d089 feat: Closes #290, implemented request body stream reading for PutObject and UploadPart actions. 2023-12-14 18:27:51 -08:00
jonaustin09
9d813def54 fix: Fixes 275, Changed the gateway request URL encoding, to accept some more special characters 2023-10-06 15:51:38 -04:00
Ben McClelland
ae4e382e61 feat: refactor internal iam service
This moves the internal iam service from the posix backend so
that we can start implementing new iam services right in the auth
module.

The internal iam service has same behavior as before, but now
must be enabled with the --iam-dir cli option.

New single user service is the default when no other iam service
is selected. This just runs the gateway in single user mode with
just the root account.
2023-10-02 11:12:18 -07:00
Ben McClelland
4661af11dd feat: replace access/role context locals with full account info 2023-10-02 10:59:59 -07:00
jonaustin09
cf1c44969b fix: Fixes #239, Change SigV4 date comparison with UTC 2023-09-13 15:16:50 -04:00
jonaustin09
53cf4f342f feat: Added more integration test cases for the authentication and md5 checker 2023-08-30 23:21:09 +04:00
jonaustin09
a58ce0c238 feat: Added 8 integration test cases for authentication 2023-08-29 23:46:54 +04:00
jonaustin09
3573a31ae6 fix: Closes #192, Fixed authentication errors returned, created integration test cases for it 2023-08-25 21:50:21 +04:00
jonaustin09
7814979efa feat: Fixes #181, Added support to add object with special character keys, disabled URI path escaping in v4 signing, add a middleware to parse the URL and store the decoded version as a new URL, added test cases for adding/getting/listing objects with special characters 2023-08-08 00:41:06 +04:00
jonaustin09
3f8c218431 fix: Fixes #143. Fixed action name in bucket creation admin checker response handler 2023-07-31 20:54:16 +04:00
Ben McClelland
884fd029c3 feat: add context to backend calls
This adds a context to the backend interface calls so that the backend
can enable request cancellation. This change isn't acutally implementing
any backend handling, but just putting the pieces into place to pass the
context to the backend.
2023-07-26 21:54:12 -07:00
jonaustin09
7fb2a7f9ba feat: ACL refactoring, moved ace parsing from controllers to middleware 2023-07-26 20:54:50 +04:00
jonaustin09
fe547a19e9 feat: bucket event notifications
Set up Bucket event notifications interface to send aws compatible format event messages to a configured event service.
First integrated service is kafka message broker as an option for bucket event notifications.
2023-07-20 11:37:14 -07:00
jonaustin09
87d61a1eb3 feat: Setup audit loggin with webhook url and root level access.log file. CLI enables either webhook or server access logs by providing the flags 2023-07-14 23:40:05 +04:00
jonaustin09
a10729b3ff fix: Fixed staticcheck error 2023-07-06 19:14:01 +04:00
jonaustin09
0330685c5c feat: Added support for unsigned, streamable and trailign payload header in sigv4 authentication 2023-07-06 19:03:19 +04:00
jonaustin09
c01d3ed542 feat: control over logging in debug mode and control logging for specific actions 2023-06-20 19:39:58 +04:00
jonaustin09
0209ca4bc0 fix: fixed merge conflicts 2023-06-19 23:20:33 +04:00
jonaustin09
127b79e148 feat: Logging system set up 2023-06-19 23:18:16 +04:00
Ben McClelland
0f733ae0c8 refactor move auth to top level 2023-06-19 11:15:19 -07:00
Ben McClelland
94808bb4a9 refactor iam service for blind backend store 2023-06-19 09:53:19 -07:00
jonaustin09
49af6f0049 feat: ACL set up finished: added VerifyACL function, added admin checker function on list buckets, fixed all the unit tests 2023-06-16 20:55:23 +04:00
jonaustin09
21f0fea5a7 feat: Implemented GetBucketACL, PutBucketACL posix functions, fixed authentication middleware signed headers bug 2023-06-14 22:39:27 +04:00
Ben McClelland
33e1d39138 cleanup responses to split out expected xml body response 2023-06-12 07:46:09 -07:00
Jon Austin
160a99cbbd feat: Added admin CLI, created api endpoint for creating new user, cr… (#68)
* feat: Added admin CLI, created api endpoint for creating new user, created action for admin CLI to create a new user, changed the authentication middleware to verify the users from db

* feat: Added both single and multi user support, added caching layer for getting IAM users

* fix: Added all the files
2023-06-09 10:30:20 -07:00
Ben McClelland
002c427e7d fix signature check when content length not included 2023-06-07 08:37:14 -07:00
Ben McClelland
1717d45664 fix: v4 auth signature to only use specified signed headers 2023-06-06 13:28:17 -07:00
Ben McClelland
bec87757a3 verify payload md5 when Content-Md5 set 2023-06-06 08:39:24 -07:00
Ben McClelland
b6f3ea3350 fix upload from aws cli 2023-06-05 11:38:52 -07:00
jonaustin09
489bb3e899 feat: Server side region added to AdminConfig, v4 signature calculation implemented with server side region 2023-06-01 00:23:50 +04:00
jonaustin09
04bbe61826 fix: Removed root user flags 2023-06-01 00:16:01 +04:00
jonaustin09
f174308e3f fix: Merge conflicts resolved 2023-05-31 22:41:52 +04:00
jonaustin09
ecd28bc2f7 feat: Completed SigV4 authentication for the root user 2023-05-31 22:20:58 +04:00
jonaustin09
510cf6ed57 feat: Added root user flags on application start 2023-05-31 15:26:19 +04:00