Commit Graph

81 Commits

Author SHA1 Message Date
jonaustin09
4d6ec783bf feat: Implements pagination for ListBuckets 2024-10-28 16:26:08 -04:00
jonaustin09
c803af4688 fix: Prevents bucket deletion when it contains object versions by returning ErrVersionedBucketNotEmpty error. Enabled object deletion with versionId and delete markers creation with DeleteObject when the versioning status is Suspended 2024-10-18 15:36:52 -04:00
jonaustin09
7b5765bd59 fix: Changed the GetBucketVersioning action return type, to return empty result for unset versioning configuration 2024-09-27 18:14:53 -04:00
jonaustin09
8252ecd452 feat: basic logic implementation of bucket object versioning in posix backend
New posix backend option --versioning-dir will enable storing object versions
in specified directory.
2024-09-18 13:04:34 -07:00
jonaustin09
684ab2371b fix: Changed ListObjects and ListObjectsV2 actions return types
Changed ListObjectsV2 and ListObjects actions return types from
*s3.ListObjects(V2)Output to s3response.ListObjects(V2)Result.

Changed the listing objects timestamp to RFC3339 to match AWS
S3 objects timestamp.

Fixes #752
2024-08-26 15:46:45 -07:00
jonaustin09
cc3c62cd9d fix: Change CreateMultipartUpload return type to match expected xml response
The AWS spec for the create multipart upload response is:
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult>
   <Bucket>string</Bucket>
   <Key>string</Key>
   <UploadId>string</UploadId>
</InitiateMultipartUploadResult>

So we need the return type to marshal to this xml format.
2024-08-21 14:49:39 -07:00
jonaustin09
23fd0d3fdd fix: Fixed PutBucketAcl action error handling, removed the bucket owner check for all the acl options 2024-08-12 15:27:03 -04:00
jonaustin09
e773872c48 feat: Implemented response body streaming for GetObject action 2024-07-08 15:56:24 -04:00
jonaustin09
7545e6236c feat: Implement bucket ownership controls
Bucket ACLs are now disabled by default the same as AWS.
By default the object ownership is BucketOwnerEnforced
which means that bucket ACLs are disabled. If one attempts
to set bucket ACL the following error is returned both in
the gateway and on AWS:
	ErrAclNotSupported: {
		Code:           "AccessControlListNotSupported",
		Description:    "The bucket does not allow ACLs",
		HTTPStatusCode: http.StatusBadRequest,
	},

ACls can be enabled with PutBucketOwnershipControls

Changed bucket canned ACL translation

New backend interface methods:
PutBucketOwnershipControls
GetBucketOwnershipControls
DeleteBucketOwnershipControls

Added these to metrics
2024-06-28 21:03:09 -07:00
Ben McClelland
f0005a0047 fix: restore object request handler and scoutfs glacier enable
The restore object api request handler was incorrectly trying to
unmarshal the request body, but for the stadnard (all?) case the
request body is emtpy. We only need the bucket and opbject params
for now.

This also adds a fix to actually honor the enable glacier mode
in scoutfs.
2024-06-11 12:57:46 -07:00
jonaustin09
dbfd9e5171 fix: Removed required request body check for PutBucketAcl action 2024-05-29 14:13:38 -07:00
jonaustin09
be6f9a86cd feat: Integrated metric manager into the gateway 2024-05-28 12:50:50 -07:00
jonaustin09
fb27e2703e feat: Implemented to logic to bypass governance retention 2024-05-24 13:50:41 -04:00
jonaustin09
4bde84eafd fix: Added status property validation for PutObjectLegalHold action 2024-05-17 12:39:27 -04:00
jonaustin09
2a2f9c827c feat: Closes #484. Added support to run the gateway on read only mode 2024-05-06 16:41:39 -04:00
jonaustin09
0c3771ae2d feat: Added GetObjectAttributes actions implementation in posix, azure and s3 backends. Added integration tests for GetObjectAttributes action 2024-04-29 15:31:53 -04:00
jonaustin09
89755ea5aa feat: Changed object lock actions interface to put/get []byte 2024-04-22 13:19:09 -07:00
jonaustin09
fbaba0b944 feat: Added object WORM protection by object-lock feature from AWS with the following actions support: PutObjectLockConfiguration, GetObjectLockConfiguration, PutObjectRetention, GetObjectRetention, PutObjectLegalHold, GetObjectLegalHold 2024-04-22 13:13:40 -07:00
jonaustin09
84ce40fb54 fix: Fixes #487, added response headers for HeadBucket action 2024-04-18 13:27:45 -04:00
Ben McClelland
4be4dc2971 feat: add more debug logging for api handler errors
There are a few cases where parsing, validations checks, etc
error details are getting lost with the more generic error
responses. This add some opt-in debug logging to log more
info for these various error cases.
2024-04-06 20:08:16 -07:00
jonaustin09
af641e5368 feat: Added integration test cases for Put/Get/DeleteBucketPolicy actions. Made some bug fixes in these actions implementations 2024-03-20 17:31:52 -04:00
Ben McClelland
f7655dab9b fix: delete object xml response should be DeleteResult instead of DeleteObjectsResult 2024-03-09 10:20:15 -08:00
jonaustin09
d4f17bf32f feat: Added bucket policy actions implementation in FE 2024-03-06 13:56:29 -05:00
jonaustin09
e6852b3a99 feat: Closes 417, Added the following versioning related actions: PutBucketVersioning, GetBucketVersioning, ListObjectVersions. Added versionId support in FE for the following actions: GetObject, DeleteObject 2024-02-28 09:48:05 -05:00
jonaustin09
3cce3a5201 feat: Added unit and integration test cases for posix bucket tagging related actions 2024-01-31 10:09:48 -08:00
Ben McClelland
6481e2aac5 fix: cleanup backend ACLs
This adds the default ACL to the CreateBucket backend method so
that the backend doesn't need to know how to construct and ACL.

This also moves the s3proxy ACLs to a tag key/value because the
gateway ACLs are not the same accounts as the backend s3 server.
TODO: we may need to mask this tag key/value if we add support
for the Get/PutBucketTagging API.
2024-01-10 09:36:00 -08:00
jonaustin09
48818927bb feat: Fixes #286, Created a struct which handles s3 select event streaming and event message construction 2023-12-06 14:02:36 -08:00
Ben McClelland
cd8ad7d482 fix: breaking changes with aws sdk updates 2023-11-28 13:51:32 -08:00
Ben McClelland
4661af11dd feat: replace access/role context locals with full account info 2023-10-02 10:59:59 -07:00
jonaustin09
099ac39f22 fix: Fixes #259, Changed delete actions response statuses from 200 to 204 2023-09-23 21:21:47 -07:00
jonaustin09
4be5d64c8b fix: Object tag actions cleanup 2023-09-23 21:00:45 -07:00
jonaustin09
e0c09ad4d9 fix: Fixes #247, Changed DeleteObjectTagging action successful response status from 200 to 204 2023-09-20 12:07:52 -04:00
jonaustin09
23c26d802c fix: Fixes #216, Fixed vet warnings, removed the code snippet which copied fiber.Ctx 2023-09-08 11:57:17 -04:00
jonaustin09
8d2e2a4106 fix: Fixes #204, Change ListBuckets action logic to return all the buckets for admin users and the buckets owned by a user for regular users. Added integration test cases for ListBuckets action 2023-09-07 14:49:47 -04:00
jonaustin09
3573a31ae6 fix: Closes #192, Fixed authentication errors returned, created integration test cases for it 2023-08-25 21:50:21 +04:00
jonaustin09
e8b06a72f9 fix: Fixes #197, Fixed PutBucketAcl action input validation 2023-08-25 00:40:17 +04:00
jonaustin09
009ceee748 feat: Added FE support for SelectObjectContent action 2023-08-02 00:08:28 +04:00
jonaustin09
97847735c8 fix: s3response action responses naming cleanup 2023-07-31 21:41:10 -07:00
Jon Austin
091375fa00 Issue 151 (#174)
* fix: Fixes #151. Fixed DeleteObjects action bugs: Corrected request body serialization type, added return type
2023-07-31 21:36:33 -07:00
jonaustin09
838a7f9ef9 fix: Fixes #152. Changed CompleteMultiPartUpload invalid payload error to MalformedXML 2023-07-28 18:19:15 +04:00
Jon Austin
77080328c1 Issue 156 (#167)
* fix: Fixes #156, Added bucket name validation on bucket creation
2023-07-27 11:04:50 -07:00
Ben McClelland
884fd029c3 feat: add context to backend calls
This adds a context to the backend interface calls so that the backend
can enable request cancellation. This change isn't acutally implementing
any backend handling, but just putting the pieces into place to pass the
context to the backend.
2023-07-26 21:54:12 -07:00
jonaustin09
7fb2a7f9ba feat: ACL refactoring, moved ace parsing from controllers to middleware 2023-07-26 20:54:50 +04:00
Ben McClelland
2291c22eaa fix: standardize Backend interface args for s3 types 2023-07-22 22:45:24 -07:00
jonaustin09
fe547a19e9 feat: bucket event notifications
Set up Bucket event notifications interface to send aws compatible format event messages to a configured event service.
First integrated service is kafka message broker as an option for bucket event notifications.
2023-07-20 11:37:14 -07:00
jonaustin09
87d61a1eb3 feat: Setup audit loggin with webhook url and root level access.log file. CLI enables either webhook or server access logs by providing the flags 2023-07-14 23:40:05 +04:00
Ben McClelland
47dea2db7c feat: implement posix UploadCopyPart 2023-07-05 19:06:19 -07:00
jonaustin09
39803cb158 feat: Some cleanup in controller unit tests, removed backend unsupported unit tests, added test cases for admin controller functions 2023-07-03 20:35:40 +04:00
jonaustin09
f63545c9b7 feat: Added UploadPartCopy action in FE 2023-07-03 17:14:46 +04:00
jonaustin09
9db01362a0 feat: increased unit testing coverage in controllers, utility functions and server functions. Fixed bucket owner bug in putbucketacl. 2 more minor changes in controllers 2023-06-30 22:04:46 +04:00