mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-08-19 05:36:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,339 Commits 36 Branches 53 Tags
e7b9db1a1fdd6c031120fc5edcabbaabf45a3240
Commit Graph
86 Commits
Author SHA1 Message Date
niksis02 6956757557 feat: Integrates object integrity checksums(CRC32, CRC32C, SHA1, SHA256) into the gateway 2025-02-14 14:14:00 +04:00
Ben McClelland 66ed32baca chore: remove failing unit test to get pipeline running 2025-01-06 10:51:32 -08:00
niksis02 7c5258e6e9 fix: Adds a check to ensure that the CompleteMultipartUpload parts are not empty. 2024-12-17 18:50:57 +04:00
jonaustin09 7bd32a2cfa fix: Changes the PutBucketTagging action response status code from 200(OK) to 204(No Content) 2024-10-31 18:30:07 -04:00
jonaustin09 06e2f2183d fix: Changes GetObjectAttributes action xml encoding root element to GetObjectAttributesResponse. Adds input validation for x-amz-object-attributes header. Adds x-amz-delete-marker and x-maz-version-id headers for GetObjectAttributes action. Adds VersionId in HeadObject response, if it's not specified in the request 2024-10-30 15:42:15 -04:00
jonaustin09 4d6ec783bf feat: Implements pagination for ListBuckets 2024-10-28 16:26:08 -04:00
jonaustin09 c803af4688 fix: Prevents bucket deletion when it contains object versions by returning ErrVersionedBucketNotEmpty error. Enabled object deletion with versionId and delete markers creation with DeleteObject when the versioning status is Suspended 2024-10-18 15:36:52 -04:00
jonaustin09 7b5765bd59 fix: Changed the GetBucketVersioning action return type, to return empty result for unset versioning configuration 2024-09-27 18:14:53 -04:00
jonaustin09andBen McClelland 8252ecd452 feat: basic logic implementation of bucket object versioning in posix backend 
New posix backend option --versioning-dir will enable storing object versions
in specified directory.
 2024-09-18 13:04:34 -07:00
jonaustin09andBen McClelland 684ab2371b fix: Changed ListObjects and ListObjectsV2 actions return types 
Changed ListObjectsV2 and ListObjects actions return types from
*s3.ListObjects(V2)Output to s3response.ListObjects(V2)Result.

Changed the listing objects timestamp to RFC3339 to match AWS
S3 objects timestamp.

Fixes #752
 2024-08-26 15:46:45 -07:00
jonaustin09andBen McClelland cc3c62cd9d fix: Change CreateMultipartUpload return type to match expected xml response 
The AWS spec for the create multipart upload response is:
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult>
   <Bucket>string</Bucket>
   <Key>string</Key>
   <UploadId>string</UploadId>
</InitiateMultipartUploadResult>

So we need the return type to marshal to this xml format.
 2024-08-21 14:49:39 -07:00
jonaustin09 23fd0d3fdd fix: Fixed PutBucketAcl action error handling, removed the bucket owner check for all the acl options 2024-08-12 15:27:03 -04:00
jonaustin09 e773872c48 feat: Implemented response body streaming for GetObject action 2024-07-08 15:56:24 -04:00
jonaustin09andBen McClelland 7545e6236c feat: Implement bucket ownership controls 
Bucket ACLs are now disabled by default the same as AWS.
By default the object ownership is BucketOwnerEnforced
which means that bucket ACLs are disabled. If one attempts
to set bucket ACL the following error is returned both in
the gateway and on AWS:
	ErrAclNotSupported: {
		Code:           "AccessControlListNotSupported",
		Description:    "The bucket does not allow ACLs",
		HTTPStatusCode: http.StatusBadRequest,
	},

ACls can be enabled with PutBucketOwnershipControls

Changed bucket canned ACL translation

New backend interface methods:
PutBucketOwnershipControls
GetBucketOwnershipControls
DeleteBucketOwnershipControls

Added these to metrics
 2024-06-28 21:03:09 -07:00
Ben McClelland f0005a0047 fix: restore object request handler and scoutfs glacier enable 
The restore object api request handler was incorrectly trying to
unmarshal the request body, but for the stadnard (all?) case the
request body is emtpy. We only need the bucket and opbject params
for now.

This also adds a fix to actually honor the enable glacier mode
in scoutfs.
 2024-06-11 12:57:46 -07:00
jonaustin09andBen McClelland dbfd9e5171 fix: Removed required request body check for PutBucketAcl action 2024-05-29 14:13:38 -07:00
jonaustin09andBen McClelland be6f9a86cd feat: Integrated metric manager into the gateway 2024-05-28 12:50:50 -07:00
jonaustin09 fb27e2703e feat: Implemented to logic to bypass governance retention 2024-05-24 13:50:41 -04:00
jonaustin09 4bde84eafd fix: Added status property validation for PutObjectLegalHold action 2024-05-17 12:39:27 -04:00
jonaustin09 2a2f9c827c feat: Closes #484. Added support to run the gateway on read only mode 2024-05-06 16:41:39 -04:00
jonaustin09 0c3771ae2d feat: Added GetObjectAttributes actions implementation in posix, azure and s3 backends. Added integration tests for GetObjectAttributes action 2024-04-29 15:31:53 -04:00
jonaustin09andBen McClelland 89755ea5aa feat: Changed object lock actions interface to put/get []byte 2024-04-22 13:19:09 -07:00
jonaustin09andBen McClelland fbaba0b944 feat: Added object WORM protection by object-lock feature from AWS with the following actions support: PutObjectLockConfiguration, GetObjectLockConfiguration, PutObjectRetention, GetObjectRetention, PutObjectLegalHold, GetObjectLegalHold 2024-04-22 13:13:40 -07:00
jonaustin09 84ce40fb54 fix: Fixes #487, added response headers for HeadBucket action 2024-04-18 13:27:45 -04:00
Ben McClelland 4be4dc2971 feat: add more debug logging for api handler errors 
There are a few cases where parsing, validations checks, etc
error details are getting lost with the more generic error
responses. This add some opt-in debug logging to log more
info for these various error cases.
 2024-04-06 20:08:16 -07:00
jonaustin09 af641e5368 feat: Added integration test cases for Put/Get/DeleteBucketPolicy actions. Made some bug fixes in these actions implementations 2024-03-20 17:31:52 -04:00
Ben McClelland f7655dab9b fix: delete object xml response should be DeleteResult instead of DeleteObjectsResult 2024-03-09 10:20:15 -08:00
jonaustin09 d4f17bf32f feat: Added bucket policy actions implementation in FE 2024-03-06 13:56:29 -05:00
jonaustin09 e6852b3a99 feat: Closes 417, Added the following versioning related actions: PutBucketVersioning, GetBucketVersioning, ListObjectVersions. Added versionId support in FE for the following actions: GetObject, DeleteObject 2024-02-28 09:48:05 -05:00
jonaustin09andBen McClelland 3cce3a5201 feat: Added unit and integration test cases for posix bucket tagging related actions 2024-01-31 10:09:48 -08:00
Ben McClelland 6481e2aac5 fix: cleanup backend ACLs 
This adds the default ACL to the CreateBucket backend method so
that the backend doesn't need to know how to construct and ACL.

This also moves the s3proxy ACLs to a tag key/value because the
gateway ACLs are not the same accounts as the backend s3 server.
TODO: we may need to mask this tag key/value if we add support
for the Get/PutBucketTagging API.
 2024-01-10 09:36:00 -08:00
jonaustin09andBen McClelland 48818927bb feat: Fixes #286, Created a struct which handles s3 select event streaming and event message construction 2023-12-06 14:02:36 -08:00
Ben McClelland cd8ad7d482 fix: breaking changes with aws sdk updates 2023-11-28 13:51:32 -08:00
Ben McClelland 4661af11dd feat: replace access/role context locals with full account info 2023-10-02 10:59:59 -07:00
jonaustin09andBen McClelland 099ac39f22 fix: Fixes #259, Changed delete actions response statuses from 200 to 204 2023-09-23 21:21:47 -07:00
jonaustin09andBen McClelland 4be5d64c8b fix: Object tag actions cleanup 2023-09-23 21:00:45 -07:00
jonaustin09 e0c09ad4d9 fix: Fixes #247, Changed DeleteObjectTagging action successful response status from 200 to 204 2023-09-20 12:07:52 -04:00
jonaustin09 23c26d802c fix: Fixes #216, Fixed vet warnings, removed the code snippet which copied fiber.Ctx 2023-09-08 11:57:17 -04:00
jonaustin09 8d2e2a4106 fix: Fixes #204, Change ListBuckets action logic to return all the buckets for admin users and the buckets owned by a user for regular users. Added integration test cases for ListBuckets action 2023-09-07 14:49:47 -04:00
jonaustin09 3573a31ae6 fix: Closes #192, Fixed authentication errors returned, created integration test cases for it 2023-08-25 21:50:21 +04:00
jonaustin09 e8b06a72f9 fix: Fixes #197, Fixed PutBucketAcl action input validation 2023-08-25 00:40:17 +04:00
jonaustin09 009ceee748 feat: Added FE support for SelectObjectContent action 2023-08-02 00:08:28 +04:00
jonaustin09andBen McClelland 97847735c8 fix: s3response action responses naming cleanup 2023-07-31 21:41:10 -07:00
Jon AustinandGitHub 091375fa00 Issue 151 (#174) 
* fix: Fixes #151. Fixed DeleteObjects action bugs: Corrected request body serialization type, added return type
 2023-07-31 21:36:33 -07:00
jonaustin09 838a7f9ef9 fix: Fixes #152. Changed CompleteMultiPartUpload invalid payload error to MalformedXML 2023-07-28 18:19:15 +04:00
Jon AustinandGitHub 77080328c1 Issue 156 (#167) 
* fix: Fixes #156, Added bucket name validation on bucket creation
 2023-07-27 11:04:50 -07:00
Ben McClelland 884fd029c3 feat: add context to backend calls 
This adds a context to the backend interface calls so that the backend
can enable request cancellation. This change isn't acutally implementing
any backend handling, but just putting the pieces into place to pass the
context to the backend.
 2023-07-26 21:54:12 -07:00
jonaustin09 7fb2a7f9ba feat: ACL refactoring, moved ace parsing from controllers to middleware 2023-07-26 20:54:50 +04:00
Ben McClelland 2291c22eaa fix: standardize Backend interface args for s3 types 2023-07-22 22:45:24 -07:00
jonaustin09andBen McClelland fe547a19e9 feat: bucket event notifications 
Set up Bucket event notifications interface to send aws compatible format event messages to a configured event service.
First integrated service is kafka message broker as an option for bucket event notifications.
 2023-07-20 11:37:14 -07:00