Commit Graph

71 Commits

Author SHA1 Message Date
Ben McClelland
f295df2217 fix: add new auth method to update ownership within acl
Add helper util auth.UpdateBucketACLOwner() that sets new
default ACL based on new owner and removes old bucket policy.

The ChangeBucketOwner() remains in the backend.Backend
interface in case there is ever a backend that needs to manage
ownership in some other way than with bucket ACLs. The arguments
are changing to clarify the updated owner. This will break any
plugins implementing the old interface. They should use the new
auth.UpdateBucketACLOwner() or implement the corresponding
change specific for the backend.
2025-07-09 16:16:34 -07:00
niksis02
323717bcf1 fix: fixes the LastModified date formatting in CopyObject result.
Fixes #1276

Creates the custom `s3response.CopyObjectOutput` type to handle the `LastModified` date property formatting correctly. It uses `time.RFC3339` to format the date to match the format that s3 uses.
2025-05-12 23:30:47 +04:00
Ben McClelland
9244e9100d fix: xml response field names for complete multipart upload
The xml encoding for the s3.CompleteMultipartUploadOutput response
type was not producing exactly the right field names for the
expected complete multipart upload result.

This change follows the pattern we have had to do for other xml
responses to create our own type that will encode better to the
expected response.

This will change the backend.Backend interface, so plugins and
other backends will have to make the corresponding changes.
2025-04-30 14:36:48 -07:00
niksis02
90033845ad feat: Implements bucket cors actions in FE to return not implemented.
Implements the bucket cors s3 actions in FE to return `NotImplemented` error.
Actions implemented:
- `PutBucketCors`
- `GetBucketCors`
- `DeleteBucketCors`

`Note`: no logic is implemented for the actions in any backend and no input or output data validation is added.
2025-03-27 20:47:51 +04:00
niksis02
cfb2d6d87d feat: Implements object meta properties for CopyObject in azure and posix backends.
Fixes #998
Closes #1125
Closes #1126
Closes #1127

Implements objects meta properties(Content-Disposition, Content-Language, Content-Encoding, Cache-Control, Expires) and tagging besed on the directives(metadata, tagging) in CopyObject in posix and azure backends. The properties/tagging should be coppied from the source object if "COPY" directive is provided and it should be replaced otherwise.

Changes the object copy principle in azure: instead of using the `CopyFromURL` method from azure sdk, it first loads the object then creates one, to be able to compare and store the meta properties.
2025-03-17 09:37:05 -07:00
niksis02
65261a9753 feat: Adds the Content-Disposition, Content-Language, Cache-Control and Expires object meta properties support in the gateway.
Closes #1128

Adds `Content-Disposition`, `Content-Language`, `Cache-Control` and `Expires` object meta properties support in posix and azure backends.
Changes the `PutObject` and `CreateMultipartUpload` actions backend input type to custom `s3response` types to be able to store `Expires` as any string.
2025-03-12 16:01:56 +04:00
niksis02
6956757557 feat: Integrates object integrity checksums(CRC32, CRC32C, SHA1, SHA256) into the gateway 2025-02-14 14:14:00 +04:00
jonaustin09
06e2f2183d fix: Changes GetObjectAttributes action xml encoding root element to GetObjectAttributesResponse. Adds input validation for x-amz-object-attributes header. Adds x-amz-delete-marker and x-maz-version-id headers for GetObjectAttributes action. Adds VersionId in HeadObject response, if it's not specified in the request 2024-10-30 15:42:15 -04:00
jonaustin09
4d6ec783bf feat: Implements pagination for ListBuckets 2024-10-28 16:26:08 -04:00
jonaustin09
c803af4688 fix: Prevents bucket deletion when it contains object versions by returning ErrVersionedBucketNotEmpty error. Enabled object deletion with versionId and delete markers creation with DeleteObject when the versioning status is Suspended 2024-10-18 15:36:52 -04:00
jonaustin09
7b5765bd59 fix: Changed the GetBucketVersioning action return type, to return empty result for unset versioning configuration 2024-09-27 18:14:53 -04:00
jonaustin09
8252ecd452 feat: basic logic implementation of bucket object versioning in posix backend
New posix backend option --versioning-dir will enable storing object versions
in specified directory.
2024-09-18 13:04:34 -07:00
jonaustin09
684ab2371b fix: Changed ListObjects and ListObjectsV2 actions return types
Changed ListObjectsV2 and ListObjects actions return types from
*s3.ListObjects(V2)Output to s3response.ListObjects(V2)Result.

Changed the listing objects timestamp to RFC3339 to match AWS
S3 objects timestamp.

Fixes #752
2024-08-26 15:46:45 -07:00
jonaustin09
cc3c62cd9d fix: Change CreateMultipartUpload return type to match expected xml response
The AWS spec for the create multipart upload response is:
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult>
   <Bucket>string</Bucket>
   <Key>string</Key>
   <UploadId>string</UploadId>
</InitiateMultipartUploadResult>

So we need the return type to marshal to this xml format.
2024-08-21 14:49:39 -07:00
jonaustin09
2843cdbd45 fix: Fixed ChangeBucketOwnership action implementation to update the bucket acl 2024-07-11 13:45:01 -04:00
jonaustin09
e773872c48 feat: Implemented response body streaming for GetObject action 2024-07-08 15:56:24 -04:00
jonaustin09
7545e6236c feat: Implement bucket ownership controls
Bucket ACLs are now disabled by default the same as AWS.
By default the object ownership is BucketOwnerEnforced
which means that bucket ACLs are disabled. If one attempts
to set bucket ACL the following error is returned both in
the gateway and on AWS:
	ErrAclNotSupported: {
		Code:           "AccessControlListNotSupported",
		Description:    "The bucket does not allow ACLs",
		HTTPStatusCode: http.StatusBadRequest,
	},

ACls can be enabled with PutBucketOwnershipControls

Changed bucket canned ACL translation

New backend interface methods:
PutBucketOwnershipControls
GetBucketOwnershipControls
DeleteBucketOwnershipControls

Added these to metrics
2024-06-28 21:03:09 -07:00
jonaustin09
fb27e2703e feat: Implemented to logic to bypass governance retention 2024-05-24 13:50:41 -04:00
jonaustin09
0c3771ae2d feat: Added GetObjectAttributes actions implementation in posix, azure and s3 backends. Added integration tests for GetObjectAttributes action 2024-04-29 15:31:53 -04:00
jonaustin09
89755ea5aa feat: Changed object lock actions interface to put/get []byte 2024-04-22 13:19:09 -07:00
jonaustin09
fbaba0b944 feat: Added object WORM protection by object-lock feature from AWS with the following actions support: PutObjectLockConfiguration, GetObjectLockConfiguration, PutObjectRetention, GetObjectRetention, PutObjectLegalHold, GetObjectLegalHold 2024-04-22 13:13:40 -07:00
jonaustin09
d469a72213 feat: Implemented Put/Get/DeletBucketPolicy s3 actions in posix backend. Implemented policy document validation function 2024-03-15 15:47:10 -04:00
Ben McClelland
f7655dab9b fix: delete object xml response should be DeleteResult instead of DeleteObjectsResult 2024-03-09 10:20:15 -08:00
jonaustin09
d4f17bf32f feat: Added bucket policy actions implementation in FE 2024-03-06 13:56:29 -05:00
jonaustin09
e6852b3a99 feat: Closes 417, Added the following versioning related actions: PutBucketVersioning, GetBucketVersioning, ListObjectVersions. Added versionId support in FE for the following actions: GetObject, DeleteObject 2024-02-28 09:48:05 -05:00
jonaustin09
d70ea61830 feat: Added the following actions support in posix backend: PutBucketTagging, GetBucketTagging, DeleteBucketTagging 2024-01-31 10:09:48 -08:00
Ben McClelland
6481e2aac5 fix: cleanup backend ACLs
This adds the default ACL to the CreateBucket backend method so
that the backend doesn't need to know how to construct and ACL.

This also moves the s3proxy ACLs to a tag key/value because the
gateway ACLs are not the same accounts as the backend s3 server.
TODO: we may need to mask this tag key/value if we add support
for the Get/PutBucketTagging API.
2024-01-10 09:36:00 -08:00
jonaustin09
48818927bb feat: Fixes #286, Created a struct which handles s3 select event streaming and event message construction 2023-12-06 14:02:36 -08:00
jonaustin09
4be5d64c8b fix: Object tag actions cleanup 2023-09-23 21:00:45 -07:00
jonaustin09
6ac69b3198 feat: Closes #217, Created an admin API and CLI action to list all the buckets and its owners as a table 2023-09-12 08:29:34 -04:00
jonaustin09
8d2e2a4106 fix: Fixes #204, Change ListBuckets action logic to return all the buckets for admin users and the buckets owned by a user for regular users. Added integration test cases for ListBuckets action 2023-09-07 14:49:47 -04:00
jonaustin09
4c7584c99f feat: Closes #206, Added an admin api endpoint and a CLI action to change buckets owner 2023-09-06 17:41:47 -04:00
Ben McClelland
d058dcb898 fix: cleanup backend interface functions ordering 2023-08-25 09:15:01 -07:00
jonaustin09
009ceee748 feat: Added FE support for SelectObjectContent action 2023-08-02 00:08:28 +04:00
jonaustin09
97847735c8 fix: s3response action responses naming cleanup 2023-07-31 21:41:10 -07:00
Jon Austin
091375fa00 Issue 151 (#174)
* fix: Fixes #151. Fixed DeleteObjects action bugs: Corrected request body serialization type, added return type
2023-07-31 21:36:33 -07:00
Ben McClelland
884fd029c3 feat: add context to backend calls
This adds a context to the backend interface calls so that the backend
can enable request cancellation. This change isn't acutally implementing
any backend handling, but just putting the pieces into place to pass the
context to the backend.
2023-07-26 21:54:12 -07:00
Ben McClelland
2291c22eaa fix: standardize Backend interface args for s3 types 2023-07-22 22:45:24 -07:00
Ben McClelland
47dea2db7c feat: implement posix UploadCopyPart 2023-07-05 19:06:19 -07:00
jonaustin09
39803cb158 feat: Some cleanup in controller unit tests, removed backend unsupported unit tests, added test cases for admin controller functions 2023-07-03 20:35:40 +04:00
Ben McClelland
7e34078d6a posix: cleanup extra debug output 2023-06-29 11:18:00 -07:00
Ben McClelland
2427c67171 refactor ACLs to separate out ACL logic from backend 2023-06-16 16:47:05 -07:00
Ben McClelland
5c61604e82 fix list buckets response for single bucket entry
The xml encoding of the s3.ListBucketsOutput return type was not giving
correct results when there is only a single bucket. This revives the
old aws xsd schema and generated types that will give more accurate xml
encoding results.
2023-06-16 10:22:25 -07:00
Jon Austin
ad09d98891 feat: Implemented GetBucketACL, PutBucketACL posix functions, fixed a… (#92)
* feat: Implemented GetBucketACL, PutBucketACL posix functions, fixed authentication middleware signed headers bug

* fix: Fixed GetBucketAcl return type, fixed staticcheck uppercase error, fixed unit tests for PutActions
2023-06-15 10:49:17 -07:00
Ben McClelland
7157280627 cleanup unused backend interface 2023-06-12 11:49:57 -07:00
Ben McClelland
ef06d11d7c fix: get simple multipart upload tests passing 2023-06-12 07:37:21 -07:00
Jon Austin
8f27e88198 feat: GetObject range calculation moved to backend, created utility function for it in the backend (#61) 2023-06-06 11:13:45 -07:00
jonaustin09
f174308e3f fix: Merge conflicts resolved 2023-05-31 22:41:52 +04:00
Ben McClelland
5cbcf0c900 add copyright headers to source files 2023-05-28 14:38:45 -07:00
Ben McClelland
8b79fb24de update module/import paths to new name, add cli framework 2023-05-28 12:10:12 -07:00