Commit Graph

120 Commits

Author SHA1 Message Date
jonaustin09
6d4ff09d6f feat: Added integration tests for bucket object versioning. Made a couple of bug fixes in the versioning implementation 2024-09-19 13:29:14 -07:00
Ben McClelland
d9d3a16051 fix: azure list objects trim common prefixes that match marker prefix 2024-09-18 12:03:21 -07:00
Ben McClelland
180df62134 fix: list objects trim common prefixes that match marker prefix
This checks to see if the common prefix is before the marker and
thus would have been returned in earlier list objects request.

The error case was aws cli listing multiple entries for the same
common prefix when the listing required multiple pagination
requests.

Fixes #778
2024-09-18 12:02:25 -07:00
Ben McClelland
4cace00d8e fix: get object range min=max return single byte min
The range min-max where min=max was incorrectly returning invalid
object range.  On AWS, this returns the byte at offset min.

Fixes #810
2024-09-18 10:00:57 -07:00
jonaustin09
600aca8bdc fix: Fixed the request uri path escape to support object key special characters 2024-09-17 13:28:30 -04:00
Ben McClelland
cecf563d92 Merge pull request #798 from versity/fix/azure-obj-content-enc
fix: Fixed the mechanism to put/get object content-type, content-enco…
2024-09-11 13:30:39 -07:00
jonaustin09
47d1a799f6 fix: Fixed the mechanism to put/get object content-type, content-encoding in azure 2024-09-11 15:11:37 -04:00
jonaustin09
a6fd1322f7 fix: Removed multipart upload part from GetObjectAttributes action 2024-09-11 11:57:32 -04:00
jonaustin09
8a46de8e3b fix: change multipart upload implementation with .sgwtmp namespace in azure 2024-09-10 13:07:15 -07:00
Ben McClelland
dc71365bab fix: set content type/encoding on put/mutlipart object
This fixes put object with setting a content type. If no content
type is set, then we will return a default content type for
following requests. Mutli-part upload appears to be ok.

Also fixed content eincoding and multipart uploads.

Fixes #783
2024-09-09 16:04:54 -07:00
Ben McClelland
ccd4166b2e fix: copy object missing prefix on metadata delete
When using the REPLACE directive, we were incorrectly removing the
old metadata on the object due to missing the metadata prefix on
the key.  Fix this to remove the correct metadata before setting
new metadata.

Fixes #787
2024-09-07 17:06:45 -07:00
jonaustin09
9be4f27550 feat: Added an integration test to cover the case to CopyObject with the copysource starting with / 2024-08-29 15:32:20 -04:00
jonaustin09
201777c819 fix: Added an integration test case for ListObjectsV2 to specify max-keys as the exact number of objects in the bucket 2024-08-29 11:14:29 -04:00
jonaustin09
227fdaa00b fix: Fixed the pagination for common prefixes in ListObjects & ListObjectsV2 actions 2024-08-28 11:07:44 -04:00
Jon Austin
d79f978df9 feat: Added the standard storage class to all the available get/list actions responses in posix. (#765) 2024-08-27 15:28:40 -07:00
Ben McClelland
453136bd5a fix: return KeyTooLongError when filenames exceed allowed length
The posix limits wont exactly match up with the AWS key length
limits because posix has component length limits as well as path
length limits.

This reponds with the aws compatible KeyTooLongError under these
conditions now.

Note that delete object returns success even in the error cases.

Fixes #755
2024-08-24 14:53:42 -07:00
jonaustin09
71df685fb7 fix: Fixed directory objects listing with delimiter 2024-08-23 11:28:52 -04:00
Ben McClelland
77aa4366b5 fix: unescape copy source before handing to backend
We were handing the URL escaped string to the backend as the
copysource which includes "%<hex>" for spaces and other special
characters. The backend would then interpret this as the source
path. This fixes the copyobject and upload part copy.

Fixes #749
2024-08-22 10:06:38 -07:00
Ben McClelland
54490f55cc chore: cleanup staticcheck errors 2024-08-13 11:09:14 -07:00
Ben McClelland
a36d974942 fix: copy-object with replace metadata-directive
In copy-object, if the source and destination are the same then
X-Amz-Metadata-Directive must be set to "REPLACE" in order to use
this api call to update the metadata of the object in place.

The default X-Amz-Metadata-Directive is "COPY" if not specified.
"COPY" is only valid if source and destination are not the same
object.

When "REPLACE" selected, metadata does not have to differ for the
call to be successful. The "REPLACE" always sets the incoming
metadata (even if empty or the same as the source).

Fixes #734
2024-08-13 10:52:47 -07:00
jonaustin09
23fd0d3fdd fix: Fixed PutBucketAcl action error handling, removed the bucket owner check for all the acl options 2024-08-12 15:27:03 -04:00
Ben McClelland
cbf03c30ce Merge pull request #726 from versity/fix/iam-get-root-user
Root user credentials in IAM services
2024-08-12 10:18:36 -07:00
jonaustin09
34b7fd6ee7 fix: Added the root user data in the iam services records 2024-08-09 16:14:51 -04:00
Ben McClelland
e92b36a12c fix: non-existing object delete response
The expected response code for deleting non-existing objects is
expected to be 204 (No Content) instead of NoSuchKey. The tests
are updated to validate expected responses.

Fixes #724
2024-08-08 11:46:36 -07:00
Ben McClelland
797376a235 fix: head/get/delete/copy directory object should fail when corresponding file object exists
The API hanlders and backend were stripping trailing "/" in object
paths. So if an object exists and a request came in for head/get/delete/copy
for that same name but with a trailing "/" indicating request should
be for directory object, the "/" would be stripped and the request
would be handlied for the incorrect file object.

This fix adds in checks to handle the case with the training "/"
in the request.

Fixes #709
2024-08-05 11:55:32 -07:00
Ben McClelland
61a97e94db fix: set default content type to binary/octet-stream
AWS uses binary/octet-stream for the default content type if the
client doesn't specify the content type. Change the default for
the gateway to match this behavior.

Fixes #697
2024-08-02 09:02:57 -07:00
jonaustin09
5e484f2355 fix: Fixed CopySource parsing to handle the values starting with '/' in CopyObject action in posix and azure backends. 2024-07-22 11:30:32 -04:00
jonaustin09
23a40d86a2 fix: Removed the bucket ACL owner check for admin and root users 2024-07-17 09:39:00 -04:00
jonaustin09
828eb93bee fix: Added 'Type' property support in bucket ACL Grantee schema 2024-07-16 18:17:16 -04:00
jonaustin09
2843cdbd45 fix: Fixed ChangeBucketOwnership action implementation to update the bucket acl 2024-07-11 13:45:01 -04:00
jonaustin09
e773872c48 feat: Implemented response body streaming for GetObject action 2024-07-08 15:56:24 -04:00
jonaustin09
7545e6236c feat: Implement bucket ownership controls
Bucket ACLs are now disabled by default the same as AWS.
By default the object ownership is BucketOwnerEnforced
which means that bucket ACLs are disabled. If one attempts
to set bucket ACL the following error is returned both in
the gateway and on AWS:
	ErrAclNotSupported: {
		Code:           "AccessControlListNotSupported",
		Description:    "The bucket does not allow ACLs",
		HTTPStatusCode: http.StatusBadRequest,
	},

ACls can be enabled with PutBucketOwnershipControls

Changed bucket canned ACL translation

New backend interface methods:
PutBucketOwnershipControls
GetBucketOwnershipControls
DeleteBucketOwnershipControls

Added these to metrics
2024-06-28 21:03:09 -07:00
jonaustin09
329fae5203 fix: Changed bucket policy validation error messages 2024-06-24 13:14:28 -04:00
jonaustin09
fe19bfaed9 fix: Changed exceeding range error for the source object in UploadPartCopy action 2024-06-20 14:12:03 -04:00
jonaustin09
7ea386aec9 fix: Bug fixing for azure backend. Added a new integration test case for ListParts 2024-06-11 16:14:35 -04:00
Ben McClelland
a7110c28b6 Merge pull request #607 from versity/fix/get-obj-exc-range
GetObject exceeding range
2024-06-04 11:54:39 -07:00
Ben McClelland
a896b3660b fix: remove deprecated WithEndpointResolver s3 client option 2024-06-04 11:09:29 -07:00
jonaustin09
0fb6bf6267 fix: Removed exceeding invalid range error when calling GetObject action 2024-06-04 13:25:56 -04:00
jonaustin09
dbfd9e5171 fix: Removed required request body check for PutBucketAcl action 2024-05-29 14:13:38 -07:00
jonaustin09
d10ffd8707 fix: Fixed GetObject action invalid range error: when the same index of bytes is specified: bytes=0-0 2024-05-29 16:32:01 -04:00
jonaustin09
9e8458a09f feat: Added integration tests for bypass governance retention functionality 2024-05-28 15:17:25 -04:00
jonaustin09
fb27e2703e feat: Implemented to logic to bypass governance retention 2024-05-24 13:50:41 -04:00
jonaustin09
6fb102056d fix: Changed WORM protection implementation to prevent root/admin users to overwrite objects in governance mode or if legal hold is set up 2024-05-23 16:56:21 -04:00
jonaustin09
43f509d971 fix: Added missing properties support for CreateMultipartUpload action: ContentType, ObjectLock, Tagging, Metadata 2024-05-22 12:16:55 -07:00
jonaustin09
8eac24c78c fix: Changed bucket policy document validation to handle object containing 'AWS' prop in principal field 2024-05-21 10:48:38 -04:00
Ben McClelland
069ff181d6 Merge pull request #578 from versity/fix/566-getobject-by-range-resp-status
GetObject by range success response status code
2024-05-20 13:36:43 -07:00
jonaustin09
e38c63448d fix: Changed GetObject by range success status code from 200 to 206 2024-05-20 14:48:02 -04:00
jonaustin09
b971467446 fix: Changed the logic to return BucketAlreadyOwnedByYou error when user tries to create an existing bucket owned by him 2024-05-20 12:07:33 -04:00
Ben McClelland
28f901ef0e Merge pull request #576 from versity/fix/560-put-obj-legal-hold-status-validation
PutObjectLegalHold status validation
2024-05-17 10:01:07 -07:00
jonaustin09
4bde84eafd fix: Added status property validation for PutObjectLegalHold action 2024-05-17 12:39:27 -04:00