Commit Graph

290 Commits

Author SHA1 Message Date
Luke McCrone
f630392718 test: allow for pre-configured direct users, ListParts update 2025-04-12 12:20:34 -03:00
niksis02
66b979ee86 fix: Sets limit to tag set count to 10 for PutObjectTagging and 50 for PutBucketTagging
Fixes #1204
Fixes #1205

Tag count in `PutBucketTagging` and `PutObjectTagging` is limited.
`PutBucketTagging`: 50
`PutObjectTagging`: 10

Adds the changes to return errors respectively
2025-04-11 21:07:08 +04:00
niksis02
aaee7afa98 fix: Changes the backend.Walk implementation to return non-truncated result if both delimiter and max-keys are provided
Fixes #816

`ListObjects(V2)` used to return truncated response, if delimiter is provided and the result is limited by max-keys and the number of common prefixes is the same as `max-keys`.
e.g
PUT -> `foo/bar`
PUT -> `foo/quxx`
LIST: `max-keys=1;delim=/` -> foo/

`ListObjects(V2)` should return `foo/` as common prefix and `truncated` should be `false`.

The PR makes this fix to return `non-truncated` response for the above described case.
2025-04-09 21:29:58 +04:00
Ben McClelland
eb72905401 Merge pull request #1200 from versity/sis/obj-lock-actions-invalid-body
fix: Changes the error type to MalformedXML for PutObjectRetention and PutObjectLegalHold empty or invalid body
2025-04-08 20:55:15 -07:00
Ben McClelland
4f70a16245 Merge pull request #1198 from versity/sis/tests-context-deadline
fix: Increases the context cancelation deadline in integration tests.
2025-04-08 10:19:53 -07:00
Ben McClelland
7866c326e3 Merge pull request #1190 from versity/sis/empty-checksum-headers
fix: Handles the error cases for empty checksum headers for PutObject and UploadPart
2025-04-08 10:19:34 -07:00
Ben McClelland
c88d45ceff Merge pull request #1196 from versity/sis/sig-v2-not-supported
fix: Returns ErrSignatureVersionNotSupported for sigV2 signed requests.
2025-04-08 10:17:49 -07:00
niksis02
20d00f7f6d fix: Changes the error type to MalformedXML for PutObjectRetention and PutObjectLegalHold empty or invalid body
Fixes #1185
Fixes #1191

`PutObjectLegalHold` and `PutObjectRetention` should return `MalformedXML` if the request body is empty or invalid.`
2025-04-08 19:01:00 +04:00
niksis02
fca1cf9fd6 fix: Increases the context cancelation deadline in integration tests.
Increases the context cancelation deadline to `30` seconds to avoid timeouts in relatively large tests.
2025-04-08 18:12:36 +04:00
niksis02
5560e5ebc4 fix: Returns ErrSignatureVersionNotSupported for sigV2 signed requests.
Fixes #1171

As signature v2 is depracated the gateway doesn't support it.
AWS S3 supports signature version 2 in some regions. For some regions the request fails with error:
```
InvalidRequest: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.
```

 The PR makes this change to return unsupported authorization mechanism for `sigV2` requests.
2025-04-08 01:05:59 +04:00
Luke McCrone
8431ba2a21 test: chunked upload - invalid checksums, retention, legal hold tests 2025-04-06 22:24:26 -03:00
niksis02
ed44fe1969 fix: Handles the error cases for empty checksum headers for PutObject and UploadPart
Fixes #1186
Fixes #1188
Fixes #1189

If multiple checksum headers are provided, no matter if they are empty or not, the gateway should return `(InvalidRequest): Expecting a single x-amz-checksum- header. Multiple checksum Types are not allowed.`
An empty checksum header is considered as invalid, because it's not valid crc32, crc32c ...
2025-04-04 23:17:22 +04:00
niksis02
f627b2efec fix: Makes the crc64nvme checksum as default for PutObject, even if no checksum is provided
Fixes #1182

S3 calculates the `CRC64NVME` checksum of an object on object upload(`PutObject`), when no checksum algorithm or precalculated checksum header is provided.
Makes the `CRC64NVME` checksum as default for `PutObject`, when no checksum is provided.
2025-04-04 00:08:54 +04:00
Ben McClelland
90a67272a7 Merge pull request #1184 from versity/sis/deleteobject-non-empty-dir-obj
fix: Fixes non empty directory objects deletion
2025-04-02 13:09:48 -07:00
niksis02
8705905319 fix: Fixes non empty directory objects deletion
Fixes #1181

`DeleteObjects` should remove non-empty directory objects, which has been uploaded as a separate object.
e.g
Upload -> `foo/bar`
Upload -> `foo/`

Delete -> `foo/`

The last action call should succeed.

The PR introduces changes which removes `ETag` from the directory object attempting to `delete`, which has been uploaded as a separate object.
2025-04-02 22:31:05 +04:00
Ben McClelland
dadfc97d46 Merge pull request #1183 from versity/sis/listobjectsv2-owner
feat: Adds Owner data in ListObjects(V2) result
2025-04-02 10:52:14 -07:00
niksis02
cb97fb589b feat: Adds Ownder data in ListObjects(V2) result.
Closes #819

ListObjects returns object owner data in each object entity in the result, while ListObjectsV2 has fetch-owner query param, which indicates if the objects owner data should be fetched.
Adds these changes in the gateway to add `Owner` data in `ListObjects` and `ListObjectsV2` result. In aws the objects can be owned by different users in the same bucket. In the gateway all the objects are owned by the bucket owner.
2025-04-02 18:28:32 +04:00
niksis02
03c7c432ad fis: Fixes the trailing slash issue in CopyObject destination and source object paths in posix.
Fixes #1021

`foo` and `foo/` object paths were considered as the same in `CopyObject` source and destination object paths in posix.
Implements the `joinPathWithTrailer` function, which calls `filepath.Join` and adds trailing `/` if it existed in the original path. This way the implementation puts separation
 between directory and file objects with the same name.
2025-04-01 17:31:02 +04:00
niksis02
30f319bc92 fix: Hanldes the case when Statement is missing in PutBucketPolicy json document.
If `Statement` field is missing in the json document in `PutBucketPolicy` body, the gateway returns `Missing required field Statement` error description alongside with `MalformedPolicy` error code.
2025-03-31 18:23:33 +04:00
Luke McCrone
605e6b83f6 uppercase fix for versitygw 2025-03-28 19:19:19 -03:00
niksis02
832371afb1 fix: Fixes the case for GetObjectAttributes to return InvalidArgument if a single invalid object attribute is provided.
Fixes #1000

`GetObjectAttributes` returned `InvalidRequest` instead of `InvalidArgument` with description `Invalid attribute name specified.`.
Fixes the logic in `ParseObjectAttributes` to ignore empty values for `X-Amz-Object-Attributes` headers to return `InvalidArgument` if all the specified object attributes are invalid.
2025-03-28 07:27:35 +04:00
Luke McCrone
e0236226fc test: chunked uploads w/trailers 2025-03-24 16:32:41 -03:00
niksis02
d82da64942 fix: Fixes the bucket policy json document validation errors for invalid jsons.
Fixes #965

Changes the returned error description to `Policies must be valid JSON and the first byte must be '{'` for invalid bucket policy json documents, which doesn't start with `{`.
The gateway returns `This policy contains invalid Json` error description, if the document starts with `{`, but still isn't valid json.

Implements the `policyErr` string type which implements the `error` interface, to handle the policy json document validation errors, by avoiding staticchecker warnings.
2025-03-20 01:08:47 +04:00
Ben McClelland
9433986121 Merge pull request #1155 from versity/test/chunked_incorrect_middle
Test/chunked incorrect final signature
2025-03-19 12:34:33 -07:00
Luke McCrone
75c25ec789 test: chunked upload - more tests, final signature test 2025-03-19 15:36:40 -03:00
Ben McClelland
4dad42e4e8 Merge pull request #1150 from versity/sis/unnecessary-int-tests
fix: Removes 2 unnecessary integration tests for HeadObject and GetOb…
2025-03-18 08:25:16 -07:00
niksis02
607c6d2308 fix: Removes 2 unnecessary integration tests for HeadObject and GetObject.
Removes the `HeadObject_with_contenttype` and `GetObject_with_meta` integration test as the logic is covered in `HeadObject_success` and `GetObject_success` accordingly.
2025-03-18 18:50:01 +04:00
niksis02
9e0f56f807 fix: Fixes the returned error type for object legal hold status and object lock mode in PutObject, CopyObject and CreateMultipartUpload.
Fixes #1141
Fixes #1142

Changes the error type to `InvalidArgument` for `x-amz-object-lock-legal-hold` and `x-amz-object-lock-mode` headers invalid values.
2025-03-18 13:58:49 +04:00
Ben McClelland
e635306bfb Merge pull request #1143 from versity/test/rest_signature_invalid_user
Test/rest signature invalid user
2025-03-17 18:01:36 -07:00
niksis02
cfb2d6d87d feat: Implements object meta properties for CopyObject in azure and posix backends.
Fixes #998
Closes #1125
Closes #1126
Closes #1127

Implements objects meta properties(Content-Disposition, Content-Language, Content-Encoding, Cache-Control, Expires) and tagging besed on the directives(metadata, tagging) in CopyObject in posix and azure backends. The properties/tagging should be coppied from the source object if "COPY" directive is provided and it should be replaced otherwise.

Changes the object copy principle in azure: instead of using the `CopyFromURL` method from azure sdk, it first loads the object then creates one, to be able to compare and store the meta properties.
2025-03-17 09:37:05 -07:00
Luke McCrone
ff305d0efc test: user REST permission, signature tests 2025-03-14 18:45:34 -03:00
Luke McCrone
976e44bb57 test: expires header test 2025-03-12 18:16:51 -03:00
niksis02
65261a9753 feat: Adds the Content-Disposition, Content-Language, Cache-Control and Expires object meta properties support in the gateway.
Closes #1128

Adds `Content-Disposition`, `Content-Language`, `Cache-Control` and `Expires` object meta properties support in posix and azure backends.
Changes the `PutObject` and `CreateMultipartUpload` actions backend input type to custom `s3response` types to be able to store `Expires` as any string.
2025-03-12 16:01:56 +04:00
Luke McCrone
acb33f608e test: checksum mode test 2025-03-10 19:37:22 -03:00
Luke McCrone
485f2abbaa test: crc32c checksum 2025-03-08 18:56:42 -03:00
Luke McCrone
4c5cd918d8 test: crc64nvme checksum 2025-03-08 17:56:14 -03:00
Ben McClelland
293e7faf31 Merge pull request #1124 from versity/sis/getobject-exceeding-range
fix: Fixes GetObject and UploadPartCopy actions data range parsing.
2025-03-07 15:45:15 -08:00
Ben McClelland
d6d4f304e1 Merge pull request #1096 from versity/test/rest_checksum_crc64nvme
Test/rest checksum crc64nvme
2025-03-07 15:43:52 -08:00
niksis02
96af2b6471 fix: Fixes GetObject and UploadPartCopy actions data range parsing.
Fixes #1004
Fixes #1122
Fixes #1120

Separates `GetObject` and `UploadPartCopy` range parsing/validation.

`GetObject` returns a successful response if acceptRange is invalid.
Adjusts the range upper limit, if it exceeds the actual objects size for `GetObject`.
Corrects the `ContentRange` in the `GetObject` response.

Fixes the `UploadPartCopy` action copy source range parsing/validation.
`UploadPartCopy` returns `InvalidArgument` if the copy source range is not valid.
2025-03-08 01:39:21 +04:00
niksis02
4b0dd64836 fix: removes ETag quotes for in GetObjectAttributes response for azure and posix backends 2025-03-06 00:28:55 +04:00
Luke McCrone
11f646b051 test: crc64nvme checksum 2025-03-05 16:22:34 -03:00
Luke McCrone
dea316bb26 test: initial checksum tests, REST script additions 2025-03-05 14:53:44 -03:00
niksis02
d59ee87b10 fix: Fixes ListObjects direcotry objects listing issue. Fixes DeleteObject directory objects parents deletion 2025-03-04 21:29:08 +04:00
niksis02
92dee62298 fix: Allows checksum-type to be empty in CompleteMultipartUpload input parameters 2025-03-03 22:14:47 +04:00
niksis02
9a02c474b1 fix: Implements a generic wildcard match for bucket policy resources 2025-03-03 20:04:31 +04:00
niksis02
30f3fac4e1 fix: Prioritize explicit deny in bucket policy statements 2025-03-01 01:14:12 +04:00
niksis02
d13791f5ce fix: Adds quotes for object Etags 2025-02-28 00:43:51 +04:00
niksis02
fcafb57abc feat: Implements checksums for ListObjectVersions and CopyObject actions. 2025-02-27 00:24:38 +04:00
niksis02
1bb07fd78e fix: Changes HeadObject returned error type from MethodNotAllowed to NotFound if the latest object version is a delete marker. 2025-02-24 20:44:26 +04:00
niksis02
b53bbb025f fix: Adds crc64nvme checksum cases in the integration tests 2025-02-21 21:01:41 +04:00