mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-07-20 06:52:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,415 Commits 37 Branches 53 Tags
4dad42e4e80af3feb0cfcece74d263b0abdcadda
Commit Graph

147 Commits

Author SHA1 Message Date
Ben McClelland 4dad42e4e8 Merge pull request #1150 from versity/sis/unnecessary-int-tests 
fix: Removes 2 unnecessary integration tests for HeadObject and GetOb…
 2025-03-18 08:25:16 -07:00
niksis02 607c6d2308 fix: Removes 2 unnecessary integration tests for HeadObject and GetObject. 
Removes the `HeadObject_with_contenttype` and `GetObject_with_meta` integration test as the logic is covered in `HeadObject_success` and `GetObject_success` accordingly.
 2025-03-18 18:50:01 +04:00
niksis02 9e0f56f807 fix: Fixes the returned error type for object legal hold status and object lock mode in PutObject, CopyObject and CreateMultipartUpload. 
Fixes #1141
Fixes #1142

Changes the error type to `InvalidArgument` for `x-amz-object-lock-legal-hold` and `x-amz-object-lock-mode` headers invalid values.
 2025-03-18 13:58:49 +04:00
niksis02 cfb2d6d87d feat: Implements object meta properties for CopyObject in azure and posix backends. 
Fixes #998
Closes #1125
Closes #1126
Closes #1127

Implements objects meta properties(Content-Disposition, Content-Language, Content-Encoding, Cache-Control, Expires) and tagging besed on the directives(metadata, tagging) in CopyObject in posix and azure backends. The properties/tagging should be coppied from the source object if "COPY" directive is provided and it should be replaced otherwise.

Changes the object copy principle in azure: instead of using the `CopyFromURL` method from azure sdk, it first loads the object then creates one, to be able to compare and store the meta properties.
 2025-03-17 09:37:05 -07:00
niksis02 65261a9753 feat: Adds the Content-Disposition, Content-Language, Cache-Control and Expires object meta properties support in the gateway. 
Closes #1128

Adds `Content-Disposition`, `Content-Language`, `Cache-Control` and `Expires` object meta properties support in posix and azure backends.
Changes the `PutObject` and `CreateMultipartUpload` actions backend input type to custom `s3response` types to be able to store `Expires` as any string.
 2025-03-12 16:01:56 +04:00
niksis02 96af2b6471 fix: Fixes GetObject and UploadPartCopy actions data range parsing. 
Fixes #1004
Fixes #1122
Fixes #1120

Separates `GetObject` and `UploadPartCopy` range parsing/validation.

`GetObject` returns a successful response if acceptRange is invalid.
Adjusts the range upper limit, if it exceeds the actual objects size for `GetObject`.
Corrects the `ContentRange` in the `GetObject` response.

Fixes the `UploadPartCopy` action copy source range parsing/validation.
`UploadPartCopy` returns `InvalidArgument` if the copy source range is not valid.
 2025-03-08 01:39:21 +04:00
niksis02 4b0dd64836 fix: removes ETag quotes for in GetObjectAttributes response for azure and posix backends 2025-03-06 00:28:55 +04:00
niksis02 d59ee87b10 fix: Fixes ListObjects direcotry objects listing issue. Fixes DeleteObject directory objects parents deletion 2025-03-04 21:29:08 +04:00
niksis02 92dee62298 fix: Allows checksum-type to be empty in CompleteMultipartUpload input parameters 2025-03-03 22:14:47 +04:00
niksis02 9a02c474b1 fix: Implements a generic wildcard match for bucket policy resources 2025-03-03 20:04:31 +04:00
niksis02 30f3fac4e1 fix: Prioritize explicit deny in bucket policy statements 2025-03-01 01:14:12 +04:00
niksis02 d13791f5ce fix: Adds quotes for object Etags 2025-02-28 00:43:51 +04:00
niksis02 fcafb57abc feat: Implements checksums for ListObjectVersions and CopyObject actions. 2025-02-27 00:24:38 +04:00
niksis02 1bb07fd78e fix: Changes HeadObject returned error type from MethodNotAllowed to NotFound if the latest object version is a delete marker. 2025-02-24 20:44:26 +04:00
niksis02 b53bbb025f fix: Adds crc64nvme checksum cases in the integration tests 2025-02-21 21:01:41 +04:00
Ben McClelland e256a1aa0d Merge pull request #1079 from versity/fix/versioning-nested-object-deletion 
fix: Adds removeParents in DeleteObject in case of deleting an object…
 2025-02-20 18:01:20 -08:00
niksis02 a30b8c2320 fix: Adds removeParents in DeleteObject in case of deleting an object in nested directories 2025-02-21 00:20:49 +04:00
Ben McClelland 549289c581 Merge pull request #1078 from versity/fix/listparts-issues 
ListParts refactoring
 2025-02-20 08:12:36 -08:00
niksis02 e5811e4ce7 fix: Fixes the entity limiter validation for ListObjects(V2), ListParts, ListMultipartUploads, ListBuckets actions 2025-02-20 15:45:42 +04:00
niksis02 64a72a2dee feat: Adds 'x-amz-mp-object-size' request header support for CompleteMultipartUpload 2025-02-19 19:26:03 +04:00
Ben McClelland ff0cf29d0a Merge pull request #1075 from versity/fix/complete-mp-parts-order 
fix: Adds PartNumber validation for CompleteMultipartUploads parts. A…
 2025-02-18 09:51:02 -08:00
niksis02 3cae3fced9 fix: Adds PartNumber validation for CompleteMultipartUploads parts. Adds a check to validate the parts order to be ascending. 2025-02-18 20:02:01 +04:00
niksis02 4517b292b9 fix: Changes UploadPart returned error from ErrInvalidPart to ErrInvalidPartNumber 2025-02-17 19:38:50 +04:00
niksis02 132d0ae631 feat: Adds the CRC64NVME checksum support in the gateway. Adds checksum-type support for the checksum implementation 2025-02-16 17:10:06 +04:00
niksis02 6956757557 feat: Integrates object integrity checksums(CRC32, CRC32C, SHA1, SHA256) into the gateway 2025-02-14 14:14:00 +04:00
niksis02 b5b418e4b3 fix: Adds the minimum allowed size(5 Mib) check for mp parts sizes in CompleteMultipartUpload 2025-02-03 21:49:15 +04:00
niksis02 f35cdfb20c feat: Adds a flag on the test command to run the integration tests in insecure mode: by skiping the tls verification 2025-02-03 09:13:01 -08:00
niksis02 c094086d83 fix: Fixes the response body streaming for GetObject, implementing a chunk streamer 2025-01-15 23:11:04 +04:00
niksis02 c37a22ffe1 fix: Fixes the AccessControlPolicy Grantee Type unmarshalling, Adds request body validation for the PutBucketAcl action 2025-01-13 23:44:46 +04:00
niksis02 7c5258e6e9 fix: Adds a check to ensure that the CompleteMultipartUpload parts are not empty. 2024-12-17 18:50:57 +04:00
niksis02 2a61489e4c fix: Uncommentes two WORM protection integration tests that were accidentally commented out. 2024-12-11 00:33:45 +04:00
dependabot[bot] adb69ed041 chore(deps): bump the dev-dependencies group across 1 directory with 16 updates 
Bumps the dev-dependencies group with 9 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.32.5` | `1.32.6` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.67.1` | `1.70.0` |
| [github.com/aws/aws-sdk-go-v2/feature/ec2/imds](https://github.com/aws/aws-sdk-go-v2) | `1.16.20` | `1.16.21` |
| [github.com/aws/aws-sdk-go-v2/service/sso](https://github.com/aws/aws-sdk-go-v2) | `1.24.6` | `1.24.7` |
| [github.com/aws/aws-sdk-go-v2/service/ssooidc](https://github.com/aws/aws-sdk-go-v2) | `1.28.5` | `1.28.6` |
| [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) | `1.33.1` | `1.33.2` |
| [github.com/nats-io/nkeys](https://github.com/nats-io/nkeys) | `0.4.7` | `0.4.8` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.28.5` | `1.28.6` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.17.39` | `1.17.42` |

Updates `github.com/aws/aws-sdk-go-v2` from 1.32.5 to 1.32.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.32.5...v1.32.6)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.67.1 to 1.70.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.67.1...service/s3/v1.70.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/ec2/imds` from 1.16.20 to 1.16.21
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/ram/v1.16.20...service/ram/v1.16.21)

Updates `github.com/aws/aws-sdk-go-v2/service/sso` from 1.24.6 to 1.24.7
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/pi/v1.24.6...service/pi/v1.24.7)

Updates `github.com/aws/aws-sdk-go-v2/service/ssooidc` from 1.28.5 to 1.28.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.28.5...config/v1.28.6)

Updates `github.com/aws/aws-sdk-go-v2/service/sts` from 1.33.1 to 1.33.2
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.33.1...service/sfn/v1.33.2)

Updates `github.com/nats-io/nkeys` from 0.4.7 to 0.4.8
- [Release notes](https://github.com/nats-io/nkeys/releases)
- [Changelog](https://github.com/nats-io/nkeys/blob/main/.goreleaser.yml)
- [Commits](https://github.com/nats-io/nkeys/compare/v0.4.7...v0.4.8)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.28.5 to 1.28.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.28.5...config/v1.28.6)

Updates `github.com/aws/aws-sdk-go-v2/credentials` from 1.17.46 to 1.17.47
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.46...credentials/v1.17.47)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.39 to 1.17.42
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.39...credentials/v1.17.42)

Updates `github.com/aws/aws-sdk-go-v2/internal/configsources` from 1.3.24 to 1.3.25
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/ini/v1.3.24...internal/ini/v1.3.25)

Updates `github.com/aws/aws-sdk-go-v2/internal/endpoints/v2` from 2.6.24 to 2.6.25
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/endpoints/v2.6.24...internal/endpoints/v2.6.25)

Updates `github.com/aws/aws-sdk-go-v2/internal/v4a` from 1.3.24 to 1.3.25
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/ini/v1.3.24...internal/ini/v1.3.25)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/checksum` from 1.4.5 to 1.4.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/m2/v1.4.5...service/m2/v1.4.6)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/presigned-url` from 1.12.5 to 1.12.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/credentials/v1.12.6/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/sso/v1.12.5...credentials/v1.12.6)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/s3shared` from 1.18.5 to 1.18.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/config/v1.18.6/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.5...config/v1.18.6)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/ec2/imds
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sso
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/ssooidc
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/nats-io/nkeys
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/configsources
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/endpoints/v2
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/v4a
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/checksum
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/presigned-url
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/s3shared
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-12-04 14:48:03 -08:00
Ben McClelland d9591f694e fix: azure admin list-buckets 
There were two issues that were preventing correct behavior here.
One was that we need to specifically request the container metadata
when listing containers, and then we also need to handle the case
where the container does not include the acl metadata.

This fixes both of these cases by adding in the metadata request
option for this container listing, and will return a default acl
if not provided in the container metadaata.

Fixes #948
 2024-12-02 16:57:49 -08:00
Ben McClelland 1e5c1780c9 Merge pull request #956 from versity/ben/list-buckets-empty 
feat: add list-buckets test for no buckets
 2024-11-19 08:16:27 -08:00
Ben McClelland 568f8346bf feat: add list-buckets test for no buckets 
We need to check to make sure list-buckets returns a correct
empty list when no buckets created.
 2024-11-18 21:35:19 -08:00
Ben McClelland a6d61e1dde fix: listobjects internal server error for prefix not directory 
We need to treat a prefix that has a parent component as a file
instead of a directory in the filesystem as a non existing prefix
instead of an internal server error. This error was caused
because we were not handling ENOTDIR within the fs walk.

Fixes #949
 2024-11-18 21:15:16 -08:00
Ben McClelland db305142f1 fix: return better error when trying to delete non empty directory object 
The posix backend will return ENOTEMPTY when trying to delete a
directory that is not empty. This normally would run successfully
on object systems. So we need to create another non-standard error
for this case. We mainly just don't want to return InternalError
for this case.

Fixes #946
 2024-11-14 22:33:08 -08:00
Ben McClelland 0312a1e3dc fix: internal server error when object parent dir is a file 
The fileystem will return ENOTDIR if we try to access a file path
where a parent directory within the path already exists as a file.
In this case we need to return a standard 404 no such key since
the request object does not exist within the filesytem.

Fixes #942
 2024-11-08 08:21:14 -08:00
Ben McClelland 5d8899baf4 Merge pull request #936 from versity/fix/putbuckettagging-status 
fix: Changes the PutBucketTagging action response status code from 20…
 2024-11-04 08:46:44 -08:00
jonaustin09 7bd32a2cfa fix: Changes the PutBucketTagging action response status code from 200(OK) to 204(No Content) 2024-10-31 18:30:07 -04:00
jonaustin09 66c13ef982 fix: Adds a check to ensure the x-amz-object-attributes header is set and non-empty. 2024-10-31 17:05:54 -04:00
jonaustin09 06e2f2183d fix: Changes GetObjectAttributes action xml encoding root element to GetObjectAttributesResponse. Adds input validation for x-amz-object-attributes header. Adds x-amz-delete-marker and x-maz-version-id headers for GetObjectAttributes action. Adds VersionId in HeadObject response, if it's not specified in the request 2024-10-30 15:42:15 -04:00
jonaustin09 4d6ec783bf feat: Implements pagination for ListBuckets 2024-10-28 16:26:08 -04:00
Ben McClelland 0aa62f16c9 Merge pull request #913 from versity/feat/admin-apis-refactoring 
Admin APIs refactoring
 2024-10-25 16:04:04 -07:00
jonaustin09 c6359a7050 feat: Refactoring admin APIs: changes i/o data transfer encoding to xml, implements traditional aws-like error handling, adds admin role checker middleware. Refactoring admin CLI actions to handle aws-like error responses 2024-10-25 11:40:23 -04:00
jonaustin09 3b903f6044 fix: Fixes max-parts, max-keys, max-uploads validation defaulting to 1000 2024-10-22 14:28:50 -04:00
jonaustin09 858236150c fix: Fixes multiple null versionId delete markers creation with DeleteObject 2024-10-21 16:10:34 -04:00
jonaustin09 c803af4688 fix: Prevents bucket deletion when it contains object versions by returning ErrVersionedBucketNotEmpty error. Enabled object deletion with versionId and delete markers creation with DeleteObject when the versioning status is Suspended 2024-10-18 15:36:52 -04:00
Ben McClelland a78c826d0f Merge pull request #897 from versity/fix/getobject-null-versionid-obj 
fix: Resolves the null object version get issue from posix directory
 2024-10-16 13:38:27 -07:00
jonaustin09 cdcbc9c9cb fix: Resolves the null object version get issue from posix directory 2024-10-16 15:58:40 -04:00