Commit Graph

2550 Commits

Author SHA1 Message Date
Ben McClelland d05d29010d Merge pull request #1739 from versity/sis/create-bucket-and-owner
feat: implements admin CreateBucket endpoint/cli command
2026-01-12 10:09:58 -08:00
Ben McClelland b1e9dead5d Merge pull request #1748 from loktionovam/fix-meta-sidecar-cleanup-performance
fix: optimize sidecar empty-dir checks
2026-01-12 08:56:33 -08:00
Ben McClelland bf5b0b85d8 Merge pull request #1746 from dch/fix/freebsd-zfs-xattr
Fix ZFS xattr namespace usage on FreeBSD vs other platforms
2026-01-12 08:40:25 -08:00
niksis02 2561ef9708 feat: implements admin CreateBucket endpoint/cli command
Closes #1731

Implements the admin `CreateBucket` (`PATCH /:bucket/create`) endpoint and CLI command, which create a new bucket with the provided owner access key ID. The endpoint internally calls the S3 `CreateBucket` API, storing the new owner information in the request context under the `bucket-owner` key. This value is then retrieved by the S3 API layer and the backends.

The endpoint uses the custom `x-vgw-owner` HTTP header to pass the bucket owner access key ID.

The admin CLI command mirrors `aws s3api create-bucket` and supports all flags implemented by the gateway (for example, `--create-bucket-configuration`, `--acl`, `--object-ownership`, etc.).
2026-01-12 14:32:52 +04:00
Aleksandr Loktionov b78d21c3db fix: optimize sidecar empty-dir checks 2026-01-12 06:51:18 -03:00
Dave Cottlehuber 0cab42d9fe xattr: use different namespace prefixes for FreeBSD vs other platforms
Go's stdlib seems to handle the FreeBSD user. namespace directly, or
FreeBSD itself doesn't require it. Make this a platform-specific
feature.

Fixes: #1745
2026-01-10 16:43:33 +00:00
Ben McClelland 12f0b5c43c Merge pull request #1737 from versity/ben/default-cors
feat: add option for default global cors allow origin headers
2026-01-09 15:34:23 -08:00
Ben McClelland e81b87f71c Merge pull request #1744 from versity/sis/invalid-retain-until-date-header-err
fix: fixes invalid/expired x-amz-object-lock-retain-until-date errors
2026-01-09 15:33:30 -08:00
Ben McClelland ff00e42538 Merge pull request #1743 from versity/sis/obj-lock-retention-period-err
fix: fixes the InvalidRetentionPeriod error code and message
2026-01-09 15:32:47 -08:00
niksis02 cf99b3e036 fix: fixes invalid/expired x-amz-object-lock-retain-until-date errors
Fixes #1733
Fixes #1734

The `x-amz-object-lock-retain-until-date` request header appears in the `PutObject`, `CopyObject`, and `CreateMultipartUpload` operations. This PR fixes the two types of error codes and messages returned when the header value is invalid or expired and adds the corresponding integration tests.
2026-01-09 17:13:02 +04:00
niksis02 c91e5dc3f2 fix: fixes the InvalidRetentionPeriod error code and message
Fixes #1738

Corrects(S3 compatible) the `InvalidRetentionPeriod` error `Code` and `Message`.
2026-01-09 15:21:26 +04:00
Ben McClelland d446102f69 feat: add option for default global cors allow origin headers
There is some desire to have a web dashboard for the gateway. So
that we dont have to proxy all requests through the webserver
and expose credentials over the wire, the better approach would
be to enable CORS headers to allow browser requests directly to
the s3/admin service.

The default for these headers is off, so that they are only
enabled for instances that specfically want to support this
workload.
2026-01-08 16:23:23 -08:00
Ben McClelland f2a75708e4 Merge pull request #1718 from versity/test/performance_improvements
test: matrix update, speed up tests, some region fixes
2026-01-06 15:41:06 -08:00
Ben McClelland 6fd939386c Merge pull request #1735 from versity/dependabot/go_modules/dev-dependencies-67992aebd4
chore(deps): bump github.com/valyala/fasthttp from 1.68.0 to 1.69.0 in the dev-dependencies group
2026-01-06 08:25:55 -08:00
Luke McCrone dff20b5b9d test: test matrix setup overhaul, fix some region/iam issues 2026-01-06 12:55:18 -03:00
dependabot[bot] 7a4dd59c81 chore(deps): bump github.com/valyala/fasthttp
Bumps the dev-dependencies group with 1 update: [github.com/valyala/fasthttp](https://github.com/valyala/fasthttp).


Updates `github.com/valyala/fasthttp` from 1.68.0 to 1.69.0
- [Release notes](https://github.com/valyala/fasthttp/releases)
- [Commits](https://github.com/valyala/fasthttp/compare/v1.68.0...v1.69.0)

---
updated-dependencies:
- dependency-name: github.com/valyala/fasthttp
  dependency-version: 1.69.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-05 21:06:58 +00:00
Ben McClelland 6f74d2cddb Merge pull request #1732 from versity/sis/getbucketlocation-us-east-1
fix: return null in GetBucketLocation for us-east-1
2026-01-05 08:22:33 -08:00
niksis02 8e0eec0201 fix: return null in GetBucketLocation for us-east-1
Fixes #1643

`GetBucketLocation` in S3 returns empty `LocationConstraint` if the bucket is in the `us-east-1` region. This fix returns empty `LocationConstraint` if the gateway region is `us-east-1`.
2026-01-05 16:24:01 +04:00
Ben McClelland 0cfacfc049 Merge pull request #1730 from versity/sis/create-bucket-location-constraint-us-east-1
fix: fixes CreateBucket LocationConstraint validation
2026-01-03 20:52:57 -08:00
Ben McClelland 6b017aa5cd Merge pull request #1729 from versity/sis/versions-query-with-key
fix: adds an error route for ?versions subresource with key
2026-01-03 20:52:09 -08:00
Ben McClelland 841a012ce0 Merge pull request #1728 from versity/sis/get-object-empty-tagging
fix: removes the NoSuchTagSet error in GetObjectTagging
2026-01-03 20:51:08 -08:00
Ben McClelland 067de184a9 Merge pull request #1726 from versity/sis/object-tagging-response-versionid
fix: adds versionId in put/get/delete object tagging actions response.
2026-01-03 20:50:24 -08:00
Ben McClelland 10ab569277 Merge pull request #1725 from versity/sis/sigv4-expect-header
fix: removes Expect from sigv4 ignored headers list
2026-01-03 20:49:21 -08:00
Ben McClelland 01552b78c7 Merge pull request #1713 from versity/test/get_bucket_policy_status
Test/get bucket policy status
2026-01-03 20:48:40 -08:00
niksis02 d0158420ee fix: fixes CreateBucket LocationConstraint validation
Fixes #1654
Fixes #1644

CreateBucket `LocationConstraint` rejects empty values with an `InvalidLocationConstraint` error.
The `us-east-1` `LocationConstraint` is considered invalid because it is the default value and must not be present in the `CreateBucketConfiguration` request body.

This PR fixes both issues by returning `InvalidLocationConstraint` in both cases.
2026-01-03 23:21:19 +04:00
niksis02 c2c2306d37 fix: adds an error route for ?versions subresource with key
Fixes #1688

S3 returns a specific `InvalidRequest` error for the requests with `?versions` query param against `object` resource. The PR implements this error route.
2026-01-03 16:23:11 +04:00
Luke McCrone 841b3d61a4 test: policy status, policy command updates 2026-01-02 19:15:00 -03:00
Luke McCrone fa2e677370 test: empty tag test correction 2026-01-02 17:02:50 -03:00
niksis02 9f6bf183f4 fix: removes Expect from sigv4 ignored headers list
Fixes #1707

The `Expect` HTTP header is ignored by the AWS SDK SigV4 signer and is omitted during signature calculation. As a result, the signature is computed incorrectly when the `Expect` header is included in the signed headers. This PR removes the `Expect` header from the SigV4 ignored headers list in the SDK-derived source code.
2026-01-02 23:47:45 +04:00
niksis02 12e1308d1f fix: adds versionId in put/get/delete object tagging actions response.
Fixes #1698

`PutObjectTagging`, `GetObjectTagging` and `DeleteObjectTagging` return the `x-amz-version-id` in the response headers. The PR adds this header in the responses.
2026-01-02 23:46:52 +04:00
Ben McClelland f235b62b70 Merge pull request #1727 from versity/sis/object-write-if-none-match 2026-01-02 11:40:10 -08:00
niksis02 06a45124b1 fix: removes the NoSuchTagSet error in GetObjecTagging
Fixes #1686

GetObjectTagging previously returned a `NoSuchTagSet` error when no object tags were set. This has been fixed, and an empty tag set is now returned instead.
2026-01-02 23:31:35 +04:00
niksis02 a75aa9bad5 fix: fixes if-none-match precondition header logic in object write operations
Fixes #1708

This PR focuses on evaluating the `x-amz-if-none-match` precondition header for object PUT operations. If any value other than `*` is provided, a `NotImplemented` error is returned. If `If-Match` is used together with `If-None-Match`, regardless of the value combination, a `NotImplemented` error is returned. When only `If-None-Match: *` is specified, a `PreconditionFailed` error is returned if the object already exists in `PutObject` or `CompleteMultipartUpload`; if the object does not exist, object creation is allowed.
2026-01-02 22:59:13 +04:00
Ben McClelland 4cbd58cc66 Merge pull request #1717 from loktionovam/fix-meta-sidecar-cleanup
fix: cleanup sidecar metadata empty dirs
2025-12-31 00:44:19 -08:00
Ben McClelland e5343cf611 Merge pull request #1723 from versity/sis/if-match-size-debug-log-fix
fix: fixes x-amz-if-match-size parsing
2025-12-31 00:34:08 -08:00
Ben McClelland 0a2c7ac7cb Merge pull request #1722 from versity/sis/authorizationheadermalformed-err-status
fix: changes AuthorizationHeaderMalformed error status to 400
2025-12-31 00:32:49 -08:00
Ben McClelland b1fed810a7 Merge pull request #1721 from versity/sis/precondition-ifmatch-obj-not-exist
fix: return NoSuchKey if a precondition header is present and object doesn't exist in PutObject, CompleteMultipartUpload
2025-12-31 00:32:02 -08:00
Ben McClelland 48b590fcb8 Merge pull request #1720 from versity/dependabot/go_modules/dev-dependencies-92da416cd8
chore(deps): bump the dev-dependencies group with 2 updates
2025-12-31 00:30:01 -08:00
Ben McClelland f835ef1772 Merge pull request #1719 from versity/sis/putbucketpolicy-status
fix: fixes the PutBucketPolicy response status
2025-12-31 00:29:33 -08:00
Ben McClelland d819fa8665 Merge pull request #1716 from versity/sis/complete-mp-location
feat: adds Location in CompleteMultipartUpload response
2025-12-31 00:28:27 -08:00
Ben McClelland 0240bb922c Merge pull request #1715 from versity/sis/precondition-string-quotes
fix: Makes precondition headers insensitive to whether the value is quoted
2025-12-31 00:23:17 -08:00
Ben McClelland 0b3722bd09 Merge pull request #1703 from versity/test/add_skips_checker
tests: add skips checker, remove skips, add new issues
2025-12-31 00:21:44 -08:00
Ben McClelland 7c454d230e Merge pull request #1701 from versity/sis/signed-streaming-upload-error-handling
fix: bunch of fixes in signed streaming requests
2025-12-31 00:20:27 -08:00
niksis02 981a34e9d5 fix: fixes x-amz-if-match-size parsing
The `x-amz-if-match-size` parsing debug log used to appear for all `DeleteObject` calls when the header was missing. An empty-string check was missing, which led to attempting to parse an empty string into an `int64`, causing a failure and triggering the debug log. This check has now been added, and the debug log is emitted only when the header is present and contains an invalid `int64` value.
2025-12-30 12:35:14 +04:00
niksis02 657b9ac046 fix: changes AuthorizationHeaderMalformed error status to 400
Fixes #1706

Changes the `AuthorizationHeaderMalformed` error http status code from `403` to `400`.
2025-12-30 12:21:54 +04:00
niksis02 61308d2fbf fix: return NoSuchKey if a precondition header is present and object doesn't exist in PutObject, CompleteMultipartUpload
Fixes #1709

If any precondition header is present(`If-Match`, `If-None-Match`) in `PutObject` and `CompleteMultipartUpload` and there's no object in the bucket with the given key, a `NoSuchKey` error is now returned. Previously the headers were simply ignored and new object creation was allowed.
2025-12-30 12:02:49 +04:00
dependabot[bot] 8d16bff8ce chore(deps): bump the dev-dependencies group with 2 updates
Bumps the dev-dependencies group with 2 updates: [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.94.0 to 1.95.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.94.0...service/s3/v1.95.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.20.17 to 1.20.18
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/emr/v1.20.17...service/emr/v1.20.18)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.95.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-version: 1.20.18
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-29 21:07:20 +00:00
Luke McCrone 35596b38ae test: change 200 in test to 204 2025-12-29 14:29:08 -03:00
niksis02 39ee175484 fix: fixes the PutBucketPolicy response status
Fixes #1712

A successful `PutBucketPolicy` call returns `204`(No Content) http status. This PR fixes the response status code, by sending correct `204`, insted of `200`.
2025-12-29 21:10:52 +04:00
Aleksandr Loktionov edac345c23 fix: cleanup sidecar metadata empty dirs 2025-12-29 08:24:05 -03:00