mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-02-13 05:31:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,134 Commits 23 Branches 46 Tags
e702a4860a7d5f4e010a12ceb6a96cbd841eb6ab
Commit Graph

2134 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ben McClelland
e702a4860a fix: CopyObject with URL-encoded special chars 
CopyObject was failing with NoSuchKey when source keys contained special
characters like {} or spaces. The X-Amz-Copy-Source header is URL-encoded
by clients, but ParseCopySource wasn't decoding before filesystem access.

Added url.QueryUnescape() to properly decode bucket and object names,
fixing copy operations for keys with special characters.

Fixing this also uncovered an errors with azure blob url encoding with
similar special character handling. Added this fix in for the integration
tests to pass.

Fixes #1832
Fixes #1637
 2026-02-10 14:55:18 -08:00
Ben McClelland
a4323d5338 Merge pull request #1831 from versity/dependabot/github_actions/actions/download-artifact-7 2026-02-10 01:55:54 -08:00
dependabot[bot]
d4ea895a9c chore(deps): bump actions/download-artifact from 4 to 7 
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 7.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-02-10 00:12:44 +00:00
Ben McClelland
76cb616b96 Merge pull request #1830 from versity/dependabot/github_actions/actions/upload-artifact-6 
chore(deps): bump actions/upload-artifact from 4 to 6
 2026-02-09 16:11:39 -08:00
Ben McClelland
afbea9f878 Merge pull request #1828 from versity/ben/readme-updates 
chore: update README.md to highlight new web gui feature
 2026-02-09 16:10:57 -08:00
dependabot[bot]
790bac2b3d chore(deps): bump actions/upload-artifact from 4 to 6 
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-02-09 23:57:05 +00:00
Ben McClelland
af76584a9d chore: update README.md to highlight new web gui feature 2026-02-09 15:36:54 -08:00
Ben McClelland
7f7b804958 Merge pull request #1827 from versity/ben/webgui-object-lock-md5 
fix: webui md5 missing error when enabling directory object lock
v1.2.0 		2026-02-09 13:56:22 -08:00
Ben McClelland
9343860321 fix: webui md5 missing error when enabling directory object lock 
The PutBucketObjectLockConfiguration now requires Content-MD5
header to match AWS behavior. This broke the GUI from being able
to set object lock configuration for a bucket.

This fix adds the Content-MD5 header to this request.
 2026-02-09 12:27:02 -08:00
Ben McClelland
324950c3e4 Merge pull request #1820 from versity/ben/cors 
fix: remove duplicate cors headers from options router
 2026-02-06 09:28:36 -08:00
Ben McClelland
e805003872 Merge pull request #1816 from versity/sis/max-limiter-errors 
fix: fixes list-limiters parsing and validation
 2026-02-06 09:28:18 -08:00
Ben McClelland
b2a9b383ae Merge pull request #1803 from versity/sis/list-mp-delimiter 
feat: adds delimiter support in ListMultipartUploads
 2026-02-06 09:27:32 -08:00
niksis02
2365f9f1ae fix: fixes list-limiters parsing and validation 
Fixes #1809
Fixes #1806
Fixes #1804
Fixes #1794

This PR focuses on correcting so-called "list-limiter" parsing and validation. The affected limiters include: `max-keys`, `max-uploads`, `max-parts`, `max-buckets`, `max-uploads` and `part-number-marker`. When a limiter value is outside the integer range, a specific `InvalidArgument` error is now returned. If the value is a valid integer but negative, a different `InvalidArgument` error is produced.

`max-buckets` has its own validation rules: completely invalid values and values outside the allowed range (`1 <= input <= 10000`) return distinct errors. For `ListObjectVersions`, negative `max-keys` values follow S3’s special-case behavior and return a different `InvalidArgument` error message.

Additionally, `GetObjectAttributes` now follows S3 semantics for `x-amz-max-parts`: S3 ignores invalid values, so the gateway now matches that behavior.
 2026-02-06 14:21:56 +04:00
niksis02
2e6794007c feat: adds delimiter support in ListMultipartUploads 
Fixes #1792
Fixes #1747
Fixes #1797
Fixes #1799

This PR primarily introduces delimiter support and several bug fixes for the `ListMultipartUploads` action in the POSIX and Azure backends. Delimiter handling is now implemented — when a delimiter is present in multipart-upload object key names, the backend collects and returns the appropriate common prefixes.
This functionality is achieved by introducing a common multipart-upload lister in the backend package. All backends (Azure, POSIX) now use this lister. The lister accepts a list that is already sorted and filtered by `KeyMarker` and `Prefix`.

Previously, the `KeyMarker` was required to exactly match an existing multipart-upload object key. This restriction is removed. The listing now relies on a lexicographical comparison between the provided `KeyMarker` and existing multipart-upload object keys.

Validation for `UploadIdMarker` is also added to correctly return an `InvalidArgument` error for invalid upload IDs. If `KeyMarker` is missing, the `UploadIdMarker` is ignored entirely. If `KeyMarker` is provided, a valid upload ID is one that matches an upload belonging to *the first object key after the KeyMarker*. For example, if the `KeyMarker` is `foo`, but the provided `UploadIdMarker` corresponds to an upload under `quxx`, it is invalid. It must match one of the uploads for the next object key equal to `foo`.

Finally, this PR fixes multipart-upload sorting. Multipart uploads must be sorted primarily lexicographically by their object key, and secondarily—when multiple uploads share the same object key—by their initiation time in ascending order.
 2026-02-06 14:16:16 +04:00
Ben McClelland
e15bd4a5bc Merge pull request #1818 from versity/test/rest_missing_tests 
test: some not-implemented's, add list-object-v2 working test
 2026-02-05 17:01:25 -08:00
Ben McClelland
11e5049573 fix: remove duplicate cors headers from options router 
In the refactor for being able to set global CORS headers, the
options router was incorrectly set to use both CORS middlewares
casuing duplicate headers to be set. The ApplyBucketCORS()
middleware is not needed for options since this is already handled
by the CORSOoptions controller.

Fixes #1819
 2026-02-05 16:54:46 -08:00
Luke McCrone
44fa5fa178 test: some not-implemented's, add list-object-v2 working test 2026-02-05 10:14:08 -03:00
Ben McClelland
772e33be70 Merge pull request #1801 from wishdev/walker 
fix: Address slowness with lexical sorting
 2026-02-04 08:47:10 -08:00
Ben McClelland
74a0396802 Merge pull request #1800 from versity/test/reporting 
Test/reporting
 2026-02-04 08:44:34 -08:00
Ben McClelland
26490d07b1 Merge pull request #1817 from versity/ben/pin-nats-dep 
fix: pin github.com/nats-io/nkeys version to v0.4.12
 2026-02-04 08:43:51 -08:00
Ben McClelland
63fcdb38e2 fix: pin github.com/nats-io/nkeys version to v0.4.12 
The go mod cache is listing a newer version than the latest tag in
the repo. So we need to pin this older version to prevent dependabot
from updating until the repo tags a newer version.
 2026-02-04 08:30:45 -08:00
Ben McClelland
35605683cf Merge pull request #1807 from versity/dependabot/go_modules/dev-dependencies-50b95ed59f 
chore(deps): bump the dev-dependencies group with 7 updates
 2026-02-04 08:11:55 -08:00
dependabot[bot]
82878f40e1 chore(deps): bump the dev-dependencies group with 7 updates 
Bumps the dev-dependencies group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/DataDog/datadog-go/v5](https://github.com/DataDog/datadog-go) | `5.8.2` | `5.8.3` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.95.1` | `1.96.0` |
| [github.com/gofiber/fiber/v2](https://github.com/gofiber/fiber) | `2.52.10` | `2.52.11` |
| [github.com/clipperhouse/uax29/v2](https://github.com/clipperhouse/uax29) | `2.4.0` | `2.5.0` |
| [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt) | `5.3.0` | `5.3.1` |
| [github.com/nats-io/nkeys](https://github.com/nats-io/nkeys) | `0.4.12` | `0.4.15` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.21.0` | `1.21.1` |


Updates `github.com/DataDog/datadog-go/v5` from 5.8.2 to 5.8.3
- [Release notes](https://github.com/DataDog/datadog-go/releases)
- [Changelog](https://github.com/DataDog/datadog-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/DataDog/datadog-go/compare/v5.8.2...v5.8.3)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.95.1 to 1.96.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.95.1...service/s3/v1.96.0)

Updates `github.com/gofiber/fiber/v2` from 2.52.10 to 2.52.11
- [Release notes](https://github.com/gofiber/fiber/releases)
- [Commits](https://github.com/gofiber/fiber/compare/v2.52.10...v2.52.11)

Updates `github.com/clipperhouse/uax29/v2` from 2.4.0 to 2.5.0
- [Release notes](https://github.com/clipperhouse/uax29/releases)
- [Commits](https://github.com/clipperhouse/uax29/compare/v2.4.0...v2.5.0)

Updates `github.com/golang-jwt/jwt/v5` from 5.3.0 to 5.3.1
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Commits](https://github.com/golang-jwt/jwt/compare/v5.3.0...v5.3.1)

Updates `github.com/nats-io/nkeys` from 0.4.12 to 0.4.15
- [Release notes](https://github.com/nats-io/nkeys/releases)
- [Commits](https://github.com/nats-io/nkeys/compare/v0.4.12...v0.4.15)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.21.0 to 1.21.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.21.0...v1.21.1)

---
updated-dependencies:
- dependency-name: github.com/DataDog/datadog-go/v5
  dependency-version: 5.8.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.96.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/gofiber/fiber/v2
  dependency-version: 2.52.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/clipperhouse/uax29/v2
  dependency-version: 2.5.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/golang-jwt/jwt/v5
  dependency-version: 5.3.1
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/nats-io/nkeys
  dependency-version: 0.4.15
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-version: 1.21.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-02-03 02:40:28 +00:00
John W Higgins
bcf341bdaa Add "lexical" sort to walker.go 2026-02-01 16:10:23 -08:00
Luke McCrone
17adbeca9e test: S3 command coverage reporting 2026-01-30 18:51:02 -03:00
John W Higgins
73e2df4105 Base import of cutdown version of io/fs/walk.go from golang as walker.go 2026-01-29 13:10:01 -08:00
John W Higgins
bef8f38f3e Revert of 34da1833 and 8e18b431 for backend/walk.go 
Revert the changes with regards to lexical sorting but only for the walk.go
code itself. Leave the tests alone.
 2026-01-29 13:10:01 -08:00
Ben McClelland
db12fb94a3 Merge pull request #1798 from versity/sis/listparts-remove-metadata-loading 2026-01-28 21:49:47 -08:00
niksis02
7017ffa2a3 fix: removes object metadata loading from posix ListParts 
In the POSIX `ListParts` implementation, there was a code snippet that loaded the object metadata, even though it wasn’t needed and never used in the response. This redundant code has now been removed.
 2026-01-28 21:12:56 +04:00
Ben McClelland
37e6d96d1b Merge pull request #1796 from versity/sis/revert-nats-nkeys-version 2026-01-27 18:01:29 -08:00
niksis02
03f0be28b6 fix: reverts the nats-io/nkeys package version to v0.4.12 
Fixes #1795
 2026-01-28 01:12:04 +04:00
Ben McClelland
086a622914 Merge pull request #1781 from versity/test/rest_put_object_acl 
test: REST - PutObject object ACLs, object lock header
 2026-01-26 15:52:01 -08:00
khambrecht
f29206afb6 fix: put object on windows when parent directories dont already exists 
The previous logic was not allowing put-object on windows when the parent directory did not already exist, and would
not always return the correct error if an ancestor in the path already existed as a file.

The problem is the different behavior of the os.Stat command in Windows compared to *nix in backend/posix/posix.go
in function PutObjectWithPostFunc. The os.Stat returns ENOTDIR on *nix if the parent object is a file instead of a
directory. On Windows, if the parent object does not exist at all, the return code of such os.Stat is ERROR_PATH_NOT_FOUND which is mapped to ENOTDIR. However this is inappropriate in this case. As a result, the
return code of the os.Stat is incorrectly interpreted as if the parent object is a file instead of the parent object does not
exist. Which then leads to a failed upload.

This fix validates the existing parent structure on put to make sure the correct error is returned or the put is successful.

Fixes #1702
 2026-01-27 08:51:06 +09:00
Ben McClelland
28c805f59e Merge pull request #1793 from versity/dependabot/go_modules/dev-dependencies-34fb423312 
chore(deps): bump github.com/clipperhouse/uax29/v2 from 2.3.1 to 2.4.0 in the dev-dependencies group
 2026-01-26 15:28:56 -08:00
dependabot[bot]
1dfc77dfd4 chore(deps): bump github.com/clipperhouse/uax29/v2 
Bumps the dev-dependencies group with 1 update: [github.com/clipperhouse/uax29/v2](https://github.com/clipperhouse/uax29).


Updates `github.com/clipperhouse/uax29/v2` from 2.3.1 to 2.4.0
- [Release notes](https://github.com/clipperhouse/uax29/releases)
- [Commits](https://github.com/clipperhouse/uax29/compare/v2.3.1...v2.4.0)

---
updated-dependencies:
- dependency-name: github.com/clipperhouse/uax29/v2
  dependency-version: 2.4.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-01-26 22:30:11 +00:00
Luke McCrone
1f9856110d test: PutObject object lock tests and updates, delete bucket test 2026-01-26 16:58:49 -03:00
Ben McClelland
01fc142c1e fix: correct spelling for debuglogger.InternalError() (#1784) 2026-01-24 06:44:54 -08:00
Ben McClelland
ca2dd9b4b3 Merge pull request #1791 from versity/dependabot/go_modules/dev-dependencies-7c76174904 
chore(deps): bump the dev-dependencies group with 2 updates
 2026-01-23 13:56:22 -08:00
Ben McClelland
10152cefbc Merge pull request #1790 from versity/dependabot/github_actions/docker/build-push-action-6 
chore(deps): bump docker/build-push-action from 5 to 6
 2026-01-23 13:56:01 -08:00
Ben McClelland
948b424ed2 Merge pull request #1789 from versity/dependabot/github_actions/actions/checkout-6 
chore(deps): bump actions/checkout from 4 to 6
 2026-01-23 13:55:33 -08:00
dependabot[bot]
d2996e1131 chore(deps): bump the dev-dependencies group with 2 updates 
Bumps the dev-dependencies group with 2 updates: [github.com/nats-io/nkeys](https://github.com/nats-io/nkeys) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/nats-io/nkeys` from 0.4.12 to 0.4.14
- [Release notes](https://github.com/nats-io/nkeys/releases)
- [Commits](https://github.com/nats-io/nkeys/commits)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.20.19 to 1.21.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.20.19...v1.21.0)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nkeys
  dependency-version: 0.4.14
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-version: 1.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-01-23 19:52:25 +00:00
dependabot[bot]
2489d876c9 chore(deps): bump docker/build-push-action from 5 to 6 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5 to 6.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-01-23 19:51:10 +00:00
dependabot[bot]
a69f5a4db7 chore(deps): bump actions/checkout from 4 to 6 
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-01-23 19:51:06 +00:00
Ben McClelland
df31eb031a Merge pull request #1788 from versity/ben/dependabot-actions 
chore: add dependabot updates for github actions
 2026-01-23 11:50:15 -08:00
Ben McClelland
b70be6116e Merge pull request #1787 from versity/ben/codeql-js 
chore: add codeql ignore for embedded 3rd party js assets
 2026-01-23 11:45:43 -08:00
Ben McClelland
e08539e909 chore: add dependabot updates for github actions 2026-01-23 11:39:53 -08:00
Ben McClelland
f78483a938 chore: add codeql ignore for embedded 3rd party js assets 
We were getting some warnings from the embedded tailwind js file.
We dont need to be scanning these 3rd party assets in this repo.
 2026-01-23 11:34:15 -08:00
Ben McClelland
cb1d469742 Merge pull request #1786 from versity/ben/default-codeql 
chore: add advanced codeql workflow for repo customizations
 2026-01-23 11:22:11 -08:00
Ben McClelland
792a3eb2c5 chore: add advanced codeql workflow for repo customizations 
This is the default generated codeql for the advanced setup
to allow for repo customization
 2026-01-23 11:11:04 -08:00
Ben McClelland
252090d9e9 Merge pull request #1783 from versity/sis/object-acl-headers-not-implemented 
fix: return not implemented in object actions, if acl header is present
 2026-01-23 09:31:59 -08:00