Commit Graph

2550 Commits

Author SHA1 Message Date
Ben McClelland e0209ebab4 Merge pull request #1997 from versity/sis/copyobject-threshold
fix: enforce 5gb copy source object size threshold.
2026-03-31 12:27:11 -07:00
Ben McClelland 89ab7a4f2b Merge pull request #1982 from versity/sis/POST-object
feat: add browser-based POST object upload support
2026-03-31 12:25:00 -07:00
Ben McClelland 56b9c67905 Merge pull request #2001 from versity/sis/headobject-response-overrides
feat: support response header overrides in HeadObject
2026-03-31 12:23:57 -07:00
niksis02 59002b2650 feat: implement integration tests for browser-based POST object 2026-03-31 22:47:04 +04:00
niksis02 285d130a47 feat: add browser-based POST object upload support
Closes #1648
Fixes #1980
Fixes #1981

This PR implements browser-based POST object uploads for S3-compatible form uploads. It adds support for handling `multipart/form-data` object uploads submitted from browsers, including streaming multipart parsing so file content is not buffered in memory, POST policy decoding and evaluation, SigV4-based form authorization, and integration with the existing `PutObject` backend flow. The implementation covers the full browser POST upload path, including validation of required form fields, credential scope and request date checks, signature verification, metadata extraction from `x-amz-meta-*` fields, checksum field parsing, object tagging conversion from XML into the query-string format expected by `PutObject`, and browser-compatible success handling through `success_action_status` and `success_action_redirect`. It also wires the new flow into the router and metrics layer and adds POST-specific error handling and debug logging across policy parsing, multipart parsing, and POST authorization. AWS S3 also accepts the `redirect` form field alongside `success_action_redirect`, but since AWS has marked `redirect` as deprecated and is planning to remove it, this gateway intentionally does not support it.
2026-03-31 22:44:54 +04:00
niksis02 bbe246e8ec fix: enforce 5gb copy source object size threshold.
Fixes #1896

Enforces the S3 `5 GiB` copy source size limit across the posix and azure
backends for `CopyObject` and `UploadPartCopy`, returning `InvalidRequest` when
the source object exceeds the threshold.

The limit is now configurable via `--copy-object-threshold`
(`VGW_COPY_OBJECT_THRESHOLD`, default 5 GiB).
A new `--mp-max-parts flag` (`VGW_MP_MAX_PARTS`, default `10000`) has been added to make multipart upload parts number limit configurable.

No integration test has been added, as GitHub Actions cannot reliably
handle large objects.
2026-03-31 22:44:03 +04:00
niksis02 3b17f05d17 feat: support response header overrides in HeadObject
Closes #1967

Add support for response header override query parameters(`response-cache-control`, `response-content-disposition`, `response-content-encoding`, `response-content-language`, `response-content-type`, `response-expires`) in `HeadObject`. Anonymous requests with override params are rejected with `ErrAnonymousResponseHeaders`.
2026-03-31 17:19:03 +04:00
Ben McClelland 69b23fe139 Merge pull request #2000 from versity/dependabot/go_modules/dev-dependencies-961b21b1c2 2026-03-30 20:42:58 -07:00
Ben McClelland 2d91e0203e Merge pull request #1999 from versity/dependabot/github_actions/sigstore/cosign-installer-4.1.1 2026-03-30 20:42:24 -07:00
dependabot[bot] fe61e3070b chore(deps): bump the dev-dependencies group with 17 updates
Bumps the dev-dependencies group with 17 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.41.4` | `1.41.5` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.32.12` | `1.32.13` |
| [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) | `1.19.12` | `1.19.13` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager](https://github.com/aws/aws-sdk-go-v2) | `0.1.11` | `0.1.12` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.97.2` | `1.97.3` |
| [github.com/nats-io/nats.go](https://github.com/nats-io/nats.go) | `1.49.0` | `1.50.0` |
| [github.com/aws/aws-sdk-go-v2/feature/ec2/imds](https://github.com/aws/aws-sdk-go-v2) | `1.18.20` | `1.18.21` |
| [github.com/aws/aws-sdk-go-v2/internal/configsources](https://github.com/aws/aws-sdk-go-v2) | `1.4.20` | `1.4.21` |
| [github.com/aws/aws-sdk-go-v2/internal/endpoints/v2](https://github.com/aws/aws-sdk-go-v2) | `2.7.20` | `2.7.21` |
| [github.com/aws/aws-sdk-go-v2/internal/v4a](https://github.com/aws/aws-sdk-go-v2) | `1.4.21` | `1.4.22` |
| [github.com/aws/aws-sdk-go-v2/service/internal/checksum](https://github.com/aws/aws-sdk-go-v2) | `1.9.12` | `1.9.13` |
| [github.com/aws/aws-sdk-go-v2/service/internal/presigned-url](https://github.com/aws/aws-sdk-go-v2) | `1.13.20` | `1.13.21` |
| [github.com/aws/aws-sdk-go-v2/service/internal/s3shared](https://github.com/aws/aws-sdk-go-v2) | `1.19.20` | `1.19.21` |
| [github.com/aws/aws-sdk-go-v2/service/signin](https://github.com/aws/aws-sdk-go-v2) | `1.0.8` | `1.0.9` |
| [github.com/aws/aws-sdk-go-v2/service/sso](https://github.com/aws/aws-sdk-go-v2) | `1.30.13` | `1.30.14` |
| [github.com/aws/aws-sdk-go-v2/service/ssooidc](https://github.com/aws/aws-sdk-go-v2) | `1.35.17` | `1.35.18` |
| [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) | `1.41.9` | `1.41.10` |


Updates `github.com/aws/aws-sdk-go-v2` from 1.41.4 to 1.41.5
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.41.4...v1.41.5)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.12 to 1.32.13
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.32.12...config/v1.32.13)

Updates `github.com/aws/aws-sdk-go-v2/credentials` from 1.19.12 to 1.19.13
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/sqs/v1.19.12...credentials/v1.19.13)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager` from 0.1.11 to 0.1.12
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/transfermanager/v0.1.11...feature/s3/transfermanager/v0.1.12)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.97.2 to 1.97.3
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.97.2...service/s3/v1.97.3)

Updates `github.com/nats-io/nats.go` from 1.49.0 to 1.50.0
- [Release notes](https://github.com/nats-io/nats.go/releases)
- [Commits](https://github.com/nats-io/nats.go/compare/v1.49.0...v1.50.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/ec2/imds` from 1.18.20 to 1.18.21
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/config/v1.18.21/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.20...config/v1.18.21)

Updates `github.com/aws/aws-sdk-go-v2/internal/configsources` from 1.4.20 to 1.4.21
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/v4a/v1.4.20...internal/v4a/v1.4.21)

Updates `github.com/aws/aws-sdk-go-v2/internal/endpoints/v2` from 2.7.20 to 2.7.21
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/endpoints/v2.7.20...internal/endpoints/v2.7.21)

Updates `github.com/aws/aws-sdk-go-v2/internal/v4a` from 1.4.21 to 1.4.22
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/v4a/v1.4.21...internal/v4a/v1.4.22)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/checksum` from 1.9.12 to 1.9.13
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/grafana/v1.9.12...service/grafana/v1.9.13)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/presigned-url` from 1.13.20 to 1.13.21
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.13.20...credentials/v1.13.21)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/s3shared` from 1.19.20 to 1.19.21
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/codebuild/v1.19.20...service/codebuild/v1.19.21)

Updates `github.com/aws/aws-sdk-go-v2/service/signin` from 1.0.8 to 1.0.9
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/m2/v1.0.8...service/m2/v1.0.9)

Updates `github.com/aws/aws-sdk-go-v2/service/sso` from 1.30.13 to 1.30.14
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/acm/v1.30.13...service/acm/v1.30.14)

Updates `github.com/aws/aws-sdk-go-v2/service/ssooidc` from 1.35.17 to 1.35.18
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/pi/v1.35.17...service/pi/v1.35.18)

Updates `github.com/aws/aws-sdk-go-v2/service/sts` from 1.41.9 to 1.41.10
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/ecs/v1.41.9...service/ecs/v1.41.10)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-version: 1.41.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-version: 1.32.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-version: 1.19.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager
  dependency-version: 0.1.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.97.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/nats-io/nats.go
  dependency-version: 1.50.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/ec2/imds
  dependency-version: 1.18.21
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/configsources
  dependency-version: 1.4.21
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/endpoints/v2
  dependency-version: 2.7.21
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/v4a
  dependency-version: 1.4.22
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/checksum
  dependency-version: 1.9.13
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/presigned-url
  dependency-version: 1.13.21
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/s3shared
  dependency-version: 1.19.21
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/signin
  dependency-version: 1.0.9
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sso
  dependency-version: 1.30.14
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/ssooidc
  dependency-version: 1.35.18
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-version: 1.41.10
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-30 22:13:59 +00:00
dependabot[bot] 1ec724b7a4 chore(deps): bump sigstore/cosign-installer from 4.1.0 to 4.1.1
Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](https://github.com/sigstore/cosign-installer/compare/v4.1.0...v4.1.1)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 4.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-30 21:59:18 +00:00
Ben McClelland 2b12a5b81c Merge pull request #1989 from versity/test/rest_list_objects_encoding
test: REST ListObjects encoding
2026-03-30 14:43:12 -07:00
Ben McClelland f6cd991d89 Merge pull request #1993 from versity/ben/racing-mp-complete
fix: serialize concurrent CompleteMultipartUpload calls via rename
2026-03-30 09:05:54 -07:00
Luke McCrone 8feed3b333 test: ListObjects encoding-type tests 2026-03-30 10:24:03 -03:00
Ben McClelland 13b3dc5267 fix: serialize concurrent CompleteMultipartUpload calls via rename
When two requests raced to complete the same multipart upload, the first
caller to finish would remove the part files and upload directory. The
second caller, already past the initial existence check, would then fail
mid-flight with confusing errors such as ErrInvalidPart or an I/O error
when trying to open a part that no longer exists.

Fix this by atomically renaming the upload directory from <uploadID> to
<uploadID>.inprogress at the very start of CompleteMultipartUploadWithCopy,
before any part data is read. A concurrent caller will now find the
original directory absent and receive a clean NoSuchUpload error. A
deferred rename restores the original name if the complete does not
succeed, allowing the client to retry.

ListMultipartUploads is updated to skip any directories whose name ends
in .inprogress so in-flight completes do not appear as pending uploads.
2026-03-27 16:14:20 -07:00
Ben McClelland fab9f1f813 Merge pull request #1994 from versity/test/rest_go_and_eval_fix
test: remove eval, go command generation overhaul
2026-03-27 15:56:13 -07:00
Ben McClelland 0234f3ecc7 Merge pull request #1992 from versity/ben/cleanup-fd
fix: cleanup file descriptor leaks with chown fails
2026-03-27 11:43:31 -07:00
Ben McClelland 52fa3f1f76 Merge pull request #1991 from versity/ben/scoutfs-fail-upload
fix: abort scoutfs multipart uploads on error after successful moveblocks
2026-03-27 11:43:17 -07:00
Luke McCrone ccd0ba2762 test: remove eval, go command generation overhaul 2026-03-27 15:00:56 -03:00
Ben McClelland 495b38a899 fix: abort scoutfs multipart uploads on error after successful moveblocks
The scoutfs backend uses the move blocks ioctl when combining parts into the
final multipart upload object. Once a move blocks from any part is successful,
the original data is no longer in the part file. If the multipart upload fails
and retries, future complete multipart upload calls will not have the correct
data within the part files anymore.

To prevent this case, once a move blocks call is successful for an upload, any
future failure for the complete upload is set to auto-abort the upload to force
clients to re-upload the part data again.
2026-03-26 16:28:42 -07:00
Ben McClelland 927d1d668a fix: cleanup file descriptor leaks with chown fails
We were missing a few cases of cleaning up temp files and file
descriptors in the openTmpFile Chown() error cases.
2026-03-26 15:45:20 -07:00
Ben McClelland 6c8eba12e3 Merge pull request #1987 from versity/dependabot/go_modules/dev-dependencies-a33a00629d
chore(deps): bump the dev-dependencies group with 5 updates
2026-03-24 08:12:38 -07:00
dependabot[bot] 3a8d658a7f chore(deps): bump the dev-dependencies group with 5 updates
Bumps the dev-dependencies group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager](https://github.com/aws/aws-sdk-go-v2) | `0.1.10` | `0.1.11` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.97.1` | `1.97.2` |
| [github.com/go-ldap/ldap/v3](https://github.com/go-ldap/ldap) | `3.4.12` | `3.4.13` |
| [github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream](https://github.com/aws/aws-sdk-go-v2) | `1.7.7` | `1.7.8` |
| [github.com/klauspost/compress](https://github.com/klauspost/compress) | `1.18.4` | `1.18.5` |


Updates `github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager` from 0.1.10 to 0.1.11
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/transfermanager/v0.1.10...feature/s3/transfermanager/v0.1.11)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.97.1 to 1.97.2
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.97.1...service/s3/v1.97.2)

Updates `github.com/go-ldap/ldap/v3` from 3.4.12 to 3.4.13
- [Release notes](https://github.com/go-ldap/ldap/releases)
- [Commits](https://github.com/go-ldap/ldap/compare/v3.4.12...v3.4.13)

Updates `github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream` from 1.7.7 to 1.7.8
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/m2/v1.7.7...service/m2/v1.7.8)

Updates `github.com/klauspost/compress` from 1.18.4 to 1.18.5
- [Release notes](https://github.com/klauspost/compress/releases)
- [Commits](https://github.com/klauspost/compress/compare/v1.18.4...v1.18.5)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager
  dependency-version: 0.1.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.97.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/go-ldap/ldap/v3
  dependency-version: 3.4.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream
  dependency-version: 1.7.8
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/klauspost/compress
  dependency-version: 1.18.5
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-23 21:08:32 +00:00
Ben McClelland 271d7377dd Merge pull request #1979 from versity/test/util_cleanup
test: cleanup of some common tests, bucket and file creation
2026-03-20 15:35:58 -07:00
Luke McCrone a96def085d test: test_common.sh updates, bucket/file creation updates, hardcode fixes 2026-03-20 14:09:00 -03:00
Jakob van Santen 22f04312a7 fix: cleanup tempfiles when error prevents calling link
When a client provides an invalid or incomplete body for a single-part
upload, the handler returns before the link stage.  Factor tempfile
removal into cleanup() to catch all cases.
2026-03-20 08:25:44 -07:00
Jakob van Santen 56cb36d45a feat: add option to disable copy-file-range for multipart uploads
The standard io.Copy() will attempt to use copy_file_range when available.
However, this can cause problems with certain filesystems. Add an option
that will prevent io.Copy() from being able to use copy_file_range to force
a standard data copy between file descriptors when needed for completing
multipart uploads.
2026-03-20 08:23:01 -07:00
Ben McClelland ae411fa3c1 Merge pull request #1974 from jvansanten/no-truncate
Always write complete files in sidecar meta provider
2026-03-19 16:01:50 -07:00
Ben McClelland 88e6396950 Merge pull request #1889 from versity/ben/fast-walk
feat: implement quicker backend/posix walk algorithm
2026-03-19 16:00:54 -07:00
tonyipm 33917ad6f3 feat: implement quicker backend/posix walk algorithm
Rewrite the posix Walk implementation to avoid the extra ReadDir per
directory that was noted as a TODO in the old code.  The new algorithm
holds all traversal state in a walkState struct and uses processDir to
interleave sibling entries in correct S3 lexicographic order without a
second syscall.

Key changes:
prefix optimisation: jump directly into the deepest matching directory
rather than scanning from the root on every call

marker short-circuit: skip entire subtrees that are lexically before
the marker, making paginated listing faster

Co-authored-by: Ben McClelland <ben.mcclelland@versity.com>
2026-03-18 17:39:16 -07:00
Ben McClelland df1c312c7a Merge pull request #1968 from versity/test/rest_list_objects
Test/rest list objects
2026-03-18 10:25:25 -07:00
Ben McClelland 02f925a84b Merge pull request #1978 from 57-Wolve/57-Wolve-patch-s3-glacier-response
Update scoutfs glacier stage constants for ongoing requests
2026-03-18 10:25:06 -07:00
Ben McClelland 766437a66d Merge pull request #1975 from jvansanten/hide-non-bucket-dirs
Do not expose directories that can't be buckets
2026-03-18 10:24:50 -07:00
57_Wolve c32ddfff1a Update stage constants for ongoing requests
Fix issue with incompatible S3 response for offline and staging status.

Resolves issue with Restic Glacier support.
2026-03-18 09:46:55 -05:00
Jakob van Santen d8f927519a Do not expose directories that can't be buckets 2026-03-18 15:33:10 +01:00
Jakob van Santen 0559807783 Always write complete files in sidecar meta provider
Some filesystems like dCache don't allow file truncation
2026-03-18 15:29:29 +01:00
Luke McCrone 7975b9bbaa test: listobjects delimiter/prefix test, skips removal, go query improvement 2026-03-17 16:07:47 -03:00
Ben McClelland feffba80fa Merge pull request #1971 from versity/dependabot/go_modules/dev-dependencies-fd5fe29f01
chore(deps): bump the dev-dependencies group with 22 updates
2026-03-16 15:50:38 -07:00
Ben McClelland 21083f6129 Merge pull request #1970 from versity/dependabot/github_actions/docker/login-action-4
chore(deps): bump docker/login-action from 3 to 4
2026-03-16 15:49:58 -07:00
Ben McClelland b179040067 Merge pull request #1969 from versity/dependabot/github_actions/docker/setup-buildx-action-4
chore(deps): bump docker/setup-buildx-action from 3 to 4
2026-03-16 15:49:39 -07:00
dependabot[bot] 504fe998b7 chore(deps): bump the dev-dependencies group with 22 updates
Bumps the dev-dependencies group with 22 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.41.3` | `1.41.4` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.32.11` | `1.32.12` |
| [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) | `1.19.11` | `1.19.12` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager](https://github.com/aws/aws-sdk-go-v2) | `0.1.8` | `0.1.10` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.96.4` | `1.97.1` |
| [github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream](https://github.com/aws/aws-sdk-go-v2) | `1.7.6` | `1.7.7` |
| [github.com/aws/aws-sdk-go-v2/feature/ec2/imds](https://github.com/aws/aws-sdk-go-v2) | `1.18.19` | `1.18.20` |
| [github.com/aws/aws-sdk-go-v2/internal/configsources](https://github.com/aws/aws-sdk-go-v2) | `1.4.19` | `1.4.20` |
| [github.com/aws/aws-sdk-go-v2/internal/endpoints/v2](https://github.com/aws/aws-sdk-go-v2) | `2.7.19` | `2.7.20` |
| [github.com/aws/aws-sdk-go-v2/internal/ini](https://github.com/aws/aws-sdk-go-v2) | `1.8.5` | `1.8.6` |
| [github.com/aws/aws-sdk-go-v2/internal/v4a](https://github.com/aws/aws-sdk-go-v2) | `1.4.20` | `1.4.21` |
| [github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding](https://github.com/aws/aws-sdk-go-v2) | `1.13.6` | `1.13.7` |
| [github.com/aws/aws-sdk-go-v2/service/internal/checksum](https://github.com/aws/aws-sdk-go-v2) | `1.9.11` | `1.9.12` |
| [github.com/aws/aws-sdk-go-v2/service/internal/presigned-url](https://github.com/aws/aws-sdk-go-v2) | `1.13.19` | `1.13.20` |
| [github.com/aws/aws-sdk-go-v2/service/internal/s3shared](https://github.com/aws/aws-sdk-go-v2) | `1.19.19` | `1.19.20` |
| [github.com/aws/aws-sdk-go-v2/service/signin](https://github.com/aws/aws-sdk-go-v2) | `1.0.7` | `1.0.8` |
| [github.com/aws/aws-sdk-go-v2/service/sso](https://github.com/aws/aws-sdk-go-v2) | `1.30.12` | `1.30.13` |
| [github.com/aws/aws-sdk-go-v2/service/ssooidc](https://github.com/aws/aws-sdk-go-v2) | `1.35.16` | `1.35.17` |
| [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) | `1.41.8` | `1.41.9` |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.48.0` | `0.49.0` |
| [golang.org/x/net](https://github.com/golang/net) | `0.51.0` | `0.52.0` |
| [golang.org/x/text](https://github.com/golang/text) | `0.34.0` | `0.35.0` |


Updates `github.com/aws/aws-sdk-go-v2` from 1.41.3 to 1.41.4
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.41.3...v1.41.4)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.11 to 1.32.12
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.32.11...config/v1.32.12)

Updates `github.com/aws/aws-sdk-go-v2/credentials` from 1.19.11 to 1.19.12
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.19.11...service/sqs/v1.19.12)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager` from 0.1.8 to 0.1.10
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/transfermanager/v0.1.8...feature/s3/transfermanager/v0.1.10)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.96.4 to 1.97.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.96.4...service/s3/v1.97.1)

Updates `github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream` from 1.7.6 to 1.7.7
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/m2/v1.7.6...service/m2/v1.7.7)

Updates `github.com/aws/aws-sdk-go-v2/feature/ec2/imds` from 1.18.19 to 1.18.20
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/config/v1.18.20/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.19...config/v1.18.20)

Updates `github.com/aws/aws-sdk-go-v2/internal/configsources` from 1.4.19 to 1.4.20
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/v4a/v1.4.19...internal/v4a/v1.4.20)

Updates `github.com/aws/aws-sdk-go-v2/internal/endpoints/v2` from 2.7.19 to 2.7.20
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/endpoints/v2.7.19...internal/endpoints/v2.7.20)

Updates `github.com/aws/aws-sdk-go-v2/internal/ini` from 1.8.5 to 1.8.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/tnb/v1.8.5...service/tnb/v1.8.6)

Updates `github.com/aws/aws-sdk-go-v2/internal/v4a` from 1.4.20 to 1.4.21
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/v4a/v1.4.20...internal/v4a/v1.4.21)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding` from 1.13.6 to 1.13.7
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/service/mq/v1.13.7/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/mq/v1.13.6...service/mq/v1.13.7)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/checksum` from 1.9.11 to 1.9.12
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/grafana/v1.9.11...service/grafana/v1.9.12)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/presigned-url` from 1.13.19 to 1.13.20
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.13.19...credentials/v1.13.20)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/s3shared` from 1.19.19 to 1.19.20
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/codebuild/v1.19.19...service/codebuild/v1.19.20)

Updates `github.com/aws/aws-sdk-go-v2/service/signin` from 1.0.7 to 1.0.8
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/m2/v1.0.7...service/m2/v1.0.8)

Updates `github.com/aws/aws-sdk-go-v2/service/sso` from 1.30.12 to 1.30.13
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/acm/v1.30.12...service/acm/v1.30.13)

Updates `github.com/aws/aws-sdk-go-v2/service/ssooidc` from 1.35.16 to 1.35.17
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/pi/v1.35.16...service/pi/v1.35.17)

Updates `github.com/aws/aws-sdk-go-v2/service/sts` from 1.41.8 to 1.41.9
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/ecs/v1.41.8...service/ecs/v1.41.9)

Updates `golang.org/x/crypto` from 0.48.0 to 0.49.0
- [Commits](https://github.com/golang/crypto/compare/v0.48.0...v0.49.0)

Updates `golang.org/x/net` from 0.51.0 to 0.52.0
- [Commits](https://github.com/golang/net/compare/v0.51.0...v0.52.0)

Updates `golang.org/x/text` from 0.34.0 to 0.35.0
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.34.0...v0.35.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-version: 1.41.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-version: 1.32.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-version: 1.19.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager
  dependency-version: 0.1.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.97.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream
  dependency-version: 1.7.7
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/ec2/imds
  dependency-version: 1.18.20
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/configsources
  dependency-version: 1.4.20
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/endpoints/v2
  dependency-version: 2.7.20
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/ini
  dependency-version: 1.8.6
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/v4a
  dependency-version: 1.4.21
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding
  dependency-version: 1.13.7
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/checksum
  dependency-version: 1.9.12
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/presigned-url
  dependency-version: 1.13.20
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/s3shared
  dependency-version: 1.19.20
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/signin
  dependency-version: 1.0.8
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sso
  dependency-version: 1.30.13
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/ssooidc
  dependency-version: 1.35.17
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-version: 1.41.9
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/crypto
  dependency-version: 0.49.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/net
  dependency-version: 0.52.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/text
  dependency-version: 0.35.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-16 21:41:24 +00:00
dependabot[bot] a71e94f2bf chore(deps): bump docker/login-action from 3 to 4
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-16 21:24:08 +00:00
dependabot[bot] 6c7b03c5f4 chore(deps): bump docker/setup-buildx-action from 3 to 4
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-16 21:24:01 +00:00
Ben McClelland 60929123a8 Merge pull request #1966 from versity/sis/remove-AmazonS3.xsd
chore: remove the unnecessary AmazonS3.xsd from s3response.
2026-03-13 13:11:38 -07:00
niksis02 2ed4d7d44a chore: remove the unnecessary AmazonS3.xsd from s3response.
Signed-off-by: niksis02 <sisnikoyan@gmail.com>
2026-03-13 22:50:17 +04:00
Ben McClelland 09b6efad78 Merge pull request #1948 from versity/test/rest_list_buckets
test: additional list-buckets tests
2026-03-13 11:25:28 -07:00
Ben McClelland 887eede892 Merge pull request #1964 from shoce/hostPath
feat: add persistence.hostPath to Helm chart
2026-03-13 11:25:05 -07:00
Sho Ce 686f58c9be chart/README.md hostPath usage warning
Co-authored-by: Jack Henschel <jackdev@mailbox.org>
2026-03-13 22:51:25 +07:00
Ben McClelland 10b69ffef2 Merge pull request #1965 from versity/sis/admin-cli-update-user-project-id 2026-03-13 05:58:41 -07:00
Luke McCrone 7308c65591 test: ListBuckets - bucket region, more default param tests 2026-03-13 09:30:39 -03:00