mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-01-08 04:35:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,597 Commits 20 Branches 44 Tags
282e875d9fdd641c18e552fb3a07894b0a9fb930
Commit Graph

1597 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Luke McCrone
282e875d9f test: logging, convert more setup commands to REST 2025-06-03 20:07:05 -03:00
Ben McClelland
f912778617 Merge pull request #1324 from versity/dependabot/go_modules/dev-dependencies-2b0e27fc16 
chore(deps): bump the dev-dependencies group with 2 updates
 2025-06-02 22:52:12 -05:00
dependabot[bot]
23169fa51d chore(deps): bump the dev-dependencies group with 2 updates 
Bumps the dev-dependencies group with 2 updates: [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.79.4 to 1.80.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.79.4...service/s3/v1.80.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.76 to 1.17.77
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.17.76...feature/s3/manager/v1.17.77)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.80.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-version: 1.17.77
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-06-02 21:55:38 +00:00
Ben McClelland
cd45a24100 Merge pull request #1294 from versity/test/more_setup_command_conversions 
Test/more setup command conversions
 2025-05-29 13:50:15 -05:00
Luke McCrone
c632e647f3 test: convert more setup commands to REST, speed up github-actions 2025-05-27 19:28:04 -03:00
Ben McClelland
9a2acceaa8 Merge pull request #1316 from versity/dependabot/go_modules/dev-dependencies-a0697c01eb v1.0.14 2025-05-27 06:01:40 -07:00
Ben McClelland
276ea75de5 Merge pull request #1315 from versity/ben/vhost-docs 2025-05-27 06:00:56 -07:00
dependabot[bot]
bbb62927a5 chore(deps): bump the dev-dependencies group with 5 updates 
Bumps the dev-dependencies group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.79.3` | `1.79.4` |
| [github.com/gofiber/fiber/v2](https://github.com/gofiber/fiber) | `2.52.7` | `2.52.8` |
| [github.com/oklog/ulid/v2](https://github.com/oklog/ulid) | `2.1.0` | `2.1.1` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.17.75` | `1.17.76` |
| [github.com/aws/aws-sdk-go-v2/service/internal/checksum](https://github.com/aws/aws-sdk-go-v2) | `1.7.1` | `1.7.2` |


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.79.3 to 1.79.4
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.79.3...service/s3/v1.79.4)

Updates `github.com/gofiber/fiber/v2` from 2.52.7 to 2.52.8
- [Release notes](https://github.com/gofiber/fiber/releases)
- [Commits](https://github.com/gofiber/fiber/compare/v2.52.7...v2.52.8)

Updates `github.com/oklog/ulid/v2` from 2.1.0 to 2.1.1
- [Release notes](https://github.com/oklog/ulid/releases)
- [Changelog](https://github.com/oklog/ulid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/oklog/ulid/compare/v2.1.0...v2.1.1)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.75 to 1.17.76
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.17.75...feature/s3/manager/v1.17.76)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/checksum` from 1.7.1 to 1.7.2
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.7.1...service/mq/v1.7.2)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.79.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/gofiber/fiber/v2
  dependency-version: 2.52.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/oklog/ulid/v2
  dependency-version: 2.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-version: 1.17.76
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/checksum
  dependency-version: 1.7.2
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-05-26 21:22:17 +00:00
Ben McClelland
60aaaa0908 Merge pull request #1287 from versity/test/begin_setup_command_conversions 
Test/begin setup command conversions
 2025-05-23 15:34:19 -07:00
Ben McClelland
e2905b6880 feat: update example service config for virtual host option 2025-05-23 15:26:03 -07:00
Luke McCrone
87ece0cc26 test: convert eight or so setup operations to REST 2025-05-23 19:10:40 -03:00
Ben McClelland
4405fb1d26 Merge pull request #1313 from versity/sis/host-style-tests-automation 
feat: automates the host-style tests in the pipeline
 2025-05-22 20:59:31 -07:00
niksis02
b9b75b58f6 feat: automates the host-style tests in the pipeline 
Adds a GitHub Actions workflow to run the `host-style` tests inside Docker containers. The tests are executed in a Docker environment using `Docker Compose` with three containers: one for running the tests, one for setting up the server, and one using the `dnsmasq` image for `DNS` server configuration.
 2025-05-23 02:22:45 +04:00
Ben McClelland
4f8b1ffb1c Merge pull request #1312 from versity/dependabot/go_modules/github.com/gofiber/fiber/v2-2.52.7 
chore(deps): bump github.com/gofiber/fiber/v2 from 2.52.6 to 2.52.7
 2025-05-22 13:44:48 -07:00
dependabot[bot]
46bde72474 chore(deps): bump github.com/gofiber/fiber/v2 from 2.52.6 to 2.52.7 
Bumps [github.com/gofiber/fiber/v2](https://github.com/gofiber/fiber) from 2.52.6 to 2.52.7.
- [Release notes](https://github.com/gofiber/fiber/releases)
- [Commits](https://github.com/gofiber/fiber/compare/v2.52.6...v2.52.7)

---
updated-dependencies:
- dependency-name: github.com/gofiber/fiber/v2
  dependency-version: 2.52.7
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-05-22 20:09:52 +00:00
Ben McClelland
14d2b8a0ed Merge pull request #1307 from versity/sis/virtual-hosted-style 
feat: implements host-style bucket addressing in the gateway.
 2025-05-21 15:35:30 -07:00
niksis02
dbc710da2d feat: implements host-style bucket addressing in the gateway. 
Closes #803

Implements host-style bucket addressing in the gateway. This feature can be enabled by running the gateway with the `--virtual-domain` flag and specifying a virtual domain name.
Example:

```bash
    ./versitygw -a user -s secret --virtual-domain localhost:7070 posix /tmp/vgw
```

The implementation follows this approach: it introduces a middleware (`HostStyleParser`) that parses the bucket name from the `Host` header and appends it to the URL path. This effectively transforms the request into a path-style bucket addressing format, which the gateway already supports. With this design, the gateway can handle both path-style and host-style requests when running in host-style mode.

For local testing, one can either set up a local DNS server to wildcard-match all subdomains of a specified domain and resolve them to the local IP address, or manually add entries to `/etc/hosts` to resolve bucket-prefixed hosts to the server IP (e.g., `127.0.0.1`).
 2025-05-22 00:36:45 +04:00
Ben McClelland
ed125c317e Merge pull request #1308 from versity/ben/bucket-empty-acl 
fix: non existing bucket acl parsing
 2025-05-20 14:49:02 -07:00
Ben McClelland
32c6f2e463 fix: non existing bucket acl parsing 
There were a couple of cases that would return an error for the
non existing bucket acl instead of treating that as the default
acl.

This also cleans up the backends that were doing their own
acl parsing instead of using the auth.ParseACL() function.

Fixes #1304
 2025-05-20 13:46:20 -07:00
Ben McClelland
845fe73b20 Merge pull request #1306 from versity/yhal-nesi/ipa 
fix: IPA IAM use http proxy from environment
 2025-05-20 10:01:52 -07:00
Yuriy Halytskyy
925f89465e fix: IPA IAM use http proxy from environment 2025-05-20 09:33:10 -07:00
Ben McClelland
12b25b7f83 Merge pull request #1302 from versity/dependabot/go_modules/dev-dependencies-de24d94eeb 
chore(deps): bump the dev-dependencies group with 2 updates
 2025-05-20 08:24:51 -07:00
dependabot[bot]
68d267e422 chore(deps): bump the dev-dependencies group with 2 updates 
Bumps the dev-dependencies group with 2 updates: [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) and [github.com/segmentio/kafka-go](https://github.com/segmentio/kafka-go).


Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.9.0 to 1.10.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/go-mgmt-sdk-release-guideline.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.9.0...sdk/azcore/v1.10.0)

Updates `github.com/segmentio/kafka-go` from 0.4.47 to 0.4.48
- [Release notes](https://github.com/segmentio/kafka-go/releases)
- [Commits](https://github.com/segmentio/kafka-go/compare/v0.4.47...v0.4.48)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-version: 1.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/segmentio/kafka-go
  dependency-version: 0.4.48
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-05-19 21:52:45 +00:00
Ben McClelland
ae7d5f677f Merge pull request #1282 from versity/test/remove_some_setup_clients 
Test/remove some setup clients
 2025-05-16 16:48:28 -07:00
Luke McCrone
35cdef1eba test: allow acl skipping, bucket setup cleanup 2025-05-14 19:02:20 -03:00
Ben McClelland
85b6437a28 Merge pull request #1281 from versity/test/remove_setup_bucket_param 
Test/remove setup bucket param
 2025-05-14 13:58:01 -07:00
Ben McClelland
132086d9d5 Merge pull request #1296 from versity/sis/iam-update-user-role 
feat: makes the user role editable in /update-user iam endpoint
 2025-05-14 13:57:41 -07:00
niksis02
4334f869f2 feat: makes the user role editable in /update-user iam endpoint 
Closes #1295

Makes the user `role` mutable in /update-user admin endpoint.
Integrates the changes in the `admin update-user` cli command, by adding the `role` flag for a user role modification.
 2025-05-14 23:10:15 +04:00
Luke McCrone
9ef7ee8254 test: remove parameter from setup_bucket 2025-05-14 13:08:48 -03:00
Ben McClelland
6b20ec96f4 Merge pull request #1293 from versity/sis/getobject_with_range-context-cancelation 
fix: fixes the early context cancelation issue in GetObject_with_range integration test.
 2025-05-14 09:06:31 -07:00
Ben McClelland
8bd5831182 Merge pull request #1292 from versity/sis/list-parts-null-checksum 
fix: overrides empty checksum type and algorithm with 'null' for ListParts
 2025-05-14 09:06:02 -07:00
niksis02
720a7e5628 fix: fixes the early context cancelation issue in GetObject_with_range integration test. 
`context` gets cancelled early before reading the full body in the `GetObject_with_range` integration test.
This change defers the context cancelation to make sure the full body is ready and the context isn't canceled in the middle of the request body read.
 2025-05-14 08:24:19 -07:00
niksis02
3e50e29306 fix: overrides empty checksum type and algorithm with 'null' for ListParts 
Fixes #1288

If the checksum algorithm/type is not specified during multipart upload initialization, it is considered `null`, and the `ListParts` result should also set it to `null`.
 2025-05-14 08:22:45 -07:00
Ben McClelland
1e91d901e7 Merge pull request #1291 from versity/sis/last-modified-formatting 
fix: fixes all the available actions date xml marshalling for response body.
 2025-05-14 08:22:09 -07:00
niksis02
afbcbcac13 fix: fixes all the available actions date xml marshalling for response body. 
Fixes the response body parsing for all available actions to correctly parse date fields (e.g., `LastModified`) into the correct format.
 2025-05-13 23:59:59 +04:00
Ben McClelland
8e2d51e501 Merge pull request #1290 from versity/dependabot/go_modules/dev-dependencies-e1f3205b40 
chore(deps): bump github.com/valyala/fasthttp from 1.61.0 to 1.62.0 in the dev-dependencies group
 2025-05-12 16:02:45 -07:00
dependabot[bot]
1f5f040840 chore(deps): bump github.com/valyala/fasthttp 
Bumps the dev-dependencies group with 1 update: [github.com/valyala/fasthttp](https://github.com/valyala/fasthttp).


Updates `github.com/valyala/fasthttp` from 1.61.0 to 1.62.0
- [Release notes](https://github.com/valyala/fasthttp/releases)
- [Commits](https://github.com/valyala/fasthttp/compare/v1.61.0...v1.62.0)

---
updated-dependencies:
- dependency-name: github.com/valyala/fasthttp
  dependency-version: 1.62.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-05-12 21:57:13 +00:00
Ben McClelland
d3bcd8ffc5 Merge pull request #1289 from versity/sis/copy-object-date 
fix: fixes the LastModified date formatting in CopyObject result.
 2025-05-12 13:15:47 -07:00
Ben McClelland
4c5f65da96 Merge pull request #1286 from ndjones/xml-omit-empty-continuationtoken 
add omit on empty for ContinuationToken
 2025-05-12 13:14:41 -07:00
niksis02
323717bcf1 fix: fixes the LastModified date formatting in CopyObject result. 
Fixes #1276

Creates the custom `s3response.CopyObjectOutput` type to handle the `LastModified` date property formatting correctly. It uses `time.RFC3339` to format the date to match the format that s3 uses.
 2025-05-12 23:30:47 +04:00
Ben McClelland
be275bbb2c Merge pull request #1284 from versity/sis/list-objects-common-prefx-optimization 
feat: optimizes backend.Walk and backend.WalkVersions to avoid sorting the common prefixes.
 2025-05-12 10:57:07 -07:00
Nick Jones
a022c3bdb6 mount-s3 has strict XML parsing which doesn't like receiving this tag empty 2025-05-12 20:59:26 +12:00
niksis02
d3585e6c1c feat: optimizes backend.Walk and backend.WalkVersions to avoid sorting the common prefixes. 
Common prefixes were originally stored in a `map[string]struct{}`, which was then converted to a slice and sorted. The new implementation stores the common prefixes in a `map[string]int`, where the map value represents the index of the common prefix. There's no need to sort the common prefixes array, as `fs.WalkDir` comes with sorted directories and files.
 2025-05-10 01:59:39 +04:00
Ben McClelland
42b03b866c Merge pull request #1278 from versity/sis/etag-quotes 
fix: adds the surrounding quotes on ETag in PutObject for dir objects and in UploadPartCopy.
 2025-05-08 14:40:39 -07:00
niksis02
3740d79173 fix: adds the surrounding quotes on ETag in PutObject for dir objects and in UploadPartCopy. 
Fixes #1277
Fixes #1235

Adds surrounding quotes on `ETag` when creating a directory object. Adds the quotes in `UploadPartCopy` as well.
 2025-05-09 00:29:23 +04:00
Ben McClelland
f4577d4af5 Merge pull request #1274 from versity/sis/versioning-getobject-success-test-fix 
fix: fixes the context cancelation issue in Versioning_GetObject_success integration test.
 2025-05-08 10:05:15 -07:00
niksis02
809d969afb fix: fixes the context cancelation issue in Versioning_GetObject_success integration test. 
Fixes #1271

In the `Versioning_GetObject_success` integration test the contexts are canceled before reading the full request body after `GetObject`.
Changes the behaviour to defer the context cancelation, to be sure it's canceled after the full request body is read.
 2025-05-08 20:34:18 +04:00
Ben McClelland
3a9f8c6525 Merge pull request #1272 from versity/sis/debug-logging-chunk-readers 
feat: adds debug logging for chunk readers.
 2025-05-07 13:55:01 -07:00
niksis02
23b5e60854 feat: adds debug logging for chunk readers. 
Closes #1221

Adds debug logging for `signed`/`unsigned` chunk readers.
Adds the `debuglogger.Infof` log method, which prints out green info logs with `[INFO]:` prefix.
The debug logging inclues some chunk details: size, signature, trailers. It also prints out stash/release stash operations.
The error cases are logged with standart yellow `[DEBUG]:` prefix.
The `String to sign` block in signed chunk reader is logged in purple horizontal borders with title.
 2025-05-08 00:22:01 +04:00
Ben McClelland
2d5d641824 Merge pull request #1270 from versity/ben/event-log-panic 
fix: panic with malformed request in event/log handlers
 2025-05-07 11:13:44 -07:00