mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-01-07 04:06:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,579 Commits 20 Branches 44 Tags
32c6f2e463c9730c1fd4385c5bacb368fdf2de41
Commit Graph

1579 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ben McClelland
32c6f2e463 fix: non existing bucket acl parsing 
There were a couple of cases that would return an error for the
non existing bucket acl instead of treating that as the default
acl.

This also cleans up the backends that were doing their own
acl parsing instead of using the auth.ParseACL() function.

Fixes #1304
 2025-05-20 13:46:20 -07:00
Ben McClelland
845fe73b20 Merge pull request #1306 from versity/yhal-nesi/ipa 
fix: IPA IAM use http proxy from environment
 2025-05-20 10:01:52 -07:00
Yuriy Halytskyy
925f89465e fix: IPA IAM use http proxy from environment 2025-05-20 09:33:10 -07:00
Ben McClelland
12b25b7f83 Merge pull request #1302 from versity/dependabot/go_modules/dev-dependencies-de24d94eeb 
chore(deps): bump the dev-dependencies group with 2 updates
 2025-05-20 08:24:51 -07:00
dependabot[bot]
68d267e422 chore(deps): bump the dev-dependencies group with 2 updates 
Bumps the dev-dependencies group with 2 updates: [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) and [github.com/segmentio/kafka-go](https://github.com/segmentio/kafka-go).


Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.9.0 to 1.10.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/go-mgmt-sdk-release-guideline.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.9.0...sdk/azcore/v1.10.0)

Updates `github.com/segmentio/kafka-go` from 0.4.47 to 0.4.48
- [Release notes](https://github.com/segmentio/kafka-go/releases)
- [Commits](https://github.com/segmentio/kafka-go/compare/v0.4.47...v0.4.48)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-version: 1.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/segmentio/kafka-go
  dependency-version: 0.4.48
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-05-19 21:52:45 +00:00
Ben McClelland
ae7d5f677f Merge pull request #1282 from versity/test/remove_some_setup_clients 
Test/remove some setup clients
 2025-05-16 16:48:28 -07:00
Luke McCrone
35cdef1eba test: allow acl skipping, bucket setup cleanup 2025-05-14 19:02:20 -03:00
Ben McClelland
85b6437a28 Merge pull request #1281 from versity/test/remove_setup_bucket_param 
Test/remove setup bucket param
 2025-05-14 13:58:01 -07:00
Ben McClelland
132086d9d5 Merge pull request #1296 from versity/sis/iam-update-user-role 
feat: makes the user role editable in /update-user iam endpoint
 2025-05-14 13:57:41 -07:00
niksis02
4334f869f2 feat: makes the user role editable in /update-user iam endpoint 
Closes #1295

Makes the user `role` mutable in /update-user admin endpoint.
Integrates the changes in the `admin update-user` cli command, by adding the `role` flag for a user role modification.
 2025-05-14 23:10:15 +04:00
Luke McCrone
9ef7ee8254 test: remove parameter from setup_bucket 2025-05-14 13:08:48 -03:00
Ben McClelland
6b20ec96f4 Merge pull request #1293 from versity/sis/getobject_with_range-context-cancelation 
fix: fixes the early context cancelation issue in GetObject_with_range integration test.
 2025-05-14 09:06:31 -07:00
Ben McClelland
8bd5831182 Merge pull request #1292 from versity/sis/list-parts-null-checksum 
fix: overrides empty checksum type and algorithm with 'null' for ListParts
 2025-05-14 09:06:02 -07:00
niksis02
720a7e5628 fix: fixes the early context cancelation issue in GetObject_with_range integration test. 
`context` gets cancelled early before reading the full body in the `GetObject_with_range` integration test.
This change defers the context cancelation to make sure the full body is ready and the context isn't canceled in the middle of the request body read.
 2025-05-14 08:24:19 -07:00
niksis02
3e50e29306 fix: overrides empty checksum type and algorithm with 'null' for ListParts 
Fixes #1288

If the checksum algorithm/type is not specified during multipart upload initialization, it is considered `null`, and the `ListParts` result should also set it to `null`.
 2025-05-14 08:22:45 -07:00
Ben McClelland
1e91d901e7 Merge pull request #1291 from versity/sis/last-modified-formatting 
fix: fixes all the available actions date xml marshalling for response body.
 2025-05-14 08:22:09 -07:00
niksis02
afbcbcac13 fix: fixes all the available actions date xml marshalling for response body. 
Fixes the response body parsing for all available actions to correctly parse date fields (e.g., `LastModified`) into the correct format.
 2025-05-13 23:59:59 +04:00
Ben McClelland
8e2d51e501 Merge pull request #1290 from versity/dependabot/go_modules/dev-dependencies-e1f3205b40 
chore(deps): bump github.com/valyala/fasthttp from 1.61.0 to 1.62.0 in the dev-dependencies group
 2025-05-12 16:02:45 -07:00
dependabot[bot]
1f5f040840 chore(deps): bump github.com/valyala/fasthttp 
Bumps the dev-dependencies group with 1 update: [github.com/valyala/fasthttp](https://github.com/valyala/fasthttp).


Updates `github.com/valyala/fasthttp` from 1.61.0 to 1.62.0
- [Release notes](https://github.com/valyala/fasthttp/releases)
- [Commits](https://github.com/valyala/fasthttp/compare/v1.61.0...v1.62.0)

---
updated-dependencies:
- dependency-name: github.com/valyala/fasthttp
  dependency-version: 1.62.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-05-12 21:57:13 +00:00
Ben McClelland
d3bcd8ffc5 Merge pull request #1289 from versity/sis/copy-object-date 
fix: fixes the LastModified date formatting in CopyObject result.
 2025-05-12 13:15:47 -07:00
Ben McClelland
4c5f65da96 Merge pull request #1286 from ndjones/xml-omit-empty-continuationtoken 
add omit on empty for ContinuationToken
 2025-05-12 13:14:41 -07:00
niksis02
323717bcf1 fix: fixes the LastModified date formatting in CopyObject result. 
Fixes #1276

Creates the custom `s3response.CopyObjectOutput` type to handle the `LastModified` date property formatting correctly. It uses `time.RFC3339` to format the date to match the format that s3 uses.
 2025-05-12 23:30:47 +04:00
Ben McClelland
be275bbb2c Merge pull request #1284 from versity/sis/list-objects-common-prefx-optimization 
feat: optimizes backend.Walk and backend.WalkVersions to avoid sorting the common prefixes.
 2025-05-12 10:57:07 -07:00
Nick Jones
a022c3bdb6 mount-s3 has strict XML parsing which doesn't like receiving this tag empty 2025-05-12 20:59:26 +12:00
niksis02
d3585e6c1c feat: optimizes backend.Walk and backend.WalkVersions to avoid sorting the common prefixes. 
Common prefixes were originally stored in a `map[string]struct{}`, which was then converted to a slice and sorted. The new implementation stores the common prefixes in a `map[string]int`, where the map value represents the index of the common prefix. There's no need to sort the common prefixes array, as `fs.WalkDir` comes with sorted directories and files.
 2025-05-10 01:59:39 +04:00
Ben McClelland
42b03b866c Merge pull request #1278 from versity/sis/etag-quotes 
fix: adds the surrounding quotes on ETag in PutObject for dir objects and in UploadPartCopy.
 2025-05-08 14:40:39 -07:00
niksis02
3740d79173 fix: adds the surrounding quotes on ETag in PutObject for dir objects and in UploadPartCopy. 
Fixes #1277
Fixes #1235

Adds surrounding quotes on `ETag` when creating a directory object. Adds the quotes in `UploadPartCopy` as well.
 2025-05-09 00:29:23 +04:00
Ben McClelland
f4577d4af5 Merge pull request #1274 from versity/sis/versioning-getobject-success-test-fix 
fix: fixes the context cancelation issue in Versioning_GetObject_success integration test.
 2025-05-08 10:05:15 -07:00
niksis02
809d969afb fix: fixes the context cancelation issue in Versioning_GetObject_success integration test. 
Fixes #1271

In the `Versioning_GetObject_success` integration test the contexts are canceled before reading the full request body after `GetObject`.
Changes the behaviour to defer the context cancelation, to be sure it's canceled after the full request body is read.
 2025-05-08 20:34:18 +04:00
Ben McClelland
3a9f8c6525 Merge pull request #1272 from versity/sis/debug-logging-chunk-readers 
feat: adds debug logging for chunk readers.
 2025-05-07 13:55:01 -07:00
niksis02
23b5e60854 feat: adds debug logging for chunk readers. 
Closes #1221

Adds debug logging for `signed`/`unsigned` chunk readers.
Adds the `debuglogger.Infof` log method, which prints out green info logs with `[INFO]:` prefix.
The debug logging inclues some chunk details: size, signature, trailers. It also prints out stash/release stash operations.
The error cases are logged with standart yellow `[DEBUG]:` prefix.
The `String to sign` block in signed chunk reader is logged in purple horizontal borders with title.
 2025-05-08 00:22:01 +04:00
Ben McClelland
2d5d641824 Merge pull request #1270 from versity/ben/event-log-panic 
fix: panic with malformed request in event/log handlers
 2025-05-07 11:13:44 -07:00
Ben McClelland
4478ed1143 fix: panic with malformed request in event/log handlers 
Sending the following malformed request with eevnt notifcations
or access logs enabled will cause a panic related to parsing the
bucket and object from the invalid request path:

printf "GET GET  HTTP/1.1\r\nHost: $HOST\r\n\r\n" | nc 127.0.0.1 7070

The fix is to add bounds checks on the slice returned from
splitting the request path to set the bucket/object.

Fixes #1269
 2025-05-06 17:42:05 -07:00
Ben McClelland
22703de0c8 Merge pull request #1267 from versity/ben/controller-bounds-check 
fix: add bounds check for ContentLength type conversion
 2025-05-06 08:27:38 -07:00
Ben McClelland
5122b8c6ed Merge pull request #1268 from sebastian-heinz/use-path-style 
use path style
 2025-05-06 08:19:49 -07:00
sebastian-heinz
42013d365b use path style 2025-05-06 10:28:16 +08:00
Ben McClelland
a77c24f61f Merge pull request #1266 from versity/dependabot/go_modules/dev-dependencies-de083807b3 
chore(deps): bump the dev-dependencies group with 7 updates
 2025-05-05 16:44:48 -07:00
Ben McClelland
e7294c631f fix: add bounds check for ContentLength type conversion 
On 32-bit systems, this value could overflow. Add a check for the
overflow and return ErrInvalidRange if it does overflow.

The type in GetObjectOutput for ContentLength is *int64, but the
fasthttp.RequestCtx.SetBodyStream() takes type int. So there is
no way to set the bodysize to the correct limit if the value
overflows.
 2025-05-05 16:36:29 -07:00
dependabot[bot]
c3334008f5 chore(deps): bump the dev-dependencies group with 7 updates 
Bumps the dev-dependencies group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/nats-io/nats.go](https://github.com/nats-io/nats.go) | `1.41.2` | `1.42.0` |
| [golang.org/x/sync](https://github.com/golang/sync) | `0.13.0` | `0.14.0` |
| [golang.org/x/sys](https://github.com/golang/sys) | `0.32.0` | `0.33.0` |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.37.0` | `0.38.0` |
| [golang.org/x/net](https://github.com/golang/net) | `0.39.0` | `0.40.0` |
| [golang.org/x/text](https://github.com/golang/text) | `0.24.0` | `0.25.0` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.17.74` | `1.17.75` |


Updates `github.com/nats-io/nats.go` from 1.41.2 to 1.42.0
- [Release notes](https://github.com/nats-io/nats.go/releases)
- [Commits](https://github.com/nats-io/nats.go/compare/v1.41.2...v1.42.0)

Updates `golang.org/x/sync` from 0.13.0 to 0.14.0
- [Commits](https://github.com/golang/sync/compare/v0.13.0...v0.14.0)

Updates `golang.org/x/sys` from 0.32.0 to 0.33.0
- [Commits](https://github.com/golang/sys/compare/v0.32.0...v0.33.0)

Updates `golang.org/x/crypto` from 0.37.0 to 0.38.0
- [Commits](https://github.com/golang/crypto/compare/v0.37.0...v0.38.0)

Updates `golang.org/x/net` from 0.39.0 to 0.40.0
- [Commits](https://github.com/golang/net/compare/v0.39.0...v0.40.0)

Updates `golang.org/x/text` from 0.24.0 to 0.25.0
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.24.0...v0.25.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.74 to 1.17.75
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.17.74...feature/s3/manager/v1.17.75)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats.go
  dependency-version: 1.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/sync
  dependency-version: 0.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/sys
  dependency-version: 0.33.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/crypto
  dependency-version: 0.38.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/net
  dependency-version: 0.40.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/text
  dependency-version: 0.25.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-version: 1.17.75
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-05-05 22:51:13 +00:00
Ben McClelland
b12b0d242e Merge pull request #1264 from versity/test/copy_object_param 2025-05-05 15:09:24 -07:00
Luke McCrone
384bb463d3 test: copy object tests (copy source, payload) 2025-05-05 17:25:56 -03:00
Ben McClelland
4b34ef1a5f Merge pull request #1263 from versity/sis/headobject-range 
fix: fixes the range parsing for GetObject. Adds range query support for HeadObject.
 2025-05-05 12:23:15 -07:00
Ben McClelland
e0999ce5a3 Merge pull request #1261 from versity/ben/scoutfs-fixes 
Ben/scoutfs fixes
 2025-05-05 11:42:36 -07:00
niksis02
dfa1ed2358 fix: fixes the range parsing for GetObject. Adds range query support for HeadObject. 
Fixes #1258
Fixes #1257
Closes #1244

Adds range queries support for `HeadObject`.
Fixes the range parsing logic for `GetObject`, which is used for `HeadObject` as well. Both actions follow the same rules for range parsing.

Fixes the error message returned by `GetObject`.
 2025-05-05 22:41:12 +04:00
Ben McClelland
98867bc731 Merge pull request #1262 from versity/ben/iam-internal 
fix: use createtemp()/rename() for iam internal files
 2025-05-05 08:12:27 -07:00
Ben McClelland
e98f7763d0 fix: use createtemp()/rename() for iam internal files 
This cleans up a previous fix to #630 to use a better temp/rename
scheme thats less likely to have bad side effects.

The test for the previous issue still passes these cases, and we
will be less liekly to find a case where the file doesnt exist
or corrpted backup files.
 2025-05-03 12:39:06 -07:00
Ben McClelland
e9286f7a23 feat: add scoutfs group tests to integration 2025-05-03 12:04:47 -07:00
Ben McClelland
a60d6a7faa fix: scoutfs racing mutlipart uploads internal error 
When multiple uploads with the same object key are racing, we can
end up with an EEXIST when trying to link the final object into
the namespace. When this happens, we should just remove the
existing file and try again since the semantics are that the
last upload should win.
 2025-05-03 09:30:45 -07:00
Ben McClelland
a29f7b1839 fix: scoutfs missing ListObjectsV2() start after 
This brings ListObjectsV2 for scoutfs in sync with posix to handle
the start after and continuation token ases.
 2025-05-03 09:15:01 -07:00
Ben McClelland
6321406008 fix: scoutfs missing ListObjects() response fields 
This fixes some tests that were fialing due to missing response
fields in ListObjects().
 2025-05-03 09:07:56 -07:00