mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-07-24 09:02:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,473 Commits 37 Branches 53 Tags
ben/plugin
Commit Graph
1473 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ben McClellandandGitHub f1c8efdaf6 Merge pull request #1156 from versity/sis/putbucketpolicy-invalid-json 
fix: Fixes the bucket policy json document validation errors for inva…
 2025-03-19 15:22:04 -07:00
niksis02 d82da64942 fix: Fixes the bucket policy json document validation errors for invalid jsons. 
Fixes #965

Changes the returned error description to `Policies must be valid JSON and the first byte must be '{'` for invalid bucket policy json documents, which doesn't start with `{`.
The gateway returns `This policy contains invalid Json` error description, if the document starts with `{`, but still isn't valid json.

Implements the `policyErr` string type which implements the `error` interface, to handle the policy json document validation errors, by avoiding staticchecker warnings.
 2025-03-20 01:08:47 +04:00
Ben McClellandandGitHub 1eb905bf14 Merge pull request #1153 from versity/ben/ldap-err 
fix: add err to ldap user/group conversion errors
 2025-03-19 13:41:47 -07:00
Ben McClellandandGitHub 9433986121 Merge pull request #1155 from versity/test/chunked_incorrect_middle 
Test/chunked incorrect final signature
 2025-03-19 12:34:33 -07:00
Luke McCrone 75c25ec789 test: chunked upload - more tests, final signature test 2025-03-19 15:36:40 -03:00
Ben McClelland 22cecea1c0 fix: add err to ldap user/group conversion errors 2025-03-19 08:48:01 -07:00
Ben McClellandandGitHub 39edb0b3b9 Merge pull request #1151 from versity/sis/posix-tag-parsing-refactoring 
fix: Reuses the ParseObjectTags method to parse the object tags in Pu…
v1.0.12 		2025-03-18 13:08:20 -07:00
niksis02 1cea4c8247 fix: Reuses the ParseObjectTags method to parse the object tags in PutObject and CreateMultipartUpload. 
Puts the `backend.ParseObjectTags` method in use to parse the object tagging for `PutObject` and `CreateMultipartUpload` to not repeat the same logic.
 2025-03-18 19:31:52 +04:00
Ben McClellandandGitHub 4dad42e4e8 Merge pull request #1150 from versity/sis/unnecessary-int-tests 
fix: Removes 2 unnecessary integration tests for HeadObject and GetOb…
 2025-03-18 08:25:16 -07:00
Ben McClellandandGitHub 0acceaeeab Merge pull request #1148 from versity/sis/object-lock-mode-validation 
fix: Fixes the returned error type for object legal hold status and o…
 2025-03-18 08:24:31 -07:00
niksis02 607c6d2308 fix: Removes 2 unnecessary integration tests for HeadObject and GetObject. 
Removes the `HeadObject_with_contenttype` and `GetObject_with_meta` integration test as the logic is covered in `HeadObject_success` and `GetObject_success` accordingly.
 2025-03-18 18:50:01 +04:00
niksis02 9e0f56f807 fix: Fixes the returned error type for object legal hold status and object lock mode in PutObject, CopyObject and CreateMultipartUpload. 
Fixes #1141
Fixes #1142

Changes the error type to `InvalidArgument` for `x-amz-object-lock-legal-hold` and `x-amz-object-lock-mode` headers invalid values.
 2025-03-18 13:58:49 +04:00
Ben McClellandandGitHub c08fd6f064 Merge pull request #1146 from versity/dependabot/go_modules/dev-dependencies-74fe486678 
chore(deps): bump the dev-dependencies group with 3 updates
 2025-03-17 18:02:01 -07:00
Ben McClellandandGitHub e635306bfb Merge pull request #1143 from versity/test/rest_signature_invalid_user 
Test/rest signature invalid user
 2025-03-17 18:01:36 -07:00
dependabot[bot]andGitHub e8e6356174 chore(deps): bump the dev-dependencies group with 3 updates 
Bumps the dev-dependencies group with 3 updates: [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2), [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/service/internal/checksum](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.78.1 to 1.78.2
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.78.1...service/s3/v1.78.2)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.65 to 1.17.66
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.17.65...feature/s3/manager/v1.17.66)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/checksum` from 1.6.2 to 1.7.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/v1.7.0/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/pi/v1.6.2...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/checksum
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-03-17 22:15:49 +00:00
Ben McClellandandGitHub 28c3b9b95e Merge pull request #1145 from versity/sis/copyobject-missing-props 
feat: Implements object meta properties for CopyObject in azure and p…
 2025-03-17 10:03:44 -07:00
niksis02andBen McClelland cfb2d6d87d feat: Implements object meta properties for CopyObject in azure and posix backends. 
Fixes #998
Closes #1125
Closes #1126
Closes #1127

Implements objects meta properties(Content-Disposition, Content-Language, Content-Encoding, Cache-Control, Expires) and tagging besed on the directives(metadata, tagging) in CopyObject in posix and azure backends. The properties/tagging should be coppied from the source object if "COPY" directive is provided and it should be replaced otherwise.

Changes the object copy principle in azure: instead of using the `CopyFromURL` method from azure sdk, it first loads the object then creates one, to be able to compare and store the meta properties.
 2025-03-17 09:37:05 -07:00
Ben McClellandandGitHub 38768a92b8 Merge pull request #1139 from versity/ben/scoutfs-updates-from-posix 
feat: sync recent posix changes to scoutfs
 2025-03-17 09:29:38 -07:00
Ben McClelland 092d3b0384 feat: sync recent posix changes to scoutfs 
This syncs recent updates to posix for scoutfs backend including
the extra metadata such as Content-Disposition,
Content-Language, Cache-Control and Expires. This also fixes the
directory object listings that have a double trailing slash due
to the change in the backend.Walk().

This also simplifies head-object to call the posix on and then
post process for glacier changes. This allows keeping in closer
sync with posix head-object over time.
 2025-03-14 15:58:31 -07:00
Luke McCrone ff305d0efc test: user REST permission, signature tests 2025-03-14 18:45:34 -03:00
Ben McClellandandGitHub 73b07eb8c5 Merge pull request #1140 from versity/test/expires_header 
Test/expires header
 2025-03-13 16:27:42 -07:00
Luke McCrone 976e44bb57 test: expires header test 2025-03-12 18:16:51 -03:00
Ben McClellandandGitHub ff80b90d32 Merge pull request #1138 from versity/sis/putobject-missing-meta 
feat: Adds the Content-Disposition, Content-Language, Cache-Control and Expires object meta properties support in the gateway.
 2025-03-12 08:44:21 -07:00
niksis02 65261a9753 feat: Adds the Content-Disposition, Content-Language, Cache-Control and Expires object meta properties support in the gateway. 
Closes #1128

Adds `Content-Disposition`, `Content-Language`, `Cache-Control` and `Expires` object meta properties support in posix and azure backends.
Changes the `PutObject` and `CreateMultipartUpload` actions backend input type to custom `s3response` types to be able to store `Expires` as any string.
 2025-03-12 16:01:56 +04:00
Ben McClellandandGitHub 39c8eddadd Merge pull request #1132 from versity/ben/scoutfs-config 
chore: update example config with scoutfs VGW_DISABLE_NOARCHIVE
 2025-03-10 17:38:14 -07:00
Ben McClelland 31e748277f chore: update example config with scoutfs VGW_DISABLE_NOARCHIVE 2025-03-10 16:14:37 -07:00
Ben McClellandandGitHub 7fd3ca27a7 Merge pull request #1121 from versity/test/rest_checksum_nomode 
Test/rest checksum nomode
v1.0.11 		2025-03-10 15:56:07 -07:00
Luke McCrone acb33f608e test: checksum mode test 2025-03-10 19:37:22 -03:00
Ben McClellandandGitHub 1421bc111a Merge pull request #1129 from versity/ben/scoutfs-noarchive 
feat: add noarchive to scoutfs part files
 2025-03-10 15:24:32 -07:00
Ben McClellandandGitHub f15f783633 Merge pull request #1131 from versity/dependabot/go_modules/dev-dependencies-c21d9491cd 
chore(deps): bump the dev-dependencies group with 14 updates
 2025-03-10 15:24:15 -07:00
dependabot[bot]andGitHub b20b4362d7 chore(deps): bump the dev-dependencies group with 14 updates 
Bumps the dev-dependencies group with 14 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.78.0` | `1.78.1` |
| [github.com/urfave/cli/v2](https://github.com/urfave/cli) | `2.27.5` | `2.27.6` |
| [golang.org/x/sync](https://github.com/golang/sync) | `0.11.0` | `0.12.0` |
| [golang.org/x/sys](https://github.com/golang/sys) | `0.30.0` | `0.31.0` |
| [github.com/aws/aws-sdk-go-v2/service/sso](https://github.com/aws/aws-sdk-go-v2) | `1.25.0` | `1.25.1` |
| [github.com/aws/aws-sdk-go-v2/service/ssooidc](https://github.com/aws/aws-sdk-go-v2) | `1.29.0` | `1.29.1` |
| [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) | `1.33.16` | `1.33.17` |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.35.0` | `0.36.0` |
| [golang.org/x/net](https://github.com/golang/net) | `0.35.0` | `0.37.0` |
| [golang.org/x/text](https://github.com/golang/text) | `0.22.0` | `0.23.0` |
| [golang.org/x/time](https://github.com/golang/time) | `0.10.0` | `0.11.0` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.29.8` | `1.29.9` |
| [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) | `1.17.61` | `1.17.62` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.17.64` | `1.17.65` |


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.78.0 to 1.78.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.78.0...service/s3/v1.78.1)

Updates `github.com/urfave/cli/v2` from 2.27.5 to 2.27.6
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.27.5...v2.27.6)

Updates `golang.org/x/sync` from 0.11.0 to 0.12.0
- [Commits](https://github.com/golang/sync/compare/v0.11.0...v0.12.0)

Updates `golang.org/x/sys` from 0.30.0 to 0.31.0
- [Commits](https://github.com/golang/sys/compare/v0.30.0...v0.31.0)

Updates `github.com/aws/aws-sdk-go-v2/service/sso` from 1.25.0 to 1.25.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.25.0...v1.25.1)

Updates `github.com/aws/aws-sdk-go-v2/service/ssooidc` from 1.29.0 to 1.29.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.29.0...config/v1.29.1)

Updates `github.com/aws/aws-sdk-go-v2/service/sts` from 1.33.16 to 1.33.17
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/sns/v1.33.16...service/sns/v1.33.17)

Updates `golang.org/x/crypto` from 0.35.0 to 0.36.0
- [Commits](https://github.com/golang/crypto/compare/v0.35.0...v0.36.0)

Updates `golang.org/x/net` from 0.35.0 to 0.37.0
- [Commits](https://github.com/golang/net/compare/v0.35.0...v0.37.0)

Updates `golang.org/x/text` from 0.22.0 to 0.23.0
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.22.0...v0.23.0)

Updates `golang.org/x/time` from 0.10.0 to 0.11.0
- [Commits](https://github.com/golang/time/compare/v0.10.0...v0.11.0)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.29.8 to 1.29.9
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.29.8...config/v1.29.9)

Updates `github.com/aws/aws-sdk-go-v2/credentials` from 1.17.61 to 1.17.62
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.61...credentials/v1.17.62)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.64 to 1.17.65
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.17.64...feature/s3/manager/v1.17.65)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sso
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/ssooidc
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/net
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/text
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/time
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-03-10 22:00:42 +00:00
Ben McClellandandGitHub 55999ec94a Merge pull request #1115 from versity/test/rest_checksum_crc32c 
Test/rest checksum crc32c
 2025-03-10 14:55:06 -07:00
Ben McClelland d034f87f60 feat: add noarchive to scoutfs part files 
The part files for multipart uploads are considered temporary
files and should not be archived by default. This adds the
noarchive attribute to the part files to prevent scoutam from
trying to archive these.

There is a new parameter, disablenoarchive, that will prevent
adding the noarchive attribute to these files for the case
where there is a desire to archive these temp files.
 2025-03-10 14:52:20 -07:00
Ben McClellandandGitHub c3c201d5b6 Merge pull request #1130 from versity/ben/scoutfs-mp-cleanup 
fix: scoutfs multipart cleanup in complete/abort mp
 2025-03-10 14:46:54 -07:00
Ben McClelland f77058b817 fix: scoutfs multipart cleanup in complete/abort mp 
This was previously not including the bucket directory for the
mutlipart temp file cleanup. This fixes leftovers in the tmp
directories after uploading multipart uploads.
 2025-03-10 13:44:24 -07:00
Luke McCrone 485f2abbaa test: crc32c checksum 2025-03-08 18:56:42 -03:00
Luke McCrone 4c5cd918d8 test: crc64nvme checksum 2025-03-08 17:56:14 -03:00
Ben McClellandandGitHub 293e7faf31 Merge pull request #1124 from versity/sis/getobject-exceeding-range 
fix: Fixes GetObject and UploadPartCopy actions data range parsing.
 2025-03-07 15:45:15 -08:00
Ben McClellandandGitHub d6d4f304e1 Merge pull request #1096 from versity/test/rest_checksum_crc64nvme 
Test/rest checksum crc64nvme
 2025-03-07 15:43:52 -08:00
niksis02 96af2b6471 fix: Fixes GetObject and UploadPartCopy actions data range parsing. 
Fixes #1004
Fixes #1122
Fixes #1120

Separates `GetObject` and `UploadPartCopy` range parsing/validation.

`GetObject` returns a successful response if acceptRange is invalid.
Adjusts the range upper limit, if it exceeds the actual objects size for `GetObject`.
Corrects the `ContentRange` in the `GetObject` response.

Fixes the `UploadPartCopy` action copy source range parsing/validation.
`UploadPartCopy` returns `InvalidArgument` if the copy source range is not valid.
 2025-03-08 01:39:21 +04:00
Ben McClellandandGitHub ca1697f1f5 Merge pull request #1119 from versity/fix/getobjectattributes-etag-quotes 
fix: removes ETag quotes for in GetObjectAttributes response for azur…
 2025-03-05 13:16:01 -08:00
niksis02 4b0dd64836 fix: removes ETag quotes for in GetObjectAttributes response for azure and posix backends 2025-03-06 00:28:55 +04:00
Ben McClellandandGitHub 12bef32d70 Merge pull request #1118 from versity/fix/checksum-headers-validation 
fix: Adds validation for x-amz-checksum- headers. Makes x-amz-sdk-che…
 2025-03-05 11:53:15 -08:00
Luke McCrone 11f646b051 test: crc64nvme checksum 2025-03-05 16:22:34 -03:00
niksis02 7d6505ec06 fix: Adds validation for x-amz-checksum- headers. Makes x-amz-sdk-checksum-algorithm header case insensitive 2025-03-05 22:06:20 +04:00
Ben McClellandandGitHub 59c392028f Merge pull request #1117 from versity/ben/readme 
feat: add Azure/S3 to supported backends in README
 2025-03-05 10:00:09 -08:00
Ben McClellandandGitHub 6361d7780b Merge pull request #1103 from versity/ben/read-only-fs 
feat: return method not allowed when uploading to read only fs
 2025-03-05 09:59:51 -08:00
Luke McCrone dea316bb26 test: initial checksum tests, REST script additions 2025-03-05 14:53:44 -03:00
Ben McClelland c6fcb8ea53 feat: add Azure/S3 to supported backends in README 2025-03-05 09:27:33 -08:00
Ben McClellandandGitHub 7b784d257f Merge pull request #1113 from versity/fix/listobjects-dir-objs 
fix: Fixes ListObjects direcotry objects listing issue. Fixes DeleteO…
 2025-03-04 11:28:06 -08:00