mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-02-06 02:10:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,473 Commits 23 Branches 45 Tags
ben/plugin
Commit Graph

1473 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ben McClelland
2d9a7cc019 Merge pull request #794 from versity/test_cmdline_xml_retrieval 
Test cmdline xml retrieval
 2024-09-11 15:25:58 -07:00
Ben McClelland
de67b1d718 Merge pull request #799 from versity/ben/dashboard 
chore: move dashboard to extra
 2024-09-11 15:25:37 -07:00
Ben McClelland
22a958bcc4 chore: move dashboard to extra 2024-09-11 13:50:27 -07:00
Ben McClelland
cecf563d92 Merge pull request #798 from versity/fix/azure-obj-content-enc 
fix: Fixed the mechanism to put/get object content-type, content-enco…
 2024-09-11 13:30:39 -07:00
jonaustin09
47d1a799f6 fix: Fixed the mechanism to put/get object content-type, content-encoding in azure 2024-09-11 15:11:37 -04:00
Luke McCrone
4ed54d9bd9 test: initial rests tests, run/assert, docker cleanup 2024-09-11 15:13:13 -03:00
Ben McClelland
7127cdeee5 Merge pull request #797 from versity/fix/getobjectattributes-mp 
fix: Removed multipart upload part from GetObjectAttributes action
 2024-09-11 10:34:46 -07:00
jonaustin09
a6fd1322f7 fix: Removed multipart upload part from GetObjectAttributes action 2024-09-11 11:57:32 -04:00
Ben McClelland
5b6f806829 Merge pull request #795 from versity/ben/update_actions 
chore: update docker-bats github actions
 2024-09-10 18:54:43 -07:00
Ben McClelland
90fb90d9a5 chore: update docker-bats github actions 2024-09-10 15:59:30 -07:00
Ben McClelland
f742a40ac2 Merge pull request #792 from versity/fix/azure-integration-tests-coverage 
Azure integration test issues
 2024-09-10 13:32:16 -07:00
jonaustin09
8a46de8e3b fix: change multipart upload implementation with .sgwtmp namespace in azure 2024-09-10 13:07:15 -07:00
Ben McClelland
448765ba04 Merge pull request #786 from versity/ben/content_type 
fix: set content type on put object
 2024-09-10 11:42:01 -07:00
Ben McClelland
dc71365bab fix: set content type/encoding on put/mutlipart object 
This fixes put object with setting a content type. If no content
type is set, then we will return a default content type for
following requests. Mutli-part upload appears to be ok.

Also fixed content eincoding and multipart uploads.

Fixes #783
 2024-09-09 16:04:54 -07:00
Ben McClelland
6ad1e25c2b Merge pull request #791 from versity/dependabot/go_modules/dev-dependencies-d5f69bb6e6 
chore(deps): bump the dev-dependencies group with 19 updates
 2024-09-09 15:36:32 -07:00
dependabot[bot]
cae5535556 chore(deps): bump the dev-dependencies group with 19 updates 
Bumps the dev-dependencies group with 19 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.30.4` | `1.30.5` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.61.0` | `1.61.2` |
| [golang.org/x/sys](https://github.com/golang/sys) | `0.24.0` | `0.25.0` |
| [github.com/aws/aws-sdk-go-v2/feature/ec2/imds](https://github.com/aws/aws-sdk-go-v2) | `1.16.12` | `1.16.13` |
| [github.com/aws/aws-sdk-go-v2/service/sso](https://github.com/aws/aws-sdk-go-v2) | `1.22.5` | `1.22.7` |
| [github.com/aws/aws-sdk-go-v2/service/ssooidc](https://github.com/aws/aws-sdk-go-v2) | `1.26.5` | `1.26.7` |
| [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) | `1.30.5` | `1.30.7` |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.26.0` | `0.27.0` |
| [golang.org/x/net](https://github.com/golang/net) | `0.28.0` | `0.29.0` |
| [golang.org/x/text](https://github.com/golang/text) | `0.17.0` | `0.18.0` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.27.31` | `1.27.33` |
| [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) | `1.17.30` | `1.17.32` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.17.16` | `1.17.18` |
| [github.com/aws/aws-sdk-go-v2/internal/configsources](https://github.com/aws/aws-sdk-go-v2) | `1.3.16` | `1.3.17` |
| [github.com/aws/aws-sdk-go-v2/internal/endpoints/v2](https://github.com/aws/aws-sdk-go-v2) | `2.6.16` | `2.6.17` |
| [github.com/aws/aws-sdk-go-v2/internal/v4a](https://github.com/aws/aws-sdk-go-v2) | `1.3.16` | `1.3.17` |
| [github.com/aws/aws-sdk-go-v2/service/internal/checksum](https://github.com/aws/aws-sdk-go-v2) | `1.3.18` | `1.3.19` |
| [github.com/aws/aws-sdk-go-v2/service/internal/presigned-url](https://github.com/aws/aws-sdk-go-v2) | `1.11.18` | `1.11.19` |
| [github.com/aws/aws-sdk-go-v2/service/internal/s3shared](https://github.com/aws/aws-sdk-go-v2) | `1.17.16` | `1.17.17` |


Updates `github.com/aws/aws-sdk-go-v2` from 1.30.4 to 1.30.5
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.30.4...v1.30.5)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.61.0 to 1.61.2
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.61.0...service/s3/v1.61.2)

Updates `golang.org/x/sys` from 0.24.0 to 0.25.0
- [Commits](https://github.com/golang/sys/compare/v0.24.0...v0.25.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/ec2/imds` from 1.16.12 to 1.16.13
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.16.12...v1.16.13)

Updates `github.com/aws/aws-sdk-go-v2/service/sso` from 1.22.5 to 1.22.7
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/pi/v1.22.5...service/mq/v1.22.7)

Updates `github.com/aws/aws-sdk-go-v2/service/ssooidc` from 1.26.5 to 1.26.7
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.26.5...service/s3/v1.26.7)

Updates `github.com/aws/aws-sdk-go-v2/service/sts` from 1.30.5 to 1.30.7
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.30.5...service/sts/v1.30.7)

Updates `golang.org/x/crypto` from 0.26.0 to 0.27.0
- [Commits](https://github.com/golang/crypto/compare/v0.26.0...v0.27.0)

Updates `golang.org/x/net` from 0.28.0 to 0.29.0
- [Commits](https://github.com/golang/net/compare/v0.28.0...v0.29.0)

Updates `golang.org/x/text` from 0.17.0 to 0.18.0
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.17.0...v0.18.0)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.27.31 to 1.27.33
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.27.31...config/v1.27.33)

Updates `github.com/aws/aws-sdk-go-v2/credentials` from 1.17.30 to 1.17.32
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.30...credentials/v1.17.32)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.16 to 1.17.18
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.16...credentials/v1.17.18)

Updates `github.com/aws/aws-sdk-go-v2/internal/configsources` from 1.3.16 to 1.3.17
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/ini/v1.3.16...internal/ini/v1.3.17)

Updates `github.com/aws/aws-sdk-go-v2/internal/endpoints/v2` from 2.6.16 to 2.6.17
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/endpoints/v2.6.16...internal/endpoints/v2.6.17)

Updates `github.com/aws/aws-sdk-go-v2/internal/v4a` from 1.3.16 to 1.3.17
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/ini/v1.3.16...internal/ini/v1.3.17)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/checksum` from 1.3.18 to 1.3.19
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/ini/v1.3.18...internal/ini/v1.3.19)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/presigned-url` from 1.11.18 to 1.11.19
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/service/dax/v1.11.19/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/dax/v1.11.18...service/dax/v1.11.19)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/s3shared` from 1.17.16 to 1.17.17
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.16...credentials/v1.17.17)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/ec2/imds
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sso
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/ssooidc
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/net
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/text
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/configsources
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/endpoints/v2
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/v4a
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/checksum
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/presigned-url
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/s3shared
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-09-09 22:07:38 +00:00
Ben McClelland
bdc8324242 Merge pull request #790 from versity/ben/response_time_format 
fix: format timestamp responses with RFC3339TimeFormat
 2024-09-09 08:16:37 -07:00
Ben McClelland
e4bc3d51e5 Merge pull request #789 from versity/ben/copy_obj_meta 
fix: copy object missing prefix on metadata delete
 2024-09-09 08:16:24 -07:00
Ben McClelland
ccd4166b2e fix: copy object missing prefix on metadata delete 
When using the REPLACE directive, we were incorrectly removing the
old metadata on the object due to missing the metadata prefix on
the key.  Fix this to remove the correct metadata before setting
new metadata.

Fixes #787
 2024-09-07 17:06:45 -07:00
Ben McClelland
3bf8b296d8 fix: format timestamp responses with RFC3339TimeFormat 
This fixes the rest of the s3reponse types to marshall the time
field in RFC3339TimeFormat when MarshalXML called.

Fixes #782
 2024-09-07 12:29:09 -07:00
Ben McClelland
66a7879b0a Merge pull request #785 from digitalglue-software/fix/delete-object-remove-parents 
fix: DeleteObject not working as expected when using `--bucketlinks` (#784)
 2024-09-06 15:12:30 -07:00
Kyle Upton
5321095de5 fix bucket dir detection 2024-09-05 16:19:07 +00:00
Ben McClelland
1adf3d9565 Merge pull request #777 from versity/test_cmdline_file_compares 
test: add better file comparisons
 2024-09-04 14:53:07 -07:00
Luke McCrone
2823676aa2 test: improve file data comparisons for testing 2024-09-04 14:24:01 -03:00
Ben McClelland
ddcc62ae0a Merge pull request #780 from versity/dependabot/go_modules/dev-dependencies-355b76d293 
chore(deps): bump the dev-dependencies group with 2 updates
 2024-09-02 15:43:39 -07:00
dependabot[bot]
151326b5d7 chore(deps): bump the dev-dependencies group with 2 updates 
Bumps the dev-dependencies group with 2 updates: [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.60.1 to 1.61.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.60.1...service/s3/v1.61.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.15 to 1.17.16
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.15...credentials/v1.17.16)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-09-02 21:33:24 +00:00
Ben McClelland
9cfc2c7b08 Merge pull request #776 from versity/copy-object-with-starting-slash 
feat: Added an integration test to cover the case to CopyObject with …
v1.0.6 		2024-08-29 12:58:57 -07:00
jonaustin09
9be4f27550 feat: Added an integration test to cover the case to CopyObject with the copysource starting with / 2024-08-29 15:32:20 -04:00
Ben McClelland
cdb5187ca2 Merge pull request #774 from cvubrugier/fix-773-copy-source-slash 
fix: handle "x-amz-copy-source" header starting with '/' in s3api
 2024-08-29 09:10:12 -07:00
Ben McClelland
c2f9e801ef Merge pull request #775 from versity/fix/list-objects-next-marker 
fix: Added an integration test case for ListObjectsV2 to specify max-…
 2024-08-29 08:56:45 -07:00
jonaustin09
201777c819 fix: Added an integration test case for ListObjectsV2 to specify max-keys as the exact number of objects in the bucket 2024-08-29 11:14:29 -04:00
Christophe Vu-Brugier
20940f0b46 fix: handle "x-amz-copy-source" header starting with '/' in s3api 
The "x-amz-copy-source" header may start with '/' as observed with
WinSCP. However, '/' is also the separator between the bucket and the
object path in "x-amz-copy-source".

Consider the following code in VerifyObjectCopyAccess():

    srcBucket, srcObject, found := strings.Cut(copySource, "/")

If `copySource` starts with '/', then `srcBucket` is set to an empty
string. Later, an error is returned because bucket "" does not exist.

This issue was fixed in the Posix and Azure backends by the following
commit:

 * 5e484f2 fix: Fixed CopySource parsing to handle the values starting with '/' in CopyObject action in posix and azure backends.

But the issue was not fixed in `VerifyObjectCopyAccess`.

This commit sanitizes "x-amz-copy-source" right after the header is
extracted in `s3api/controllers/base.go`. This ensures that the
`CopySource` argument passed to the backend functions UploadPartCopy()
and CopyObject() does not start with '/'. Since the backends no longer
need to strip away any leading '/' in `CopySource`, the parts of
commit 5e484f2 modifying the Posix and Azure backends are reverted.

Fixes issue #773.

Signed-off-by: Christophe Vu-Brugier <christophe.vu-brugier@seagate.com>
 2024-08-29 15:08:17 +02:00
Ben McClelland
26ef99c593 Merge pull request #772 from versity/ben/docs 
fix: update help description to reference repo
 2024-08-28 19:41:26 -07:00
Ben McClelland
923ee5f0db fix: update help description to reference repo 2024-08-28 19:00:46 -07:00
Izzy-B37
282213a9de Demonstration dashboard for Versity gateway metrics (#771) 
* feat: demo grafana dashboard with docker compose
 2024-08-28 18:31:38 -07:00
Ben McClelland
366993d9d1 Merge pull request #748 from versity/ben/banner 
feat: change startup banner to versitygw version
 2024-08-28 11:14:30 -07:00
Ben McClelland
3cb53d0fad Merge pull request #768 from versity/fix/s3-ls-page-size 
ListObjects(V2) common prefixes pagination
 2024-08-28 11:14:07 -07:00
Ben McClelland
810bf01871 feat: change startup banner to versitygw version 
This changes the startup banner to report the versitygw version
and build info along with interfaces configured for admin and
s3 services when quiet option not enabled.

Fixes #728
 2024-08-28 10:50:12 -07:00
Ben McClelland
1cc72e1055 Merge pull request #769 from versity/test_cmdline_copyright 
test: copyright
 2024-08-28 10:17:14 -07:00
Luke McCrone
1b4db1fd96 test: copyright 2024-08-28 12:51:39 -03:00
jonaustin09
227fdaa00b fix: Fixed the pagination for common prefixes in ListObjects & ListObjectsV2 actions 2024-08-28 11:07:44 -04:00
Ben McClelland
a2ba263d31 Merge pull request #766 from versity/ben/xml_response 
fix: add XMLName to InitiateMultipartUploadResult for consistency
 2024-08-27 17:26:13 -07:00
Ben McClelland
e1c2945fb0 fix: add XMLName to InitiateMultipartUploadResult for consistency 2024-08-27 15:32:07 -07:00
Jon Austin
d79f978df9 feat: Added the standard storage class to all the available get/list actions responses in posix. (#765) 2024-08-27 15:28:40 -07:00
Ben McClelland
3ed7c18839 Merge pull request #764 from versity/test_cmdline_policy_delete_tagging_two 
Test cmdline policy delete tagging two
 2024-08-27 13:28:08 -07:00
Ben McClelland
3afc3f9c5d Merge pull request #761 from versity/ben/time_marshal 
fix: move RFC 3339 time formatting to s3response
 2024-08-27 13:27:56 -07:00
Luke McCrone
3238aac4bd test: delete tagging test, dockerfile 2024-08-27 14:45:30 -03:00
Ben McClelland
ee202b76f3 fix: move RFC 3339 time formatting to s3response 
It is better if we let the s3response module handle the xml
formatting spec specifics, and let the backends not worry
about how to format the time fields. This should help to
prevent any future backend modifications or additions from
accidental incorrect time formatting.
 2024-08-26 21:08:24 -07:00
Ben McClelland
e065c86e62 Merge pull request #759 from versity/fix/list-objects-v2-timestamp 
ListObjects & ListObjectsV2 return types
 2024-08-26 17:39:52 -07:00
jonaustin09
684ab2371b fix: Changed ListObjects and ListObjectsV2 actions return types 
Changed ListObjectsV2 and ListObjects actions return types from
*s3.ListObjects(V2)Output to s3response.ListObjects(V2)Result.

Changed the listing objects timestamp to RFC3339 to match AWS
S3 objects timestamp.

Fixes #752
 2024-08-26 15:46:45 -07:00