mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-01-10 13:27:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
552 Commits 21 Branches 44 Tags
v0.12
Commit Graph

552 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ben McClelland
99d0d9a007 Merge pull request #384 from versity/luke/posix_test 
Luke/posix test
v0.12 		2024-01-29 15:20:54 -08:00
Luke McCrone
1409d664b4 test: initial aws cli bats tests 2024-01-29 20:07:00 -03:00
Ben McClelland
b908a4b981 Merge pull request #386 from versity/dependabot/go_modules/dev-dependencies-55f64e24bf 
chore(deps): bump the dev-dependencies group with 3 updates
 2024-01-29 14:53:14 -08:00
dependabot[bot]
ac06b5c4ae chore(deps): bump the dev-dependencies group with 3 updates 
Bumps the dev-dependencies group with 3 updates: [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2), [github.com/google/uuid](https://github.com/google/uuid) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.48.0 to 1.48.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.48.0...service/s3/v1.48.1)

Updates `github.com/google/uuid` from 1.5.0 to 1.6.0
- [Release notes](https://github.com/google/uuid/releases)
- [Changelog](https://github.com/google/uuid/blob/master/CHANGELOG.md)
- [Commits](https://github.com/google/uuid/compare/v1.5.0...v1.6.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.14 to 1.15.15
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.14...config/v1.15.15)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/google/uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-29 22:03:46 +00:00
Ben McClelland
3146556293 Merge pull request #380 from versity/ben/chunked_reader 
feat: add chunked upload support
 2024-01-25 13:41:43 -08:00
Ben McClelland
1c03fce3f5 Merge pull request #383 from versity/dependabot/go_modules/dev-dependencies-83121c2333 
chore(deps): bump the dev-dependencies group with 3 updates
 2024-01-22 15:39:54 -08:00
dependabot[bot]
b83e2393a5 chore(deps): bump the dev-dependencies group with 3 updates 
Bumps the dev-dependencies group with 3 updates: [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go), [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.4.0 to 1.5.1
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.4.0...sdk/internal/v1.5.1)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.26.3 to 1.26.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.26.3...config/v1.26.6)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.11 to 1.15.14
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.11...config/v1.15.14)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-22 21:45:22 +00:00
Ben McClelland
1366408baa feat: add chunked upload support 
As described in
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
this adds support for reading from a chunked upload encoded request
body. The chunked reader modifies the data stream to remove the
chunk encoding while validating the chunk signatures in line. This
allows the upper layers to get just the object data stream.
 2024-01-22 11:35:01 -08:00
Jon Austin
cf92b6fd80 Fix/azure copy object (#382) 
* fix: Added destination bucket acl check and metadata comparision for CopyObject action in azure backend

---------

Co-authored-by: Ben McClelland <ben.mcclelland@versity.com>
 2024-01-22 10:01:16 -08:00
Jon Austin
d956ecacd7 Fix/azure iam (#381) 
* fix: Fixed iam internal iam file removal bug
 2024-01-22 10:00:41 -08:00
Jon Austin
68e800492e Fix/azure list objects (#379) 
* fix: Added pagination to ListObjects and ListObjectsV2 actions, fixed multipart upload non existing key error handling
 2024-01-22 09:54:45 -08:00
Ben McClelland
f836d96717 Merge pull request #378 from versity/ben/signature 2024-01-17 13:18:54 -08:00
Ben McClelland
b5894dd714 fix: allow spaces in Authorization string 
This change removes all spaces after the algorithm to have
standard parsing for the following key/value pairs. This fixes
some clients that were using a slightly different format than
the example AWS request strings.
 2024-01-17 10:45:57 -08:00
Ben McClelland
17bdc58da9 Merge pull request #374 from versity/ben/test_fixup 
Ben/test fixup
 2024-01-17 10:45:43 -08:00
jonaustin09
03e4a28d57 fix: Fixed couple of bugs regarding to GetObject range errors, blob metadata reference losing 2024-01-17 08:27:37 -08:00
jonaustin09
240db54feb feat: Added ChangeBucketOwner, ListBucketsAndOwners action implementation in azure backend. Fixed acl key bug in getting container metadata. Added container owner in ListBuckets action 2024-01-17 08:27:37 -08:00
Ben McClelland
d404f96320 fix: translate azure errors to s3 for compatibility 2024-01-17 08:27:37 -08:00
Ben McClelland
1cdf0706e7 fix: fix crashes in test cases when fields missing 2024-01-17 08:27:37 -08:00
Ben McClelland
ca6d9e3c11 fix: docker env set to tests defaults 2024-01-17 08:27:37 -08:00
Ben McClelland
e16c54c1a3 Merge pull request #375 from versity/dependabot/go_modules/dev-dependencies-88fd56ff93 
chore(deps): bump the dev-dependencies group with 1 update
 2024-01-16 08:12:42 -08:00
dependabot[bot]
15daec9f51 chore(deps): bump the dev-dependencies group with 1 update 
Bumps the dev-dependencies group with 1 update: [github.com/nats-io/nats.go](https://github.com/nats-io/nats.go).


Updates `github.com/nats-io/nats.go` from 1.31.0 to 1.32.0
- [Release notes](https://github.com/nats-io/nats.go/releases)
- [Commits](https://github.com/nats-io/nats.go/compare/v1.31.0...v1.32.0)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats.go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-15 21:10:11 +00:00
Ben McClelland
c406d7069f Merge pull request #371 from versity/ben/default_acl 
fix: cleanup backend ACLs
 2024-01-11 12:30:44 -08:00
Ben McClelland
6481e2aac5 fix: cleanup backend ACLs 
This adds the default ACL to the CreateBucket backend method so
that the backend doesn't need to know how to construct and ACL.

This also moves the s3proxy ACLs to a tag key/value because the
gateway ACLs are not the same accounts as the backend s3 server.
TODO: we may need to mask this tag key/value if we add support
for the Get/PutBucketTagging API.
 2024-01-10 09:36:00 -08:00
Ben McClelland
45cf5e6373 Merge pull request #366 from versity/ben/az_ident 
feat: add azure local env auth
 2024-01-09 22:24:43 -08:00
Ben McClelland
3db43b7206 feat: add azure local env auth 
This is the recommended auth from the following:
https://github.com/Azure-Samples/storage-blobs-go-quickstart/blob/master/storage-quickstart.go
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-go?toc=%2Fazure%2Fdeveloper%2Fgo%2Ftoc.json&bc=%2Fazure%2Fdeveloper%2Fgo%2Fbreadcrumb%2Ftoc.json&tabs=roles-azure-portal#authenticate-to-azure-and-authorize-access-to-blob-data
 2024-01-09 22:21:39 -08:00
Ben McClelland
6786a6385a Merge pull request #367 from versity/azure-sas-token 
Azure sas token authentication
 2024-01-09 22:06:57 -08:00
jonaustin09
e5fc12042b feat: Added sas token authentication for azure backend 2024-01-09 22:03:13 -08:00
Ben McClelland
06ccd7496e Merge pull request #369 from versity/ben/az_cleanup 
chore: remove azure bug comment
 2024-01-09 08:29:42 -08:00
Ben McClelland
c86362b269 Merge pull request #370 from versity/dependabot/go_modules/dev-dependencies-925c4d3e9f 
chore(deps): bump the dev-dependencies group with 6 updates
 2024-01-09 08:28:59 -08:00
Ben McClelland
a86a8cbce5 fix: add azure CreateMultipartUpload to allow clients to work as expected 
The azure sdk doesnt use a separate function to initialize a
multipart upload, so CreateMultipartUpload becomes a no-op.
But we still need to have it return success so that clients
wont get an unexpected error.
 2024-01-08 13:40:20 -08:00
dependabot[bot]
328ea4f4b7 chore(deps): bump the dev-dependencies group with 6 updates 
Bumps the dev-dependencies group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.24.0` | `1.24.1` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.47.7` | `1.48.0` |
| [github.com/gofiber/fiber/v2](https://github.com/gofiber/fiber) | `2.51.0` | `2.52.0` |
| [golang.org/x/sys](https://github.com/golang/sys) | `0.15.0` | `0.16.0` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.26.2` | `1.26.3` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.15.9` | `1.15.11` |


Updates `github.com/aws/aws-sdk-go-v2` from 1.24.0 to 1.24.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.24.0...v1.24.1)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.47.7 to 1.48.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.47.7...service/s3/v1.48.0)

Updates `github.com/gofiber/fiber/v2` from 2.51.0 to 2.52.0
- [Release notes](https://github.com/gofiber/fiber/releases)
- [Commits](https://github.com/gofiber/fiber/compare/v2.51.0...v2.52.0)

Updates `golang.org/x/sys` from 0.15.0 to 0.16.0
- [Commits](https://github.com/golang/sys/compare/v0.15.0...v0.16.0)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.26.2 to 1.26.3
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.26.2...config/v1.26.3)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.9 to 1.15.11
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/config/v1.15.11/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.9...config/v1.15.11)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/gofiber/fiber/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-08 21:30:29 +00:00
Ben McClelland
bf38a03af9 chore: remove azure bug comment 
This comment references a bug that was fixed in the v1.2.1 sdk
update:
https://github.com/Azure/azure-sdk-for-go/issues/22171
 2024-01-08 13:11:41 -08:00
Ben McClelland
f237d06a01 Merge pull request #368 from versity/azure-docker 
Azure docker
 2024-01-08 10:10:18 -08:00
jonaustin09
8fc16392d1 feat: Dockerized azure backend to run 2 images: one for azurite, one for azure backend 2024-01-08 10:07:50 -08:00
Jon Austin
9bfec719f3 Azure ACL (#364) 
feat: Addded GetBucketAcl and PutBucketAcl actions implementation in azure backend. ACL is stored in the container metadata
 2024-01-03 11:15:53 -08:00
Ben McClelland
4a1d479bcb Merge pull request #365 from versity/ben/readme_update 
chore: update docs for s3 backend support
 2024-01-03 11:13:26 -08:00
Ben McClelland
9226999ae9 chore: update docs for s3 backend support 2024-01-03 11:00:09 -08:00
Ben McClelland
3f18bb5977 Merge pull request #362 from versity/dependabot/go_modules/dev-dependencies-21be33ef01 
chore(deps): bump the dev-dependencies group with 1 update
 2024-01-01 14:02:48 -08:00
dependabot[bot]
b145777340 chore(deps): bump the dev-dependencies group with 1 update 
Bumps the dev-dependencies group with 1 update: [github.com/urfave/cli/v2](https://github.com/urfave/cli).


Updates `github.com/urfave/cli/v2` from 2.26.0 to 2.27.1
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.26.0...v2.27.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-01 21:11:27 +00:00
Ben McClelland
bae716b012 Merge pull request #352 from versity/ben/azure_blob 
Ben/azure blob
 2023-12-29 21:59:23 -08:00
Ben McClelland
4343252c1f Merge pull request #361 from versity/ben/readme 
chore: update readme status and news
 2023-12-29 21:58:07 -08:00
Ben McClelland
5a3ecc2db4 fix: azure run go mod tidy 2023-12-29 21:56:47 -08:00
jonaustin09
cafa45760c feat: Added pagination for ListParts azure action and added get range support for GetObject azure action 2023-12-29 21:55:32 -08:00
jonaustin09
8cc89fa713 feat: Azure backend implementation 2023-12-29 21:55:32 -08:00
Ben McClelland
3b945f72fc feat: azure blob backend initial pass 2023-12-29 21:54:56 -08:00
Ben McClelland
111d75b5d4 chore: update readme status and news 2023-12-29 21:46:47 -08:00
Ben McClelland
8b31d6d93c Merge pull request #356 from versity/ben/s3_minor_fix 
fix: make s3 PutBucketAcl have receiver pointer like others for consi…
v0.11 		2023-12-26 10:34:23 -08:00
Ben McClelland
a6927a0947 Merge pull request #360 from versity/dependabot/go_modules/dev-dependencies-a55e97eb27 
chore(deps): bump the dev-dependencies group with 3 updates
 2023-12-26 10:33:18 -08:00
Ben McClelland
c1587e4c1c fix: make s3 PutBucketAcl have receiver pointer like others for consistency 2023-12-26 10:31:56 -08:00
Ben McClelland
6146dcff4a Merge pull request #358 from versity/ben/gw_perf 
fix: disable keepalive for better request rates under heavy load
 2023-12-26 10:28:46 -08:00