mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-06-06 11:02:34 +00:00
Code Issues Packages Projects Releases Wiki Activity
536 Commits 30 Branches 51 Tags
d404f96320bbe460018283d523ac3dfd8085c70d
Commit Graph

536 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ben McClelland
d404f96320 fix: translate azure errors to s3 for compatibility 2024-01-17 08:27:37 -08:00
Ben McClelland
1cdf0706e7 fix: fix crashes in test cases when fields missing 2024-01-17 08:27:37 -08:00
Ben McClelland
ca6d9e3c11 fix: docker env set to tests defaults 2024-01-17 08:27:37 -08:00
Ben McClelland
e16c54c1a3 Merge pull request #375 from versity/dependabot/go_modules/dev-dependencies-88fd56ff93 
chore(deps): bump the dev-dependencies group with 1 update
 2024-01-16 08:12:42 -08:00
dependabot[bot]
15daec9f51 chore(deps): bump the dev-dependencies group with 1 update 
Bumps the dev-dependencies group with 1 update: [github.com/nats-io/nats.go](https://github.com/nats-io/nats.go).


Updates `github.com/nats-io/nats.go` from 1.31.0 to 1.32.0
- [Release notes](https://github.com/nats-io/nats.go/releases)
- [Commits](https://github.com/nats-io/nats.go/compare/v1.31.0...v1.32.0)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats.go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-15 21:10:11 +00:00
Ben McClelland
c406d7069f Merge pull request #371 from versity/ben/default_acl 
fix: cleanup backend ACLs
 2024-01-11 12:30:44 -08:00
Ben McClelland
6481e2aac5 fix: cleanup backend ACLs 
This adds the default ACL to the CreateBucket backend method so
that the backend doesn't need to know how to construct and ACL.

This also moves the s3proxy ACLs to a tag key/value because the
gateway ACLs are not the same accounts as the backend s3 server.
TODO: we may need to mask this tag key/value if we add support
for the Get/PutBucketTagging API.
 2024-01-10 09:36:00 -08:00
Ben McClelland
45cf5e6373 Merge pull request #366 from versity/ben/az_ident 
feat: add azure local env auth
 2024-01-09 22:24:43 -08:00
Ben McClelland
3db43b7206 feat: add azure local env auth 
This is the recommended auth from the following:
https://github.com/Azure-Samples/storage-blobs-go-quickstart/blob/master/storage-quickstart.go
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-go?toc=%2Fazure%2Fdeveloper%2Fgo%2Ftoc.json&bc=%2Fazure%2Fdeveloper%2Fgo%2Fbreadcrumb%2Ftoc.json&tabs=roles-azure-portal#authenticate-to-azure-and-authorize-access-to-blob-data
 2024-01-09 22:21:39 -08:00
Ben McClelland
6786a6385a Merge pull request #367 from versity/azure-sas-token 
Azure sas token authentication
 2024-01-09 22:06:57 -08:00
jonaustin09
e5fc12042b feat: Added sas token authentication for azure backend 2024-01-09 22:03:13 -08:00
Ben McClelland
06ccd7496e Merge pull request #369 from versity/ben/az_cleanup 
chore: remove azure bug comment
 2024-01-09 08:29:42 -08:00
Ben McClelland
c86362b269 Merge pull request #370 from versity/dependabot/go_modules/dev-dependencies-925c4d3e9f 
chore(deps): bump the dev-dependencies group with 6 updates
 2024-01-09 08:28:59 -08:00
Ben McClelland
a86a8cbce5 fix: add azure CreateMultipartUpload to allow clients to work as expected 
The azure sdk doesnt use a separate function to initialize a
multipart upload, so CreateMultipartUpload becomes a no-op.
But we still need to have it return success so that clients
wont get an unexpected error.
 2024-01-08 13:40:20 -08:00
dependabot[bot]
328ea4f4b7 chore(deps): bump the dev-dependencies group with 6 updates 
Bumps the dev-dependencies group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.24.0` | `1.24.1` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.47.7` | `1.48.0` |
| [github.com/gofiber/fiber/v2](https://github.com/gofiber/fiber) | `2.51.0` | `2.52.0` |
| [golang.org/x/sys](https://github.com/golang/sys) | `0.15.0` | `0.16.0` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.26.2` | `1.26.3` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.15.9` | `1.15.11` |


Updates `github.com/aws/aws-sdk-go-v2` from 1.24.0 to 1.24.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.24.0...v1.24.1)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.47.7 to 1.48.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.47.7...service/s3/v1.48.0)

Updates `github.com/gofiber/fiber/v2` from 2.51.0 to 2.52.0
- [Release notes](https://github.com/gofiber/fiber/releases)
- [Commits](https://github.com/gofiber/fiber/compare/v2.51.0...v2.52.0)

Updates `golang.org/x/sys` from 0.15.0 to 0.16.0
- [Commits](https://github.com/golang/sys/compare/v0.15.0...v0.16.0)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.26.2 to 1.26.3
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.26.2...config/v1.26.3)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.9 to 1.15.11
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/config/v1.15.11/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.9...config/v1.15.11)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/gofiber/fiber/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-08 21:30:29 +00:00
Ben McClelland
bf38a03af9 chore: remove azure bug comment 
This comment references a bug that was fixed in the v1.2.1 sdk
update:
https://github.com/Azure/azure-sdk-for-go/issues/22171
 2024-01-08 13:11:41 -08:00
Ben McClelland
f237d06a01 Merge pull request #368 from versity/azure-docker 
Azure docker
 2024-01-08 10:10:18 -08:00
jonaustin09
8fc16392d1 feat: Dockerized azure backend to run 2 images: one for azurite, one for azure backend 2024-01-08 10:07:50 -08:00
Jon Austin
9bfec719f3 Azure ACL (#364) 
feat: Addded GetBucketAcl and PutBucketAcl actions implementation in azure backend. ACL is stored in the container metadata
 2024-01-03 11:15:53 -08:00
Ben McClelland
4a1d479bcb Merge pull request #365 from versity/ben/readme_update 
chore: update docs for s3 backend support
 2024-01-03 11:13:26 -08:00
Ben McClelland
9226999ae9 chore: update docs for s3 backend support 2024-01-03 11:00:09 -08:00
Ben McClelland
3f18bb5977 Merge pull request #362 from versity/dependabot/go_modules/dev-dependencies-21be33ef01 
chore(deps): bump the dev-dependencies group with 1 update
 2024-01-01 14:02:48 -08:00
dependabot[bot]
b145777340 chore(deps): bump the dev-dependencies group with 1 update 
Bumps the dev-dependencies group with 1 update: [github.com/urfave/cli/v2](https://github.com/urfave/cli).


Updates `github.com/urfave/cli/v2` from 2.26.0 to 2.27.1
- [Release notes](https://github.com/urfave/cli/releases)
- [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md)
- [Commits](https://github.com/urfave/cli/compare/v2.26.0...v2.27.1)

---
updated-dependencies:
- dependency-name: github.com/urfave/cli/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-01 21:11:27 +00:00
Ben McClelland
bae716b012 Merge pull request #352 from versity/ben/azure_blob 
Ben/azure blob
 2023-12-29 21:59:23 -08:00
Ben McClelland
4343252c1f Merge pull request #361 from versity/ben/readme 
chore: update readme status and news
 2023-12-29 21:58:07 -08:00
Ben McClelland
5a3ecc2db4 fix: azure run go mod tidy 2023-12-29 21:56:47 -08:00
jonaustin09
cafa45760c feat: Added pagination for ListParts azure action and added get range support for GetObject azure action 2023-12-29 21:55:32 -08:00
jonaustin09
8cc89fa713 feat: Azure backend implementation 2023-12-29 21:55:32 -08:00
Ben McClelland
3b945f72fc feat: azure blob backend initial pass 2023-12-29 21:54:56 -08:00
Ben McClelland
111d75b5d4 chore: update readme status and news 2023-12-29 21:46:47 -08:00
Ben McClelland
8b31d6d93c Merge pull request #356 from versity/ben/s3_minor_fix 
fix: make s3 PutBucketAcl have receiver pointer like others for consi…
v0.11 		2023-12-26 10:34:23 -08:00
Ben McClelland
a6927a0947 Merge pull request #360 from versity/dependabot/go_modules/dev-dependencies-a55e97eb27 
chore(deps): bump the dev-dependencies group with 3 updates
 2023-12-26 10:33:18 -08:00
Ben McClelland
c1587e4c1c fix: make s3 PutBucketAcl have receiver pointer like others for consistency 2023-12-26 10:31:56 -08:00
Ben McClelland
6146dcff4a Merge pull request #358 from versity/ben/gw_perf 
fix: disable keepalive for better request rates under heavy load
 2023-12-26 10:28:46 -08:00
Ben McClelland
3ba218bd9a Merge pull request #357 from versity/ben/quiet_logging 
feat: add quiet option to silence request log output
 2023-12-26 10:28:30 -08:00
Ben McClelland
60bc9a3fc5 Merge pull request #354 from versity/ben/req_body_limit 
fix: body limit no longer needed for streaming request body
 2023-12-26 10:27:44 -08:00
Ben McClelland
3a2cc8f915 Merge pull request #353 from versity/ben/s3_backend_perf 
fix: s3 backend performance increase with client reuse
 2023-12-26 10:27:27 -08:00
dependabot[bot]
15455f5028 chore(deps): bump the dev-dependencies group with 3 updates 
Bumps the dev-dependencies group with 3 updates: [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2), [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.47.6 to 1.47.7
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.47.6...service/s3/v1.47.7)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.26.1 to 1.26.2
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.26.1...config/v1.26.2)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.8 to 1.15.9
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/config/v1.15.9/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.8...config/v1.15.9)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-25 21:08:44 +00:00
Ben McClelland
216e50b9fd fix: disable keepalive for better request rates under heavy load 2023-12-23 22:56:49 -08:00
Ben McClelland
d47cbcb39f feat: add quiet option to silence request log output 2023-12-23 16:52:56 -08:00
Ben McClelland
43bfe8a869 fix: body limit no longer needed for streaming request body 2023-12-22 15:24:23 -08:00
Ben McClelland
6e37096b35 fix: s3 backend performance increase with client reuse 
The previous way of initializing the s3 client in each call was
adding a lot of overhead and would tank performance beyond about
20 simultaneous requests.

Since the backend access is through a single account, we can init
and store this client for use from each api call.
 2023-12-22 15:09:44 -08:00
Ben McClelland
6f6af8ec07 Merge pull request #344 from versity/ben/s3proxy 
Ben/s3proxy
v0.10 		2023-12-19 09:58:30 -08:00
Ben McClelland
f27162b36d Merge pull request #347 from versity/dependabot/go_modules/golang.org/x/crypto-0.17.0 
chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0
 2023-12-18 17:09:10 -08:00
dependabot[bot]
22fcabe085 chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-18 21:51:46 +00:00
Ben McClelland
89783a69f4 Merge pull request #346 from versity/dependabot/go_modules/dev-dependencies-f554204c74 
chore(deps): bump the dev-dependencies group with 4 updates
 2023-12-18 13:51:20 -08:00
dependabot[bot]
9afca13329 chore(deps): bump the dev-dependencies group with 4 updates 
Bumps the dev-dependencies group with 4 updates: [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2), [github.com/google/uuid](https://github.com/google/uuid), [github.com/segmentio/kafka-go](https://github.com/segmentio/kafka-go) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.47.5 to 1.47.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.47.5...service/s3/v1.47.6)

Updates `github.com/google/uuid` from 1.4.0 to 1.5.0
- [Release notes](https://github.com/google/uuid/releases)
- [Changelog](https://github.com/google/uuid/blob/master/CHANGELOG.md)
- [Commits](https://github.com/google/uuid/compare/v1.4.0...v1.5.0)

Updates `github.com/segmentio/kafka-go` from 0.4.46 to 0.4.47
- [Release notes](https://github.com/segmentio/kafka-go/releases)
- [Commits](https://github.com/segmentio/kafka-go/compare/v0.4.46...v0.4.47)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.7 to 1.15.8
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.7...config/v1.15.8)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/google/uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/segmentio/kafka-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-18 21:47:29 +00:00
Ben McClelland
3d6e37bbb1 fix: make iam s3 object work similar to internal files 2023-12-18 10:39:20 -08:00
Ben McClelland
ab43240b4e fix: add cli options to enable s3 iam service 2023-12-18 08:29:04 -08:00
Ben McClelland
b833e94c4b Merge pull request #343 from versity/ben/s3proxy 
chore: clarify s3 backend struct with better name
 2023-12-18 08:20:11 -08:00