Commit Graph

638 Commits

Author SHA1 Message Date
Luke McCrone
3fbc170bc7 test: bats test dockerfile, test refactoring, file compare updates 2024-02-20 14:45:45 -03:00
Ben McClelland
8d229b5878 Merge pull request #407 from versity/dependabot/go_modules/dev-dependencies-6e387e58a1 2024-02-19 14:28:11 -08:00
dependabot[bot]
7605654e47 chore(deps): bump the dev-dependencies group with 7 updates
Bumps the dev-dependencies group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.24.1` | `1.25.0` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.48.1` | `1.50.1` |
| [github.com/aws/smithy-go](https://github.com/aws/smithy-go) | `1.19.0` | `1.20.0` |
| [github.com/nats-io/nats.go](https://github.com/nats-io/nats.go) | `1.32.0` | `1.33.1` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.26.6` | `1.27.0` |
| [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) | `1.16.16` | `1.17.0` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.15.15` | `1.16.2` |


Updates `github.com/aws/aws-sdk-go-v2` from 1.24.1 to 1.25.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.24.1...v1.25.0)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.48.1 to 1.50.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.48.1...service/s3/v1.50.1)

Updates `github.com/aws/smithy-go` from 1.19.0 to 1.20.0
- [Release notes](https://github.com/aws/smithy-go/releases)
- [Changelog](https://github.com/aws/smithy-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/smithy-go/compare/v1.19.0...v1.20.0)

Updates `github.com/nats-io/nats.go` from 1.32.0 to 1.33.1
- [Release notes](https://github.com/nats-io/nats.go/releases)
- [Commits](https://github.com/nats-io/nats.go/compare/v1.32.0...v1.33.1)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.26.6 to 1.27.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.26.6...config/v1.27.0)

Updates `github.com/aws/aws-sdk-go-v2/credentials` from 1.16.16 to 1.17.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/v1.17.0/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.16.16...v1.17.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.15 to 1.16.2
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.15...v1.16.2)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/smithy-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/nats-io/nats.go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-19 22:06:30 +00:00
Ben McClelland
6dc48d2233 Merge pull request #403 from versity/test_cmdline_upload_part_copy
Test cmdline upload part copy
2024-02-19 10:43:53 -08:00
Luke McCrone
c938a32252 test: multpart - upload part copy, static bucket tests 2024-02-15 20:50:42 -03:00
Ben McClelland
1db9021aa7 Merge pull request #404 from versity/azure-bucket-tagging-actions
Bucket tagging actions in azure backend
2024-02-15 13:13:29 -08:00
jonaustin09
4d8352225b feat: Added bucket tagging actions in azure backend 2024-02-15 15:23:55 -05:00
Ben McClelland
905b283421 Merge pull request #402 from versity/ben/sign_with_user_agent v0.14 2024-02-14 10:53:53 -08:00
Ben McClelland
6fea34acda fix: request signature check with signed user-agent
This is a hack to replace the ignored headers in the aws-sdk-go-v2
internal/v4 package. The headers in the default ignore list include
User-Agent, but this is included is signed headers from some clients.

fixes #396
2024-02-13 22:56:13 -08:00
Ben McClelland
1c29fbfd81 Merge pull request #397 from versity/presigned-url-authentication
Presigned URL authentication
2024-02-13 11:33:49 -08:00
jonaustin09
a3b14d3a05 feat: Added an integration test for UploadPart action with v4 query params authentication, added unit tests for validateDate function 2024-02-13 11:38:28 -05:00
Ben McClelland
cafb57eb33 Merge pull request #399 from versity/ben/xml_responses
fix: correct xml response encoding for list-buckets
2024-02-13 08:16:55 -08:00
Ben McClelland
0760467c3d fix: correct xml response encoding for list-buckets and tagging
fixes #395
2024-02-12 16:20:07 -08:00
Ben McClelland
4d168da376 Merge pull request #401 from versity/dependabot/go_modules/dev-dependencies-0c6b2d3779
chore(deps): bump the dev-dependencies group with 4 updates
2024-02-12 16:09:21 -08:00
Ben McClelland
cde033811f Merge pull request #400 from versity/test_cmdline_list_parts
Test cmdline list parts
2024-02-12 16:08:30 -08:00
Luke McCrone
7a56c7e15e test: multipart upload - list parts, uploads 2024-02-12 19:36:24 -03:00
jonaustin09
e21e514997 feat: Added 20 integration tests for v4 authentication with query params. Fixed few bugs in v4 query params authentication 2024-02-12 16:31:01 -05:00
dependabot[bot]
660709fe6d chore(deps): bump the dev-dependencies group with 4 updates
Bumps the dev-dependencies group with 4 updates: [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go), [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go), [github.com/valyala/fasthttp](https://github.com/valyala/fasthttp) and [golang.org/x/sys](https://github.com/golang/sys).


Updates `github.com/Azure/azure-sdk-for-go/sdk/azcore` from 1.9.1 to 1.9.2
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.9.1...sdk/azcore/v1.9.2)

Updates `github.com/Azure/azure-sdk-for-go/sdk/storage/azblob` from 1.2.1 to 1.3.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azidentity/v1.2.1...sdk/azcore/v1.3.0)

Updates `github.com/valyala/fasthttp` from 1.51.0 to 1.52.0
- [Release notes](https://github.com/valyala/fasthttp/releases)
- [Commits](https://github.com/valyala/fasthttp/compare/v1.51.0...v1.52.0)

Updates `golang.org/x/sys` from 0.16.0 to 0.17.0
- [Commits](https://github.com/golang/sys/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/valyala/fasthttp
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-12 21:21:05 +00:00
Ben McClelland
5931d713f2 Merge pull request #398 from versity/test_cmdline_multipart_abort
test: multipart abort
2024-02-08 18:34:25 -08:00
Luke McCrone
08e5eb02a0 test: multipart abort 2024-02-08 18:31:19 -03:00
Ben McClelland
7cba952546 Merge pull request #394 from versity/test_cmdline_obj_tag
Test cmdline obj tag
2024-02-07 10:20:26 -08:00
Luke McCrone
5d6c0f8b67 another shellcheck fix 2024-02-07 14:58:06 -03:00
jonaustin09
be17b3fd33 feat: Closes #355. Added support for presigned URLs, particularly v4 authentication with query params 2024-02-07 09:17:35 -05:00
Ben McClelland
e6440da30a Merge pull request #393 from versity/ben/releaser_naming
fix: add release version to release artifacts
2024-02-06 08:42:26 -08:00
Ben McClelland
443da7f9a4 fix: add release version to release artifacts 2024-02-05 11:04:51 -08:00
Ben McClelland
6c56307746 Merge pull request #391 from versity/ben/docker_actions
feat: add docker images to release
v0.13
2024-02-04 10:21:07 -08:00
Ben McClelland
9765eadd84 feat: add docker images to release 2024-02-04 10:17:15 -08:00
Ben McClelland
4619171f86 Merge pull request #389 from versity/test_cmdline_head_data
Test cmdline head data
2024-02-02 11:35:35 -08:00
Luke McCrone
89b4b615ab test: cmdline tests (acls, get bucket/object info) 2024-02-02 11:32:13 -08:00
Jon Austin
0c056f935b ListObjectsV2 start-after prop (#388)
* fix: Fixes #138, Added StartAfter property in ListObjectsV2 action, added couple of integration tests for ListObjectsV2
2024-02-01 11:04:52 -08:00
Ben McClelland
bf1e2c83d5 Merge pull request #385 from versity/bucket-tagging-actions
Bucket tagging actions
2024-01-31 10:15:22 -08:00
Ben McClelland
68794518af fix: remove special proxy handling for bucket acls in posix backend 2024-01-31 10:10:12 -08:00
jonaustin09
3cce3a5201 feat: Added unit and integration test cases for posix bucket tagging related actions 2024-01-31 10:09:48 -08:00
jonaustin09
d70ea61830 feat: Added the following actions support in posix backend: PutBucketTagging, GetBucketTagging, DeleteBucketTagging 2024-01-31 10:09:48 -08:00
Ben McClelland
9d0cf77b25 Merge pull request #387 from versity/bucket-acl-on-creation
Bucket ACL on bucket creation
2024-01-31 09:55:12 -08:00
jonaustin09
0d3a238ceb feat: Implemented logic to add bucket ACL on bucket creation 2024-01-31 09:49:56 -08:00
Ben McClelland
99d0d9a007 Merge pull request #384 from versity/luke/posix_test
Luke/posix test
v0.12
2024-01-29 15:20:54 -08:00
Luke McCrone
1409d664b4 test: initial aws cli bats tests 2024-01-29 20:07:00 -03:00
Ben McClelland
b908a4b981 Merge pull request #386 from versity/dependabot/go_modules/dev-dependencies-55f64e24bf
chore(deps): bump the dev-dependencies group with 3 updates
2024-01-29 14:53:14 -08:00
dependabot[bot]
ac06b5c4ae chore(deps): bump the dev-dependencies group with 3 updates
Bumps the dev-dependencies group with 3 updates: [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2), [github.com/google/uuid](https://github.com/google/uuid) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.48.0 to 1.48.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.48.0...service/s3/v1.48.1)

Updates `github.com/google/uuid` from 1.5.0 to 1.6.0
- [Release notes](https://github.com/google/uuid/releases)
- [Changelog](https://github.com/google/uuid/blob/master/CHANGELOG.md)
- [Commits](https://github.com/google/uuid/compare/v1.5.0...v1.6.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.14 to 1.15.15
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.14...config/v1.15.15)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/google/uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-29 22:03:46 +00:00
Ben McClelland
3146556293 Merge pull request #380 from versity/ben/chunked_reader
feat: add chunked upload support
2024-01-25 13:41:43 -08:00
Ben McClelland
1c03fce3f5 Merge pull request #383 from versity/dependabot/go_modules/dev-dependencies-83121c2333
chore(deps): bump the dev-dependencies group with 3 updates
2024-01-22 15:39:54 -08:00
dependabot[bot]
b83e2393a5 chore(deps): bump the dev-dependencies group with 3 updates
Bumps the dev-dependencies group with 3 updates: [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go), [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.4.0 to 1.5.1
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.4.0...sdk/internal/v1.5.1)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.26.3 to 1.26.6
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.26.3...config/v1.26.6)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.15.11 to 1.15.14
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.15.11...config/v1.15.14)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-22 21:45:22 +00:00
Ben McClelland
1366408baa feat: add chunked upload support
As described in
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
this adds support for reading from a chunked upload encoded request
body. The chunked reader modifies the data stream to remove the
chunk encoding while validating the chunk signatures in line. This
allows the upper layers to get just the object data stream.
2024-01-22 11:35:01 -08:00
Jon Austin
cf92b6fd80 Fix/azure copy object (#382)
* fix: Added destination bucket acl check and metadata comparision for CopyObject action in azure backend

---------

Co-authored-by: Ben McClelland <ben.mcclelland@versity.com>
2024-01-22 10:01:16 -08:00
Jon Austin
d956ecacd7 Fix/azure iam (#381)
* fix: Fixed iam internal iam file removal bug
2024-01-22 10:00:41 -08:00
Jon Austin
68e800492e Fix/azure list objects (#379)
* fix: Added pagination to ListObjects and ListObjectsV2 actions, fixed multipart upload non existing key error handling
2024-01-22 09:54:45 -08:00
Ben McClelland
f836d96717 Merge pull request #378 from versity/ben/signature 2024-01-17 13:18:54 -08:00
Ben McClelland
b5894dd714 fix: allow spaces in Authorization string
This change removes all spaces after the algorithm to have
standard parsing for the following key/value pairs. This fixes
some clients that were using a slightly different format than
the example AWS request strings.
2024-01-17 10:45:57 -08:00
Ben McClelland
17bdc58da9 Merge pull request #374 from versity/ben/test_fixup
Ben/test fixup
2024-01-17 10:45:43 -08:00