Commit Graph

1119 Commits

Author SHA1 Message Date
jonaustin09
3dc654eb11 fix: Modified DeleteObject error handling to return a successful response when versionId is not specified, and to return InvalidVersionId error when it is specified, in cases where versioning is enabled. 2024-09-26 16:41:35 -04:00
Ben McClelland
034a820b99 Merge pull request #831 from versity/test_cmdline_rest_object_tagging
Test cmdline rest object tagging
2024-09-24 08:31:22 -07:00
Ben McClelland
1808cec789 Merge pull request #836 from versity/dependabot/go_modules/dev-dependencies-fcb99e703d
chore(deps): bump the dev-dependencies group with 20 updates
2024-09-23 15:28:10 -07:00
dependabot[bot]
ccf597ef68 chore(deps): bump the dev-dependencies group with 20 updates
Bumps the dev-dependencies group with 20 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go) | `1.4.0` | `1.4.1` |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.30.5` | `1.31.0` |
| [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) | `1.61.2` | `1.63.1` |
| [github.com/aws/smithy-go](https://github.com/aws/smithy-go) | `1.20.4` | `1.21.0` |
| [github.com/aws/aws-sdk-go-v2/feature/ec2/imds](https://github.com/aws/aws-sdk-go-v2) | `1.16.13` | `1.16.14` |
| [github.com/aws/aws-sdk-go-v2/service/sso](https://github.com/aws/aws-sdk-go-v2) | `1.22.7` | `1.23.1` |
| [github.com/aws/aws-sdk-go-v2/service/ssooidc](https://github.com/aws/aws-sdk-go-v2) | `1.26.7` | `1.27.1` |
| [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) | `1.30.7` | `1.31.1` |
| [github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream](https://github.com/aws/aws-sdk-go-v2) | `1.6.4` | `1.6.5` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.27.34` | `1.27.37` |
| [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) | `1.17.32` | `1.17.35` |
| [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) | `1.17.19` | `1.17.23` |
| [github.com/aws/aws-sdk-go-v2/internal/configsources](https://github.com/aws/aws-sdk-go-v2) | `1.3.17` | `1.3.18` |
| [github.com/aws/aws-sdk-go-v2/internal/endpoints/v2](https://github.com/aws/aws-sdk-go-v2) | `2.6.17` | `2.6.18` |
| [github.com/aws/aws-sdk-go-v2/internal/v4a](https://github.com/aws/aws-sdk-go-v2) | `1.3.17` | `1.3.18` |
| [github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding](https://github.com/aws/aws-sdk-go-v2) | `1.11.4` | `1.11.5` |
| [github.com/aws/aws-sdk-go-v2/service/internal/checksum](https://github.com/aws/aws-sdk-go-v2) | `1.3.19` | `1.3.20` |
| [github.com/aws/aws-sdk-go-v2/service/internal/presigned-url](https://github.com/aws/aws-sdk-go-v2) | `1.11.19` | `1.11.20` |
| [github.com/aws/aws-sdk-go-v2/service/internal/s3shared](https://github.com/aws/aws-sdk-go-v2) | `1.17.17` | `1.17.18` |
| [github.com/klauspost/compress](https://github.com/klauspost/compress) | `1.17.9` | `1.17.10` |


Updates `github.com/Azure/azure-sdk-for-go/sdk/storage/azblob` from 1.4.0 to 1.4.1
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.4.0...sdk/storage/azblob/v1.4.1)

Updates `github.com/aws/aws-sdk-go-v2` from 1.30.5 to 1.31.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.30.5...v1.31.0)

Updates `github.com/aws/aws-sdk-go-v2/service/s3` from 1.61.2 to 1.63.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.61.2...service/s3/v1.63.1)

Updates `github.com/aws/smithy-go` from 1.20.4 to 1.21.0
- [Release notes](https://github.com/aws/smithy-go/releases)
- [Changelog](https://github.com/aws/smithy-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/smithy-go/compare/v1.20.4...v1.21.0)

Updates `github.com/aws/aws-sdk-go-v2/feature/ec2/imds` from 1.16.13 to 1.16.14
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.16.13...v1.16.14)

Updates `github.com/aws/aws-sdk-go-v2/service/sso` from 1.22.7 to 1.23.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/mq/v1.22.7...v1.23.1)

Updates `github.com/aws/aws-sdk-go-v2/service/ssooidc` from 1.26.7 to 1.27.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.26.7...v1.27.1)

Updates `github.com/aws/aws-sdk-go-v2/service/sts` from 1.30.7 to 1.31.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/service/s3/v1.31.1/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/mgn/v1.30.7...service/s3/v1.31.1)

Updates `github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream` from 1.6.4 to 1.6.5
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/drs/v1.6.4...credentials/v1.6.5)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.27.34 to 1.27.37
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.27.34...config/v1.27.37)

Updates `github.com/aws/aws-sdk-go-v2/credentials` from 1.17.32 to 1.17.35
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.32...credentials/v1.17.35)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.19 to 1.17.23
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.19...credentials/v1.17.23)

Updates `github.com/aws/aws-sdk-go-v2/internal/configsources` from 1.3.17 to 1.3.18
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/ini/v1.3.17...internal/ini/v1.3.18)

Updates `github.com/aws/aws-sdk-go-v2/internal/endpoints/v2` from 2.6.17 to 2.6.18
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/endpoints/v2.6.17...internal/endpoints/v2.6.18)

Updates `github.com/aws/aws-sdk-go-v2/internal/v4a` from 1.3.17 to 1.3.18
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/ini/v1.3.17...internal/ini/v1.3.18)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding` from 1.11.4 to 1.11.5
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/service/dlm/v1.11.5/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/waf/v1.11.4...service/dlm/v1.11.5)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/checksum` from 1.3.19 to 1.3.20
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/internal/ini/v1.3.19...internal/ini/v1.3.20)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/presigned-url` from 1.11.19 to 1.11.20
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/dax/v1.11.19...service/dax/v1.11.20)

Updates `github.com/aws/aws-sdk-go-v2/service/internal/s3shared` from 1.17.17 to 1.17.18
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.17...credentials/v1.17.18)

Updates `github.com/klauspost/compress` from 1.17.9 to 1.17.10
- [Release notes](https://github.com/klauspost/compress/releases)
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/compress/compare/v1.17.9...v1.17.10)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/smithy-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/ec2/imds
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sso
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/ssooidc
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/configsources
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/endpoints/v2
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/internal/v4a
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/checksum
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/presigned-url
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/service/internal/s3shared
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/klauspost/compress
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-23 22:05:42 +00:00
Luke McCrone
9014f05bad test: tagging, more run/assert changes, dockerfile, test split-ups 2024-09-23 12:36:25 -07:00
Ben McClelland
2f0d39f44f Merge pull request #830 from versity/ben/azurite_tests
fix: azure tests and add azure ci test
2024-09-23 12:25:40 -07:00
Ben McClelland
3a9cbfcbd6 fix: azure tests and add azure ci test 2024-09-23 09:10:52 -07:00
Ben McClelland
9f9f895522 Merge pull request #829 from versity/ben/enable_tests
fix: enable TestUploadPartCopy tests in full flow
2024-09-23 08:15:22 -07:00
Ben McClelland
b2d9a58907 Merge pull request #828 from versity/ben/directory_get
fix: requests should fail for directory objects if key does not end in /
2024-09-23 08:15:07 -07:00
Ben McClelland
20f334b1f9 fix: requests should fail for directory objects if key does not end in /
The object semantics are that a key that ends with / is different
than a key that does not.  However on posix a name that ends with /
is assumed to be a directory, but it still succeeds to access the
directory by name without a trailing /. So we need to explicitly
check if the request is for a non-directory and we are trying to
access a directory.

Fixes #824
2024-09-20 15:44:10 -07:00
Ben McClelland
14595ac6f3 fix: enable TestUploadPartCopy tests in full flow 2024-09-20 15:11:23 -07:00
Ben McClelland
fba121e4aa Merge pull request #825 from versity/fix/list-objects-empty-props
fix: Removed empty poperties from ListObjects and ListObjectsV2 actio…
2024-09-20 12:36:14 -07:00
Ben McClelland
30ffccbcf6 Merge pull request #827 from versity/fix/put-object-overwrite-file-objs
fix: Fixed the error case, to return ObjectParentIsFile error when ov…
2024-09-20 12:19:38 -07:00
Ben McClelland
b777a4697e Merge pull request #826 from versity/ben/docker_directory
fix: docker invocation in Makefile
2024-09-20 12:18:54 -07:00
jonaustin09
6de3df6070 fix: Removed empty poperties from ListObjects and ListObjectsV2 actions responses. Added StartAfter in ListObjectV2 response 2024-09-20 12:17:30 -07:00
jonaustin09
9ffb70f08e fix: Fixed the error case, to return ObjectParentIsFile error when overwriting a file object with a nested file object with PutObject in posix 2024-09-20 14:45:35 -04:00
Ben McClelland
767a6615fc Merge pull request #809 from versity/test_cmdline_rest_delete_objects
Test cmdline rest delete objects
2024-09-20 09:50:50 -07:00
Ben McClelland
1e60aae841 fix: docker invocation in Makefile 2024-09-20 09:43:20 -07:00
Ben McClelland
53415cc93a Merge pull request #708 from versity/object-versioning
Bucket versioning in Posix
2024-09-20 09:38:30 -07:00
Ben McClelland
0d0de244e1 fix: name too long error for head/delete 2024-09-20 09:17:19 -07:00
Ben McClelland
20d65ea6d9 fix: version id no metadata key check 2024-09-20 08:56:03 -07:00
jonaustin09
800cf62209 feat: Implemented object versioning for multipart uploads. Implemented integration tests for the versioning implementation for multipart uploads 2024-09-19 13:29:19 -07:00
jonaustin09
6d4ff09d6f feat: Added integration tests for bucket object versioning. Made a couple of bug fixes in the versioning implementation 2024-09-19 13:29:14 -07:00
Luke McCrone
baea416311 test: REST get/put/delete object, docker updates, time reduction 2024-09-18 18:01:32 -03:00
jonaustin09
8252ecd452 feat: basic logic implementation of bucket object versioning in posix backend
New posix backend option --versioning-dir will enable storing object versions
in specified directory.
2024-09-18 13:04:34 -07:00
Ben McClelland
cf067b5d00 Merge pull request #806 from versity/ben/common_prefixes
fix: list objects trim common prefixes that match marker prefix
v1.0.7
2024-09-18 12:22:49 -07:00
Ben McClelland
d9d3a16051 fix: azure list objects trim common prefixes that match marker prefix 2024-09-18 12:03:21 -07:00
Ben McClelland
180df62134 fix: list objects trim common prefixes that match marker prefix
This checks to see if the common prefix is before the marker and
thus would have been returned in earlier list objects request.

The error case was aws cli listing multiple entries for the same
common prefix when the listing required multiple pagination
requests.

Fixes #778
2024-09-18 12:02:25 -07:00
Ben McClelland
221440e9b2 Merge pull request #811 from versity/ben/invalid_range
fix: get object range min=max return single byte min
2024-09-18 12:00:49 -07:00
Ben McClelland
4cace00d8e fix: get object range min=max return single byte min
The range min-max where min=max was incorrectly returning invalid
object range.  On AWS, this returns the byte at offset min.

Fixes #810
2024-09-18 10:00:57 -07:00
Ben McClelland
c5f31a8407 Merge pull request #807 from versity/fix/uri-path-escape
URI path escape
2024-09-17 16:44:08 -07:00
Luke McCrone
b14df4a595 test: proper upload list key comparisons 2024-09-17 18:41:06 -03:00
Luke McCrone
f7991f935a test: list multipart uploads fix 2024-09-17 18:17:57 -03:00
jonaustin09
600aca8bdc fix: Fixed the request uri path escape to support object key special characters 2024-09-17 13:28:30 -04:00
Ben McClelland
8612b75337 Merge pull request #808 from versity/dependabot/go_modules/dev-dependencies-eb57ff20b2
chore(deps): bump the dev-dependencies group with 3 updates
2024-09-16 16:23:17 -07:00
dependabot[bot]
9b7b977ecd chore(deps): bump the dev-dependencies group with 3 updates
Bumps the dev-dependencies group with 3 updates: [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2), [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) and [github.com/cpuguy83/go-md2man/v2](https://github.com/cpuguy83/go-md2man).


Updates `github.com/aws/aws-sdk-go-v2/config` from 1.27.33 to 1.27.34
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.27.33...config/v1.27.34)

Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.17.18 to 1.17.19
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.18...credentials/v1.17.19)

Updates `github.com/cpuguy83/go-md2man/v2` from 2.0.4 to 2.0.5
- [Release notes](https://github.com/cpuguy83/go-md2man/releases)
- [Commits](https://github.com/cpuguy83/go-md2man/compare/v2.0.4...v2.0.5)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
- dependency-name: github.com/cpuguy83/go-md2man/v2
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: dev-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-16 22:02:53 +00:00
Ben McClelland
21a51380f9 Merge pull request #805 from versity/ben/direct_tests
fix: separate out direct tests dockerfile for easier use
2024-09-13 16:27:09 -07:00
Ben McClelland
590295c3d1 test: separate out direct tests dockerfile for easier use
The direct tests don't need the gateway or any go compilers.
This separates out the dockerfile to minimize it a bit more
from the other tests.
2024-09-13 15:47:51 -07:00
Ben McClelland
f43040d1da Merge pull request #804 from versity/static_setup_fix
test: static bucket init fix
2024-09-13 15:47:32 -07:00
Luke McCrone
ae1c566656 test: static bucket init fix, github-actions enhancements 2024-09-13 19:11:00 -03:00
Ben McClelland
53a7abf82f Merge pull request #802 from versity/ben/utc_time
fix: time locale utc in responses
2024-09-13 08:34:06 -07:00
Ben McClelland
9dbfaeed0c fix: time locale utc in responses
Change all response times to UTC. Fix bucket create date time
formatting. This was using the incorrct xml field name before.
2024-09-12 19:17:46 -07:00
Ben McClelland
080fd0136c Merge pull request #801 from versity/ben/docker_cleanup
chore: move test docker files to tests dir
2024-09-12 16:50:02 -07:00
Ben McClelland
220819444f chore: move test docker files to tests dir 2024-09-12 16:15:33 -07:00
Ben McClelland
cf6c1b97d1 Merge pull request #800 from versity/ben/tests_direct
fix: get direct tests working against a non-versitygw endpoint
2024-09-12 16:08:23 -07:00
Ben McClelland
d50027419e fix: get direct tests working against a non-versitygw endpoint 2024-09-12 11:39:31 -07:00
Ben McClelland
2d9a7cc019 Merge pull request #794 from versity/test_cmdline_xml_retrieval
Test cmdline xml retrieval
2024-09-11 15:25:58 -07:00
Ben McClelland
de67b1d718 Merge pull request #799 from versity/ben/dashboard
chore: move dashboard to extra
2024-09-11 15:25:37 -07:00
Ben McClelland
22a958bcc4 chore: move dashboard to extra 2024-09-11 13:50:27 -07:00
Ben McClelland
cecf563d92 Merge pull request #798 from versity/fix/azure-obj-content-enc
fix: Fixed the mechanism to put/get object content-type, content-enco…
2024-09-11 13:30:39 -07:00