Commit Graph

130 Commits

Author SHA1 Message Date
Ben McClelland
590295c3d1 test: separate out direct tests dockerfile for easier use
The direct tests don't need the gateway or any go compilers.
This separates out the dockerfile to minimize it a bit more
from the other tests.
2024-09-13 15:47:51 -07:00
Luke McCrone
ae1c566656 test: static bucket init fix, github-actions enhancements 2024-09-13 19:11:00 -03:00
Ben McClelland
220819444f chore: move test docker files to tests dir 2024-09-12 16:15:33 -07:00
Ben McClelland
d50027419e fix: get direct tests working against a non-versitygw endpoint 2024-09-12 11:39:31 -07:00
Ben McClelland
2d9a7cc019 Merge pull request #794 from versity/test_cmdline_xml_retrieval
Test cmdline xml retrieval
2024-09-11 15:25:58 -07:00
Ben McClelland
cecf563d92 Merge pull request #798 from versity/fix/azure-obj-content-enc
fix: Fixed the mechanism to put/get object content-type, content-enco…
2024-09-11 13:30:39 -07:00
jonaustin09
47d1a799f6 fix: Fixed the mechanism to put/get object content-type, content-encoding in azure 2024-09-11 15:11:37 -04:00
Luke McCrone
4ed54d9bd9 test: initial rests tests, run/assert, docker cleanup 2024-09-11 15:13:13 -03:00
jonaustin09
a6fd1322f7 fix: Removed multipart upload part from GetObjectAttributes action 2024-09-11 11:57:32 -04:00
jonaustin09
8a46de8e3b fix: change multipart upload implementation with .sgwtmp namespace in azure 2024-09-10 13:07:15 -07:00
Ben McClelland
dc71365bab fix: set content type/encoding on put/mutlipart object
This fixes put object with setting a content type. If no content
type is set, then we will return a default content type for
following requests. Mutli-part upload appears to be ok.

Also fixed content eincoding and multipart uploads.

Fixes #783
2024-09-09 16:04:54 -07:00
Ben McClelland
ccd4166b2e fix: copy object missing prefix on metadata delete
When using the REPLACE directive, we were incorrectly removing the
old metadata on the object due to missing the metadata prefix on
the key.  Fix this to remove the correct metadata before setting
new metadata.

Fixes #787
2024-09-07 17:06:45 -07:00
Luke McCrone
2823676aa2 test: improve file data comparisons for testing 2024-09-04 14:24:01 -03:00
jonaustin09
9be4f27550 feat: Added an integration test to cover the case to CopyObject with the copysource starting with / 2024-08-29 15:32:20 -04:00
jonaustin09
201777c819 fix: Added an integration test case for ListObjectsV2 to specify max-keys as the exact number of objects in the bucket 2024-08-29 11:14:29 -04:00
Ben McClelland
3cb53d0fad Merge pull request #768 from versity/fix/s3-ls-page-size
ListObjects(V2) common prefixes pagination
2024-08-28 11:14:07 -07:00
Luke McCrone
1b4db1fd96 test: copyright 2024-08-28 12:51:39 -03:00
jonaustin09
227fdaa00b fix: Fixed the pagination for common prefixes in ListObjects & ListObjectsV2 actions 2024-08-28 11:07:44 -04:00
Jon Austin
d79f978df9 feat: Added the standard storage class to all the available get/list actions responses in posix. (#765) 2024-08-27 15:28:40 -07:00
Luke McCrone
3238aac4bd test: delete tagging test, dockerfile 2024-08-27 14:45:30 -03:00
Luke McCrone
a5c3332dc6 test: added put, get tagging tests, some reorganization 2024-08-26 14:24:40 -03:00
Ben McClelland
453136bd5a fix: return KeyTooLongError when filenames exceed allowed length
The posix limits wont exactly match up with the AWS key length
limits because posix has component length limits as well as path
length limits.

This reponds with the aws compatible KeyTooLongError under these
conditions now.

Note that delete object returns success even in the error cases.

Fixes #755
2024-08-24 14:53:42 -07:00
jonaustin09
71df685fb7 fix: Fixed directory objects listing with delimiter 2024-08-23 11:28:52 -04:00
Ben McClelland
77aa4366b5 fix: unescape copy source before handing to backend
We were handing the URL escaped string to the backend as the
copysource which includes "%<hex>" for spaces and other special
characters. The backend would then interpret this as the source
path. This fixes the copyobject and upload part copy.

Fixes #749
2024-08-22 10:06:38 -07:00
Ben McClelland
54490f55cc chore: cleanup staticcheck errors 2024-08-13 11:09:14 -07:00
Ben McClelland
a36d974942 fix: copy-object with replace metadata-directive
In copy-object, if the source and destination are the same then
X-Amz-Metadata-Directive must be set to "REPLACE" in order to use
this api call to update the metadata of the object in place.

The default X-Amz-Metadata-Directive is "COPY" if not specified.
"COPY" is only valid if source and destination are not the same
object.

When "REPLACE" selected, metadata does not have to differ for the
call to be successful. The "REPLACE" always sets the incoming
metadata (even if empty or the same as the source).

Fixes #734
2024-08-13 10:52:47 -07:00
jonaustin09
23fd0d3fdd fix: Fixed PutBucketAcl action error handling, removed the bucket owner check for all the acl options 2024-08-12 15:27:03 -04:00
Ben McClelland
cbf03c30ce Merge pull request #726 from versity/fix/iam-get-root-user
Root user credentials in IAM services
2024-08-12 10:18:36 -07:00
Ben McClelland
9f53d0f584 Merge pull request #725 from versity/ben/delete_error
fix: non-existing object delete response
2024-08-09 13:26:52 -07:00
jonaustin09
34b7fd6ee7 fix: Added the root user data in the iam services records 2024-08-09 16:14:51 -04:00
Luke McCrone
0facfdc9fd test: multiple policy principals, improved bucket cleanup, general cleanup 2024-08-09 16:40:44 -03:00
Ben McClelland
e92b36a12c fix: non-existing object delete response
The expected response code for deleting non-existing objects is
expected to be 204 (No Content) instead of NoSuchKey. The tests
are updated to validate expected responses.

Fixes #724
2024-08-08 11:46:36 -07:00
Ben McClelland
797376a235 fix: head/get/delete/copy directory object should fail when corresponding file object exists
The API hanlders and backend were stripping trailing "/" in object
paths. So if an object exists and a request came in for head/get/delete/copy
for that same name but with a trailing "/" indicating request should
be for directory object, the "/" would be stripped and the request
would be handlied for the incorrect file object.

This fix adds in checks to handle the case with the training "/"
in the request.

Fixes #709
2024-08-05 11:55:32 -07:00
Ben McClelland
61a97e94db fix: set default content type to binary/octet-stream
AWS uses binary/octet-stream for the default content type if the
client doesn't specify the content type. Change the default for
the gateway to match this behavior.

Fixes #697
2024-08-02 09:02:57 -07:00
Luke
18a8813ce7 Test cmdline acl hardcode fix (#712)
* test: s3cmd, s3api acls tests (direct vs s3)
* test: completed s3:PutBucketAcl tests
2024-08-01 17:24:49 -07:00
Luke McCrone
60c8eb795d test: improve command coverage for tools, retention bypass 2024-07-25 15:23:31 -03:00
jonaustin09
5e484f2355 fix: Fixed CopySource parsing to handle the values starting with '/' in CopyObject action in posix and azure backends. 2024-07-22 11:30:32 -04:00
Luke McCrone
c580947b98 test: added user tests, added command recording, re-added s3cmd tests 2024-07-17 13:45:01 -03:00
jonaustin09
23a40d86a2 fix: Removed the bucket ACL owner check for admin and root users 2024-07-17 09:39:00 -04:00
jonaustin09
828eb93bee fix: Added 'Type' property support in bucket ACL Grantee schema 2024-07-16 18:17:16 -04:00
jonaustin09
2843cdbd45 fix: Fixed ChangeBucketOwnership action implementation to update the bucket acl 2024-07-11 13:45:01 -04:00
Luke McCrone
059205c174 test: re-addition of some tests after default ACL disable, cleanup 2024-07-11 12:20:35 -03:00
jonaustin09
e773872c48 feat: Implemented response body streaming for GetObject action 2024-07-08 15:56:24 -04:00
Luke McCrone
e81fac9558 test: multipart abort policy, direct s3 user setup, ACLs on/off command 2024-07-04 15:21:08 -03:00
Luke McCrone
f66deb9b9a test: commented out failing acls-related tests
Recent change requires fixes to tests that depended on bucket
ACLs enabled by default. Comment these out until tests can be
fixed up for new behavior.
2024-06-28 21:07:29 -07:00
jonaustin09
7545e6236c feat: Implement bucket ownership controls
Bucket ACLs are now disabled by default the same as AWS.
By default the object ownership is BucketOwnerEnforced
which means that bucket ACLs are disabled. If one attempts
to set bucket ACL the following error is returned both in
the gateway and on AWS:
	ErrAclNotSupported: {
		Code:           "AccessControlListNotSupported",
		Description:    "The bucket does not allow ACLs",
		HTTPStatusCode: http.StatusBadRequest,
	},

ACls can be enabled with PutBucketOwnershipControls

Changed bucket canned ACL translation

New backend interface methods:
PutBucketOwnershipControls
GetBucketOwnershipControls
DeleteBucketOwnershipControls

Added these to metrics
2024-06-28 21:03:09 -07:00
Luke McCrone
812efe6d43 test: multiple policy-related tests, multipart upload test rewrites 2024-06-27 20:19:44 -03:00
Ben McClelland
eafa5e12db Merge pull request #653 from versity/test_cmdline_policy_more_tests
Test cmdline policy more tests
2024-06-25 09:44:29 -07:00
Luke McCrone
7f152126a4 test: more policy tests, cleanup, include s3 backend in github-actions 2024-06-25 13:23:03 -03:00
Ben McClelland
c3dbb923ba test: increase test file sizes to 5M for larger part sizes
Some backends require larger part sizes for the multipart upload
tests. Increase the test file size to 5M to meet the requirements
for scoutfs.
2024-06-24 15:39:12 -07:00