There were two issues that were preventing correct behavior here.
One was that we need to specifically request the container metadata
when listing containers, and then we also need to handle the case
where the container does not include the acl metadata.
This fixes both of these cases by adding in the metadata request
option for this container listing, and will return a default acl
if not provided in the container metadaata.
Fixes#948
The latest azurite made a change where the blob metadata must be
explicitly requested when calling NewListBlobsFlatPager(). We were
taking action on metadata iteams, and the tests were failing due
to these always missing without requesting metadata to be included
in the response.
Fix is to enable metadata for the response.
The "x-amz-copy-source" header may start with '/' as observed with
WinSCP. However, '/' is also the separator between the bucket and the
object path in "x-amz-copy-source".
Consider the following code in VerifyObjectCopyAccess():
srcBucket, srcObject, found := strings.Cut(copySource, "/")
If `copySource` starts with '/', then `srcBucket` is set to an empty
string. Later, an error is returned because bucket "" does not exist.
This issue was fixed in the Posix and Azure backends by the following
commit:
* 5e484f2 fix: Fixed CopySource parsing to handle the values starting with '/' in CopyObject action in posix and azure backends.
But the issue was not fixed in `VerifyObjectCopyAccess`.
This commit sanitizes "x-amz-copy-source" right after the header is
extracted in `s3api/controllers/base.go`. This ensures that the
`CopySource` argument passed to the backend functions UploadPartCopy()
and CopyObject() does not start with '/'. Since the backends no longer
need to strip away any leading '/' in `CopySource`, the parts of
commit 5e484f2 modifying the Posix and Azure backends are reverted.
Fixes issue #773.
Signed-off-by: Christophe Vu-Brugier <christophe.vu-brugier@seagate.com>
It is better if we let the s3response module handle the xml
formatting spec specifics, and let the backends not worry
about how to format the time fields. This should help to
prevent any future backend modifications or additions from
accidental incorrect time formatting.
Changed ListObjectsV2 and ListObjects actions return types from
*s3.ListObjects(V2)Output to s3response.ListObjects(V2)Result.
Changed the listing objects timestamp to RFC3339 to match AWS
S3 objects timestamp.
Fixes#752
The AWS spec for the create multipart upload response is:
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult>
<Bucket>string</Bucket>
<Key>string</Key>
<UploadId>string</UploadId>
</InitiateMultipartUploadResult>
So we need the return type to marshal to this xml format.
The container URL was adding an extra / when the service URL ended
with /. This results in an invalid container URI error from Azure.
This also fixes the container metadata keys to not include "-" that
the Azure service disallows. The metadata values are base64 encoded
to prevent any special chacter handling on Azure service side.
The content type is set for uploads consistent with S3 expectations.
Fixes#736
Bucket ACLs are now disabled by default the same as AWS.
By default the object ownership is BucketOwnerEnforced
which means that bucket ACLs are disabled. If one attempts
to set bucket ACL the following error is returned both in
the gateway and on AWS:
ErrAclNotSupported: {
Code: "AccessControlListNotSupported",
Description: "The bucket does not allow ACLs",
HTTPStatusCode: http.StatusBadRequest,
},
ACls can be enabled with PutBucketOwnershipControls
Changed bucket canned ACL translation
New backend interface methods:
PutBucketOwnershipControls
GetBucketOwnershipControls
DeleteBucketOwnershipControls
Added these to metrics
Added support to enable object lock on bucket creation in posix and azure
backends.
Implemented the logic to add object legal hold and retention on object creation
in azure and posix backends.
Added the functionality for HeadObject to return object lock related headers.
Added integration tests for these features.
This adds the default ACL to the CreateBucket backend method so
that the backend doesn't need to know how to construct and ACL.
This also moves the s3proxy ACLs to a tag key/value because the
gateway ACLs are not the same accounts as the backend s3 server.
TODO: we may need to mask this tag key/value if we add support
for the Get/PutBucketTagging API.
The azure sdk doesnt use a separate function to initialize a
multipart upload, so CreateMultipartUpload becomes a no-op.
But we still need to have it return success so that clients
wont get an unexpected error.
