Add meta.MetadataStorer compatibility to scoutfs so that scoutfs
is using the same interface as posix. This fixes the metadata
retrieval and adds the recently supported object lock compatibility
as well.
The complete multipart upload can be optimized in some cases
to not need to copy the full data from parts to the final
object file. If the filesystem supports it, there can be
optimizations to just clone exent references and not have to
actually copy the data.
For io.Copy() to make use of file_copy_range, we have to pass it
the *os.File file handles from the filesystem for the source and
destination. We were previously adding a layer of indirection
that was causing io.Copy() to fallback to full data copy. This
fixes the copy by exposing the underlying fd.
This also skips the falloc for the final object in complete
mutlipart upload, because some filesystems will be able to use
file_copy_range to optimize the copy and may not even need
new data allocations in the final object file.
Note that this only affects posix mode as scoutfs has special
handling for this case that is similar to but not compatible
with copy_file_range using a special ioctl.
Added support to enable object lock on bucket creation in posix and azure
backends.
Implemented the logic to add object legal hold and retention on object creation
in azure and posix backends.
Added the functionality for HeadObject to return object lock related headers.
Added integration tests for these features.
When fileystem quota exceeded, the gateway will now return the
error:
S3 error: 403 (QuotaExceeded):
Your request was denied due to quota exceeded.
This will help clients to better detect upload errors due to
quota exceeded.
Fixes#483
We had some duplicated code that we can bring into the backend
package so that we can remove duplications. This moves the mkdir
implementation into backend so that both posix and scoutfs can
call the same implementation.
This adds the default ACL to the CreateBucket backend method so
that the backend doesn't need to know how to construct and ACL.
This also moves the s3proxy ACLs to a tag key/value because the
gateway ACLs are not the same accounts as the backend s3 server.
TODO: we may need to mask this tag key/value if we add support
for the Get/PutBucketTagging API.
Since objects with trailing "/" are mapped to directories in the
posix filesystem, they must not contain data since there is no
place to store that data.
This checks both PutObject and CreateMultipartUpload for invalid
directory object types containing data.
This moves the internal iam service from the posix backend so
that we can start implementing new iam services right in the auth
module.
The internal iam service has same behavior as before, but now
must be enabled with the --iam-dir cli option.
New single user service is the default when no other iam service
is selected. This just runs the gateway in single user mode with
just the root account.
* fix: Object tag actions cleanup
* fix: Fixes#249, Changed ListObjects default max-keys from -1 to 1000
* fix: Fixes#250, Added support to provide a marker not from the objects list and list the objects after the provided marker in ListObjects(V2) actions
* feat: Closes#256, Addded a check step, to compare object metadatas and allow the copying to itself, if the metadata has been changed
* fix: Simplified range assignment in CopyObject posix function
