It is better if we let the s3response module handle the xml
formatting spec specifics, and let the backends not worry
about how to format the time fields. This should help to
prevent any future backend modifications or additions from
accidental incorrect time formatting.
Changed ListObjectsV2 and ListObjects actions return types from
*s3.ListObjects(V2)Output to s3response.ListObjects(V2)Result.
Changed the listing objects timestamp to RFC3339 to match AWS
S3 objects timestamp.
Fixes#752
The posix limits wont exactly match up with the AWS key length
limits because posix has component length limits as well as path
length limits.
This reponds with the aws compatible KeyTooLongError under these
conditions now.
Note that delete object returns success even in the error cases.
Fixes#755
The API hanlders and backend were stripping trailing "/" in object
paths. So if an object exists and a request came in for head/get/delete/copy
for that same name but with a trailing "/" indicating request should
be for directory object, the "/" would be stripped and the request
would be handlied for the incorrect file object.
This fix adds in checks to handle the case with the training "/"
in the request.
Fixes#709
For large directories, the treewalk can take longer than the
client request timeout. If the client times out the request
then we need to stop walking the filesystem and just return
the context error.
This should prevent the gateway from consuming system resources
uneccessarily after an incoming request is terminated.
This adds the ability to treat symlinks to directories at the top
level gateway directory as buckets the same as normal directories.
This could be a potential security issue allowing traversal into
other filesystems within the system, so is defaulted to off. This
can be enabled when specifically needed for both posix and scoutfs
backend systems.
Fixes#644
The restore object api request handler was incorrectly trying to
unmarshal the request body, but for the stadnard (all?) case the
request body is emtpy. We only need the bucket and opbject params
for now.
This also adds a fix to actually honor the enable glacier mode
in scoutfs.
Add meta.MetadataStorer compatibility to scoutfs so that scoutfs
is using the same interface as posix. This fixes the metadata
retrieval and adds the recently supported object lock compatibility
as well.
When fileystem quota exceeded, the gateway will now return the
error:
S3 error: 403 (QuotaExceeded):
Your request was denied due to quota exceeded.
This will help clients to better detect upload errors due to
quota exceeded.
Fixes#483
We had some duplicated code that we can bring into the backend
package so that we can remove duplications. This moves the mkdir
implementation into backend so that both posix and scoutfs can
call the same implementation.
The scoutfs backend is only supported on 64bit linux. This corrects
the build constraints to only supported linux arch, and prevents the
incompatible import for unspported arch.
We also need to adjust the body limit setting on 32 bit since this
is an int, and our default limit will overfow on 32 bit.
This adds a context to the backend interface calls so that the backend
can enable request cancellation. This change isn't acutally implementing
any backend handling, but just putting the pieces into place to pass the
context to the backend.
The aws cli will send range gets of an object with ranges like
the following:
bytes=0-8388607
bytes=8388608-16777215
bytes=16777216-25165823
bytes=25165824-
The last one with the end offset unspecified just means the rest of
the object. So this fixes that case where there is only one offset
in the range.
