Ben McClelland
cfe367da99
chore: fix spelling typos
2024-06-10 20:01:28 -07:00
Ben McClelland
576dfc5884
fix: correct metadata, tags, and lock info for scoutfs multipart objects
...
Add meta.MetadataStorer compatibility to scoutfs so that scoutfs
is using the same interface as posix. This fixes the metadata
retrieval and adds the recently supported object lock compatibility
as well.
2024-06-10 17:57:07 -07:00
Ben McClelland
fe4c9dff76
fix: part file bounds checks in posix
2024-06-04 15:20:05 -07:00
Ben McClelland
a7110c28b6
Merge pull request #607 from versity/fix/get-obj-exc-range
...
GetObject exceeding range
2024-06-04 11:54:39 -07:00
Ben McClelland
a896b3660b
fix: remove deprecated WithEndpointResolver s3 client option
2024-06-04 11:09:29 -07:00
jonaustin09
0fb6bf6267
fix: Removed exceeding invalid range error when calling GetObject action
2024-06-04 13:25:56 -04:00
jonaustin09
d10ffd8707
fix: Fixed GetObject action invalid range error: when the same index of bytes is specified: bytes=0-0
2024-05-29 16:32:01 -04:00
jonaustin09
9e8458a09f
feat: Added integration tests for bypass governance retention functionality
2024-05-28 15:17:25 -04:00
jonaustin09
fb27e2703e
feat: Implemented to logic to bypass governance retention
2024-05-24 13:50:41 -04:00
Ben McClelland
ee0f14e07a
Merge pull request #585 from versity/fix/567-create-mp-missing-props
...
CreateMultipartUpload missing properties support
2024-05-22 14:22:20 -07:00
jonaustin09
43f509d971
fix: Added missing properties support for CreateMultipartUpload action: ContentType, ObjectLock, Tagging, Metadata
2024-05-22 12:16:55 -07:00
Ben McClelland
190dd8853c
fix: expose posix tmpfile fd to enable copy_file_range
...
The complete multipart upload can be optimized in some cases
to not need to copy the full data from parts to the final
object file. If the filesystem supports it, there can be
optimizations to just clone exent references and not have to
actually copy the data.
For io.Copy() to make use of file_copy_range, we have to pass it
the *os.File file handles from the filesystem for the source and
destination. We were previously adding a layer of indirection
that was causing io.Copy() to fallback to full data copy. This
fixes the copy by exposing the underlying fd.
This also skips the falloc for the final object in complete
mutlipart upload, because some filesystems will be able to use
file_copy_range to optimize the copy and may not even need
new data allocations in the final object file.
Note that this only affects posix mode as scoutfs has special
handling for this case that is similar to but not compatible
with copy_file_range using a special ioctl.
2024-05-21 16:16:00 -07:00
jonaustin09
b971467446
fix: Changed the logic to return BucketAlreadyOwnedByYou error when user tries to create an existing bucket owned by him
2024-05-20 12:07:33 -04:00
Brian Wheeler
bd649f8c46
Set the media type for directories
2024-05-16 15:35:12 -04:00
jonaustin09
5acf1f332a
fix: Fixes #555 , Added the logic to return InvalidBucketState when calling PutObjectLockConfiguration action on not object lock enabled bucket
2024-05-16 11:30:35 -04:00
Ben McClelland
f722f515ae
chore: add missing copyright headers to files
2024-05-06 16:16:31 -07:00
jonaustin09
481c9246c6
feat: HeadObject ation multipart upload case
2024-05-03 18:10:32 -04:00
jonaustin09
b4cd35f60b
feat: error refactoring and enable object lock in backends
...
Added support to enable object lock on bucket creation in posix and azure
backends.
Implemented the logic to add object legal hold and retention on object creation
in azure and posix backends.
Added the functionality for HeadObject to return object lock related headers.
Added integration tests for these features.
2024-05-02 15:23:48 -07:00
jonaustin09
0c3771ae2d
feat: Added GetObjectAttributes actions implementation in posix, azure and s3 backends. Added integration tests for GetObjectAttributes action
2024-04-29 15:31:53 -04:00
jonaustin09
5126aedeff
feat: Added bucket policy and object lock actions implementation in s3 proxy
2024-04-24 15:49:02 -04:00
jonaustin09
4a56d570ad
feat: Added object lock actions implementation in azure backend
2024-04-23 17:05:59 -04:00
jonaustin09
89755ea5aa
feat: Changed object lock actions interface to put/get []byte
2024-04-22 13:19:09 -07:00
jonaustin09
00476ef70c
feat: Closes #490 , Added integration tests for object lock actions
2024-04-22 13:13:40 -07:00
jonaustin09
fbaba0b944
feat: Added object WORM protection by object-lock feature from AWS with the following actions support: PutObjectLockConfiguration, GetObjectLockConfiguration, PutObjectRetention, GetObjectRetention, PutObjectLegalHold, GetObjectLegalHold
2024-04-22 13:13:40 -07:00
Ben McClelland
7ab0e3ebbe
Merge pull request #530 from versity/azure-policy-actions
...
Azure bucket policy actions
2024-04-20 10:00:23 -07:00
jonaustin09
5c835c5c74
feat: Implemented GetBucketPolicy, PutBucketPolicy action in azure backend
2024-04-19 16:36:42 -04:00
Ben McClelland
4429570388
fix: use xattr.ENOATTR check for posix xattrs
...
The xattr package has a more universal error type for xattrs
not existing. Use this for better platform compatibility.
This also adds the xattr.XATTR_SUPPORTED check for platform
xattr suport in xattr package.
Fixes #527
2024-04-18 18:20:43 -07:00
Ben McClelland
c81403fe90
feat: add metadata storage abstraction layer
...
Closes #511 . This adds an abstraction layer to the metadata
storage to allow for future non-xattr metadata storage
implementations.
2024-04-15 13:57:31 -07:00
Ben McClelland
aeea61544b
feat: add s3err QuotaExceeded for posix/scoutfs
...
When fileystem quota exceeded, the gateway will now return the
error:
S3 error: 403 (QuotaExceeded):
Your request was denied due to quota exceeded.
This will help clients to better detect upload errors due to
quota exceeded.
Fixes #483
2024-04-06 11:53:40 -07:00
Ben McClelland
c228bbfd79
feat: add option to change ownership of dir/files to acct settings
...
When enabled, any new directories or files created through the
gateway will change ownership based on the account uid/gid.
Fixes #238 .
2024-03-30 22:26:29 -07:00
Ben McClelland
f72d6349fe
feat: consolidate scoutfs/posix mkdir in backend
...
We had some duplicated code that we can bring into the backend
package so that we can remove duplications. This moves the mkdir
implementation into backend so that both posix and scoutfs can
call the same implementation.
2024-03-30 22:26:29 -07:00
jonaustin09
e6203c5765
feat: Closes #441 , Added access control integration tests, fixed some bugs in bucket policy and acl access checking flow
2024-03-28 14:52:56 -04:00
Ben McClelland
4d02ac21c5
Merge pull request #460 from versity/bucket-policies
...
Bucket Policy
2024-03-25 15:19:46 -07:00
jonaustin09
754c221c4d
feat: Added bucket policy access verifier function implementation. Changed the default behaviour of bucket ACLs. Fixed the supported actions list for bucket policy. Implemented Copy* actions access checker function
2024-03-25 16:00:35 -04:00
Ben McClelland
0ea5db228d
fix: scoutfs return correct ContentRange for get request
2024-03-25 09:36:38 -07:00
Ben McClelland
7ff89af6b5
fix: scoutfs backend needs to dereference types.CompletedPart.PartNumber
...
There was a change a while back in the aws sdk that made these
pointers instead of ints. We somehow missed the pointer deref in
the scoutfs backend.
2024-03-25 08:29:03 -07:00
jonaustin09
af641e5368
feat: Added integration test cases for Put/Get/DeleteBucketPolicy actions. Made some bug fixes in these actions implementations
2024-03-20 17:31:52 -04:00
jonaustin09
d469a72213
feat: Implemented Put/Get/DeletBucketPolicy s3 actions in posix backend. Implemented policy document validation function
2024-03-15 15:47:10 -04:00
Ben McClelland
8144d90e25
feat: check for xattr support on posix init
2024-03-14 08:16:52 -07:00
Ben McClelland
d4d511cf98
feat: compile support for 32bit and bsd platforms
2024-03-11 09:42:36 -07:00
Ben McClelland
5b2beb8fc0
Merge pull request #443 from versity/ben/cleanup
...
chore: cleanup redundant nil checks and unused variable args
2024-03-11 09:01:17 -07:00
Ben McClelland
39663724a6
chore: cleanup redundant nil checks and unused variable args
2024-03-09 10:34:04 -08:00
Ben McClelland
f7655dab9b
fix: delete object xml response should be DeleteResult instead of DeleteObjectsResult
2024-03-09 10:20:15 -08:00
jonaustin09
d4f17bf32f
feat: Added bucket policy actions implementation in FE
2024-03-06 13:56:29 -05:00
jonaustin09
e6852b3a99
feat: Closes 417, Added the following versioning related actions: PutBucketVersioning, GetBucketVersioning, ListObjectVersions. Added versionId support in FE for the following actions: GetObject, DeleteObject
2024-02-28 09:48:05 -05:00
jonaustin09
4d8352225b
feat: Added bucket tagging actions in azure backend
2024-02-15 15:23:55 -05:00
Jon Austin
0c056f935b
ListObjectsV2 start-after prop ( #388 )
...
* fix: Fixes #138 , Added StartAfter property in ListObjectsV2 action, added couple of integration tests for ListObjectsV2
2024-02-01 11:04:52 -08:00
Ben McClelland
68794518af
fix: remove special proxy handling for bucket acls in posix backend
2024-01-31 10:10:12 -08:00
jonaustin09
3cce3a5201
feat: Added unit and integration test cases for posix bucket tagging related actions
2024-01-31 10:09:48 -08:00
jonaustin09
d70ea61830
feat: Added the following actions support in posix backend: PutBucketTagging, GetBucketTagging, DeleteBucketTagging
2024-01-31 10:09:48 -08:00
