Commit Graph
209 Commits
Author SHA1 Message Date
niksis02 96af2b6471 fix: Fixes GetObject and UploadPartCopy actions data range parsing.
Fixes #1004
Fixes #1122
Fixes #1120

Separates `GetObject` and `UploadPartCopy` range parsing/validation.

`GetObject` returns a successful response if acceptRange is invalid.
Adjusts the range upper limit, if it exceeds the actual objects size for `GetObject`.
Corrects the `ContentRange` in the `GetObject` response.

Fixes the `UploadPartCopy` action copy source range parsing/validation.
`UploadPartCopy` returns `InvalidArgument` if the copy source range is not valid.
2025-03-08 01:39:21 +04:00
niksis02 4b0dd64836 fix: removes ETag quotes for in GetObjectAttributes response for azure and posix backends 2025-03-06 00:28:55 +04:00
Ben McClellandandGitHub 6361d7780b Merge pull request #1103 from versity/ben/read-only-fs
feat: return method not allowed when uploading to read only fs
2025-03-05 09:59:51 -08:00
niksis02 d59ee87b10 fix: Fixes ListObjects direcotry objects listing issue. Fixes DeleteObject directory objects parents deletion 2025-03-04 21:29:08 +04:00
niksis02 92dee62298 fix: Allows checksum-type to be empty in CompleteMultipartUpload input parameters 2025-03-03 22:14:47 +04:00
Ben McClelland 003719cc74 feat: return method not allowed when uploading to read only fs
Instead of an internal server error, we should be returning
method not allowed when trying to upload to a read only
filesystem or make other modifications that are expected to
fail. This will give clearer feedback to the clients
that this is not expected to work.

Fixes #1062
2025-03-01 11:00:12 -08:00
John BerthelsandBen McClelland b4f2b0ddaf fix: admin list-buckets with bucketlinks option enabled
- extract method to find top-level dirs, optionally including links to
  dirs
- rework ListBuckets and ListBucketsAndOwners to use FileInfo rather
  than DirEntry

Fixes #1081
2025-02-27 16:31:59 -08:00
niksis02 d13791f5ce fix: Adds quotes for object Etags 2025-02-28 00:43:51 +04:00
niksis02 fcafb57abc feat: Implements checksums for ListObjectVersions and CopyObject actions. 2025-02-27 00:24:38 +04:00
niksis02 1bb07fd78e fix: Changes HeadObject returned error type from MethodNotAllowed to NotFound if the latest object version is a delete marker. 2025-02-24 20:44:26 +04:00
Ben McClellandandGitHub e256a1aa0d Merge pull request #1079 from versity/fix/versioning-nested-object-deletion
fix: Adds removeParents in DeleteObject in case of deleting an object…
2025-02-20 18:01:20 -08:00
niksis02 a30b8c2320 fix: Adds removeParents in DeleteObject in case of deleting an object in nested directories 2025-02-21 00:20:49 +04:00
Ben McClellandandGitHub 549289c581 Merge pull request #1078 from versity/fix/listparts-issues
ListParts refactoring
2025-02-20 08:12:36 -08:00
niksis02 e5811e4ce7 fix: Fixes the entity limiter validation for ListObjects(V2), ListParts, ListMultipartUploads, ListBuckets actions 2025-02-20 15:45:42 +04:00
Ben McClellandandGitHub 38366b88b0 Merge pull request #1077 from versity/feat/complete-mp-mpu-object-size
x-amz-mp-object-size header for CompleteMultipartUpload
2025-02-19 16:13:25 -08:00
niksis02 173518278e fix: refactoring the checksum implementation by avoiding many if conditions and making the code more readable 2025-02-19 23:59:34 +04:00
niksis02 60151a70b0 fix: Fixes the typ in retrieveChecksums function in posix 2025-02-19 22:07:25 +04:00
niksis02 64a72a2dee feat: Adds 'x-amz-mp-object-size' request header support for CompleteMultipartUpload 2025-02-19 19:26:03 +04:00
niksis02 3cae3fced9 fix: Adds PartNumber validation for CompleteMultipartUploads parts. Adds a check to validate the parts order to be ascending. 2025-02-18 20:02:01 +04:00
niksis02 132d0ae631 feat: Adds the CRC64NVME checksum support in the gateway. Adds checksum-type support for the checksum implementation 2025-02-16 17:10:06 +04:00
niksis02 6956757557 feat: Integrates object integrity checksums(CRC32, CRC32C, SHA1, SHA256) into the gateway 2025-02-14 14:14:00 +04:00
niksis02 b5b418e4b3 fix: Adds the minimum allowed size(5 Mib) check for mp parts sizes in CompleteMultipartUpload 2025-02-03 21:49:15 +04:00
Ben McClelland 4025172897 feat: add optional sidecar files for metadata
This adds the option to store metadata for objects and buckets
within a specified directory:
bucket: <sidecar dir>/<bucket>/meta/<attribute>
object: <sidecar dir>/bucket/<object>/meta/<attribute>

Example invocation:
./versitygw -a myaccess -s mysecret posix --sidecar /tmp/sidecar /tmp/gw

The attributes are stored by name within the hidden directory.
2025-01-08 15:47:31 -08:00
Ben McClelland db305142f1 fix: return better error when trying to delete non empty directory object
The posix backend will return ENOTEMPTY when trying to delete a
directory that is not empty. This normally would run successfully
on object systems. So we need to create another non-standard error
for this case. We mainly just don't want to return InternalError
for this case.

Fixes #946
2024-11-14 22:33:08 -08:00
Ben McClelland 0312a1e3dc fix: internal server error when object parent dir is a file
The fileystem will return ENOTDIR if we try to access a file path
where a parent directory within the path already exists as a file.
In this case we need to return a standard 404 no such key since
the request object does not exist within the filesytem.

Fixes #942
2024-11-08 08:21:14 -08:00
jonaustin09 06e2f2183d fix: Changes GetObjectAttributes action xml encoding root element to GetObjectAttributesResponse. Adds input validation for x-amz-object-attributes header. Adds x-amz-delete-marker and x-maz-version-id headers for GetObjectAttributes action. Adds VersionId in HeadObject response, if it's not specified in the request 2024-10-30 15:42:15 -04:00
Ryan HilemanandBen McClelland e7a6ce214b feat: make zero-copy GetObject possible via sendfile
From #919, This provides the *os.File handle to io.Copy() for the
case where the full file range is requested. This prevents hiding
the *os.File type for io.Copy() optimizations.

This still requires the change to valyala/fasthttp#1889 to expose
the net.Conn similarly to enable the linux sendfile optimization.
2024-10-28 15:13:30 -07:00
jonaustin09 4d6ec783bf feat: Implements pagination for ListBuckets 2024-10-28 16:26:08 -04:00
jonaustin09 858236150c fix: Fixes multiple null versionId delete markers creation with DeleteObject 2024-10-21 16:10:34 -04:00
Ben McClellandandGitHub 32f2571056 Merge pull request #901 from versity/fix/versioning-bucket-cleanup
fix: Prevents bucket deletion when it contains object versions by ret…
2024-10-18 15:28:22 -07:00
jonaustin09 c803af4688 fix: Prevents bucket deletion when it contains object versions by returning ErrVersionedBucketNotEmpty error. Enabled object deletion with versionId and delete markers creation with DeleteObject when the versioning status is Suspended 2024-10-18 15:36:52 -04:00
Ben McClelland 2c713c58f9 feat: add option to configure mode permissions on new directories
We had 0755 hard coded for newly created directories before. This
adds a user option to configure what the default mode permissions
should be for newly created directories.

Fixes #878
2024-10-16 14:31:03 -07:00
Ben McClellandandGitHub a78c826d0f Merge pull request #897 from versity/fix/getobject-null-versionid-obj
fix: Resolves the null object version get issue from posix directory
2024-10-16 13:38:27 -07:00
jonaustin09 cdcbc9c9cb fix: Resolves the null object version get issue from posix directory 2024-10-16 15:58:40 -04:00
jonaustin09 608e380e36 fix: Fixes the double entry issue for single null versionId object version in ListObjectVersions 2024-10-16 12:05:41 -04:00
jonaustin09 f061deb146 fix: Returns NoSuchKey error for GetObject, if verionId is not specified and the latest version is a delete marker 2024-10-15 15:48:18 -04:00
jonaustin09 16e8134e80 fix: Adds bucket object lock status check in GetObjectLegalHold and GetObjectRetention actions 2024-10-14 15:04:11 -04:00
jonaustin09 f03d600b56 fix: The implementation handles null versionId objects for versioning enabled buckets within the versioning directory 2024-10-11 15:39:49 -04:00
Ben McClelland b7a2e8a2c3 fix: unexpected errors during upload races
This fixes the cases for racing uploads with the same object names.
Before we were making some bad assumptions about what would cause
an error when trying to link/rename the final object name into
the namespace, but missed the case that another upload for the
same name could be racing with this upload and causing an incorrect
error.

This also changes the order of setting metadata to prevent
accidental setting of metadata for the current upload to another
racing upload.

This also fix auth.CheckObjectAccess() when objects are removed
while this runs.

Fixes #854
2024-10-07 17:24:44 -07:00
Ben McClellandandGitHub d2b0d24520 Merge pull request #876 from versity/fix/put-bucket-versioning-suspended
fix: Removed ObjectLockConfigurationNotFoundError, when attempting to…
2024-10-07 16:40:01 -07:00
jonaustin09 b5b592c683 fix: Removed ObjectLockConfigurationNotFoundError, when attempting to set bucket versioning status to suspended 2024-10-07 18:26:33 -04:00
jonaustin09 b39b5e2373 fix: Fixed object versions listing with ListObjectVersions when versioning is not configured for the gateway 2024-10-07 18:16:11 -04:00
jonaustin09 768983be34 fix: Added VersioningNotConfigured error in Put/GetBucketVersioning acitons 2024-10-04 20:52:31 -04:00
jonaustin09 e502a15306 fix: Added the implementation to automatically enable bucket versioning when enabling object lock. Added error response when attempting to set bucket versioning status to Suspended when object lock is enabled 2024-10-01 16:59:04 -04:00
jonaustin09 7d368be82e feat: Implemented object locking for object versions 2024-09-30 17:26:49 -04:00
Ben McClellandandGitHub 5c40de231d Merge pull request #849 from versity/fix/get-bucket-versioning-empty-response
fix: Changed the GetBucketVersioning action return type, to return em…
2024-09-27 19:12:31 -07:00
jonaustin09 7b5765bd59 fix: Changed the GetBucketVersioning action return type, to return empty result for unset versioning configuration 2024-09-27 18:14:53 -04:00
jonaustin09 82592d97f4 fix: Added the versionId prop in GetObject response, when attempting to get the latest object version without specifying the versionId 2024-09-27 17:52:02 -04:00
Ben McClellandandGitHub 44d51b787d Merge pull request #847 from versity/fix/versioning-delete_object-nonexistent
fix: Modified DeleteObject error handling to return a successful resp…
2024-09-26 14:45:45 -07:00
jonaustin09 3dc654eb11 fix: Modified DeleteObject error handling to return a successful response when versionId is not specified, and to return InvalidVersionId error when it is specified, in cases where versioning is enabled. 2024-09-26 16:41:35 -04:00