49 Commits

Author SHA1 Message Date
niksis02
d0158420ee fix: fixes CreateBucket LocationConstraint validation
Fixes #1654
Fixes #1644

CreateBucket `LocationConstraint` rejects empty values with an `InvalidLocationConstraint` error.
The `us-east-1` `LocationConstraint` is considered invalid because it is the default value and must not be present in the `CreateBucketConfiguration` request body.

This PR fixes both issues by returning `InvalidLocationConstraint` in both cases.
2026-01-03 23:21:19 +04:00
niksis02
9bde1ddb3a feat: implements tagging support for CreateBucket
Closes #1595

This implementation diverges from AWS S3 behavior. The `CreateBucket` request body is no longer ignored. Based on the S3 request body schema, the gateway parses only the `LocationConstraint` and `Tags` fields. If the `LocationConstraint` does not match the gateway’s region, it returns an `InvalidLocationConstraint` error.

In AWS S3, tagging during bucket creation is supported only for directory buckets. The gateway extends this support to general-purpose buckets.

If the request body is malformed, the gateway returns a `MalformedXML` error.
2025-10-31 00:59:56 +04:00
niksis02
818e91ebde feat: adds x-amz-object-size in PutObject response headers
Closes #1518

Adds the `x-amz-object-size` header to the `PutObject` response, indicating the size of the uploaded object. This change is applied to the POSIX, Azure, and S3 proxy backends.
2025-09-05 21:40:46 +04:00
Ben McClelland
58117c011a feat: add get bucket location frontend handlers
GetBucketLocation is being deprecated by AWS, but is still used
by some clients. We don't need any backend handlers for this since
the region is managed by the frontend. All we need is to test for
bucket existence, so we can use HeadBucket for this.

Fixes #1499
2025-08-30 12:29:26 -07:00
niksis02
7dc213e68e fix: adds bucket region in ListBuckets result
Fixes #1374

Hardcodes the gateway region for each bucket entry in `ListBuckets` result as bucket region.
2025-07-26 00:45:18 +04:00
niksis02
67d0750ee0 feat: adds unit tests for object DELETE and POST operations 2025-07-22 20:55:22 +04:00
niksis02
afbcbcac13 fix: fixes all the available actions date xml marshalling for response body.
Fixes the response body parsing for all available actions to correctly parse date fields (e.g., `LastModified`) into the correct format.
2025-05-13 23:59:59 +04:00
Ben McClelland
d3bcd8ffc5 Merge pull request #1289 from versity/sis/copy-object-date
fix: fixes the LastModified date formatting in CopyObject result.
2025-05-12 13:15:47 -07:00
niksis02
323717bcf1 fix: fixes the LastModified date formatting in CopyObject result.
Fixes #1276

Creates the custom `s3response.CopyObjectOutput` type to handle the `LastModified` date property formatting correctly. It uses `time.RFC3339` to format the date to match the format that s3 uses.
2025-05-12 23:30:47 +04:00
Nick Jones
a022c3bdb6 mount-s3 has strict XML parsing which doesn't like receiving this tag empty 2025-05-12 20:59:26 +12:00
Ben McClelland
9244e9100d fix: xml response field names for complete multipart upload
The xml encoding for the s3.CompleteMultipartUploadOutput response
type was not producing exactly the right field names for the
expected complete multipart upload result.

This change follows the pattern we have had to do for other xml
responses to create our own type that will encode better to the
expected response.

This will change the backend.Backend interface, so plugins and
other backends will have to make the corresponding changes.
2025-04-30 14:36:48 -07:00
niksis02
2b7a71568c fix: Changes the GetObjectLegalHold result root tag to LegalHold
Fixes #1193

Changes the xml root tag element to `LegalHold` in `GetObjectLegalHold` response.
2025-04-08 18:26:26 +04:00
niksis02
cfb2d6d87d feat: Implements object meta properties for CopyObject in azure and posix backends.
Fixes #998
Closes #1125
Closes #1126
Closes #1127

Implements objects meta properties(Content-Disposition, Content-Language, Content-Encoding, Cache-Control, Expires) and tagging besed on the directives(metadata, tagging) in CopyObject in posix and azure backends. The properties/tagging should be coppied from the source object if "COPY" directive is provided and it should be replaced otherwise.

Changes the object copy principle in azure: instead of using the `CopyFromURL` method from azure sdk, it first loads the object then creates one, to be able to compare and store the meta properties.
2025-03-17 09:37:05 -07:00
niksis02
65261a9753 feat: Adds the Content-Disposition, Content-Language, Cache-Control and Expires object meta properties support in the gateway.
Closes #1128

Adds `Content-Disposition`, `Content-Language`, `Cache-Control` and `Expires` object meta properties support in posix and azure backends.
Changes the `PutObject` and `CreateMultipartUpload` actions backend input type to custom `s3response` types to be able to store `Expires` as any string.
2025-03-12 16:01:56 +04:00
niksis02
132d0ae631 feat: Adds the CRC64NVME checksum support in the gateway. Adds checksum-type support for the checksum implementation 2025-02-16 17:10:06 +04:00
niksis02
6956757557 feat: Integrates object integrity checksums(CRC32, CRC32C, SHA1, SHA256) into the gateway 2025-02-14 14:14:00 +04:00
jonaustin09
06e2f2183d fix: Changes GetObjectAttributes action xml encoding root element to GetObjectAttributesResponse. Adds input validation for x-amz-object-attributes header. Adds x-amz-delete-marker and x-maz-version-id headers for GetObjectAttributes action. Adds VersionId in HeadObject response, if it's not specified in the request 2024-10-30 15:42:15 -04:00
jonaustin09
4d6ec783bf feat: Implements pagination for ListBuckets 2024-10-28 16:26:08 -04:00
jonaustin09
c6359a7050 feat: Refactoring admin APIs: changes i/o data transfer encoding to xml, implements traditional aws-like error handling, adds admin role checker middleware. Refactoring admin CLI actions to handle aws-like error responses 2024-10-25 11:40:23 -04:00
jonaustin09
b39b5e2373 fix: Fixed object versions listing with ListObjectVersions when versioning is not configured for the gateway 2024-10-07 18:16:11 -04:00
jonaustin09
b0aee40f21 fix: Implemented the AmzDate type to handle iso8601 date parsing and validation. Used it to parse/validate the RetainUntilDate prop in PutObjectRetention data input 2024-10-03 13:13:18 -04:00
jonaustin09
7b5765bd59 fix: Changed the GetBucketVersioning action return type, to return empty result for unset versioning configuration 2024-09-27 18:14:53 -04:00
jonaustin09
800cf62209 feat: Implemented object versioning for multipart uploads. Implemented integration tests for the versioning implementation for multipart uploads 2024-09-19 13:29:19 -07:00
jonaustin09
8252ecd452 feat: basic logic implementation of bucket object versioning in posix backend
New posix backend option --versioning-dir will enable storing object versions
in specified directory.
2024-09-18 13:04:34 -07:00
Ben McClelland
9dbfaeed0c fix: time locale utc in responses
Change all response times to UTC. Fix bucket create date time
formatting. This was using the incorrct xml field name before.
2024-09-12 19:17:46 -07:00
Ben McClelland
3bf8b296d8 fix: format timestamp responses with RFC3339TimeFormat
This fixes the rest of the s3reponse types to marshall the time
field in RFC3339TimeFormat when MarshalXML called.

Fixes #782
2024-09-07 12:29:09 -07:00
Ben McClelland
e1c2945fb0 fix: add XMLName to InitiateMultipartUploadResult for consistency 2024-08-27 15:32:07 -07:00
Jon Austin
d79f978df9 feat: Added the standard storage class to all the available get/list actions responses in posix. (#765) 2024-08-27 15:28:40 -07:00
Ben McClelland
ee202b76f3 fix: move RFC 3339 time formatting to s3response
It is better if we let the s3response module handle the xml
formatting spec specifics, and let the backends not worry
about how to format the time fields. This should help to
prevent any future backend modifications or additions from
accidental incorrect time formatting.
2024-08-26 21:08:24 -07:00
jonaustin09
684ab2371b fix: Changed ListObjects and ListObjectsV2 actions return types
Changed ListObjectsV2 and ListObjects actions return types from
*s3.ListObjects(V2)Output to s3response.ListObjects(V2)Result.

Changed the listing objects timestamp to RFC3339 to match AWS
S3 objects timestamp.

Fixes #752
2024-08-26 15:46:45 -07:00
jonaustin09
cc3c62cd9d fix: Change CreateMultipartUpload return type to match expected xml response
The AWS spec for the create multipart upload response is:
<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult>
   <Bucket>string</Bucket>
   <Key>string</Key>
   <UploadId>string</UploadId>
</InitiateMultipartUploadResult>

So we need the return type to marshal to this xml format.
2024-08-21 14:49:39 -07:00
jonaustin09
7545e6236c feat: Implement bucket ownership controls
Bucket ACLs are now disabled by default the same as AWS.
By default the object ownership is BucketOwnerEnforced
which means that bucket ACLs are disabled. If one attempts
to set bucket ACL the following error is returned both in
the gateway and on AWS:
	ErrAclNotSupported: {
		Code:           "AccessControlListNotSupported",
		Description:    "The bucket does not allow ACLs",
		HTTPStatusCode: http.StatusBadRequest,
	},

ACls can be enabled with PutBucketOwnershipControls

Changed bucket canned ACL translation

New backend interface methods:
PutBucketOwnershipControls
GetBucketOwnershipControls
DeleteBucketOwnershipControls

Added these to metrics
2024-06-28 21:03:09 -07:00
jonaustin09
0c3771ae2d feat: Added GetObjectAttributes actions implementation in posix, azure and s3 backends. Added integration tests for GetObjectAttributes action 2024-04-29 15:31:53 -04:00
Ben McClelland
0d8a4f5791 fix: remove namespace restrictions on tag xml input
Fixes #447. Previously we required XML namespace and got these
errors with this input:
DEBUG:  <Tagging><TagSet><Tag><Key>mykey</Key><Value>myvalue</Value></Tag></TagSet></Tagging>
DEBUG: expected element <Tagging> in name space http://s3.amazonaws.com/doc/2006-03-01/ but have no name space
2024-03-11 21:01:40 -07:00
Ben McClelland
f7655dab9b fix: delete object xml response should be DeleteResult instead of DeleteObjectsResult 2024-03-09 10:20:15 -08:00
Ben McClelland
0760467c3d fix: correct xml response encoding for list-buckets and tagging
fixes #395
2024-02-12 16:20:07 -08:00
jonaustin09
8cc89fa713 feat: Azure backend implementation 2023-12-29 21:55:32 -08:00
jonaustin09
4f8e4714ee fix: Fixes #286, Fixed SelectObjectContent CLI bug, added Content-Length header in response 2023-10-13 13:23:19 -04:00
jonaustin09
6ac69b3198 feat: Closes #217, Created an admin API and CLI action to list all the buckets and its owners as a table 2023-09-12 08:29:34 -04:00
jonaustin09
a2439264b2 fix: Fixes #195, fixed DeleteObjects action response structure 2023-08-24 18:37:01 +04:00
jonaustin09
009ceee748 feat: Added FE support for SelectObjectContent action 2023-08-02 00:08:28 +04:00
jonaustin09
97847735c8 fix: s3response action responses naming cleanup 2023-07-31 21:41:10 -07:00
Jon Austin
091375fa00 Issue 151 (#174)
* fix: Fixes #151. Fixed DeleteObjects action bugs: Corrected request body serialization type, added return type
2023-07-31 21:36:33 -07:00
Jon Austin
45e3c0922d Tag actions FE (#113)
* feat: Added get-object-tagging, put-object-tagging, delete-object-tagging actions in fe
2023-06-26 12:29:56 -07:00
Ben McClelland
ef06d11d7c fix: get simple multipart upload tests passing 2023-06-12 07:37:21 -07:00
jonaustin09
346a05b49a feat: removed s3 xsd schema 2023-05-17 00:46:21 +05:00
jonaustin09
6a3254c29f feat: add s3 xsd new schemas, create new routes
add: DeleteObjects new xsd schema
add: HeadObject, DeleteObjects api actions
2023-05-12 23:17:36 +04:00
jonaustin09
8c6e016109 feat: add s3 xsd new schemas, create new routes
add: GetBucketAcl, GetObjectAcl, GetObjectAttributes, HeadBucket, HeadObject new xsd schemas
add: GetBucketAcl, GetObjectAcl, HeadBucket api actions
2023-05-12 04:19:33 +04:00
Ben McClelland
f156a78dee fill out basic project layout 2023-05-05 17:16:59 -07:00