Fixes#1909
Previously, the mapping between object metadata and posix object was as follows: for each metadata key, we stored a separate xattr with the `user.X-Amz-Meta.<key>` prefix. This resulted in syscall overhead when storing and deleting large numbers of metadata keys.
In addition, very long metadata keys caused failures because most posix filesystems limit xattr key lengths to 127–255 bytes, while S3 does not enforce such a per-key limit.
The logic has now been changed so that all object metadata is stored in a single xattr, `user.metadata`, as a JSON key/value object. For backward compatibility, metadata GET operations still fall back to the old mechanism (`metadata key -> xattr key`) when `user.metadata` is not present.
A new CLI utility has been added to convert all legacy object metadata to the new metadata format within the provided directory.
**Example usage:**
```
versitygw utils convert-xattr-metadata path/to/bucket
```
or
```
versitygw utils cxm path/to/bucket
```
It is recommended to run this command on bucket directories to convert all legacy metadata for every object in the bucket.
The fifth parameter of request() is useAdminEndpoint (boolean), but
putBucketPolicy was passing a Content-Type header object instead.
This caused useAdminEndpoint to be truthy and contentType to default
to 'application/xml' instead of 'application/json'.
Fixed by passing false for useAdminEndpoint and 'application/json'
as the contentType argument.
Fixes#1928
Use net.JoinHostPort instead of manual string formatting in
buildServiceURLs so IPv6 addresses are properly wrapped in brackets.
This fixes malformed URLs like "http://::1:7070" being presented as
the default server URL in the WebUI login form when listening on an
IPv6 address; they now correctly render as "http://[::1]:7070".
Fixes#1926
Fixes#1917
When the `--webui` flag uses `localhost:<port>`, the hostname `localhost` resolves to both IPv6 and IPv4 addresses. Previously, the Fiber web server was initialized without an explicit network setting, and Fiber defaults to `tcp4` (IPv4-only). Because `tcp4` cannot bind to IPv6 addresses, any resolved IPv6 address (e.g., `::1`) was treated as invalid for the listener.
The network mode is now changed to `tcp`, which enables dual-stack behavior and allows binding to both IPv4 and IPv6 addresses.
The logic to return a `NotImplemented` error on object upload operations, when any ACL header is present has been removed. Now all object ACL headers are by default ignored. The `-noacl` flag is preserved to disabled bucket ACLs.
**Testing**
The Put/Get object ACL tests are moved to `NotImplemented` integration tests group as a default gateway behavior. The existing `_acl_not_supported` tests are modified to expect no error, when ACLs are used on object uploads.
Closes#1857
Adds object Tagging support for directory objects in `PutObject` posix. Updates the integration tests to test object metadata and tagging both for file and directory objects.
In sidecar mode, the three StoreAttribute/storeChecksums calls after the
copy loop were using objPath (the source object path) instead of the
destination part's bucket and partPath. This caused checksums and the
internal part-crc64nvme to be written under the source object's sidecar
directory, making them unresolvable when CompleteMultipartUploadWithCopy
tried to retrieve them. All three stores now use *upi.Bucket and partPath,
consistent with the etag store directly below them.
Replace the length-comparison condition with an explicit predicate that
is both more readable and correctly scoped: skip only when the visited
directory is a strict ancestor of the specified prefix (not a descendant
and not when prefix is empty).
Adds tests from the original bug report (#1864) to verify the fix and
guard against future regressions.
Closes#1847
This PR introduces a global optional gateway CLI flag `--disable-acl` (`VGW_DISABLE_ACL`) to disable ACL handling. When this flag is enabled, the gateway ignores all ACL-related headers, particularly in `CreateBucket`, `PutObject`, `CopyObject`, and `CreateMultipartUpload`.
`GetBucketAcl` behavior is unchanged simply returning the bucket ACL config.
There's no change in object ACL actions(`PutObjectACL`, `GetObjectACL`). They return a`NotImplemented` error as before.
A new custom error is added for PutBucketAcl calls when ACLs are disabled at the gateway level. Its HTTP status code and error code match AWS S3’s behavior, with only a slightly different error message.
In the access-control checker, ACL evaluation is fully bypassed. If ACLs are disabled only the bucket owner gets access to the bucket and all grantee checks are ignored.
The PR also includes minor refactoring of the S3 API server and router. The growing list of parameters passed to the router’s Init method has been consolidated into fields within the router struct, initialized during router construction. Parameters not needed by the S3 server are no longer stored in the server configuration and are instead forwarded directly to the router.
This PR optimizes multipart upload checksum handling. When a checksum algorithm/type is specified at multipart-upload initiation, each `UploadPart` request computes, validates, and stores the corresponding part checksum. During `CompleteMultipartUpload`, the final checksum is derived either via composite checksum calculation or by composing the CRC-family checksums.
When **no** checksum algorithm is specified during multipart-upload initiation, each `UploadPart` may supply a different checksum algorithm for data-integrity verification. To support this scenario, a new mechanism has been implemented: for every `UploadPart`, a **crc64nvme** checksum is always computed.
* If the client uses crc64nvme for the part upload, a single hash reader is used.
* Otherwise, two hash readers are used—one for crc64nvme and one for the user-provided checksum.
The crc64nvme value is stored in part xattrs under `user.part-crc64nvme` and later used during `CompleteMultipartUpload` as a composable checksum source.
In `CompleteMultipartUpload`, the hash reader is entirely removed; the gateway no longer re-reads part data to compute the final checksum. The logic now follows two distinct paths:
1. **Checksum algorithm/type specified at MP initiation**
* All required per-part checksums have already been stored.
* If the checksum type is `FULL_OBJECT`, the gateway uses the composable path.
* If the type is `COMPOSITE`, the gateway follows the checksum-combining path.
2. **No checksum algorithm specified at MP initiation**
* The gateway loads the stored per-part `crc64nvme` values and composes them to compute the final checksum.
The previous `composableCRC` check has been removed because all `FULL_OBJECT` algorithms are inherently composable (`crc32`, `crc32c`, `crc64nvme`). Validation now relies solely on `checksum.Type`.
Previously, if no object checksum type/algorithm was specified when initiating a multipart upload, the CompleteMultipartUpload request would compute the final object’s CRC64NVME checksum but not persist it. This logic has now been fixed, and in the scenario described above the checksum is stored on the final object. There should no longer be any case where a CompleteMultipartUpload request finishes without persisting the final object checksum.
This brings scoutfs in-line with the posix concurrency limiter.
This fixes a hang with scoutfs due to not correctly initializing
the concurrency in posix leading to a concurrency of 0 allowed.
This also adds a sane default to the posix concurrency when not
initialized.
