When versioning uses sidecar metadata, deleting the current object can leave
the delete-marker attribute behind after the data file is removed. A later
PUT or multipart completion using the same key then inherits that stale
marker and the object remains hidden from the normal object view.
Clear the marker when publishing a replacement, including orphaned sidecar
and suspended-versioning cases, and remove all object metadata when deleting
the final version without an older version to restore.
Co-authored-by: Ben McClelland <ben.mcclelland@versity.com>
Closes#2322
The admin API could not be served under a path prefix. Admin requests are SigV4-signed over the full path, so a reverse proxy or Gateway API route cannot strip a prefix before forwarding. That made it impossible to serve the WebUI and the admin API under one hostname. The new `--admin-path-prefix` (`VGW_ADMIN_PATH_PREFIX`) option mounts the admin routes under a single-segment prefix such as `/admin`, on `--admin-port` or, when that is unset, on the S3 port. Admin clients include the prefix in their endpoint URL, which the admin CLI and the WebUI already support.
The prefix is limited to unreserved characters, because clients and the gateway encode other characters differently when signing. When the admin API shares the S3 port, the prefix must also differ from `--webui-s3-prefix`, otherwise the WebUI mount answers the admin requests. Auto-detected WebUI admin gateway URLs and the startup banner now include the prefix. The admin routes served on the S3 port now reuse `S3AdminRouter` instead of a duplicated route list. This also stops the standalone admin server from recording bucket creation under the `ActionAdminListBuckets` action.
The Helm chart now exposes `webui.pathPrefix`, `webui.s3Prefix` and `admin.pathPrefix`. The WebUI gateway lists are also passed when only `webui.s3Prefix` is set, since the WebUI hosted on the S3 port does not require `webui.enabled`.
Internal IAM data can now live on its own volume. `iam.dir` sets the IAM directory, and `iam.persistence` creates or references a dedicated PVC when `iam.enabled` is true and `iam.type` is `internal`. Otherwise IAM data stays in the `iam` subdirectory of the backend data volume, so existing releases are unaffected.
The chart now rejects overlapping storage directories. `iam.dir` must be an absolute path outside `/mnt/data`, `gateway.backend.sidecarDir` and `gateway.backend.versioningDir`, and those three directories must not overlap each other. Deleting a bucket removes `<dir>/<bucket>` from the sidecar and versioning directories, so with nested directories, deleting a suitably named bucket could wipe IAM accounts, object versions or the entire backend data directory.
A posix gateway serving a preexisting dataset treats every directory under the root as a bucket. But the bucket checks only stat the path, so a regular file, FIFO or symlink at the root passed for a bucket. Only `ListBuckets` skipped these entries.
With a root-level file as the bucket, `HeadBucket`, `ListObjects`, `ListObjectsV2`, `ListMultipartUploads`, `GetBucketAcl`, `GetBucketLocation`, `GetBucketVersioning` and the bucket config getters returned a successful response. `PutBucketAcl`, `PutBucketTagging`, `PutBucketPolicy`, `PutBucketCors`, ... and the admin `ChangeBucketOwner` also succeeded, storing gateway metadata on the file itself or in the sidecar directory. `DeleteObject` and `DeleteObjects` reported success. `CreateMultipartUpload`, `UploadPart`, `UploadPartCopy`, `ListParts`, `CompleteMultipartUpload`, `ListObjectVersions` and `DeleteBucket` returned `InternalError` with not a directory. `PutObject` and `CopyObject` returned `ObjectParentIsFile`. `GetObject`, `HeadObject`, `GetObjectAttributes` and the object tagging, legal hold and retention actions returned `NoSuchKey`, and `AbortMultipartUpload` returned `NoSuchUpload`. A symlink loop at the root returned `InternalError` for every action.
Bucket existence is now checked in one place, `doesBucketExist`, and `ScoutFS` uses it too through the exported `DoesBucketExist`. A bucket is a directory under the root, or a symlink to a directory when `--bucketlinks` is enabled. Any other entry returns `NoSuchBucket` for all bucket and object actions. `CreateBucket` on such an entry returns `BucketAlreadyExists` without reading its metadata.
This also enforces `--bucketlinks` for all actions. Before, it only affected `ListBuckets`: a symlinked directory was hidden from the listing, but every other action followed the link and used it as a bucket even with the option disabled. Without `--bucketlinks` such symlinks now return `NoSuchBucket`.
* fix: omit ObjectLockConfiguration Rule when there is no default retention
ParseBucketLockConfigurationOutput always set Rule, so a bucket with object lock enabled and no default retention answered GET ?object-lock with an empty <Rule></Rule>. AWS S3 omits the element, and the AWS SDK v2 clients that read the rule from it follow the empty element with a malformed request.
Fixes#2397
* test: cover GetObjectLockConfiguration with no default retention in the integration suite and drop the stale non-nil Rule expectation from the controller unit test
---------
Co-authored-by: Tung Lam <lamphamabtung96@gmail.com>
The azurite workflow waited a hardcoded `sleep 40` for the gateway container to compile and start listening before running tests. The container cold-builds `versitygw` from source on every run via `CompileDaemon`, and as the codebase grew that build increasingly exceeded the 40s budget, so tests started against a port with nothing
listening yet and failed with connection resets. Poll `http://127.0.0.1:7070/` with `curl` until it responds, bounded by a `timeout`, so the wait scales with actual build time instead of a stale constant.
The unsupported-platform cuObject stubs returned freshly constructed error
values, which let staticcheck prove caller error checks were always true on
non-Linux builds. Return shared package-level errors instead so the stubs
keep the same runtime behavior without triggering SA4023.
Gate native RDMA, cuObject, and cuobjclient implementations behind the
rdma build tag while keeping fallback stubs available for standard builds.
Preserve the separate cuobjclient_host configuration, clarify unsupported
platform errors, and update Makefile RDMA targets to pass the required tags
and disable VCS stamping.
Co-authored-by: Ben McClelland <ben.mcclelland@versity.com>
Closes#2304
Adds `versitygw utils convert-posix-dataset` (alias `cpd`), which makes a posix dataset not created by the gateway fully compatible with the posix backend. Every top level directory is treated as a bucket and gets a private ACL owned by `--access-key-id` (defaults to the root `--access`) and `BucketOwnerEnforced` object ownership; root level files are ignored. Metadata is stored in xattrs, or in `--sidecar-dir` when set.
`--calculate-etag` and `--checksum-algorithm` optionally compute the MD5 ETag and the `FULL_OBJECT` checksum of every object in a single streamed read. Objects are converted concurrently, tunable with `--concurrency` and `--read-buffer-size`. Existing bucket and object metadata is never overwritten.
`AssumeRoleWithWebIdentity` always fetched a provider's discovery document from `<provider url>/.well-known/openid-configuration`, so an identity provider that issues tokens naming a public issuer while serving its metadata and keys on a cluster-internal path could not be used: reaching it meant relaxing the endpoint checks for every registered provider. `--oidc-discovery-url` moves that one fetch to an operator-named endpoint, which is how keys can be looked up over an optimized private path while the tokens themselves stay verifiable from the public internet against the issuer alone, as the JWT spec requires.
The flag takes `<provider url>=<discovery url>` pairs, can be repeated once per provider, and is also read from `VGW_IAM_OIDC_DISCOVERY_URLS` as a comma-separated list; the Helm chart exposes the same list as `iamServer.oidc.discoveryUrls`. The discovery URL is fetched exactly as written, so it must carry the `/.well-known/openid-configuration` path when the provider serves it there. A malformed pair is rejected at startup rather than at the first assume-role call.
Only the fetch moves. The provider URL is still what a token's `iss` claim is matched against, the fetched document's own `issuer` field must still equal it, and the key set still comes from the `jwks_uri` that document publishes. A configured discovery endpoint is named by the operator at startup rather than by a request, so it and the `jwks_uri` it publishes waive the private-address check for that provider's fetch chain only, without `--oidc-allow-private-endpoints` and its far broader effect on every other provider. Transport rules are unchanged: a plaintext discovery URL still requires `--oidc-allow-insecure-transport`.
Thumbprint auto-fetch follows the override and pins the discovery endpoint's certificate chain, since that is the host every later fetch is verified against.
Uploading a large file through the Explorer gives no feedback at all
today — the toast only fires once at the start ("Uploading...") and
once at the end ("Uploaded"), so a multi-GB multipart upload looks
completely stalled to the user for however long it actually takes,
with no way to tell it apart from a hung connection.
Adds a small persistent widget (bottom-right, in the same visual
style as the existing toasts) that tracks bytes uploaded / total
bytes across the whole upload batch, updating after each part of a
multipart upload completes (or once a small single-PUT file
finishes). It disappears once the batch is done, right before the
existing success/failure toast.
No changes to any upload/network behavior — this only observes
progress already being made by the existing uploadMultipart/putObject
calls and renders it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Originated from #2302
When the Azure credential is valid but its RBAC role does not grant a data action, Azure answers with `403 AuthorizationPermissionMismatch`. This code had no mapping, so the gateway returned `500 InternalError` to the S3 client. A common case is `CompleteMultipartUpload` under a managed identity with `Storage Blob Data Contributor`: the Get Blob Tags call on the `.sgwtmp` multipart staging blob needs `blobs/tags/read`, which that role does not include.
`azErrToS3err` now maps `AuthorizationPermissionMismatch` to `AccessDenied`. `parseMpError` used to return the raw Azure error for every code except `NoSuchKey`, so the new mapping never reached the multipart paths. It now also passes `AccessDenied` through, and the client gets a `403 AccessDenied` instead of a `500 InternalError`.
Conditional PUTs require a lock primitive that excludes competing gateway
processes sharing a backend filesystem. A successful flock call does not prove
that property: some clustered filesystem configurations accept flock but scope
it to one node, silently leaving cross-gateway check-and-publish races open.
Add an object-lock mode that lets operators select flock or fcntl for
filesystems where that primitive is cluster-coherent, local for the existing
per-process behavior, or none to reject conditional writes with NotImplemented.
Keep the legacy disable flag as an alias for local.
Shared lock modes now verify the selected primitive on the root lock filesystem
during startup and fail closed if it cannot be used. Runtime lock failures no
longer silently downgrade to process-local exclusion. The startup check cannot
establish cross-node coherence, so that remains an explicit operator
requirement.
ScoutFS defaults to none since posix locks are not cluster consistent, but
allow setting local for single node deployments.
Fixes#2351
Problem:
- `posix.New` calls `os.Chdir(rootdir)` and uses cwd-relative paths for
every bucket and object. That is the cheapest way to address files, but
the cwd is process-wide: embedding the gateway (`embedgw`) silently moves
the host program's cwd. In particular, Go unit tests that embed the
gateway can no longer read their test data files by relative path.
Change:
- New `PosixOpts.AbsolutePaths`. When set, `New` leaves the working
directory alone and builds every path from the absolute root; a relative
`VersioningDir`/`SideCarDir` is then resolved against the working
directory rather than the root. The default is unchanged: chdir and
relative paths.
- All bucket and object paths go through new `BucketPath`/`ObjectPath`,
which return the name as-is by default and prefix the root with
`AbsolutePaths`. An absolute "bucket" (the versioning directory
substitution) is passed through unchanged.
- `tmpfile` records the bucket directory path so `link()` and its fallbacks
use the same addressing; `ListBuckets` reads the root through the same
helper.
- `meta.XattrMeta` needs the same root with `AbsolutePaths`. New
`meta.RootDirSetter` interface; `posix.New` calls `WithRootDir` on
storers that implement it in that mode. A zero `XattrMeta` keeps
resolving against the cwd. `SideCar`/`NoMeta` unchanged. A type that
embeds `XattrMeta` inherits a `WithRootDir` that returns a bare
`XattrMeta`, so it needs its own (documented on `RootDirSetter`).
- `DeleteObject` (directory object), `ListParts`, and `UploadPartCopy`
passed filesystem paths where the metadata API expects bucket/object
names; they now pass names, so the sidecar layout is unchanged in both
modes.
- Windows `handleParentDirError` walks up until `filepath.Dir` is a fixed
point, which works for relative and absolute paths.
- scoutfs used cwd-relative bucket/object paths in `CreateBucket`,
`GetObject`, `HeadObject`, `RestoreObject` and the glacier walk; they now
go through `BucketPath`/`ObjectPath`. `scoutfs.New` resolves `rootdir`
before `posix.New` so a relative root no longer reopens `rootdir/rootdir`
after the chdir.
- `isBucketValid` unconditionally rejects names that do not denote a single
entry under the root: `""`, `.`, `..`, names containing a path separator,
and absolute paths. `XattrMeta` rejects `""`, `.` and `..` likewise.
With relative paths `os.Stat("")` and `os.RemoveAll(".")` failed by
accident; with absolute paths they would act on the root directory itself
(reachable with strict bucket names disabled, or via the admin
`change-bucket-owner` endpoint which does not validate `bucket`).
- scoutfs had its own `isBucketValid` whose `validateBucketName` flag was
never set, so it accepted everything. It now delegates to the new exported
`Posix.IsBucketValid`.
- `UploadPartCopy` did not validate the copy source's bucket name (unlike
`CopyObject`); it does now.
- `New` opens the root after validating the versioning and sidecar
directories, so those error paths no longer leak the root handle. The
chdir still happens first, so a relative directory resolves against the
root as before.
Tests:
- New `TestDefaultModeChangesWorkingDirectory` documents the default.
- New `TestRootDirIndependentOfWorkingDirectory`: `AbsolutePaths` with a
relative root from an unrelated cwd, checks cwd is untouched and that
put/get/list/delete, copy, multipart upload with checksums and part copy,
directory-object delete, and invalid bucket names behave correctly under
the root, for both metadata storers.
- New `TestVersioningDirIndependentOfWorkingDirectory`: same setup with a
relative versioning directory; versions land there and not under the
root or cwd.
- New `TestXattrMetaPath` covers cwd-relative and root resolution, absolute
pass-through and the rejected names.
- New `BenchmarkPosix*` benchmarks (small-object head/get/put/list, both
storers, both path modes). The default mode matches `main` within noise
on both Linux and macOS. `AbsolutePaths` costs about 0.2µs (Linux) to
0.4µs (macOS) per path lookup; on Linux (arm64 VM, overlayfs) that is
+2-3% on PutObject and +10-27% on the metadata-heavy small-object
HeadObject/GetObject/ListObjectsV2 with xattr metadata, which is why it
is opt-in.
NVIDIA's rolling cuda-rhel9 repository bumped libcuobjserver from `1.2.0` to `2.0.0` and `build/vgwrdma-builder/Dockerfile` installs it unpinned, so `make vgwrdma-docker` broke: `setTelemFlags` gained a second mask and `initRDMAConfigParams` vanished along with the `RDMAConnection` base class that 2.0.0 deletes entirely. Rather than pin the package, this ports the C wrapper and the Go `rdma` package to the new API.
Telemetry now calls `setTelemFlags(flags, 0)`, where 0 reproduces the old behaviour. `Server.InitRDMAConfig` and its C entry point are removed, which drops a method from the exported API of `github.com/versity/versitygw/rdma`; it had no callers, and 2.0.0 only accepts tunables through the four-argument constructor that `NewServer` already uses. The `dlsym` lookups for `startRDMASession` and `closeRDMASession` go too, as dead code: 1.2.0 never exported those symbols either, so the fallback paths were always what ran. `HandleGet` and `HandlePut` now wrap the library's negative return in a `syscall.Errno` so 2.0.0's new `-EPROTO` is legible, and the wrapper compile rule gains `-std=c++17`. No defaults change.
`make vgwrdma-docker` passes and the binary links `libcuobjserver.so.2`, so it requires a 2.x install at runtime. `cuobjtest-gpu` and `cuobjtest-host` are unaffected. The RDMA data path itself is not verified here; that needs Mellanox hardware with DC transport.
S3 returns `InvalidArgument` when an object action receives a `versionId` query parameter with no value. The gateway silently treated it as an unversioned request instead.
Added a shared `versionId` validation helper and apply it to the object actions that accept the parameter, so malformed requests are rejected up front rather than reaching the backend.
The body stream is now wrapped in a `bodyStreamTracker` before the handler touches it, which remembers the stream's first terminal result rather than asking fasthttp a second, unsafe question. `io.EOF` means the body was read out in full and the connection is still in sync; no terminal result means the handler stopped partway, so the leftovers are drained the way a
`Content-Length` body already was; a framing error means nothing decodable is left and the connection cannot carry another request.
`fasthttp.Request.SetBodyStream` cannot install the wrapper, as it releases the current `*requestStream` back to its pool, so `requestBodyStream` is now the accessor every body reader takes the stream from.
Broken framing no longer gives up on draining either. The connection is closed either way, so a bounded read off the raw socket costs nothing and lets the client finish its write and read the S3 error instead of a reset.
Ben McClellandandGitHub
iRioxeandBen McClelland
niksis02
Eric EntzelandBen McClelland
Luke McCrone
Michał "DualFroz" FoxandGitHub
dependabot[bot]andGitHub
JoshPigott
RaduBerindeandGitHub