Enough people are making use of sidecar that we need to add
a CI test to make sure we have some coverage with this mode.
This add a couple small functional test fixes found wtih
enabling sidecar tests as well.
Unlike xattr metadata which is tied to the filesystem object and removed
automatically, sidecar metadata lives in a parallel directory tree and
must be deleted explicitly. Add DeleteAttributes calls after removing
bucket directories, null-version files, and versioned object files.
Also add an os.Stat existence check in GetObjectTagging and
PutObjectTagging when no versionId is given, since sidecar's
StoreAttribute/RetrieveAttribute do not naturally return ErrNotExist
for missing objects the way xattr operations do.
The dl.min.io download site has been having stability issues
possibly related to github action runners getting rate limited.
Disable these for now until we can find a better place to host
this client.
Global flags must appear before the backend subcommand in the
versitygw CLI. Previously VGW_ARGS was appended after the backend,
causing global flags to be silently ignored.
Reorder argument assembly to: VGW_ARGS <backend> VGW_BACKEND_ARG VGW_BACKEND_ARGS
Fixes#2082
Most actions within each page is stateless (show modals) or change the
URL hash. As it is, those are not tracked and using the back button has
no effect.
This commits implements two things:
- Tracking of the URL hash in the explorer to move from bucket/folders
on history change.
- Add a history state when a modal is open, so the back button closes
the modal.
The object listing now fetches pages from S3 directly using ListObjectsV2
continuation tokens rather than loading all objects at once. Users can
navigate forward and back with first/prev/next buttons and choose how many
rows to show per page (10/20/50/100/1000, defaulting to 10), which keeps
the listing fast and responsive even in buckets with thousands of objects.
Pagination resets automatically when navigating into a folder, running a
search, or changing page size. While a search is active, forward navigation
is disabled since search filters within the current page only, and the item
count shows "(filtered)" to make that clear. When versioning is enabled,
delete-marker rows are scoped to the current page's key range so they don't
bleed in from adjacent pages.
Fixes#2055
Add a --socket-perm flag (VGW_SOCKET_PERM env var) to control the
file-mode permissions on file-backed UNIX domain sockets. This allows
operators to limit access permission without relying on process umask.
The option applies to S3, admin, and WebUI sockets and has no effect
on TCP/IP addresses or Linux abstract namespace sockets.
Fixes#2010
Users may have access to buckets that don't appear in their owned-bucket
list. Previously there was no way to reach those buckets from the explorer
without knowing and manually editing the URL.
This adds a "go to bucket by name" input to the buckets view. Typing a
name and pressing Enter (or clicking Open) navigates to the bucket if
the user has permissions to allow it.
To avoid re-typing bucket names on every visit, users can now save buckets
to a favorites panel that persists in localStorage. Favorites are keyed by
access key so different users sharing the same browser each see only their
own list. Any bucket in the owned list can be starred directly from its row.
Favorites chips are clickable with the same access check, and a hover ×
removes them. The panel hides itself automatically when the list is empty.
Fixes#2052Fixes#2056Fixes#2057
Previously, GetObject and HeadObject used the request's `Range` header to determine the response status code, which caused incorrect 206 responses for invalid Range header values.
The status is now driven by whether res.ContentRange is set in the response, rather than by the presence of a range in the request. Backends (posix and azure) now set Content-Range for PartNumber=1 on non-multipart objects, skipping zero-size objects where no range applies.
HeadObject was also fixed to return 206 when Content-Range is present, and to only return checksums when the full object is requested.
Replace `encodeURIComponent()` with a custom `awsUriEncode()` that also
percent-encodes characters not in the SigV4 unreserved set.
This fixes invalid signatures for object keys or query values containing
those characters.
Fixes#2060.
Add a modal dialog to the object explorer that generates presigned GET
URLs for S3 objects. Supports configurable expiration in minutes, hours,
or days, and includes a copy-to-clipboard button for the result.
Reuses the existing `api.presignUrl()` function; adds only the UI layer.
Closes#1064
Use the multipart ETag as the in-progress directory suffix instead of the static `.inprogress` marker so that concurrent CompleteMultipartUpload calls for the same upload ID are all treated as successful (idempotent) rather than racing, where only one succeeded and the rest returned NoSuchUpload.
After finalizing the multipart upload, store an `mp-metadata` xattr on the assembled object that records the upload ID and cumulative byte offsets for each part. GetObject and HeadObject now use this metadata to serve individual part ranges via the `partNumber` query parameter, returning a successful response instead of returning NotImplemented.
Add two new S3 error codes:
- `ErrInvalidPartNumberRange` (416 RequestedRangeNotSatisfiable) — returned
when the requested part number exceeds the number of parts in the upload.
- `ErrRangeAndPartNumber` (400 BadRequest) — returned when both a Range header
and a partNumber query parameter are specified on the same request.
As multipart uploads are translated to blobs in azure blob storage, they were visible in ListObjects(V2) as complete objects. Now the blobs with multipart prefix are filtered out during listing.
The listing logic is rewritten client-side to implement proper S3 semantics: flat blob enumeration with manual delimiter handling, correct truncation (IsTruncated only set when more items genuinely exist beyond maxKeys), and StartAfter/Marker/ContinuationToken applied via the lexicographic max of both constraints in ListObjectsV2.
For the same reason bucket deletion was not allowed. Now multipart objects are explicitly checked on bucket deletion and any pending multipart upload doesn't block the bucket deletion anymore.
encodeURIComponent encodes the trailing slash as %2F, producing a path
like /bucket/folder%2F. S3 servers (e.g. Ceph) reject this with a
SignatureDoesNotMatch error because the canonical URI used for signing
differs from what the server reconstructs.
encodeS3Key encodes each path segment individually, leaving slashes as
literal /, which is consistent with all other object-key operations in
the API client.
Fixes#2029
On Windows, a file apparently cannot be renamed while an open
handle to it exists.
The previous code used defer tempfile.Close(), which meant the
handle was still open when os.Rename was called, producing:
failed to rename temporary file: The process cannot access
the file because it is being used by another process.
Fix by closing the file explicitly before the rename.
Fixes#2021
The AWS SDK v2 includes Accept-Encoding in SigV4 signed headers
which causes GCS to return a SignatureDoesNotMatch error because
GCS rewrites that header internally before verifying the signature.
Add a --gcs-compatibility / VGW_S3_GCS_COMPATIBILITY option for the s3proxy
backend that injects two Smithy finalize-layer middlewares: one removes
Accept-Encoding from the request immediately before the Signing step, and
a second restores it after signing so the header is still sent on the wire.
see: https://github.com/aws/aws-sdk-go-v2/issues/1816
This can be removed once GCS fixes this incompatibility.
The urfave/cli/v2 has built-in auto-completion support for bash.
This enables generating the auto completions with a cli option
that can be added to environment setup scripts.
Also adding some distro support scripts for rpm and deb for the
auto completion setup.
To enable auto completion in zsh, this still needs to be added
to user .zshrc:
fpath=(/usr/share/zsh/site-functions $fpath)
autoload -Uz compinit && compinit
Fixes#2015
fasthttp v1.70.0 now enforces the HTTP/1.1 requirement of exactly
one Host header, rejecting requests that omit it. Fix tests that
were failing due to missing host.
Fixes#2030
When an object has offline blocks, the restore status was incorrectly set to `ongoing-request="false"` instead of omitting the header entirely, which causes s3 clients fail on parsing the x-amz-restore header.
Remove the incorrect `stageNotInProgress` constant and simplify the `requestOngoing` initialization to reflect the correct default.
