The gateway currently supports only Signature Version 4 (SigV4) authorization. Deprecated AWS SigV2 requests are now rejected with an AWS-specific `InvalidRequest` error for both Authorization-header requests and query-string requests(presigned URLs).
This also fixes SigV4 Authorization-header handling for date headers. SigV4 accepts two date headers: `Date` and `X-Amz-Date`. `X-Amz-Date` takes precedence, but when it is missing, `Date` should be used. The gateway now uses the `Date` header with lower precedence when `X-Amz-Date` is not present. No SDK integration test was added for this case because the SDK always sets `X-Amz-Date`, and this behavior is not configurable.
Remove the client-side search that filtered already-loaded objects by
name. Replace it with a prefix input that is appended to the current
path prefix and passed directly to the S3 ListObjectsV2 API, so
filtering is performed server-side and works correctly across all pages.
Fixes#2091
Integrate the new S3 checksum types in the gateway, including `SHA512`, `MD5`, `XXHASH64`, `XXHASH3`, and `XXHASH128`. This adds checksum calculation, validation, schema handling, and test coverage for the expanded checksum support.
These external packages have been used:
- `github.com/zeebo/xxh3` for `XXHASH3` and `XXHASH128`
- `github.com/cespare/xxhash/v2` for `XXHASH64`
Adjust integration tests because `aws-sdk-go-v2/service/s3` does not support automatic checksum calculation for the new checksum algorithms and returns an SDK-level error when only the checksum algorithm is provided. Only precalculated checksum values are acceptable for these checksum types.
References:
- `https://github.com/aws/aws-sdk-go-v2/issues/3404`
- `https://github.com/aws/aws-sdk-go-v2/issues/3403`
Add options for embedders to register Fiber routes and middleware before the S3 route table is initialized.
WithRoute registers a top-level route with explicit method and path matching. WithMiddleware registers prefix middleware that can handle a request or call ctx.Next() to continue into the S3 stack.
Add coverage for route registration order when a top-level route and catch-all middleware are both configured.
Closes#1273
Rewrite UnsignedChunkReader to stream the payload bytes directly into the caller buffer instead of allocating and stashing full chunks. With this implementation, no stash is held by the reader and the chunk reader doesn't allocate any memory.
Make debug logging more descriptive, which records reader state on all error paths and logs read progress whenever a Read call fills the caller buffer.
Some unit tests were added to cover the main moving parts of the reader flow.
The PATCH /:bucket/create admin route was missing
middlewares.ApplyDefaultCORS, while every other admin PATCH route
applies it. The OPTIONS preflight handler already sets CORS headers,
so browsers pass preflight but block the actual response for lacking
Access-Control-Allow-Origin. This caused the WebUI bucket-creation
flow to fail with ERR_FAILED even though the server returned 201.
Fixes#2105. Introduced in #1739 when the endpoint was added.
Co-Authored-By: Claude <noreply@anthropic.com>
The MoveData() requires that all but the last part be 4k aligned.
We accidentally were including the alignment check for the last
part causing large uploads where the total object was not a
multiple of 4k to fallback to copying the last part. For very large
part sizes this was triggering timeouts in some clients.
Add a new non-AWS error ErrNoSpaceLeftOnDevice (HTTP 507 Insufficient
Storage) to s3err. Update all call sites in the posix backend that
could return ENOSPC and return the new error when the underlying
filesystem has no space remaining.
Fixes#2093
Fixes#1986
When a client includes tagging, legal hold, or retention headers in a PutObject, CopyObject or CreateMultipartUpload request, the corresponding bucket policy permissions must be verified in addition to s3:PutObject:
`X-Amz-Tagging` - `s3:PutObjectTagging`
`X-Amz-Object-Lock-Legal-Hold` - `s3:PutObjectLegalHold`
`X-Amz-Object-Lock-Mode` - `s3:PutObjectRetention`
Previously, only s3:PutObject was checked, allowing users to set tagging, legal hold, and retention without having the required permissions. Now each action permission is check, if user tries to add them.
For CopyObject these permissions are checked on destination object.
Enough people are making use of sidecar that we need to add
a CI test to make sure we have some coverage with this mode.
This add a couple small functional test fixes found wtih
enabling sidecar tests as well.
Unlike xattr metadata which is tied to the filesystem object and removed
automatically, sidecar metadata lives in a parallel directory tree and
must be deleted explicitly. Add DeleteAttributes calls after removing
bucket directories, null-version files, and versioned object files.
Also add an os.Stat existence check in GetObjectTagging and
PutObjectTagging when no versionId is given, since sidecar's
StoreAttribute/RetrieveAttribute do not naturally return ErrNotExist
for missing objects the way xattr operations do.
The dl.min.io download site has been having stability issues
possibly related to github action runners getting rate limited.
Disable these for now until we can find a better place to host
this client.
Global flags must appear before the backend subcommand in the
versitygw CLI. Previously VGW_ARGS was appended after the backend,
causing global flags to be silently ignored.
Reorder argument assembly to: VGW_ARGS <backend> VGW_BACKEND_ARG VGW_BACKEND_ARGS
Fixes#2082
Most actions within each page is stateless (show modals) or change the
URL hash. As it is, those are not tracked and using the back button has
no effect.
This commits implements two things:
- Tracking of the URL hash in the explorer to move from bucket/folders
on history change.
- Add a history state when a modal is open, so the back button closes
the modal.
The object listing now fetches pages from S3 directly using ListObjectsV2
continuation tokens rather than loading all objects at once. Users can
navigate forward and back with first/prev/next buttons and choose how many
rows to show per page (10/20/50/100/1000, defaulting to 10), which keeps
the listing fast and responsive even in buckets with thousands of objects.
Pagination resets automatically when navigating into a folder, running a
search, or changing page size. While a search is active, forward navigation
is disabled since search filters within the current page only, and the item
count shows "(filtered)" to make that clear. When versioning is enabled,
delete-marker rows are scoped to the current page's key range so they don't
bleed in from adjacent pages.
Fixes#2055
Add a --socket-perm flag (VGW_SOCKET_PERM env var) to control the
file-mode permissions on file-backed UNIX domain sockets. This allows
operators to limit access permission without relying on process umask.
The option applies to S3, admin, and WebUI sockets and has no effect
on TCP/IP addresses or Linux abstract namespace sockets.
Fixes#2010
Users may have access to buckets that don't appear in their owned-bucket
list. Previously there was no way to reach those buckets from the explorer
without knowing and manually editing the URL.
This adds a "go to bucket by name" input to the buckets view. Typing a
name and pressing Enter (or clicking Open) navigates to the bucket if
the user has permissions to allow it.
To avoid re-typing bucket names on every visit, users can now save buckets
to a favorites panel that persists in localStorage. Favorites are keyed by
access key so different users sharing the same browser each see only their
own list. Any bucket in the owned list can be starred directly from its row.
Favorites chips are clickable with the same access check, and a hover ×
removes them. The panel hides itself automatically when the list is empty.
Ben McClelland
niksis02
Luke McCrone
dependabot[bot]
Yevgeniy Miretskiy
Dylan Dellett-Wion
alice nodelman
Antoine POPINEAU