Problem:
- `posix.New` calls `os.Chdir(rootdir)` and uses cwd-relative paths for
every bucket and object. That is the cheapest way to address files, but
the cwd is process-wide: embedding the gateway (`embedgw`) silently moves
the host program's cwd. In particular, Go unit tests that embed the
gateway can no longer read their test data files by relative path.
Change:
- New `PosixOpts.AbsolutePaths`. When set, `New` leaves the working
directory alone and builds every path from the absolute root; a relative
`VersioningDir`/`SideCarDir` is then resolved against the working
directory rather than the root. The default is unchanged: chdir and
relative paths.
- All bucket and object paths go through new `BucketPath`/`ObjectPath`,
which return the name as-is by default and prefix the root with
`AbsolutePaths`. An absolute "bucket" (the versioning directory
substitution) is passed through unchanged.
- `tmpfile` records the bucket directory path so `link()` and its fallbacks
use the same addressing; `ListBuckets` reads the root through the same
helper.
- `meta.XattrMeta` needs the same root with `AbsolutePaths`. New
`meta.RootDirSetter` interface; `posix.New` calls `WithRootDir` on
storers that implement it in that mode. A zero `XattrMeta` keeps
resolving against the cwd. `SideCar`/`NoMeta` unchanged. A type that
embeds `XattrMeta` inherits a `WithRootDir` that returns a bare
`XattrMeta`, so it needs its own (documented on `RootDirSetter`).
- `DeleteObject` (directory object), `ListParts`, and `UploadPartCopy`
passed filesystem paths where the metadata API expects bucket/object
names; they now pass names, so the sidecar layout is unchanged in both
modes.
- Windows `handleParentDirError` walks up until `filepath.Dir` is a fixed
point, which works for relative and absolute paths.
- scoutfs used cwd-relative bucket/object paths in `CreateBucket`,
`GetObject`, `HeadObject`, `RestoreObject` and the glacier walk; they now
go through `BucketPath`/`ObjectPath`. `scoutfs.New` resolves `rootdir`
before `posix.New` so a relative root no longer reopens `rootdir/rootdir`
after the chdir.
- `isBucketValid` unconditionally rejects names that do not denote a single
entry under the root: `""`, `.`, `..`, names containing a path separator,
and absolute paths. `XattrMeta` rejects `""`, `.` and `..` likewise.
With relative paths `os.Stat("")` and `os.RemoveAll(".")` failed by
accident; with absolute paths they would act on the root directory itself
(reachable with strict bucket names disabled, or via the admin
`change-bucket-owner` endpoint which does not validate `bucket`).
- scoutfs had its own `isBucketValid` whose `validateBucketName` flag was
never set, so it accepted everything. It now delegates to the new exported
`Posix.IsBucketValid`.
- `UploadPartCopy` did not validate the copy source's bucket name (unlike
`CopyObject`); it does now.
- `New` opens the root after validating the versioning and sidecar
directories, so those error paths no longer leak the root handle. The
chdir still happens first, so a relative directory resolves against the
root as before.
Tests:
- New `TestDefaultModeChangesWorkingDirectory` documents the default.
- New `TestRootDirIndependentOfWorkingDirectory`: `AbsolutePaths` with a
relative root from an unrelated cwd, checks cwd is untouched and that
put/get/list/delete, copy, multipart upload with checksums and part copy,
directory-object delete, and invalid bucket names behave correctly under
the root, for both metadata storers.
- New `TestVersioningDirIndependentOfWorkingDirectory`: same setup with a
relative versioning directory; versions land there and not under the
root or cwd.
- New `TestXattrMetaPath` covers cwd-relative and root resolution, absolute
pass-through and the rejected names.
- New `BenchmarkPosix*` benchmarks (small-object head/get/put/list, both
storers, both path modes). The default mode matches `main` within noise
on both Linux and macOS. `AbsolutePaths` costs about 0.2µs (Linux) to
0.4µs (macOS) per path lookup; on Linux (arm64 VM, overlayfs) that is
+2-3% on PutObject and +10-27% on the metadata-heavy small-object
HeadObject/GetObject/ListObjectsV2 with xattr metadata, which is why it
is opt-in.
This fixes the cases for racing uploads with the same object names.
Before we were making some bad assumptions about what would cause
an error when trying to link/rename the final object name into
the namespace, but missed the case that another upload for the
same name could be racing with this upload and causing an incorrect
error.
This also changes the order of setting metadata to prevent
accidental setting of metadata for the current upload to another
racing upload.
This also fix auth.CheckObjectAccess() when objects are removed
while this runs.
Fixes#854
Radu Berinde
Anael ORLINSKI
Ben McClelland