Files
versitygw/backend/meta/xattr.go
T
Radu Berinde 7ae119d654 posix: add AbsolutePaths option for embedding without chdir
Problem:
- `posix.New` calls `os.Chdir(rootdir)` and uses cwd-relative paths for
  every bucket and object. That is the cheapest way to address files, but
  the cwd is process-wide: embedding the gateway (`embedgw`) silently moves
  the host program's cwd. In particular, Go unit tests that embed the
  gateway can no longer read their test data files by relative path.

Change:
- New `PosixOpts.AbsolutePaths`. When set, `New` leaves the working
  directory alone and builds every path from the absolute root; a relative
  `VersioningDir`/`SideCarDir` is then resolved against the working
  directory rather than the root. The default is unchanged: chdir and
  relative paths.
- All bucket and object paths go through new `BucketPath`/`ObjectPath`,
  which return the name as-is by default and prefix the root with
  `AbsolutePaths`. An absolute "bucket" (the versioning directory
  substitution) is passed through unchanged.
- `tmpfile` records the bucket directory path so `link()` and its fallbacks
  use the same addressing; `ListBuckets` reads the root through the same
  helper.
- `meta.XattrMeta` needs the same root with `AbsolutePaths`. New
  `meta.RootDirSetter` interface; `posix.New` calls `WithRootDir` on
  storers that implement it in that mode. A zero `XattrMeta` keeps
  resolving against the cwd. `SideCar`/`NoMeta` unchanged. A type that
  embeds `XattrMeta` inherits a `WithRootDir` that returns a bare
  `XattrMeta`, so it needs its own (documented on `RootDirSetter`).
- `DeleteObject` (directory object), `ListParts`, and `UploadPartCopy`
  passed filesystem paths where the metadata API expects bucket/object
  names; they now pass names, so the sidecar layout is unchanged in both
  modes.
- Windows `handleParentDirError` walks up until `filepath.Dir` is a fixed
  point, which works for relative and absolute paths.
- scoutfs used cwd-relative bucket/object paths in `CreateBucket`,
  `GetObject`, `HeadObject`, `RestoreObject` and the glacier walk; they now
  go through `BucketPath`/`ObjectPath`. `scoutfs.New` resolves `rootdir`
  before `posix.New` so a relative root no longer reopens `rootdir/rootdir`
  after the chdir.
- `isBucketValid` unconditionally rejects names that do not denote a single
  entry under the root: `""`, `.`, `..`, names containing a path separator,
  and absolute paths. `XattrMeta` rejects `""`, `.` and `..` likewise.
  With relative paths `os.Stat("")` and `os.RemoveAll(".")` failed by
  accident; with absolute paths they would act on the root directory itself
  (reachable with strict bucket names disabled, or via the admin
  `change-bucket-owner` endpoint which does not validate `bucket`).
- scoutfs had its own `isBucketValid` whose `validateBucketName` flag was
  never set, so it accepted everything. It now delegates to the new exported
  `Posix.IsBucketValid`.
- `UploadPartCopy` did not validate the copy source's bucket name (unlike
  `CopyObject`); it does now.
- `New` opens the root after validating the versioning and sidecar
  directories, so those error paths no longer leak the root handle. The
  chdir still happens first, so a relative directory resolves against the
  root as before.

Tests:
- New `TestDefaultModeChangesWorkingDirectory` documents the default.
- New `TestRootDirIndependentOfWorkingDirectory`: `AbsolutePaths` with a
  relative root from an unrelated cwd, checks cwd is untouched and that
  put/get/list/delete, copy, multipart upload with checksums and part copy,
  directory-object delete, and invalid bucket names behave correctly under
  the root, for both metadata storers.
- New `TestVersioningDirIndependentOfWorkingDirectory`: same setup with a
  relative versioning directory; versions land there and not under the
  root or cwd.
- New `TestXattrMetaPath` covers cwd-relative and root resolution, absolute
  pass-through and the rejected names.
- New `BenchmarkPosix*` benchmarks (small-object head/get/put/list, both
  storers, both path modes). The default mode matches `main` within noise
  on both Linux and macOS. `AbsolutePaths` costs about 0.2µs (Linux) to
  0.4µs (macOS) per path lookup; on Linux (arm64 VM, overlayfs) that is
  +2-3% on PutObject and +10-27% on the metadata-heavy small-object
  HeadObject/GetObject/ListObjectsV2 with xattr metadata, which is why it
  is opt-in.
2026-09-10 13:28:01 -07:00

185 lines
5.5 KiB
Go

// Copyright 2024 Versity Software
// This file is licensed under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package meta
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/pkg/xattr"
"github.com/versity/versitygw/s3err"
)
var (
// ErrNoSuchKey is returned when the key does not exist.
ErrNoSuchKey = errors.New("no such key")
)
// XattrMeta stores metadata as extended attributes on the bucket and object
// files. The zero value resolves bucket names against the process working
// directory, which the posix backend sets to its root directory; a backend
// that keeps the working directory supplies the root through WithRootDir.
type XattrMeta struct {
// rootdir is the absolute path bucket names are resolved under, or ""
// to resolve them against the process working directory.
rootdir string
}
var _ RootDirSetter = XattrMeta{}
// WithRootDir returns a copy of x that resolves bucket names against rootdir.
func (x XattrMeta) WithRootDir(rootdir string) MetadataStorer {
x.rootdir = rootdir
return x
}
// path returns the filesystem path holding the attributes of object in
// bucket (of the bucket itself when object is empty).
//
// The bucket argument is normally a bucket name, resolved under the root
// directory (or the working directory when no root is set). The versioning
// code instead passes the absolute path of a bucket's versioning directory;
// an absolute bucket is used as given.
func (x XattrMeta) path(bucket, object string) (string, error) {
if filepath.IsAbs(bucket) {
return filepath.Join(bucket, object), nil
}
if bucket == "" || bucket == "." || bucket == ".." {
// Would resolve to the root directory itself or its parent.
return "", fmt.Errorf("xattr metadata: invalid bucket name %q", bucket)
}
return filepath.Join(x.rootdir, bucket, object), nil
}
// RetrieveAttribute retrieves the value of a specific attribute for an object in a bucket.
func (x XattrMeta) RetrieveAttribute(f *os.File, bucket, object, attribute string) ([]byte, error) {
if f != nil {
b, err := xattr.FGet(f, xattrPrefix+attribute)
if errors.Is(err, xattr.ENOATTR) {
return nil, ErrNoSuchKey
}
return b, err
}
name, err := x.path(bucket, object)
if err != nil {
return nil, err
}
b, err := xattr.Get(name, xattrPrefix+attribute)
if errors.Is(err, xattr.ENOATTR) {
return nil, ErrNoSuchKey
}
return b, err
}
// StoreAttribute stores the value of a specific attribute for an object in a bucket.
func (x XattrMeta) StoreAttribute(f *os.File, bucket, object, attribute string, value []byte) error {
if f != nil {
err := xattr.FSet(f, xattrPrefix+attribute, value)
if errors.Is(err, syscall.EROFS) {
return s3err.GetAPIError(s3err.ErrMethodNotAllowed)
}
if errors.Is(err, syscall.ENOSPC) {
return s3err.GetAPIError(s3err.ErrNoSpaceLeftOnDevice)
}
return err
}
name, err := x.path(bucket, object)
if err != nil {
return err
}
err = xattr.Set(name, xattrPrefix+attribute, value)
if errors.Is(err, syscall.EROFS) {
return s3err.GetAPIError(s3err.ErrMethodNotAllowed)
}
if errors.Is(err, syscall.ENOSPC) {
return s3err.GetAPIError(s3err.ErrNoSpaceLeftOnDevice)
}
return err
}
// DeleteAttribute removes the value of a specific attribute for an object in a bucket.
func (x XattrMeta) DeleteAttribute(bucket, object, attribute string) error {
name, err := x.path(bucket, object)
if err != nil {
return err
}
err = xattr.Remove(name, xattrPrefix+attribute)
if errors.Is(err, xattr.ENOATTR) {
return ErrNoSuchKey
}
if errors.Is(err, syscall.EROFS) {
return s3err.GetAPIError(s3err.ErrMethodNotAllowed)
}
return err
}
// DeleteAttributes is not implemented for xattr since xattrs
// are automatically removed when the file is deleted.
func (x XattrMeta) DeleteAttributes(bucket, object string) error {
return nil
}
// RenameObject is a no-op for xattr because extended attributes are stored
// on the inodes and follow the file/directory when it is renamed.
func (x XattrMeta) RenameObject(_, _, _ string) error {
return nil
}
// ListAttributes lists all attributes for an object in a bucket.
func (x XattrMeta) ListAttributes(bucket, object string) ([]string, error) {
name, err := x.path(bucket, object)
if err != nil {
return nil, err
}
attrs, err := xattr.List(name)
if err != nil {
return nil, err
}
attributes := make([]string, 0, len(attrs))
for _, attr := range attrs {
if !isUserAttr(attr) {
continue
}
attributes = append(attributes, strings.TrimPrefix(attr, xattrPrefix))
}
return attributes, nil
}
func isUserAttr(attr string) bool {
return strings.HasPrefix(attr, xattrPrefix)
}
// Test is a helper function to test if xattrs are supported.
func (x XattrMeta) Test(path string) error {
// check for platform support
if !xattr.XATTR_SUPPORTED {
return fmt.Errorf("xattrs are not supported on this platform")
}
// check if the filesystem supports xattrs
_, err := xattr.Get(path, "user.test")
if errors.Is(err, syscall.ENOTSUP) {
return fmt.Errorf("xattrs are not supported on this filesystem")
}
return nil
}