From bf31f404bc8895e3bfcb91e1a7962253a332217a Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Fri, 10 Apr 2026 09:52:16 -0700 Subject: [PATCH] test: add pjdfstest POSIX compliance suite (#9013) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test: add pjdfstest POSIX compliance suite Adds a script and CI workflow that runs the upstream pjdfstest POSIX compliance test suite against a SeaweedFS FUSE mount. The script starts a self-contained `weed mini` server, mounts the filesystem with `weed mount`, builds pjdfstest from source, and runs it under prove(1). * fix: address review feedback on pjdfstest setup - Use github.ref instead of github.head_ref in concurrency group so push events get a stable group key - Add explicit timeout check after filer readiness polling loop - Refresh pjdfstest checkout when PJDFSTEST_REPO or PJDFSTEST_REF are overridden instead of silently reusing stale sources * test: add Docker-based pjdfstest for faster iteration Adds a docker-compose setup that reuses the existing e2e image pattern: - master, volume, filer services from chrislusf/seaweedfs:e2e - mount service extended with pjdfstest baked in (Dockerfile extends e2e) - Tests run via `docker compose exec mount /run.sh` - CI workflow gains a parallel `pjdfstest (docker)` job This avoids building Go from scratch on each iteration — just rebuild the e2e image once and iterate on the compose stack. * fix: address second round of review feedback - Use mktemp for WORK_DIR so each run starts with a clean filer state - Pin PJDFSTEST_REF to immutable commit (03eb257) instead of master - Use cp -r instead of cp -a to avoid preserving ownership during setup * fix: address CI failure and third round of review feedback - Fix docker job: fall back to plain docker build when buildx cache export is not supported (default docker driver in some CI runners) - Use /healthz endpoint for filer healthcheck in docker-compose - Copy logs to a fixed path (/tmp/seaweedfs-pjdfstest-logs/) for reliable CI artifact upload when WORK_DIR is a mktemp path * fix(mount): improve POSIX compliance for FUSE mount Address several POSIX compliance gaps surfaced by the pjdfstest suite: 1. Filename length limit: reduce from 4096 to 255 bytes (NAME_MAX), returning ENAMETOOLONG for longer names. 2. SUID/SGID clearing on write: clear setuid/setgid bits when a non-root user writes to a file (POSIX requirement). 3. SUID/SGID clearing on chown: clear setuid/setgid bits when file ownership changes by a non-root user. 4. Sticky bit enforcement: add checkStickyBit helper and enforce it in Unlink, Rmdir, and Rename — only file owner, directory owner, or root may delete entries in sticky directories. 5. ctime (inode change time) tracking: add ctime field to the FuseAttributes protobuf message and filer.Attr struct. Update ctime on all metadata-modifying operations (SetAttr, Write/flush, Link, Create, Mkdir, Mknod, Symlink, Truncate). Fall back to mtime for backward compatibility when ctime is 0. * fix: add -T flag to docker compose exec for CI Disable TTY allocation in the pjdfstest docker job since GitHub Actions runners have no interactive TTY. * fix(mount): update parent directory mtime/ctime on entry changes POSIX requires that a directory's st_mtime and st_ctime be updated whenever entries are created or removed within it. Add touchDirMtimeCtime() helper and call it after: - mkdir, rmdir - create (including deferred creates), mknod, unlink - symlink, link - rename (both source and destination directories) This fixes pjdfstest failures in mkdir/00, mkfifo/00, mknod/00, mknod/11, open/00, symlink/00, link/00, and rmdir/00. * fix(mount): enforce sticky bit on destination directory during rename POSIX requires sticky-bit enforcement on both source and destination directories during rename. When the destination directory has the sticky bit set and a target entry already exists, only the file owner, directory owner, or root may replace it. * fix(mount): add in-memory atime tracking for POSIX compliance Track atime separately from mtime using a bounded in-memory map (capped at 8192 entries with random eviction). atime is not persisted to the filer — it's only kept in mount memory to satisfy POSIX stat requirements for utimensat and related syscalls. This fixes utimensat/00, utimensat/02, utimensat/04, utimensat/05, and utimensat/09 pjdfstest failures where atime was incorrectly aliased to mtime. * fix(mount): restore long filename support, fix permission checks - Restore 4096-byte filename limit (was incorrectly reduced to 255). SeaweedFS stores names as protobuf strings with no ext4-style constraint — the 255 limit is not applicable. - Fix AcquireHandle permission check to map filer uid/gid to local space before calling hasAccess, matching the pattern used in Access(). - Fix hasAccess fallback when supplementary group lookup fails: fall through to "other" permissions instead of requiring both group AND other to match, which was overly restrictive for non-existent UIDs. * fix(mount): fix permission checks and enforce NAME_MAX=255 - Fix AcquireHandle to map uid/gid from filer-space to local-space before calling hasAccess, consistent with the Access handler. - Fix hasAccess fallback when supplementary group lookup fails: use "other" permissions only instead of requiring both group AND other. - Enforce NAME_MAX=255 with a comment explaining the Linux FUSE kernel module's VFS-layer limit. Files >255 bytes can be created via direct FUSE protocol calls but can't be stat'd/chmod'd via normal syscalls. - Don't call touchDirMtimeCtime for deferred creates to avoid invalidating the just-cached entry via filer metadata events. * ci: mark pjdfstest steps as continue-on-error The pjdfstest suite has known failures (Linux FUSE NAME_MAX=255 limitation, hard link nlink/ctime tracking, nanosecond precision) that cannot be fixed in the mount layer. Mark the test steps as continue-on-error so the CI job reports results without blocking. * ci: increase pjdfstest bare metal timeout to 90 minutes * fix: use full commit hash for PJDFSTEST_REF in run.sh Short hashes cannot be resolved by git fetch --depth 1 on shallow clones. Use the full 40-char SHA. * test: add pjdfstest known failures skip list Add known_failures.txt listing 33 test files that cannot pass due to: - Linux FUSE kernel NAME_MAX=255 (26 files) - Hard link nlink/ctime tracking requiring filer changes (3 files) - Parent dir mtime on deferred create (1 file) - Directory rename permission edge case (1 file) - rmdir after hard link unlink (1 file) - Nanosecond timestamp precision (1 file) Both run.sh and run_inside_container.sh now skip these tests when running the full suite. Any failure in a non-skipped test will cause CI to fail, catching regressions immediately. Remove continue-on-error from CI steps since the skip list handles known failures. Result: 204 test files, 8380 tests, all passing. * ci: remove bare metal pjdfstest job, keep Docker only The bare metal job consistently gets stuck past its timeout due to weed processes not exiting cleanly. The Docker job covers the same tests reliably and runs faster. --- .github/workflows/pjdfstest.yml | 87 +++++++++++++ test/pjdfstest/Dockerfile | 28 ++++ test/pjdfstest/docker-compose.yml | 57 ++++++++ test/pjdfstest/known_failures.txt | 67 ++++++++++ test/pjdfstest/run.sh | 173 +++++++++++++++++++++++++ test/pjdfstest/run_inside_container.sh | 48 +++++++ weed/filer/entry.go | 1 + weed/filer/entry_codec.go | 7 + weed/mount/weedfs.go | 3 + weed/mount/weedfs_access.go | 24 +++- weed/mount/weedfs_attr.go | 68 +++++++++- weed/mount/weedfs_dir_mkrm.go | 19 ++- weed/mount/weedfs_file_mkrm.go | 18 ++- weed/mount/weedfs_file_sync.go | 4 +- weed/mount/weedfs_file_write.go | 5 + weed/mount/weedfs_filehandle.go | 6 +- weed/mount/weedfs_link.go | 5 +- weed/mount/weedfs_rename.go | 30 +++++ weed/mount/weedfs_symlink.go | 7 +- weed/mount/wfs_save.go | 5 +- weed/pb/filer.proto | 1 + weed/pb/filer_pb/filer.pb.go | 13 +- 22 files changed, 656 insertions(+), 20 deletions(-) create mode 100644 .github/workflows/pjdfstest.yml create mode 100644 test/pjdfstest/Dockerfile create mode 100644 test/pjdfstest/docker-compose.yml create mode 100644 test/pjdfstest/known_failures.txt create mode 100755 test/pjdfstest/run.sh create mode 100755 test/pjdfstest/run_inside_container.sh diff --git a/.github/workflows/pjdfstest.yml b/.github/workflows/pjdfstest.yml new file mode 100644 index 000000000..c23ef5bb1 --- /dev/null +++ b/.github/workflows/pjdfstest.yml @@ -0,0 +1,87 @@ +name: "pjdfstest POSIX Compliance" + +on: + push: + branches: [ master, main ] + paths: + - 'weed/mount/**' + - 'weed/filer/**' + - 'test/pjdfstest/**' + - '.github/workflows/pjdfstest.yml' + pull_request: + branches: [ master, main ] + paths: + - 'weed/mount/**' + - 'weed/filer/**' + - 'test/pjdfstest/**' + - '.github/workflows/pjdfstest.yml' + workflow_dispatch: + +concurrency: + group: pjdfstest/${{ github.ref }} + cancel-in-progress: true + +permissions: + contents: read + +jobs: + pjdfstest: + name: pjdfstest + runs-on: ubuntu-22.04 + timeout-minutes: 60 + + steps: + - name: Checkout code + uses: actions/checkout@v6 + + - name: Set up Go + uses: actions/setup-go@v6 + with: + go-version-file: 'go.mod' + + - name: Build SeaweedFS e2e image + run: | + cd docker + make build_e2e || { + echo "Retrying without buildx cache..." + make binary_race + docker build --no-cache -t chrislusf/seaweedfs:e2e -f Dockerfile.e2e . + } + + - name: Build pjdfstest image + run: | + docker build -t chrislusf/seaweedfs:pjdfstest test/pjdfstest/ + + - name: Start SeaweedFS cluster + run: | + docker compose -f test/pjdfstest/docker-compose.yml up --wait + + - name: Run pjdfstest + run: | + set -o pipefail + docker compose -f test/pjdfstest/docker-compose.yml exec -T mount \ + /run.sh 2>&1 | tee /tmp/pjdfstest-output.log + + - name: Collect logs + if: always() + run: | + mkdir -p /tmp/pjdfstest-docker-logs + for svc in master volume filer mount; do + docker compose -f test/pjdfstest/docker-compose.yml logs "$svc" \ + > "/tmp/pjdfstest-docker-logs/${svc}.log" 2>&1 || true + done + + - name: Tear down + if: always() + run: | + docker compose -f test/pjdfstest/docker-compose.yml down -v + + - name: Upload logs + if: always() + uses: actions/upload-artifact@v7 + with: + name: pjdfstest-results + path: | + /tmp/pjdfstest-output.log + /tmp/pjdfstest-docker-logs/ + retention-days: 7 diff --git a/test/pjdfstest/Dockerfile b/test/pjdfstest/Dockerfile new file mode 100644 index 000000000..862ee4ea4 --- /dev/null +++ b/test/pjdfstest/Dockerfile @@ -0,0 +1,28 @@ +FROM chrislusf/seaweedfs:e2e + +RUN apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get install -y \ + --no-install-recommends \ + --no-install-suggests \ + autoconf \ + automake \ + build-essential \ + git \ + libtap-harness-archive-perl \ + perl \ + && apt-get clean \ + && rm -rf /var/lib/apt/lists/* + +ARG PJDFSTEST_REPO=https://github.com/pjd/pjdfstest.git +ARG PJDFSTEST_REF=03eb25706d8dbf3611c3f820b45b7a5e09a36c06 + +RUN git clone "${PJDFSTEST_REPO}" /opt/pjdfstest && \ + cd /opt/pjdfstest && \ + git checkout "${PJDFSTEST_REF}" && \ + autoreconf -ifs && \ + ./configure && \ + make pjdfstest + +COPY run_inside_container.sh /run.sh +COPY known_failures.txt /known_failures.txt +RUN chmod +x /run.sh diff --git a/test/pjdfstest/docker-compose.yml b/test/pjdfstest/docker-compose.yml new file mode 100644 index 000000000..8b706a9da --- /dev/null +++ b/test/pjdfstest/docker-compose.yml @@ -0,0 +1,57 @@ +services: + master: + image: chrislusf/seaweedfs:e2e + command: "-v=4 master -ip=master -ip.bind=0.0.0.0 -raftBootstrap" + healthcheck: + test: ["CMD", "curl", "--fail", "-I", "http://localhost:9333/cluster/healthz"] + interval: 2s + timeout: 10s + retries: 30 + start_period: 10s + + volume: + image: chrislusf/seaweedfs:e2e + command: "-v=4 volume -master=master:9333 -ip=volume -ip.bind=0.0.0.0 -preStopSeconds=1" + healthcheck: + test: ["CMD", "curl", "--fail", "-I", "http://localhost:8080/healthz"] + interval: 2s + timeout: 10s + retries: 15 + start_period: 5s + depends_on: + master: + condition: service_healthy + + filer: + image: chrislusf/seaweedfs:e2e + command: "-v=4 filer -master=master:9333 -ip=filer -ip.bind=0.0.0.0" + healthcheck: + test: ["CMD", "curl", "--fail", "-I", "http://localhost:8888/healthz"] + interval: 2s + timeout: 10s + retries: 15 + start_period: 5s + depends_on: + volume: + condition: service_healthy + + mount: + image: chrislusf/seaweedfs:pjdfstest + build: + context: . + command: "-v=4 mount -filer=filer:8888 -filer.path=/ -dirAutoCreate -dir=/mnt/seaweedfs -allowOthers" + cap_add: + - SYS_ADMIN + devices: + - /dev/fuse + security_opt: + - apparmor:unconfined + healthcheck: + test: ["CMD", "mountpoint", "-q", "--", "/mnt/seaweedfs"] + interval: 2s + timeout: 10s + retries: 15 + start_period: 10s + depends_on: + filer: + condition: service_healthy diff --git a/test/pjdfstest/known_failures.txt b/test/pjdfstest/known_failures.txt new file mode 100644 index 000000000..a15ddfdc0 --- /dev/null +++ b/test/pjdfstest/known_failures.txt @@ -0,0 +1,67 @@ +# Known pjdfstest failures for SeaweedFS FUSE mount. +# +# Tests listed here are skipped during CI runs. Each entry must be a path +# relative to the pjdfstest root (e.g. tests/chmod/02.t). +# +# A failure in any test NOT listed here will cause the CI job to fail, +# catching regressions immediately. + +# ── Linux FUSE NAME_MAX=255 limitation ────────────────────────────────── +# The Linux FUSE kernel module enforces NAME_MAX=255 at the VFS layer. +# These tests create filenames >255 bytes which cannot be looked up via +# normal syscalls (stat, chmod, etc.) after creation. +tests/chmod/02.t +tests/chmod/03.t +tests/chown/02.t +tests/chown/03.t +tests/ftruncate/02.t +tests/ftruncate/03.t +tests/link/02.t +tests/link/03.t +tests/mkdir/02.t +tests/mkdir/03.t +tests/mkfifo/02.t +tests/mkfifo/03.t +tests/mknod/02.t +tests/mknod/03.t +tests/open/02.t +tests/open/03.t +tests/rename/01.t +tests/rename/02.t +tests/rmdir/02.t +tests/rmdir/03.t +tests/symlink/02.t +tests/symlink/03.t +tests/truncate/02.t +tests/truncate/03.t +tests/unlink/02.t +tests/unlink/03.t + +# ── Hard link nlink/ctime tracking (requires filer changes) ──────────── +# The filer does not update ctime on remaining hard link entries when one +# link is removed, and nlink counts are not correctly maintained across +# rename operations. +tests/rename/23.t +tests/rename/24.t +tests/unlink/00.t + +# ── Parent directory mtime/ctime on deferred file create ─────────────── +# When file creation is deferred (not flushed to filer immediately), +# the parent directory mtime/ctime cannot be updated without invalidating +# the just-cached child entry in the metadata cache. +tests/open/00.t + +# ── Directory rename permission edge case ────────────────────────────── +# Cross-directory rename of a subdirectory with restricted permissions +# causes cascading test failures within the test file. +tests/rename/21.t + +# ── rmdir after hard link unlink ─────────────────────────────────────── +# The filer may still report a directory as non-empty after all hard-linked +# entries have been unlinked. +tests/unlink/14.t + +# ── Nanosecond timestamp precision ───────────────────────────────────── +# The protobuf schema stores timestamps as int64 seconds. Nanosecond +# precision would require adding new fields. +tests/utimensat/08.t diff --git a/test/pjdfstest/run.sh b/test/pjdfstest/run.sh new file mode 100755 index 000000000..7374ec0ba --- /dev/null +++ b/test/pjdfstest/run.sh @@ -0,0 +1,173 @@ +#!/usr/bin/env bash +# +# Run the pjdfstest POSIX compliance suite against a SeaweedFS FUSE mount. +# +# This script: +# 1. Starts a self-contained "weed mini" server (master+volume+filer in one) +# 2. Mounts the filesystem with "weed mount" +# 3. Builds pjdfstest from upstream and runs it under prove(1) +# +# Requirements: weed in $PATH, fusermount3, perl with TAP::Harness::Archive, +# autoconf, make, sudo (pjdfstest exercises chown/chmod which need root). +# +# Usage: +# test/pjdfstest/run.sh # runs full suite +# PJDFSTEST_TESTS=tests/chmod ./run.sh # runs a subset + +set -euo pipefail + +WEED_BIN="${WEED_BIN:-weed}" +WORK_DIR="${WORK_DIR:-$(mktemp -d /tmp/seaweedfs-pjdfstest.XXXXXX)}" +MOUNT_DIR="${MOUNT_DIR:-${WORK_DIR}/mnt}" +DATA_DIR="${DATA_DIR:-${WORK_DIR}/data}" +LOG_DIR="${LOG_DIR:-${WORK_DIR}/logs}" +FILER_PORT="${FILER_PORT:-28888}" +FILER_ADDR="127.0.0.1:${FILER_PORT}" + +# Pin to an immutable upstream commit so CI is reproducible. Override via env +# if you want to test against a different ref or fork. +PJDFSTEST_REPO="${PJDFSTEST_REPO:-https://github.com/pjd/pjdfstest.git}" +PJDFSTEST_REF="${PJDFSTEST_REF:-03eb25706d8dbf3611c3f820b45b7a5e09a36c06}" +PJDFSTEST_TESTS="${PJDFSTEST_TESTS:-tests/}" + +mini_pid="" +mount_pid="" + +CI_LOG_DIR="/tmp/seaweedfs-pjdfstest-logs" + +cleanup() { + set +e + if [[ -n "${mount_pid}" ]] && kill -0 "${mount_pid}" 2>/dev/null; then + kill -TERM "${mount_pid}" 2>/dev/null || true + wait "${mount_pid}" 2>/dev/null || true + fi + if mountpoint -q "${MOUNT_DIR}" 2>/dev/null; then + fusermount3 -u "${MOUNT_DIR}" 2>/dev/null || \ + fusermount -u "${MOUNT_DIR}" 2>/dev/null || \ + sudo umount "${MOUNT_DIR}" 2>/dev/null || true + fi + if [[ -n "${mini_pid}" ]] && kill -0 "${mini_pid}" 2>/dev/null; then + kill -TERM "${mini_pid}" 2>/dev/null || true + wait "${mini_pid}" 2>/dev/null || true + fi + # Copy logs to a fixed path for CI artifact upload. + mkdir -p "${CI_LOG_DIR}" + cp "${LOG_DIR}"/*.log "${CI_LOG_DIR}/" 2>/dev/null || true +} +trap cleanup EXIT INT TERM + +mkdir -p "${MOUNT_DIR}" "${DATA_DIR}" "${LOG_DIR}" + +echo "==> Starting weed mini on ${FILER_ADDR}" +"${WEED_BIN}" mini \ + -dir="${DATA_DIR}" \ + -ip=127.0.0.1 \ + -filer.port="${FILER_PORT}" \ + -s3=false \ + -webdav=false \ + -admin.ui=false \ + >"${LOG_DIR}/mini.log" 2>&1 & +mini_pid=$! + +# Wait for filer to accept connections. +for i in $(seq 1 60); do + if (echo > "/dev/tcp/127.0.0.1/${FILER_PORT}") 2>/dev/null; then + break + fi + if ! kill -0 "${mini_pid}" 2>/dev/null; then + echo "weed mini exited early; log tail:" >&2 + tail -n 100 "${LOG_DIR}/mini.log" >&2 || true + exit 1 + fi + sleep 0.5 +done + +if ! (echo > "/dev/tcp/127.0.0.1/${FILER_PORT}") 2>/dev/null; then + echo "weed mini filer did not become reachable within 30s; log tail:" >&2 + tail -n 100 "${LOG_DIR}/mini.log" >&2 || true + exit 1 +fi + +echo "==> Mounting SeaweedFS at ${MOUNT_DIR}" +# allowOthers is required so that pjdfstest's setuid/setgid sub-tests (run via +# sudo) can access files created as the invoking user. +sudo "${WEED_BIN}" mount \ + -filer="${FILER_ADDR}" \ + -dir="${MOUNT_DIR}" \ + -filer.path=/ \ + -dirAutoCreate \ + -allowOthers=true \ + >"${LOG_DIR}/mount.log" 2>&1 & +mount_pid=$! + +for i in $(seq 1 60); do + if mountpoint -q "${MOUNT_DIR}"; then + break + fi + if ! kill -0 "${mount_pid}" 2>/dev/null; then + echo "weed mount exited early; log tail:" >&2 + tail -n 100 "${LOG_DIR}/mount.log" >&2 || true + exit 1 + fi + sleep 0.5 +done + +if ! mountpoint -q "${MOUNT_DIR}"; then + echo "FUSE mount did not come up within 30s" >&2 + tail -n 100 "${LOG_DIR}/mount.log" >&2 || true + exit 1 +fi + +echo "==> Cloning and building pjdfstest" +PJDFSTEST_DIR="${WORK_DIR}/pjdfstest" +if [[ ! -d "${PJDFSTEST_DIR}/.git" ]]; then + git clone "${PJDFSTEST_REPO}" "${PJDFSTEST_DIR}" +fi +git -C "${PJDFSTEST_DIR}" remote set-url origin "${PJDFSTEST_REPO}" +git -C "${PJDFSTEST_DIR}" fetch --depth 1 origin "${PJDFSTEST_REF}" +git -C "${PJDFSTEST_DIR}" checkout --detach FETCH_HEAD +( + cd "${PJDFSTEST_DIR}" + autoreconf -ifs + ./configure + make pjdfstest +) + +# pjdfstest must run inside the filesystem under test. +TEST_ROOT="${MOUNT_DIR}/pjdfstest-root" +sudo mkdir -p "${TEST_ROOT}" +sudo cp -r "${PJDFSTEST_DIR}/." "${TEST_ROOT}/" + +cd "${TEST_ROOT}" + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +KNOWN_FAILURES="${SCRIPT_DIR}/known_failures.txt" + +# Build the list of tests to run, excluding known failures. +if [[ -f "${KNOWN_FAILURES}" ]] && [[ "${PJDFSTEST_TESTS}" == "tests/" ]]; then + mapfile -t skip < <(grep -v '^#' "${KNOWN_FAILURES}" | grep -v '^$') + all_tests=() + while IFS= read -r -d '' t; do + all_tests+=("$t") + done < <(find tests/ -name '*.t' -print0 | sort -z) + + run_tests=() + for t in "${all_tests[@]}"; do + is_skipped=false + for s in "${skip[@]}"; do + if [[ "$t" == "$s" ]]; then + is_skipped=true + break + fi + done + if ! $is_skipped; then + run_tests+=("$t") + fi + done + + echo "==> Running pjdfstest (${#run_tests[@]} tests, ${#skip[@]} skipped)" + sudo prove -rv "${run_tests[@]}" +else + echo "==> Running pjdfstest (${PJDFSTEST_TESTS})" + sudo prove -rv "${PJDFSTEST_TESTS}" +fi diff --git a/test/pjdfstest/run_inside_container.sh b/test/pjdfstest/run_inside_container.sh new file mode 100755 index 000000000..4777f6bf9 --- /dev/null +++ b/test/pjdfstest/run_inside_container.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# +# Runs pjdfstest inside the mount container against /mnt/seaweedfs. +# Invoked via: docker compose exec mount /run.sh +# +# Uses known_failures.txt to skip tests with known issues. Any failure +# in a test NOT in the skip list causes a non-zero exit (regression). +set -euo pipefail + +MOUNT_DIR="/mnt/seaweedfs" +PJDFSTEST_TESTS="${PJDFSTEST_TESTS:-tests/}" +KNOWN_FAILURES="/known_failures.txt" + +# Copy pjdfstest into the mounted filesystem so tests exercise the FS under test. +TEST_ROOT="${MOUNT_DIR}/pjdfstest-root" +mkdir -p "${TEST_ROOT}" +cp -r /opt/pjdfstest/. "${TEST_ROOT}/" + +cd "${TEST_ROOT}" + +# Build the list of tests to run, excluding known failures. +if [[ -f "${KNOWN_FAILURES}" ]] && [[ "${PJDFSTEST_TESTS}" == "tests/" ]]; then + mapfile -t skip < <(grep -v '^#' "${KNOWN_FAILURES}" | grep -v '^$') + all_tests=() + while IFS= read -r -d '' t; do + all_tests+=("$t") + done < <(find tests/ -name '*.t' -print0 | sort -z) + + run_tests=() + for t in "${all_tests[@]}"; do + is_skipped=false + for s in "${skip[@]}"; do + if [[ "$t" == "$s" ]]; then + is_skipped=true + break + fi + done + if ! $is_skipped; then + run_tests+=("$t") + fi + done + + echo "==> Running pjdfstest (${#run_tests[@]} tests, ${#skip[@]} skipped)" + prove -rv "${run_tests[@]}" +else + echo "==> Running pjdfstest (${PJDFSTEST_TESTS})" + prove -rv "${PJDFSTEST_TESTS}" +fi diff --git a/weed/filer/entry.go b/weed/filer/entry.go index 237ddc964..43ceec51c 100644 --- a/weed/filer/entry.go +++ b/weed/filer/entry.go @@ -12,6 +12,7 @@ import ( type Attr struct { Mtime time.Time // time of last modification Crtime time.Time // time of creation (OS X only) + Ctime time.Time // time of last inode change Mode os.FileMode // file mode Uid uint32 // owner uid Gid uint32 // group gid diff --git a/weed/filer/entry_codec.go b/weed/filer/entry_codec.go index 1c096c911..bff56c008 100644 --- a/weed/filer/entry_codec.go +++ b/weed/filer/entry_codec.go @@ -82,6 +82,7 @@ func EntryAttributeToPb(entry *Entry) *filer_pb.FuseAttributes { return &filer_pb.FuseAttributes{ Crtime: entry.Attr.Crtime.Unix(), Mtime: entry.Attr.Mtime.Unix(), + Ctime: entry.Attr.Ctime.Unix(), FileMode: uint32(entry.Attr.Mode), Uid: entry.Uid, Gid: entry.Gid, @@ -105,6 +106,7 @@ func EntryAttributeToExistingPb(entry *Entry, attr *filer_pb.FuseAttributes) { } attr.Crtime = entry.Attr.Crtime.Unix() attr.Mtime = entry.Attr.Mtime.Unix() + attr.Ctime = entry.Attr.Ctime.Unix() attr.FileMode = uint32(entry.Attr.Mode) attr.Uid = entry.Uid attr.Gid = entry.Gid @@ -129,6 +131,11 @@ func PbToEntryAttribute(attr *filer_pb.FuseAttributes) Attr { t.Crtime = time.Unix(attr.Crtime, 0) t.Mtime = time.Unix(attr.Mtime, 0) + if attr.Ctime != 0 { + t.Ctime = time.Unix(attr.Ctime, 0) + } else { + t.Ctime = t.Mtime + } t.Mode = os.FileMode(attr.FileMode) t.Uid = attr.Uid t.Gid = attr.Gid diff --git a/weed/mount/weedfs.go b/weed/mount/weedfs.go index 509e6d813..2f6dd07d1 100644 --- a/weed/mount/weedfs.go +++ b/weed/mount/weedfs.go @@ -123,6 +123,8 @@ type WFS struct { filerClient *wdclient.FilerClient // Cached volume location client refreshMu sync.Mutex refreshingDirs map[util.FullPath]struct{} + atimeMu sync.Mutex + atimeMap map[uint64]int64 // inode -> atime (unix seconds), in-memory only, bounded dirHotWindow time.Duration dirHotThreshold int dirIdleEvict time.Duration @@ -202,6 +204,7 @@ func NewSeaweedFileSystem(option *Option) *WFS { fhLockTable: util.NewLockTable[FileHandleId](), posixLocks: NewPosixLockTable(), refreshingDirs: make(map[util.FullPath]struct{}), + atimeMap: make(map[uint64]int64, 8192), dirHotWindow: dirHotWindow, dirHotThreshold: dirHotThreshold, dirIdleEvict: dirIdleEvict, diff --git a/weed/mount/weedfs_access.go b/weed/mount/weedfs_access.go index f61c0593e..d701d5782 100644 --- a/weed/mount/weedfs_access.go +++ b/weed/mount/weedfs_access.go @@ -69,12 +69,11 @@ func hasAccess(callerUid, callerGid, fileUid, fileGid uint32, perm uint32, mask if !isMember { groupIDs, err := lookupSupplementaryGroupIDs(callerUid) if err != nil { - // Cannot determine group membership; require both group and - // other permission classes to satisfy the mask so we never - // overgrant when the lookup fails. - groupMatch := ((perm >> 3) & mask) == mask - otherMatch := (perm & mask) == mask - return groupMatch && otherMatch + // Cannot determine supplementary group membership. + // Fall through to "other" permission check since we already + // know the caller is not the owner (checked above) and not + // in the primary group. + return (perm & mask) == mask } fileGidStr := strconv.Itoa(int(fileGid)) for _, gidStr := range groupIDs { @@ -92,6 +91,19 @@ func hasAccess(callerUid, callerGid, fileUid, fileGid uint32, perm uint32, mask return (perm & mask) == mask } +// checkStickyBit enforces the POSIX sticky-bit rule: when a directory has the +// sticky bit set, only the file owner, the directory owner, or root may +// delete or rename entries within it. +func checkStickyBit(dirMode, dirUid, targetUid, callerUid uint32) fuse.Status { + if dirMode&0o1000 == 0 { + return fuse.OK + } + if callerUid == 0 || callerUid == dirUid || callerUid == targetUid { + return fuse.OK + } + return fuse.EPERM +} + // openFlagsToAccessMask converts open(2) flags to an access permission mask. func openFlagsToAccessMask(flags uint32) uint32 { switch flags & uint32(syscall.O_ACCMODE) { diff --git a/weed/mount/weedfs_attr.go b/weed/mount/weedfs_attr.go index 534402998..9157df8e1 100644 --- a/weed/mount/weedfs_attr.go +++ b/weed/mount/weedfs_attr.go @@ -24,6 +24,7 @@ func (wfs *WFS) GetAttr(cancel <-chan struct{}, input *fuse.GetAttrIn, out *fuse if status == fuse.OK { out.AttrValid = 1 wfs.setAttrByPbEntry(&out.Attr, inode, entry, true) + wfs.applyInMemoryAtime(&out.Attr, inode) return status } else { if fh, found := wfs.fhMap.FindFileHandle(inode); found { @@ -32,6 +33,7 @@ func (wfs *WFS) GetAttr(cancel <-chan struct{}, input *fuse.GetAttrIn, out *fuse fhActiveLock := wfs.fhLockTable.AcquireLock("GetAttr", fh.fh, util.SharedLock) wfs.setAttrByPbEntry(&out.Attr, inode, fh.entry.GetEntry(), true) wfs.fhLockTable.ReleaseLock(fh.fh, fhActiveLock) + wfs.applyInMemoryAtime(&out.Attr, inode) out.Nlink = 0 return fuse.OK } @@ -108,8 +110,10 @@ func (wfs *WFS) SetAttr(cancel <-chan struct{}, input *fuse.SetAttrIn, out *fuse } } + ownerChanged := false if uid, ok := input.GetUID(); ok { entry.Attributes.Uid = uid + ownerChanged = true if input.NodeId == 1 { wfs.option.MountUid = uid } @@ -117,25 +121,35 @@ func (wfs *WFS) SetAttr(cancel <-chan struct{}, input *fuse.SetAttrIn, out *fuse if gid, ok := input.GetGID(); ok { entry.Attributes.Gid = gid + ownerChanged = true if input.NodeId == 1 { wfs.option.MountGid = gid } } + // POSIX: clear SUID/SGID bits when ownership changes (unless caller is root). + if ownerChanged && input.Uid != 0 { + entry.Attributes.FileMode &^= 0o6000 + } + if atime, ok := input.GetATime(); ok { - entry.Attributes.Mtime = atime.Unix() + wfs.setAtime(input.NodeId, atime.Unix()) } if mtime, ok := input.GetMTime(); ok { entry.Attributes.Mtime = mtime.Unix() } + // POSIX: update ctime on any metadata change. + entry.Attributes.Ctime = time.Now().Unix() + out.AttrValid = 1 size, includeSize := input.GetSize() if includeSize { out.Attr.Size = size } wfs.setAttrByPbEntry(&out.Attr, input.NodeId, entry, !includeSize) + wfs.applyInMemoryAtime(&out.Attr, input.NodeId) if fh != nil { fh.dirtyMetadata = true @@ -177,8 +191,13 @@ func (wfs *WFS) setAttrByPbEntry(out *fuse.Attr, inode uint64, entry *filer_pb.E } out.Blocks = (out.Size + blockSize - 1) / blockSize out.Mtime = uint64(entry.Attributes.Mtime) - out.Ctime = uint64(entry.Attributes.Mtime) + if entry.Attributes.Ctime != 0 { + out.Ctime = uint64(entry.Attributes.Ctime) + } else { + out.Ctime = uint64(entry.Attributes.Mtime) + } out.Atime = uint64(entry.Attributes.Mtime) + // In-memory atime overlay is applied by the caller via applyInMemoryAtime. out.Mode = toSyscallMode(os.FileMode(entry.Attributes.FileMode)) if entry.HardLinkCounter > 0 { out.Nlink = uint32(entry.HardLinkCounter) @@ -200,7 +219,11 @@ func (wfs *WFS) setAttrByFilerEntry(out *fuse.Attr, inode uint64, entry *filer.E setBlksize(out, blockSize) out.Atime = uint64(entry.Attr.Mtime.Unix()) out.Mtime = uint64(entry.Attr.Mtime.Unix()) - out.Ctime = uint64(entry.Attr.Mtime.Unix()) + if !entry.Attr.Ctime.IsZero() { + out.Ctime = uint64(entry.Attr.Ctime.Unix()) + } else { + out.Ctime = uint64(entry.Attr.Mtime.Unix()) + } out.Mode = toSyscallMode(entry.Attr.Mode) if entry.HardLinkCounter > 0 { out.Nlink = uint32(entry.HardLinkCounter) @@ -228,6 +251,45 @@ func (wfs *WFS) outputFilerEntry(out *fuse.EntryOut, inode uint64, entry *filer. wfs.setAttrByFilerEntry(&out.Attr, inode, entry) } +// touchDirMtimeCtime updates a directory's mtime and ctime on the filer. +// POSIX requires this when entries are created or removed in the directory. +func (wfs *WFS) touchDirMtimeCtime(dirPath util.FullPath) { + dirEntry, code := wfs.maybeLoadEntry(dirPath) + if code != fuse.OK || dirEntry == nil || dirEntry.Attributes == nil { + return + } + now := time.Now().Unix() + dirEntry.Attributes.Mtime = now + dirEntry.Attributes.Ctime = now + wfs.saveEntry(dirPath, dirEntry) +} + +const atimeMapMaxSize = 8192 + +// setAtime stores an in-memory atime for an inode. The map is bounded; +// when full, a random entry is evicted. +func (wfs *WFS) setAtime(inode uint64, t int64) { + wfs.atimeMu.Lock() + defer wfs.atimeMu.Unlock() + if len(wfs.atimeMap) >= atimeMapMaxSize { + // evict one random entry + for k := range wfs.atimeMap { + delete(wfs.atimeMap, k) + break + } + } + wfs.atimeMap[inode] = t +} + +// applyInMemoryAtime overlays the in-memory atime onto a fuse.Attr if present. +func (wfs *WFS) applyInMemoryAtime(out *fuse.Attr, inode uint64) { + wfs.atimeMu.Lock() + if t, ok := wfs.atimeMap[inode]; ok { + out.Atime = uint64(t) + } + wfs.atimeMu.Unlock() +} + func chmod(existing uint32, mode uint32) uint32 { return existing&^07777 | mode&07777 } diff --git a/weed/mount/weedfs_dir_mkrm.go b/weed/mount/weedfs_dir_mkrm.go index d966258e7..d569e265d 100644 --- a/weed/mount/weedfs_dir_mkrm.go +++ b/weed/mount/weedfs_dir_mkrm.go @@ -30,12 +30,14 @@ func (wfs *WFS) Mkdir(cancel <-chan struct{}, in *fuse.MkdirIn, name string, out return s } + now := time.Now().Unix() newEntry := &filer_pb.Entry{ Name: name, IsDirectory: true, Attributes: &filer_pb.FuseAttributes{ - Mtime: time.Now().Unix(), - Crtime: time.Now().Unix(), + Mtime: now, + Crtime: now, + Ctime: now, FileMode: uint32(os.ModeDir) | in.Mode&^uint32(wfs.option.Umask), Uid: in.Uid, Gid: in.Gid, @@ -76,6 +78,7 @@ func (wfs *WFS) Mkdir(cancel <-chan struct{}, in *fuse.MkdirIn, name string, out wfs.inodeToPath.InvalidateChildrenCache(dirFullPath) } wfs.inodeToPath.TouchDirectory(dirFullPath) + wfs.touchDirMtimeCtime(dirFullPath) } glog.V(3).Infof("mkdir %s: %v", entryFullPath, err) @@ -114,6 +117,17 @@ func (wfs *WFS) Rmdir(cancel <-chan struct{}, header *fuse.InHeader, name string } entryFullPath := dirFullPath.Child(name) + // POSIX: enforce sticky bit on the parent directory. + if dirEntry, dirCode := wfs.maybeLoadEntry(dirFullPath); dirCode == fuse.OK && dirEntry != nil && dirEntry.Attributes != nil { + targetUid := uint32(0) + if targetEntry, targetCode := wfs.maybeLoadEntry(entryFullPath); targetCode == fuse.OK && targetEntry != nil && targetEntry.Attributes != nil { + targetUid = targetEntry.Attributes.Uid + } + if code := checkStickyBit(dirEntry.Attributes.FileMode, dirEntry.Attributes.Uid, targetUid, header.Uid); code != fuse.OK { + return code + } + } + glog.V(3).Infof("remove directory: %v", entryFullPath) deleteReq := &filer_pb.DeleteEntryRequest{ Directory: string(dirFullPath), @@ -141,6 +155,7 @@ func (wfs *WFS) Rmdir(cancel <-chan struct{}, header *fuse.InHeader, name string } wfs.inodeToPath.RemovePath(entryFullPath) wfs.inodeToPath.TouchDirectory(dirFullPath) + wfs.touchDirMtimeCtime(dirFullPath) return fuse.OK diff --git a/weed/mount/weedfs_file_mkrm.go b/weed/mount/weedfs_file_mkrm.go index 8e9007203..d3636af80 100644 --- a/weed/mount/weedfs_file_mkrm.go +++ b/weed/mount/weedfs_file_mkrm.go @@ -186,6 +186,17 @@ func (wfs *WFS) Unlink(cancel <-chan struct{}, header *fuse.InHeader, name strin return fuse.EPERM } + // POSIX: enforce sticky bit on the parent directory. + if dirEntry, dirCode := wfs.maybeLoadEntry(dirFullPath); dirCode == fuse.OK && dirEntry != nil && dirEntry.Attributes != nil { + targetUid := uint32(0) + if entry != nil && entry.Attributes != nil { + targetUid = entry.Attributes.Uid + } + if code := checkStickyBit(dirEntry.Attributes.FileMode, dirEntry.Attributes.Uid, targetUid, header.Uid); code != fuse.OK { + return code + } + } + // Before deleting from the filer, mark any draining async-flush handle // as deleted and wait for it to complete. Without this, the async flush // can race with the filer delete and recreate the just-unlinked entry @@ -233,6 +244,7 @@ func (wfs *WFS) Unlink(cancel <-chan struct{}, header *fuse.InHeader, name strin wfs.inodeToPath.InvalidateChildrenCache(dirFullPath) } wfs.inodeToPath.TouchDirectory(dirFullPath) + wfs.touchDirMtimeCtime(dirFullPath) wfs.inodeToPath.RemovePath(entryFullPath) @@ -279,6 +291,7 @@ func (wfs *WFS) createRegularFile(dirFullPath util.FullPath, name string, mode u Attributes: &filer_pb.FuseAttributes{ Mtime: now, Crtime: now, + Ctime: now, FileMode: uint32(fileMode), Uid: uid, Gid: gid, @@ -327,6 +340,7 @@ func (wfs *WFS) createRegularFile(dirFullPath util.FullPath, name string, mode u wfs.inodeToPath.InvalidateChildrenCache(dirFullPath) } wfs.inodeToPath.TouchDirectory(dirFullPath) + wfs.touchDirMtimeCtime(dirFullPath) } glog.V(3).Infof("createFile %s: %v", entryFullPath, err) @@ -349,7 +363,9 @@ func (wfs *WFS) truncateEntry(entryFullPath util.FullPath, entry *filer_pb.Entry entry.Content = nil entry.Chunks = nil entry.Attributes.FileSize = 0 - entry.Attributes.Mtime = time.Now().Unix() + now := time.Now().Unix() + entry.Attributes.Mtime = now + entry.Attributes.Ctime = now if code := wfs.saveEntry(entryFullPath, entry); code != fuse.OK { return code diff --git a/weed/mount/weedfs_file_sync.go b/weed/mount/weedfs_file_sync.go index 55db0b165..0dcf0e87c 100644 --- a/weed/mount/weedfs_file_sync.go +++ b/weed/mount/weedfs_file_sync.go @@ -189,7 +189,9 @@ func (wfs *WFS) flushMetadataToFiler(fh *FileHandle, dir, name string, uid, gid if entry.Attributes.Gid == 0 { entry.Attributes.Gid = gid } - entry.Attributes.Mtime = time.Now().Unix() + now := time.Now().Unix() + entry.Attributes.Mtime = now + entry.Attributes.Ctime = now } glog.V(4).Infof("%s set chunks: %v", fileFullPath, len(entry.GetChunks())) diff --git a/weed/mount/weedfs_file_write.go b/weed/mount/weedfs_file_write.go index c1ddb70d3..63c621abe 100644 --- a/weed/mount/weedfs_file_write.go +++ b/weed/mount/weedfs_file_write.go @@ -88,6 +88,11 @@ func (wfs *WFS) Write(cancel <-chan struct{}, in *fuse.WriteIn, data []byte) (wr fh.dirtyMetadata = true + // POSIX: clear SUID/SGID bits on write by non-root users. + if in.Uid != 0 { + entry.Attributes.FileMode &^= 0o6000 + } + if IsDebugFileReadWrite { // print("+") fh.mirrorFile.WriteAt(data, offset) diff --git a/weed/mount/weedfs_filehandle.go b/weed/mount/weedfs_filehandle.go index 0d5ea2ef8..6f9cd376e 100644 --- a/weed/mount/weedfs_filehandle.go +++ b/weed/mount/weedfs_filehandle.go @@ -26,7 +26,11 @@ func (wfs *WFS) AcquireHandle(inode uint64, flags, uid, gid uint32) (fileHandle } // Check unix permission bits for the requested access mode. if entry != nil && entry.Attributes != nil { - if mask := openFlagsToAccessMask(flags); mask != 0 && !hasAccess(uid, gid, entry.Attributes.Uid, entry.Attributes.Gid, entry.Attributes.FileMode, mask) { + fileUid, fileGid := entry.Attributes.Uid, entry.Attributes.Gid + if wfs.option.UidGidMapper != nil { + fileUid, fileGid = wfs.option.UidGidMapper.FilerToLocal(fileUid, fileGid) + } + if mask := openFlagsToAccessMask(flags); mask != 0 && !hasAccess(uid, gid, fileUid, fileGid, entry.Attributes.FileMode, mask) { return nil, fuse.EACCES } } diff --git a/weed/mount/weedfs_link.go b/weed/mount/weedfs_link.go index 613450b00..b4f399fcc 100644 --- a/weed/mount/weedfs_link.go +++ b/weed/mount/weedfs_link.go @@ -72,7 +72,9 @@ func (wfs *WFS) Link(cancel <-chan struct{}, in *fuse.LinkIn, name string, out * } // CreateLink 1.2 : update new file to hardlink mode - oldEntry.Attributes.Mtime = time.Now().Unix() + now := time.Now().Unix() + oldEntry.Attributes.Mtime = now + oldEntry.Attributes.Ctime = now request := &filer_pb.CreateEntryRequest{ Directory: string(newParentPath), Entry: &filer_pb.Entry{ @@ -127,6 +129,7 @@ func (wfs *WFS) Link(cancel <-chan struct{}, in *fuse.LinkIn, name string, out * glog.Warningf("link %s: best-effort metadata apply failed: %v", newParentPath.Child(name), applyErr) wfs.inodeToPath.InvalidateChildrenCache(newParentPath) } + wfs.touchDirMtimeCtime(newParentPath) } } diff --git a/weed/mount/weedfs_rename.go b/weed/mount/weedfs_rename.go index 7e56a3e27..88c02215c 100644 --- a/weed/mount/weedfs_rename.go +++ b/weed/mount/weedfs_rename.go @@ -199,6 +199,32 @@ func (wfs *WFS) Rename(cancel <-chan struct{}, in *fuse.RenameIn, oldName string return status } + // POSIX: enforce sticky bit on the source directory. + if oldDirEntry, dirCode := wfs.maybeLoadEntry(oldDir); dirCode == fuse.OK && oldDirEntry != nil && oldDirEntry.Attributes != nil { + targetUid := uint32(0) + if oldEntry != nil && oldEntry.Attributes != nil { + targetUid = oldEntry.Attributes.Uid + } + if code := checkStickyBit(oldDirEntry.Attributes.FileMode, oldDirEntry.Attributes.Uid, targetUid, in.Uid); code != fuse.OK { + return code + } + } + + // POSIX: enforce sticky bit on the destination directory when replacing an existing entry. + if in.Flags != RenameNoReplace { + if newEntry, newStatus := wfs.maybeLoadEntry(newPath); newStatus == fuse.OK && newEntry != nil { + if newDirEntry, dirCode := wfs.maybeLoadEntry(newDir); dirCode == fuse.OK && newDirEntry != nil && newDirEntry.Attributes != nil { + targetUid := uint32(0) + if newEntry.Attributes != nil { + targetUid = newEntry.Attributes.Uid + } + if code := checkStickyBit(newDirEntry.Attributes.FileMode, newDirEntry.Attributes.Uid, targetUid, in.Uid); code != fuse.OK { + return code + } + } + } + } + if wormEnforced, _ := wfs.wormEnforcedForEntry(oldPath, oldEntry); wormEnforced { return fuse.EPERM } @@ -297,6 +323,10 @@ func (wfs *WFS) Rename(cancel <-chan struct{}, in *fuse.RenameIn, oldName string } wfs.inodeToPath.TouchDirectory(oldDir) wfs.inodeToPath.TouchDirectory(newDir) + wfs.touchDirMtimeCtime(oldDir) + if oldDir != newDir { + wfs.touchDirMtimeCtime(newDir) + } return fuse.OK diff --git a/weed/mount/weedfs_symlink.go b/weed/mount/weedfs_symlink.go index c30588cbe..d6ba244a6 100644 --- a/weed/mount/weedfs_symlink.go +++ b/weed/mount/weedfs_symlink.go @@ -28,14 +28,16 @@ func (wfs *WFS) Symlink(cancel <-chan struct{}, header *fuse.InHeader, target st } entryFullPath := dirPath.Child(name) + now := time.Now().Unix() request := &filer_pb.CreateEntryRequest{ Directory: string(dirPath), Entry: &filer_pb.Entry{ Name: name, IsDirectory: false, Attributes: &filer_pb.FuseAttributes{ - Mtime: time.Now().Unix(), - Crtime: time.Now().Unix(), + Mtime: now, + Crtime: now, + Ctime: now, FileMode: uint32(os.FileMode(0777) | os.ModeSymlink), Uid: header.Uid, Gid: header.Gid, @@ -58,6 +60,7 @@ func (wfs *WFS) Symlink(cancel <-chan struct{}, header *fuse.InHeader, target st glog.Warningf("symlink %s: best-effort metadata apply failed: %v", entryFullPath, applyErr) wfs.inodeToPath.InvalidateChildrenCache(dirPath) } + wfs.touchDirMtimeCtime(dirPath) } // Map back to local uid/gid before writing to the kernel. diff --git a/weed/mount/wfs_save.go b/weed/mount/wfs_save.go index 3e2464117..54ede4152 100644 --- a/weed/mount/wfs_save.go +++ b/weed/mount/wfs_save.go @@ -66,7 +66,10 @@ func (wfs *WFS) mapPbIdFromLocalToFiler(entry *filer_pb.Entry) { } func checkName(name string) fuse.Status { - if len(name) >= 4096 { + // The Linux FUSE kernel module enforces NAME_MAX=255 at the VFS layer. + // Return ENAMETOOLONG early to avoid creating entries that cannot be + // looked up via normal syscalls (stat, chmod, etc.). + if len(name) > 255 { return fuse.Status(syscall.ENAMETOOLONG) } return fuse.OK diff --git a/weed/pb/filer.proto b/weed/pb/filer.proto index d7067def0..0d31a5e0f 100644 --- a/weed/pb/filer.proto +++ b/weed/pb/filer.proto @@ -197,6 +197,7 @@ message FuseAttributes { bytes md5 = 14; uint32 rdev = 16; uint64 inode = 17; + int64 ctime = 18; // unix time in seconds, inode change time } message CreateEntryRequest { diff --git a/weed/pb/filer_pb/filer.pb.go b/weed/pb/filer_pb/filer.pb.go index 45ea561a8..2f52defad 100644 --- a/weed/pb/filer_pb/filer.pb.go +++ b/weed/pb/filer_pb/filer.pb.go @@ -961,6 +961,7 @@ type FuseAttributes struct { Md5 []byte `protobuf:"bytes,14,opt,name=md5,proto3" json:"md5,omitempty"` Rdev uint32 `protobuf:"varint,16,opt,name=rdev,proto3" json:"rdev,omitempty"` Inode uint64 `protobuf:"varint,17,opt,name=inode,proto3" json:"inode,omitempty"` + Ctime int64 `protobuf:"varint,18,opt,name=ctime,proto3" json:"ctime,omitempty"` // unix time in seconds, inode change time unknownFields protoimpl.UnknownFields sizeCache protoimpl.SizeCache } @@ -1093,6 +1094,13 @@ func (x *FuseAttributes) GetInode() uint64 { return 0 } +func (x *FuseAttributes) GetCtime() int64 { + if x != nil { + return x.Ctime + } + return 0 +} + type CreateEntryRequest struct { state protoimpl.MessageState `protogen:"open.v1"` Directory string `protobuf:"bytes,1,opt,name=directory,proto3" json:"directory,omitempty"` @@ -5136,7 +5144,7 @@ const file_filer_proto_rawDesc = "" + "\x06FileId\x12\x1b\n" + "\tvolume_id\x18\x01 \x01(\rR\bvolumeId\x12\x19\n" + "\bfile_key\x18\x02 \x01(\x04R\afileKey\x12\x16\n" + - "\x06cookie\x18\x03 \x01(\aR\x06cookie\"\xe8\x02\n" + + "\x06cookie\x18\x03 \x01(\aR\x06cookie\"\xfe\x02\n" + "\x0eFuseAttributes\x12\x1b\n" + "\tfile_size\x18\x01 \x01(\x04R\bfileSize\x12\x14\n" + "\x05mtime\x18\x02 \x01(\x03R\x05mtime\x12\x1b\n" + @@ -5153,7 +5161,8 @@ const file_filer_proto_rawDesc = "" + "\x0esymlink_target\x18\r \x01(\tR\rsymlinkTarget\x12\x10\n" + "\x03md5\x18\x0e \x01(\fR\x03md5\x12\x12\n" + "\x04rdev\x18\x10 \x01(\rR\x04rdev\x12\x14\n" + - "\x05inode\x18\x11 \x01(\x04R\x05inode\"\x82\x02\n" + + "\x05inode\x18\x11 \x01(\x04R\x05inode\x12\x14\n" + + "\x05ctime\x18\x12 \x01(\x03R\x05ctime\"\x82\x02\n" + "\x12CreateEntryRequest\x12\x1c\n" + "\tdirectory\x18\x01 \x01(\tR\tdirectory\x12%\n" + "\x05entry\x18\x02 \x01(\v2\x0f.filer_pb.EntryR\x05entry\x12\x15\n" +