6 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandGitHub	bfb2661fec	fix(tests): make 32-bit GOARCH tests build and run (#9507) ... fix(tests): make 32-bit GOARCH tests build and run (#9503) verifyTestFilerClient had bare int64 atomic counters after a map header, so atomic.AddInt64 panicked with "unaligned 64-bit atomic operation" on linux/386. Switch to atomic.Int64, which the stdlib guarantees is 8-byte aligned on all platforms. rpc_version_filter_test.go passed the untyped constant 0xdeadbeef to t.Errorf, where it default-promoted to int and overflowed 32-bit int. Bind it to a typed uint32 const used in both the comparison and the error message.	2026-05-14 20:55:37 -07:00
Chris LuandGitHub	d265274e13	fix(nfs): accept dirpath any-where under the export, mirroring rclone (#9291) ... * fix(nfs): accept any MOUNT3 dirpath, mirroring rclone's permissive policy weed nfs has exactly one export per process, so the MOUNT3 dirpath argument has no second export to disambiguate against. Strict comparison only translated PV-path typos into the inconsistent "mount succeeds but empty" / "mount fails completely" split that operators see. Match rclone's serve nfs Handler.Mount: ignore the dirpath, log an INFO line when it differs from the configured export, and always serve the export root. Apply the same change to the UDP MOUNT3 path so kernel clients defaulting to mountproto=udp see identical behaviour. Access control still goes through -allowedClients / -ip.bind, and file-handle scoping in FromHandle is unchanged so handles still cannot escape the export. Replace the prior single-path reject tests with table tests covering the shapes operators commonly hit: root, parent, sibling, deeper child, unrelated, empty, relative form, exact match, and trailing slash, at the Handler.Mount, UDP MOUNT3, and full RPC layers. * feat(nfs): mount at subdirectory when MOUNT3 dirpath is under the export Make the dirpath argument meaningful when the client asks for a subtree of the configured export. With -filer.path=/buckets, a client mounting <server>:/buckets/data lands directly inside /buckets/data instead of at the export root. - dirpath equals the export root: serve the export root. - dirpath strictly under the export, directory entry: serve that subdirectory; the returned filehandle encodes its inode. - dirpath strictly under the export, missing or non-directory: reject with NoEnt or NotDir. - dirpath outside the export: keep the rclone-style fallback to the export root. TCP returns a sub-rooted seaweedFileSystem and lets go-nfs's onMount call ToHandle to encode the FH; UDP encodes the FH itself. FromHandle is unchanged: handles are content-addressed by inode and resolve via the inode index, so they remain stable across mounts and across process restarts. The trimmed permissive tests keep their outside-export shapes; new subexport tests cover under-export directories, missing entries, and non-directory entries on Handler.Mount, the UDP MOUNT3 wire, and through the full RPC stack. * nfs: propagate request context through MOUNT3 resolution Mount now accepts the gonfs context and threads it through resolveMountFilesystem and lstatExportStatus so a slow filer call during MOUNT cannot outlive a cancelled or timed-out request. lstatExportStatus uses fileInfoForVirtualPath(ctx, "/") directly instead of billy.Filesystem.Lstat, which would otherwise drop the context on the floor by calling fileInfoForVirtualPathWithOptions with context.Background(). Lower the successful subexport-mount log from V(0) to V(1). The fallback log stays at V(0) so operator typos still surface; the success line is per-mount churn that adds up on NFS-CSI deployments. * nfs: mirror TCP defensive checks on the UDP MOUNT3 path Two transport-parity bugs the rabbit caught: (1) The exact-export-root and outside-export branches were returning MNT3_OK unconditionally, while the TCP handler runs lstatExportStatus on those same branches. If the configured -filer.path has been removed from the filer, TCP returns NoEnt/ServerFault but UDP would still hand out a synthetic root handle pointing at nothing. Add rootMountStatus as the UDP analogue and call it on both branches. (2) resolveSubexportFileHandle did filer I/O on the single UDP serve loop with context.Background(). One slow filer round-trip would block every later MOUNT packet. Wrap each MOUNT call's filer work in context.WithTimeout(mountUDPLookupTimeout) and thread that ctx through both rootMountStatus and resolveSubexportFileHandle. Lower the successful subexport log to V(1) to match the TCP side. * nfs: assert TCP/UDP MOUNT3 produce byte-identical filehandles The existing UDP subexport assertions only checked the decoded inode and kind. A regression that drifted the generation, exportID, or encoding format on one transport but not the other would have slipped through. Build the TCP Handler from the same Server, drive its Mount with the same dirpath, and require ToHandle to match the raw UDP FH bytes for every OK case. * nfs: take MOUNT3 dirpath as string in resolveMountFilesystem Convert req.Dirpath to string once at the call site instead of sprinkling string(...) casts through every log line and conversion inside the function. Behavior unchanged. * nfs: share rootFS lifecycle between TCP and UDP MOUNT handlers Server.rootFilesystem() lazily constructs the seaweedFileSystem rooted at the configured export the first time anything asks for it, then hands the same instance to every subsequent caller. newHandler() and mountUDPServer.rootMountStatus() now both go through it, so: - Both transports observe the same chunk reader cache and chunk invalidator without depending on call order during startup. - The UDP defensive Lstat doesn't allocate a fresh wrapper per MOUNT request anymore; one struct lives for the life of the Server. The sub-rooted seaweedFileSystem the subexport branch builds in resolveSubexportFileHandle is still per-request because actualRoot varies with the requested dirpath. * nfs: drive rootFilesystem before reading sharedReaderCache on UDP The UDP listener is started before serve() calls newHandler(), so an under-export MOUNT3 request can reach resolveSubexportFileHandle before Server.sharedReaderCache has been assigned. Reading it directly would hand newSeaweedFileSystem a nil cache and the sub-fs would build a throwaway ReaderCache that never gets shared with the TCP path. Take rootFS off Server.rootFilesystem() (which drives the sync.Once that initializes the shared cache) and read readerCache off that instead, so subexport sub-fs instances always share the same reader cache as rootFS regardless of which transport sees the first MOUNT. * nfs: collapse exact-match and outside-export MOUNT branches The two branches return the same filesystem (export root) and the same status; only the log line differs. Combine the conditions and guard the fallback log inline. Behavior unchanged.	2026-04-30 10:06:44 -07:00
Chris LuandGitHub	35fe3c801b	feat(nfs): UDP MOUNT v3 responder + real-Linux e2e mount harness (#9267) ... * feat(nfs): add UDP MOUNT v3 responder The upstream willscott/go-nfs library only serves the MOUNT protocol over TCP. Linux's mount.nfs and the in-kernel NFS client default mountproto to UDP in many configurations, so against a stock weed nfs deployment the kernel queries portmap for "MOUNT v3 UDP", gets port=0 ("not registered"), and either falls back inconsistently or surfaces EPROTONOSUPPORT — surfacing as the user-visible "requested NFS version or transport protocol is not supported" reported in #9263. The user has to add `mountproto=tcp` or `mountport=2049` to mount options to coerce TCP just for the MOUNT phase. Add a small UDP responder that speaks just enough of MOUNT v3 to handle the procedures the kernel actually invokes during mount setup and teardown: NULL, MNT, and UMNT. The wire layout for MNT mirrors handler.go's TCP path so both transports produce the same root filehandle and the same auth flavor list for the same export. Other v3 procedures (DUMP, EXPORT, UMNTALL) cleanly return PROC_UNAVAIL. This commit only adds the responder; portmap-advertise and Server.Start wire-up follow in subsequent commits so each step stays independently reviewable. References: RFC 1813 §5 (NFSv3/MOUNTv3), RFC 5531 (RPC). Existing constants and parseRPCCall / encodeAcceptedReply helpers from portmap.go are reused so behaviour stays consistent across both UDP listening goroutines. * feat(nfs): advertise UDP MOUNT v3 in the portmap responder The portmap responder advertised TCP-only entries because go-nfs only serves TCP, but with the new UDP MOUNT responder in place we can now honestly advertise MOUNT v3 over UDP as well. Linux clients whose default mountproto is UDP query portmap during mount setup; if the answer is "not registered" some kernels translate the result to EPROTONOSUPPORT instead of falling back to TCP, which is exactly the failure pattern reported in #9263. Add the entry, refresh the doc comment, and extend the existing GETPORT and DUMP unit tests so a regression that drops the entry shows up at unit-test granularity rather than only in an end-to-end mount. * feat(nfs): start UDP MOUNT v3 responder alongside the TCP NFS listener Plug the new mountUDPServer into Server.Start so it comes up on the same bind/port as the TCP NFS listener. Started before portmap so a portmap query that races a fast client never returns a UDP MOUNT entry the responder isn't actually answering, and shut down via the same defer chain so a portmap-or-listener startup failure doesn't leave the UDP responder dangling. The portmap startup log now reflects all three advertised entries (NFS v3 tcp, MOUNT v3 tcp, MOUNT v3 udp) so operators can confirm at a glance that the UDP MOUNT path is up. Verified end-to-end: built a Linux/arm64 binary, ran weed nfs in a container with -portmap.bind, and mounted from another container using both the user-reported failing setup from #9263 (vers=3 + tcp without mountport) and an explicit mountproto=udp to force the new code path. The trace `mount.nfs: trying ... prog 100005 vers 3 prot UDP port 2049` now leads to a successful mount instead of EPROTONOSUPPORT. * docs(nfs): note that the plain mount form works on UDP-default clients With UDP MOUNT v3 now served alongside TCP, the only path that ever required mountproto=tcp / mountport=2049 — clients whose default mountproto is UDP — works against the plain mount example. Update the startup mount hint and the `weed nfs` long help so users don't go hunting for a mount-option workaround that no longer applies. The "without -portmap.bind" branch is unchanged: that path still has to bypass portmap entirely because there is no portmap responder for the kernel to query. * test(nfs): add kernel-mount e2e tests under test/nfs The existing test/nfs/ harness boots a real master + volume + filer + weed nfs subprocess stack and drives it via go-nfs-client. That covers protocol behaviour from a Go client's perspective, but anything mis-coded once a real Linux kernel parses the wire bytes is invisible: both ends of the test use the same RPC library, so identical bugs round-trip cleanly. The two NFS issues hit recently were exactly that shape — NFSv4 mis-routed to v3 SETATTR (#9262) and missing UDP MOUNT v3 — and only surfaced in a real client. Add three end-to-end tests that mount the harness's running NFS server through the in-tree Linux client: - TestKernelMountV3TCP: NFSv3 + MOUNT v3 over TCP (baseline). - TestKernelMountV3MountProtoUDP: NFSv3 over TCP, MOUNT v3 over UDP only — regression test for the new UDP MOUNT v3 responder. - TestKernelMountV4RejectsCleanly: vers=4 against the v3-only server, asserting the kernel surfaces a protocol/version-level error rather than a generic "mount system call failed" — regression test for the PROG_MISMATCH path from #9262. The tests pass explicit port=/mountport= mount options so the kernel never queries portmap, which means the harness doesn't need to bind the privileged port 111 and won't collide with a system rpcbind on a shared CI runner. They t.Skip cleanly when the host isn't Linux, when mount.nfs isn't installed, or when the test process isn't running as root. Run locally with: cd test/nfs sudo go test -v -run TestKernelMount ./... CI wiring follows in the next commit. * ci(nfs): run kernel-mount e2e tests in nfs-tests workflow Wire the new TestKernelMount* tests from test/nfs into the existing NFS workflow: - Existing protocol-layer step now skips '^TestKernelMount' so a "skipped because not root" line doesn't appear on every run. - New "Install kernel NFS client" step pulls nfs-common (mount.nfs + helpers) and netbase (/etc/protocols, which mount.nfs's protocol- name lookups need to resolve `tcp`/`udp`). - New privileged step runs only the kernel-mount tests under sudo, preserving PATH and pointing GOMODCACHE/GOCACHE at the user's caches so the second `go test` invocation reuses already-built test binaries instead of redownloading modules under root. The summary block now lists the three kernel-mount cases explicitly so a regression on either of #9262 or this PR's UDP MOUNT change is traceable from the workflow run page.	2026-04-28 14:06:35 -07:00
Chris LuandGitHub	e2c8791441	fix(nfs): reject NFSv4 calls with PROG_MISMATCH so clients fall back to v3 (#9262) ... * feat(nfs): add NFSv3-only RPC version filter The upstream willscott/go-nfs library dispatches RPC calls by (program, procedure) only — it does not validate the program version. A client sending NFSv4 (prog 100003 vers 4 proc 1 COMPOUND) lands on the same handler map as NFSv3 and gets routed to v3 SETATTR, which parses the COMPOUND args as SETATTR3args and writes a malformed reply. The kernel then returns EPROTONOSUPPORT and mount.nfs prints "requested NFS version or transport protocol is not supported" without retrying v3. This commit adds a listener wrapper that peeks the first RPC frame on each new TCP connection. If the program is NFS or MOUNT and the version is not 3, it writes a protocol-correct PROG_MISMATCH reply (supported range 3..3, per RFC 5531) directly to the socket and closes the connection. v3 frames are replayed unchanged via a bufio reader so go-nfs sees the original bytes. Unknown programs pass through so go-nfs's own PROG_UNAVAIL handling stays in charge. The filter is not yet wired into the server; the next commit activates it. Tests cover NFSv4 reject, MOUNTv4 reject, NFSv3 pass-through, and unknown-program pass-through. * fix(nfs): wire NFSv3 version filter into the listener chain Place the version filter after the optional client allowlist so that unauthorized peers are still rejected first by IP/CIDR before we look at RPC content. With the filter active, a Linux client doing the default v4-first probe gets a clean PROG_MISMATCH reply pointing at v3, which lets mount.nfs (and the in-kernel client) skip v4 and reuse the same v3 mountOptions that already work for rclone serve nfs against this deployment. * test(nfs): exercise MOUNT v4 in the v4-rejection test, not v1 TestVersionFilterRejectsMOUNTv4WithProgMismatch was sending mountProgramID with version 1, so the test never actually covered the "reject MOUNT v4" path it claims to exercise. The filter does reject any non-v3 version uniformly, so the test still passed, but a future change that tightened the version check (for example, only rejecting v4) would let this test silently lie about coverage. Bump the call to version 4 so the name matches what is actually exercised. * refactor(nfs): reuse package RPC constants and io.ReadFull in version filter The RPC numeric constants (msg_type=CALL/REPLY, MSG_ACCEPTED, PROG_MISMATCH, AUTH_NONE, the NFS/MOUNT program numbers) are already named in portmap.go alongside the portmap responder. Reuse them here instead of defining a parallel set in rpc_version_filter.go: keeping one source of truth per package means a future correction in one spot can't drift away from the other. The filter-only constants (peek timeout, peek length, supportedNFSVer) stay local because they have no portmap analog. In the test, drop the bespoke readFull loop in favor of io.ReadFull. The custom version was a near-identical reimplementation that did not return io.ErrUnexpectedEOF on short reads, so the standard library is both shorter and more diagnostic-friendly. * fix(nfs): move RPC peek off the Accept path The previous wrapper called filterFirstRPCFrame inline inside versionFilterListener.Accept, which meant a single slow or idle TCP connect could hold rpcVersionFilterPeekTimeout (10s) of head-of-line blocking against every other accept: gonfs.Serve calls Accept serially, so each in-flight peek stalled the next legitimate client until the deadline expired. An attacker who simply opens a TCP connection without sending any RPC payload could trivially throttle accept throughput. Restructure the wrapper so a background goroutine drives the inner Accept loop and hands each raw conn to its own short-lived goroutine that runs the peek. Validated conns are sent on a buffered-once channel, which the wrapper's Accept reads from; rejected conns finish their PROG_MISMATCH reply and disappear without ever reaching the channel. This means N concurrent slow clients only block themselves, not the N+1th fast client that connects after them. Add Close coordination — sync.WaitGroup for the accept loop and per-conn peek goroutines, plus a closed channel so Accept unblocks immediately on shutdown — so the wrapper now satisfies the full net.Listener contract instead of relying on the embedded listener. Add a regression test that opens a slow conn (TCP only, never writes) and a fast conn (sends a v3 frame) and asserts the fast conn reaches the inner accept handler well below the peek timeout. * test(nfs): assert io.EOF (not just any error) after PROG_MISMATCH close The post-rejection check was only failing when conn.Read succeeded; any error — including a deadline timeout because the server kept the socket open — let the test pass. That defeats the point of the assertion: a regression where the filter replies but forgets to close would slip through silently. Match against io.EOF explicitly. The TCP semantics are deterministic here: the server writes PROG_MISMATCH, calls conn.Close(), the client reads what's left in flight and then sees a clean FIN, which surfaces as io.EOF on the next zero-byte read. * fix(nfs): reject short first fragments before parsing RPC header fields bufio.Reader.Peek(28) is willing to read across record boundaries to satisfy the requested length, so a final fragment whose body is shorter than the 24-byte fixed RPC CALL header (xid + msg_type + rpcvers + prog + vers + proc) leaves the trailing peek bytes pointing at the next RPC's framing or whatever bytes happen to follow on the wire. Indexing hdr[16:24] for prog/vers in that state can spuriously reject (or pass through) traffic based on data that doesn't belong to the request being classified. Drop those frames out of the filter early: if the first fragment can't possibly hold a full CALL header, pass the connection straight to go-nfs, which has its own framing-error handling for malformed input. Add a regression test that crafts a 12-byte first fragment whose trailing peek bytes are deliberately shaped like an NFSv4 CALL — without the length check the filter sends a PROG_MISMATCH; with it, the conn passes through silently. Verified by stashing the production-code change and running the test in isolation: it fails as expected without the fix. * fix(nfs): retry transient Accept() errors instead of treating any error as terminal acceptLoop previously exited on the first error returned by the inner listener's Accept(). That conflates two very different failure modes: permanent shutdown (the listener was Close()d, OS-level fatal failure) and transient resource pressure (EMFILE, EAGAIN, ECONNABORTED on accept). The transient case should not take the entire NFS server down — a single fd-table-full event would leave the deployment offline until restart. Classify the error: errors.Is(err, net.ErrClosed) is the permanent signal we already wanted to surface to Accept(); everything else is transient. Log at V(1) and back off rpcVersionFilterAcceptBackoff (50ms, mirroring portmap.go's portmapRetryBackoff) before retrying. The backoff sleep is interruptible via the closed channel so Close() still shuts the loop down promptly. Add a regression test that wraps a real listener with one that injects 3 fake transient errors before delegating, and asserts Accept() still delivers the next real connection. Verified the test fails on the old "any error is terminal" loop and passes with this change. * fix(nfs): only synthesize PROG_MISMATCH for ONC RPC v2 traffic The filter was rejecting any CALL-shaped record with prog=100003 or 100005 and vers!=3, regardless of the rpcvers field. If the caller is speaking some other protocol that happens to share the port — or just sending garbled bytes — pretending to be an NFSv3 server replying PROG_MISMATCH is misleading at best, and at worst fabricates a coherent RPC reply for traffic we don't actually understand. Add an rpcvers==2 check between the msg_type and prog/vers parses. Any non-v2 record now passes through to go-nfs, whose RFC 5531 §9 RPC_MISMATCH handling is the correct place to reject mis-versioned RPC. Regression test takes a normal v3 NFS CALL frame, overwrites the rpcvers field with 99, and asserts no PROG_MISMATCH-shaped reply lands on the client and that the conn is delivered to the inner accept handler. Verified the test fails on the previous code (filter still rejected on prog/vers alone) and passes with the guard in place. * fix(nfs): bound Close() latency by evicting in-flight prefilter conns Close() does wg.Wait() to drain handleConn goroutines, but each of those goroutines can be parked inside filterFirstRPCFrame's bufio.Peek for up to rpcVersionFilterPeekTimeout (10s) waiting for the very first RPC header. A client that completes the TCP handshake but never sends a byte therefore stretched shutdown by 10s per such conn — a real regression for stop/restart paths and for tests that just want to tear the listener down. Track raw (pre-peek) conns in versionFilterListener.inFlight as handleConn enters, untrack on exit, and have Close() forcibly close every tracked conn before wg.Wait. Closing the underlying conn breaks its Peek immediately, so handleConn returns within a single scheduler hop. trackInFlight also short-circuits if shutdown has already started, so a conn accepted after signalClose can't slip past the eviction. Black-box regression test opens 4 idle TCP-handshake-only conns, lets their handleConn goroutines settle into Peek, and asserts Close() returns under 2s. Verified: same test fails on the previous code with Close taking ~9.9s; passes here at ~100ms.	2026-04-28 12:17:54 -07:00
Chris LuandGitHub	3d39324bc1	fix(nfs): make Linux mount -t nfs work without client workaround (#9199) (#9201) ... * fix(nfs): make Linux `mount -t nfs` work without client-side workaround (#9199) The upstream go-nfs library serves NFSv3 + MOUNT on a single TCP port and does not register with portmap. Linux mount.nfs queries portmap on port 111 first, so the plain `mount -t nfs host:/export /mnt` form failed with "portmap query failed" / "requested NFS version or transport protocol is not supported" against a default `weed nfs` deployment. - Add a minimal PORTMAP v2 responder (weed/server/nfs/portmap.go) with TCP+UDP listeners implementing PMAP_NULL, PMAP_GETPORT, PMAP_DUMP, and proper PROG_MISMATCH / PROG_UNAVAIL / PROC_UNAVAIL responses. Advertises NFS v3 TCP and MOUNT v3 TCP at the configured NFS port. - New CLI flag `-portmap.bind` (empty, disabled by default) to opt into the responder. Binding port 111 requires root or CAP_NET_BIND_SERVICE and must not collide with a system rpcbind. - Extended `weed nfs -h` help with the two supported ways to mount from Linux (client-side portmap bypass, or server-side `-portmap.bind`). - Startup log now prints a copy-pasteable mount command tailored to whether portmap is enabled. Unit tests cover RPC/XDR parsing, accept-stat paths, and a TCP+UDP round-trip against the real listener. Verified in a privileged Debian 12 container: with `-portmap.bind=0.0.0.0` the exact command from #9199 (`mount -t nfs -o nfsvers=3,nolock host:/export /mnt`) now succeeds and both read and write work. * fix(nfs): harden portmap responder per review feedback (#9201) Addresses three review findings on the portmap responder: - parseRPCCall: validate opaque_auth length against the record limit before applying the XDR 4-byte padding, so a near-uint32-max authLen can no longer overflow (authLen + 3) and bypass the bounds check. (gemini-code-assist) - serveTCP/Close: track live TCP connections and evict them on Close() so shutdown does not block on idle clients waiting for the read deadline to trip. serveTCP also no longer tears the listener down on a non-fatal Accept error (e.g. EMFILE); it logs and retries after a small back-off. Replaces the atomic.Bool closed flag with a mutex-guarded one so closed, conns, and the shutdown transition stay consistent. (coderabbit, minor) - handleTCPConn: apply per-IO read/write deadlines (30s idle, 10s in-flight) so a peer that opens the privileged port 111 and stalls cannot pin a goroutine indefinitely. (coderabbit, major) Adds TestPortmapServer_CloseEvictsIdleTCPConn, which holds a TCP connection idle and asserts Close() returns within 2s (well under the 30s idle deadline) and that the client sees the eviction. All existing tests still pass, including under -race. * fix(nfs): keep portmap UDP responder alive on transient read errors (#9201) - serveUDP: on a non-shutdown ReadFromUDP error, log, back off, and continue instead of returning. Matches how serveTCP now treats non-fatal Accept errors so a transient network blip doesn't take UDP portmap down until restart. (coderabbit) - Rename portmapAcceptBackoff -> portmapRetryBackoff now that both paths use it. - pmapProcDump: fix the pre-allocation capacity to match the actual encoding (20 bytes per entry + 4-byte terminator), replacing the old over-estimate of 24 per entry. No behavior change; just documents intent. (coderabbit nit) * docs(nfs): clarify encodeAcceptedReply body semantics (#9201) The prior comment said body is "nil when the accept_stat is itself an error", which was misleading: the PROG_MISMATCH branch already passes an 8-byte mismatch_info body. Rewrite to enumerate which error accept_stat values omit the body and call out PROG_MISMATCH as the exception, referencing RFC 5531 §9. Comment-only. (coderabbit nit) * fix(nfs): make portmap retry backoff interruptible by Close() (#9201) serveTCP and serveUDP both sleep portmapRetryBackoff (50ms) after a non-fatal listener error. If Close() races in during that sleep, the goroutine can't be interrupted, so Close() has to wait out the remaining backoff before wg.Wait() returns. Add a done channel that Close() closes once, and replace both time.Sleep calls with a select on ps.done + time.After. The window was tiny in practice but the select makes shutdown strictly bounded by Close()'s own work. (coderabbit nit)	2026-04-23 13:53:53 -07:00
Chris LuandGitHub	08d9193fe1	[nfs] Add NFS (#9067) ... * add filer inode foundation for nfs * nfs command skeleton * add filer inode index foundation for nfs * make nfs inode index hardlink aware * add nfs filehandle and inode lookup plumbing * add read-only nfs frontend foundation * add nfs namespace mutation support * add chunk-backed nfs write path * add nfs protocol integration tests * add stale handle nfs coverage * complete nfs hardlink and failover coverage * add nfs export access controls * add nfs metadata cache invalidation * fix nfs chunk read lookup routing * fix nfs review findings and rename regression * address pr 9067 review comments - filer_inode: fail fast if the snowflake sequencer cannot start, and let operators override the 10-bit node id via SEAWEEDFS_FILER_SNOWFLAKE_ID to avoid multi-filer collisions - filer_inode: drop the redundant retry loop in nextInode - filerstore_wrapper: treat inode-index writes/removals as best-effort so a primary store success no longer surfaces as an operation failure - filer_grpc_server_rename: defer overwritten-target chunk deletion until after CommitTransaction so a rolled-back rename does not strand live metadata pointing at freshly deleted chunks - command/nfs: default ip.bind to loopback and require an explicit filer.path, so the experimental server does not expose the entire filer namespace on first run - nfs integration_test: document why LinkArgs matches go-nfs's on-the-wire layout rather than RFC 1813 LINK3args * mount: pre-allocate inode in Mkdir and Symlink Mkdir and Symlink used to send filer_pb.CreateEntryRequest with Attributes.Inode = 0. After PR 9067, the filer's CreateEntry now assigns its own inode in that case, so the filer-side entry ends up with a different inode than the one the mount allocates via inodeToPath.Lookup and returns to the kernel. Once applyLocalMetadataEvent stores the filer's entry in the meta cache, subsequent GetAttr calls read the cached entry and hit the setAttrByPbEntry override at line 197 of weedfs_attr.go, returning the filer-assigned inode instead of the mount's local one. pjdfstest tests/rename/00.t (subtests 81/87/91) caught this — it lstat'd a freshly-created directory/symlink, renamed it, lstat'd again, and saw a different inode the second time. createRegularFile already pre-allocates via inodeToPath.AllocateInode and stamps it into the create request. Do the same thing in Mkdir and Symlink so both sides agree on the object identity from the very first request, and so GetAttr's cache path returns the same value as Mkdir / Symlink's initial response. * sequence: mask snowflake node id on int→uint32 conversion CodeQL flagged the unchecked uint32(snowflakeId) cast in NewSnowflakeSequencer as a potential truncation bug when snowflakeId is sourced from user input (e.g. via SEAWEEDFS_FILER_SNOWFLAKE_ID). Mask to the 10 bits the snowflake library actually uses so any caller- supplied int is safely clamped into range. * add test/nfs integration suite Boots a real SeaweedFS cluster (master + volume + filer) plus the experimental `weed nfs` frontend as subprocesses and drives it through the NFSv3 wire protocol via go-nfs-client, mirroring the layout of test/sftp. The tests run without a kernel NFS mount, privileged ports, or any platform-specific tooling. Coverage includes read/write round-trip, mkdir/rmdir, nested directories, rename content preservation, overwrite + explicit truncate, 3 MiB binary file, all-byte binary and empty files, symlink round-trip, ReadDirPlus listing, missing-path remove, FSInfo sanity, sequential appends, and readdir-after-remove. Framework notes: - Picks ephemeral ports with net.Listen("127.0.0.1:0") and passes -port.grpc explicitly so the default port+10000 convention cannot overflow uint16 on macOS. - Pre-creates the /nfs_export directory via the filer HTTP API before starting the NFS server — the NFS server's ensureIndexedEntry check requires the export root to exist with a real entry, which filer.Root does not satisfy when the export path is "/". - Reuses the same rpc.Client for mount and target so go-nfs-client does not try to re-dial via portmapper (which concatenates ":111" onto the address). * ci: add NFS integration test workflow Mirror test/sftp's workflow for the new test/nfs suite so PRs that touch the NFS server, the inode filer plumbing it depends on, or the test harness itself run the 14 NFSv3-over-RPC integration tests on Ubuntu 22.04 via `make test`. * nfs: use append for buffer growth in Write and Truncate The previous make+copy pattern reallocated the full buffer on every extending write or truncate, giving O(N^2) behaviour for sequential write loops. Switching to `append(f.content, make([]byte, delta)...)` lets Go's amortized growth strategy absorb the repeated extensions. Called out by gemini-code-assist on PR 9067. * filer: honor caller cancellation in collectInodeIndexEntries Dropping the WithoutCancel wrapper lets DeleteFolderChildren bail out of the inode-index scan if the client disconnects mid-walk. The cleanup is already treated as best-effort by the caller (it logs on error and continues), so a cancelled walk just means the partial index rebuild is skipped — the same failure mode as any other index write error. Flagged as a DoS concern by gemini-code-assist on PR 9067. * nfs: skip filer read on open when O_TRUNC is set openFile used to unconditionally loadWritableContent for every writable open and then discard the buffer if O_TRUNC was set. For large files that is a pointless 64 MiB round-trip. Reorder the branches so we only fetch existing content when the caller intends to keep it, and mark the file dirty right away so the subsequent Close still issues the truncating write. Called out by gemini-code-assist on PR 9067. * nfs: allow Seek on O_APPEND files and document buffered write cap Two related cleanups on filesystem.go: - POSIX only restricts Write on an O_APPEND fd, not lseek. The existing Seek error ("append-only file descriptors may only seek to EOF") prevented read-and-write workloads that legitimately reposition the read cursor. Write already snaps the offset to EOF before persisting (see seaweedFile Write), so Seek can unconditionally accept any offset. Update the unit test that was asserting the old behaviour. - Add a doc comment on maxBufferedWriteSize explaining that it is a per-file ceiling, the memory footprint it implies, and that the real fix for larger whole-file rewrites is streaming / multi-chunk support. Both changes flagged by gemini-code-assist on PR 9067. * nfs: guard offset before casting to int in Write CodeQL flagged `int(f.offset) + len(p)` inside the Write growth path as a potential overflow on architectures where `int` is 32-bit. The existing check only bounded the post-cast value, which is too late. Clamp f.offset against maxBufferedWriteSize before the cast and also reject negative/overflowed endOffset results. Both branches fall through to billy.ErrNotSupported, the same behaviour the caller gets today for any out-of-range buffered write. * nfs: compute Write endOffset in int64 to satisfy CodeQL The previous guard bounded f.offset but left len(p) unchecked, so CodeQL still flagged `int(f.offset) + len(p)` as a possible int-width overflow path. Bound len(p) against maxBufferedWriteSize first, do the addition in int64, and only cast down after the total has been clamped against the buffer ceiling. Behaviour is unchanged: any out-of-range write still returns billy.ErrNotSupported. * ci: drop emojis from nfs-tests workflow summary Plain-text step summary per user preference — no decorative glyphs in the NFS CI output or checklist. * nfs: annotate remaining DEV_PLAN TODOs with status Three of the unchecked items are genuine follow-up PRs rather than missing work in this one, and one was actually already done: - Reuse chunk cache and mutation stream helpers without FUSE deps: checked off — the NFS server imports weed/filer.ReaderCache and weed/util/chunk_cache directly with no weed/mount or go-fuse imports. - Extract shared read/write helpers from mount/WebDAV/SFTP: annotated as deferred to a separate refactor PR (touches four packages). - Expand direct data-path writes beyond the 64 MiB buffered fallback: annotated as deferred — requires a streaming WRITE path. - Shared lock state + lock tests: annotated as blocked upstream on go-nfs's missing NLM/NFSv4 lock state RPCs, matching the existing "Current Blockers" note. * test/nfs: share port+readiness helpers with test/testutil Drop the per-suite mustPickFreePort and waitForService re-implementations in favor of testutil.MustAllocatePorts (atomic batch allocation; no close-then-hope race) and testutil.WaitForPort / SeaweedMiniStartupTimeout. Pull testutil in via a local replace directive so this standalone seaweedfs-nfs-tests module can import the in-repo package without a separate release. Subprocess startup is still master + volume + filer + nfs — no switch to weed mini yet, since mini does not know about the nfs frontend. * nfs: stream writes to volume servers instead of buffering the whole file Before this change the NFS write path held the full contents of every writable open in memory: - OpenFile(write) called loadWritableContent which read the existing file into seaweedFile.content up to maxBufferedWriteSize (64 MiB) - each Write() extended content in-place - Close() uploaded the whole buffer as a single chunk via persistContent + AssignVolume The 64 MiB ceiling made large NFS writes return NFS3ERR_NOTSUPP, and even below the cap every Write paid a whole-file-in-memory cost. This PR rewrites the write path to match how `weed filer` and the S3 gateway persist data: - openFile(write) no longer loads the existing content at all; it only issues an UpdateEntry when O_TRUNC is set *and* the file is non-empty (so a fresh create+trunc is still zero-RPC) - Write() streams the caller's bytes straight to a volume server via one AssignVolume + one chunk upload, then atomically appends the resulting chunk to the filer entry through mutateEntry. Any previously inlined entry.Content is migrated to a chunk in the same update so the chunk list becomes the authoritative representation. - Truncate() becomes a direct mutateEntry (drop chunks past the new size, clip inline content, update FileSize) instead of resizing an in-memory buffer. - Close() is a no-op because everything was flushed inline. The small-file fast path that the filer HTTP handler uses is preserved: if the post-write size still fits in maxInlineWriteSize (4 MiB) and the file has no existing chunks, we rewrite entry.Content directly and skip the volume-server round-trip. This keeps single-shot tiny writes (echo, small edits) cheap while completely removing the 64 MiB cap on larger files. Read() now always reads through the chunk reader instead of a local byte slice, so reads inside the same session see the freshly appended data. Drops the unused seaweedFile.content / dirty fields, the maxBufferedWriteSize constant, and the loadWritableContent helper. Updates TestSeaweedFileSystemSupportsNamespaceMutations expectations to match the new "no extra O_TRUNC UpdateEntry on an empty file" behavior (still 3 updates: Write + Chmod + Truncate). * filer: extract shared gateway upload helper for NFS and WebDAV Three filer-backed gateways (NFS, WebDAV, and mount) each had a local saveDataAsChunk that wrapped operation.NewUploader().UploadWithRetry with near-identical bodies: build AssignVolumeRequest, build UploadOption, build genFileUrlFn with optional filerProxy rewriting, call UploadWithRetry, validate the result, and call ToPbFileChunk. Pull that body into filer.SaveGatewayDataAsChunk with a GatewayChunkUploadRequest struct so both NFS and WebDAV can delegate to one implementation. - NFS's saveDataAsChunk is now a thin adapter that assembles the GatewayChunkUploadRequest from server options and calls the helper. The chunkUploader interface keeps working for test injection because the new GatewayChunkUploader interface is structurally identical. - WebDAV's saveDataAsChunk is similarly a thin adapter — it drops the local operation.NewUploader call plus the AssignVolume/UploadOption scaffolding. - mount is intentionally left alone. mount's saveDataAsChunk has two features that do not fit the shared helper (a pre-allocated file-id pool used to skip AssignVolume entirely, and a chunkCache write-through at offset 0 so future reads hit the mount's local cache), both of which are mount-specific. Marks the Phase 2 "extract shared read/write helpers from mount, WebDAV, and SFTP" DEV_PLAN item as done. The filer-level chunk read path (NonOverlappingVisibleIntervals + ViewFromVisibleIntervals + NewChunkReaderAtFromClient) was already shared. * nfs: remove DESIGN.md and DEV_PLAN.md The planning documents have served their purpose — all phase 1 and phase 2 items are landed, phase 3 streaming writes are landed, phase 2 shared helpers are extracted, and the two remaining phase 4 items (shared lock state + lock tests) are blocked upstream on github.com/willscott/go-nfs which exposes no NLM or NFSv4 lock state RPCs. The running decision log no longer reflects current code and would just drift. The NFS wiki page (https://github.com/seaweedfs/seaweedfs/wiki/NFS-Server) now carries the overview, configuration surface, architecture notes, and known limitations; the source is the source of truth for the rest.	2026-04-14 20:48:24 -07:00