mirrors/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-08-16 04:06:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
22ebe9feb09c82f9610a42e9ff229ca3d513da7e
seaweedfs/weed/server/nfs/integration_test.go
T
Chris LuandGitHub d265274e13 fix(nfs): accept dirpath any-where under the export, mirroring rclone (#9291) 
* fix(nfs): accept any MOUNT3 dirpath, mirroring rclone's permissive policy

weed nfs has exactly one export per process, so the MOUNT3 dirpath
argument has no second export to disambiguate against. Strict
comparison only translated PV-path typos into the inconsistent
"mount succeeds but empty" / "mount fails completely" split that
operators see.

Match rclone's serve nfs Handler.Mount: ignore the dirpath, log an INFO
line when it differs from the configured export, and always serve the
export root. Apply the same change to the UDP MOUNT3 path so kernel
clients defaulting to mountproto=udp see identical behaviour. Access
control still goes through -allowedClients / -ip.bind, and file-handle
scoping in FromHandle is unchanged so handles still cannot escape the
export.

Replace the prior single-path reject tests with table tests covering
the shapes operators commonly hit: root, parent, sibling, deeper child,
unrelated, empty, relative form, exact match, and trailing slash, at
the Handler.Mount, UDP MOUNT3, and full RPC layers.

* feat(nfs): mount at subdirectory when MOUNT3 dirpath is under the export

Make the dirpath argument meaningful when the client asks for a subtree
of the configured export. With -filer.path=/buckets, a client mounting
<server>:/buckets/data lands directly inside /buckets/data instead of
at the export root.

  - dirpath equals the export root: serve the export root.
  - dirpath strictly under the export, directory entry: serve that
    subdirectory; the returned filehandle encodes its inode.
  - dirpath strictly under the export, missing or non-directory: reject
    with NoEnt or NotDir.
  - dirpath outside the export: keep the rclone-style fallback to the
    export root.

TCP returns a sub-rooted seaweedFileSystem and lets go-nfs's onMount
call ToHandle to encode the FH; UDP encodes the FH itself. FromHandle
is unchanged: handles are content-addressed by inode and resolve via
the inode index, so they remain stable across mounts and across
process restarts.

The trimmed permissive tests keep their outside-export shapes; new
subexport tests cover under-export directories, missing entries, and
non-directory entries on Handler.Mount, the UDP MOUNT3 wire, and
through the full RPC stack.

* nfs: propagate request context through MOUNT3 resolution

Mount now accepts the gonfs context and threads it through
resolveMountFilesystem and lstatExportStatus so a slow filer call
during MOUNT cannot outlive a cancelled or timed-out request.

lstatExportStatus uses fileInfoForVirtualPath(ctx, "/") directly
instead of billy.Filesystem.Lstat, which would otherwise drop the
context on the floor by calling fileInfoForVirtualPathWithOptions
with context.Background().

Lower the successful subexport-mount log from V(0) to V(1). The
fallback log stays at V(0) so operator typos still surface; the
success line is per-mount churn that adds up on NFS-CSI deployments.

* nfs: mirror TCP defensive checks on the UDP MOUNT3 path

Two transport-parity bugs the rabbit caught:

(1) The exact-export-root and outside-export branches were returning
MNT3_OK unconditionally, while the TCP handler runs lstatExportStatus
on those same branches. If the configured -filer.path has been
removed from the filer, TCP returns NoEnt/ServerFault but UDP would
still hand out a synthetic root handle pointing at nothing. Add
rootMountStatus as the UDP analogue and call it on both branches.

(2) resolveSubexportFileHandle did filer I/O on the single UDP serve
loop with context.Background(). One slow filer round-trip would
block every later MOUNT packet. Wrap each MOUNT call's filer work in
context.WithTimeout(mountUDPLookupTimeout) and thread that ctx
through both rootMountStatus and resolveSubexportFileHandle.

Lower the successful subexport log to V(1) to match the TCP side.

* nfs: assert TCP/UDP MOUNT3 produce byte-identical filehandles

The existing UDP subexport assertions only checked the decoded inode
and kind. A regression that drifted the generation, exportID, or
encoding format on one transport but not the other would have slipped
through. Build the TCP Handler from the same Server, drive its Mount
with the same dirpath, and require ToHandle to match the raw UDP FH
bytes for every OK case.

* nfs: take MOUNT3 dirpath as string in resolveMountFilesystem

Convert req.Dirpath to string once at the call site instead of
sprinkling string(...) casts through every log line and conversion
inside the function. Behavior unchanged.

* nfs: share rootFS lifecycle between TCP and UDP MOUNT handlers

Server.rootFilesystem() lazily constructs the seaweedFileSystem rooted
at the configured export the first time anything asks for it, then
hands the same instance to every subsequent caller. newHandler() and
mountUDPServer.rootMountStatus() now both go through it, so:

  - Both transports observe the same chunk reader cache and chunk
    invalidator without depending on call order during startup.
  - The UDP defensive Lstat doesn't allocate a fresh wrapper per
    MOUNT request anymore; one struct lives for the life of the
    Server.

The sub-rooted seaweedFileSystem the subexport branch builds in
resolveSubexportFileHandle is still per-request because actualRoot
varies with the requested dirpath.

* nfs: drive rootFilesystem before reading sharedReaderCache on UDP

The UDP listener is started before serve() calls newHandler(), so an
under-export MOUNT3 request can reach resolveSubexportFileHandle before
Server.sharedReaderCache has been assigned. Reading it directly would
hand newSeaweedFileSystem a nil cache and the sub-fs would build a
throwaway ReaderCache that never gets shared with the TCP path.

Take rootFS off Server.rootFilesystem() (which drives the sync.Once
that initializes the shared cache) and read readerCache off that
instead, so subexport sub-fs instances always share the same reader
cache as rootFS regardless of which transport sees the first MOUNT.

* nfs: collapse exact-match and outside-export MOUNT branches

The two branches return the same filesystem (export root) and the
same status; only the log line differs. Combine the conditions and
guard the fallback log inline. Behavior unchanged.
2026-04-30 10:06:44 -07:00

881 lines
24 KiB
Go
Raw Blame History

package nfs
import (
"bytes"
"context"
"crypto/md5"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"math/rand"
"mime/multipart"
"net"
"net/http"
"net/http/httptest"
"path"
"strconv"
"strings"
"sync"
"testing"
"time"
"github.com/seaweedfs/seaweedfs/weed/filer"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/util"
util_http "github.com/seaweedfs/seaweedfs/weed/util/http"
"github.com/seaweedfs/seaweedfs/weed/wdclient"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
gonfs "github.com/willscott/go-nfs"
nfsclient "github.com/willscott/go-nfs-client/nfs"
"github.com/willscott/go-nfs-client/nfs/rpc"
"github.com/willscott/go-nfs-client/nfs/xdr"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
)
type fakeVolumeBlob struct {
data []byte
contentEncoding string
}
type fakeVolumeServer struct {
mu sync.Mutex
blobs map[string]fakeVolumeBlob
server *httptest.Server
}
type fakeVolumeControlPlane struct {
filer_pb.UnimplementedSeaweedFilerServer
mu sync.Mutex
host string
nextID int
assigns []*filer_pb.AssignVolumeRequest
lookups []*filer_pb.LookupVolumeRequest
}
var initIntegrationHTTPClient sync.Once
const nfsProc3Link = 15
func newFakeVolumeServer(t *testing.T) *fakeVolumeServer {
t.Helper()
fake := &fakeVolumeServer{
blobs: make(map[string]fakeVolumeBlob),
}
fake.server = httptest.NewServer(http.HandlerFunc(fake.serveHTTP))
t.Cleanup(fake.server.Close)
return fake
}
func (f *fakeVolumeServer) host() string {
return strings.TrimPrefix(f.server.URL, "http://")
}
func (f *fakeVolumeServer) serveHTTP(w http.ResponseWriter, r *http.Request) {
fileID := strings.TrimPrefix(r.URL.Path, "/")
if fileID == "" {
http.NotFound(w, r)
return
}
switch r.Method {
case http.MethodPost:
part, err := firstMultipartFile(r)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
defer part.Close()
data, err := io.ReadAll(part)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
contentEncoding := part.Header.Get("Content-Encoding")
sum := md5.Sum(data)
f.mu.Lock()
f.blobs[fileID] = fakeVolumeBlob{
data: bytes.Clone(data),
contentEncoding: contentEncoding,
}
f.mu.Unlock()
w.Header().Set("Content-MD5", base64.StdEncoding.EncodeToString(sum[:]))
w.Header().Set("ETag", `"`+base64.StdEncoding.EncodeToString(sum[:])+`"`)
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{
"name": path.Base(fileID),
"size": len(data),
})
case http.MethodGet:
f.mu.Lock()
blob, found := f.blobs[fileID]
f.mu.Unlock()
if !found {
http.NotFound(w, r)
return
}
if blob.contentEncoding != "" {
w.Header().Set("Content-Encoding", blob.contentEncoding)
}
http.ServeContent(w, r, fileID, time.Unix(0, 0), bytes.NewReader(blob.data))
default:
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
}
}
func firstMultipartFile(r *http.Request) (*multipart.Part, error) {
reader, err := r.MultipartReader()
if err != nil {
return nil, err
}
for {
part, err := reader.NextPart()
if err == io.EOF {
return nil, io.ErrUnexpectedEOF
}
if err != nil {
return nil, err
}
if part.FormName() == "file" {
return part, nil
}
part.Close()
}
}
func (f *fakeVolumeControlPlane) AssignVolume(_ context.Context, req *filer_pb.AssignVolumeRequest) (*filer_pb.AssignVolumeResponse, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.assigns = append(f.assigns, req)
f.nextID++
fileID := fmt.Sprintf("7,%08x", f.nextID)
return &filer_pb.AssignVolumeResponse{
FileId: fileID,
Count: 1,
Location: &filer_pb.Location{
Url: f.host,
},
}, nil
}
func (f *fakeVolumeControlPlane) LookupVolume(_ context.Context, req *filer_pb.LookupVolumeRequest) (*filer_pb.LookupVolumeResponse, error) {
f.mu.Lock()
f.lookups = append(f.lookups, req)
f.mu.Unlock()
locations := make(map[string]*filer_pb.Locations, len(req.GetVolumeIds()))
for _, volumeID := range req.GetVolumeIds() {
locations[volumeID] = &filer_pb.Locations{
Locations: []*filer_pb.Location{
{Url: f.host},
},
}
}
return &filer_pb.LookupVolumeResponse{LocationsMap: locations}, nil
}
func startFakeVolumeControlPlane(t *testing.T, controlPlane *fakeVolumeControlPlane) string {
t.Helper()
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
grpcServer := grpc.NewServer()
filer_pb.RegisterSeaweedFilerServer(grpcServer, controlPlane)
done := make(chan error, 1)
go func() {
done <- grpcServer.Serve(listener)
}()
t.Cleanup(func() {
grpcServer.Stop()
_ = listener.Close()
select {
case err := <-done:
if err != nil && !isClosedNetworkErr(err) {
t.Errorf("fake control plane exited with error: %v", err)
}
case <-time.After(time.Second):
t.Errorf("timed out waiting for fake control plane shutdown")
}
})
return listener.Addr().String()
}
func mountTestTarget(t *testing.T, server *Server) (*nfsclient.Target, func()) {
t.Helper()
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
handler, err := server.newHandler()
require.NoError(t, err)
done := make(chan error, 1)
go func() {
done <- gonfs.Serve(listener, handler)
}()
var client *rpc.Client
for attempt := 0; attempt < 10; attempt++ {
client, err = rpc.DialTCP(listener.Addr().Network(), listener.Addr().String(), false)
if err == nil {
break
}
if attempt == 9 {
require.NoError(t, err)
}
time.Sleep(10 * time.Millisecond)
}
require.NoError(t, err)
mounter := &nfsclient.Mount{Client: client}
target, err := mounter.Mount(string(server.exportRoot), rpc.AuthNull)
require.NoError(t, err)
cleanup := func() {
_ = mounter.Unmount()
client.Close()
_ = listener.Close()
select {
case err := <-done:
if err != nil && !isClosedNetworkErr(err) {
t.Errorf("nfs server exited with error: %v", err)
}
case <-time.After(time.Second):
t.Errorf("timed out waiting for nfs server shutdown")
}
}
return target, cleanup
}
func isClosedNetworkErr(err error) bool {
if err == nil {
return false
}
if strings.Contains(err.Error(), "use of closed network connection") {
return true
}
return strings.Contains(err.Error(), "listener closed")
}
func nfsLink(target *nfsclient.Target, sourceHandle []byte, linkPath string) error {
parentDir, linkName := path.Split(path.Clean(linkPath))
if linkName == "" {
return fmt.Errorf("invalid hard link path %q", linkPath)
}
if parentDir == "" {
parentDir = "/"
}
_, parentHandle, err := target.Lookup(parentDir)
if err != nil {
return err
}
// Field layout matches the go-nfs server's onLink handler
// (vendor: github.com/willscott/go-nfs/nfs_onlink.go), which reads
// DirOpArg + SetFileAttributes + opaque target handle. That wire
// order differs from RFC 1813 §3.3.15 LINK3args {nfs_fh3 file;
// diropargs3 link;} — the go-nfs library is not strictly compliant
// here, and we mirror its layout so the integration test exercises
// the same parser the server uses. Do not reorder fields to match
// the RFC: the test would then fail against a correctly-functioning
// server.
type LinkArgs struct {
rpc.Header
Link nfsclient.Diropargs3
Sattr nfsclient.Sattr3
Target []byte
}
res, err := target.Call(&LinkArgs{
Header: rpc.Header{
Rpcvers: 2,
Prog: nfsclient.Nfs3Prog,
Vers: nfsclient.Nfs3Vers,
Proc: nfsProc3Link,
Cred: rpc.AuthNull,
Verf: rpc.AuthNull,
},
Link: nfsclient.Diropargs3{
FH: parentHandle,
Filename: linkName,
},
Target: sourceHandle,
})
if err != nil {
return err
}
status, err := xdr.ReadUint32(res)
if err != nil {
return err
}
return nfsclient.NFS3Error(status)
}
func TestSeaweedNFSAcceptsAnyMountPathOverRPC(t *testing.T) {
const exportRoot = "/buckets/data"
client := &fakeNFSFilerClient{
entries: map[util.FullPath]*filer_pb.Entry{
"/buckets": testEntry("buckets", true, 100, uint32(0755), nil),
"/buckets/data": testEntry("data", true, 101, uint32(0755), nil),
},
kv: map[string][]byte{
string(filer.InodeIndexKey(100)): testIndexRecord(t, 100, 1, "/buckets"),
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/buckets/data"),
},
}
server := newTestServer(t, exportRoot, client)
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
handler, err := server.newHandler()
require.NoError(t, err)
serveDone := make(chan error, 1)
go func() {
serveDone <- gonfs.Serve(listener, handler)
}()
t.Cleanup(func() {
_ = listener.Close()
select {
case err := <-serveDone:
if err != nil && !isClosedNetworkErr(err) {
t.Errorf("nfs server exited with error: %v", err)
}
case <-time.After(time.Second):
t.Errorf("timed out waiting for nfs server shutdown")
}
})
dirpaths := []string{
"/",
"/buckets",
"/buckets/other",
"/wrong/path",
exportRoot,
exportRoot + "/",
}
for _, dirpath := range dirpaths {
t.Run(dirpath, func(t *testing.T) {
var rpcClient *rpc.Client
var dialErr error
for attempt := 0; attempt < 10; attempt++ {
rpcClient, dialErr = rpc.DialTCP(listener.Addr().Network(), listener.Addr().String(), false)
if dialErr == nil {
break
}
time.Sleep(10 * time.Millisecond)
}
require.NoError(t, dialErr)
defer rpcClient.Close()
mounter := &nfsclient.Mount{Client: rpcClient}
target, err := mounter.Mount(dirpath, rpc.AuthNull)
require.NoErrorf(t, err, "Mount(%q)", dirpath)
defer target.Close()
entries, err := target.ReadDirPlus("/")
require.NoError(t, err)
assert.Empty(t, entries, "Mount(%q) should land at the empty export root", dirpath)
})
}
}
func TestSeaweedNFSSubexportMountOverRPC(t *testing.T) {
const exportRoot = "/buckets"
client := &fakeNFSFilerClient{
entries: map[util.FullPath]*filer_pb.Entry{
"/buckets": testEntry("buckets", true, 100, uint32(0755), nil),
"/buckets/data": testEntry("data", true, 101, uint32(0755), nil),
"/buckets/data/inner": testEntry("inner", false, 104, uint32(0644), []byte("payload")),
"/buckets/other": testEntry("other", true, 105, uint32(0755), nil),
"/buckets/file.txt": testEntry("file.txt", false, 103, uint32(0644), []byte("hi")),
},
kv: map[string][]byte{
string(filer.InodeIndexKey(100)): testIndexRecord(t, 100, 1, "/buckets"),
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/buckets/data"),
string(filer.InodeIndexKey(103)): testIndexRecord(t, 103, 1, "/buckets/file.txt"),
string(filer.InodeIndexKey(104)): testIndexRecord(t, 104, 1, "/buckets/data/inner"),
string(filer.InodeIndexKey(105)): testIndexRecord(t, 105, 1, "/buckets/other"),
},
}
server := newTestServer(t, exportRoot, client)
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
handler, err := server.newHandler()
require.NoError(t, err)
serveDone := make(chan error, 1)
go func() {
serveDone <- gonfs.Serve(listener, handler)
}()
t.Cleanup(func() {
_ = listener.Close()
select {
case err := <-serveDone:
if err != nil && !isClosedNetworkErr(err) {
t.Errorf("nfs server exited with error: %v", err)
}
case <-time.After(time.Second):
t.Errorf("timed out waiting for nfs server shutdown")
}
})
dial := func(t *testing.T) *rpc.Client {
t.Helper()
var rpcClient *rpc.Client
var dialErr error
for attempt := 0; attempt < 10; attempt++ {
rpcClient, dialErr = rpc.DialTCP(listener.Addr().Network(), listener.Addr().String(), false)
if dialErr == nil {
break
}
time.Sleep(10 * time.Millisecond)
}
require.NoError(t, dialErr)
t.Cleanup(func() { rpcClient.Close() })
return rpcClient
}
t.Run("mounts_under_export_at_subdirectory", func(t *testing.T) {
mounter := &nfsclient.Mount{Client: dial(t)}
target, err := mounter.Mount("/buckets/data", rpc.AuthNull)
require.NoError(t, err)
defer target.Close()
entries, err := target.ReadDirPlus("/")
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, "inner", entries[0].Name())
readFile, err := target.Open("/inner")
require.NoError(t, err)
defer readFile.Close()
data, err := io.ReadAll(readFile)
require.NoError(t, err)
assert.Equal(t, []byte("payload"), data)
})
t.Run("missing_entry_under_export_rejects", func(t *testing.T) {
mounter := &nfsclient.Mount{Client: dial(t)}
_, err := mounter.Mount("/buckets/missing", rpc.AuthNull)
require.Error(t, err)
})
t.Run("regular_file_under_export_rejects", func(t *testing.T) {
mounter := &nfsclient.Mount{Client: dial(t)}
_, err := mounter.Mount("/buckets/file.txt", rpc.AuthNull)
require.Error(t, err)
})
}
func TestSeaweedNFSServesInlineRoundTripOverRPC(t *testing.T) {
client := &fakeNFSFilerClient{
kv: map[string][]byte{
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/exports"),
},
entries: map[util.FullPath]*filer_pb.Entry{
"/exports": testEntry("exports", true, 101, uint32(0755), nil),
},
}
server := newTestServer(t, "/exports", client)
target, cleanup := mountTestTarget(t, server)
defer cleanup()
defer target.Close()
_, err := target.Mkdir("/docs", 0o755)
require.NoError(t, err)
file, err := target.OpenFile("/docs/note.txt", 0o644)
require.NoError(t, err)
payload := []byte("hello over rpc")
_, err = file.Write(payload)
require.NoError(t, err)
require.NoError(t, file.Close())
readFile, err := target.Open("/docs/note.txt")
require.NoError(t, err)
defer readFile.Close()
data, err := io.ReadAll(readFile)
require.NoError(t, err)
assert.Equal(t, payload, data)
entry := client.entries["/exports/docs/note.txt"]
require.NotNil(t, entry)
assert.Equal(t, payload, entry.Content)
assert.Empty(t, entry.Chunks)
_, beforeRenameHandle, err := target.Lookup("/docs/note.txt")
require.NoError(t, err)
entries, err := target.ReadDirPlus("/docs")
require.NoError(t, err)
require.Len(t, entries, 1)
assert.Equal(t, "note.txt", entries[0].Name())
require.NoError(t, target.Rename("/docs/note.txt", "/docs/final.txt"))
_, err = target.GetAttr(beforeRenameHandle)
require.NoError(t, err)
_, _, err = target.Lookup("/docs/final.txt")
require.NoError(t, err)
_, _, err = target.Lookup("/docs/note.txt")
require.Error(t, err)
require.NoError(t, target.Remove("/docs/final.txt"))
_, _, err = target.Lookup("/docs/final.txt")
require.Error(t, err)
}
func TestSeaweedNFSReadOnlyRejectsMutations(t *testing.T) {
client := &fakeNFSFilerClient{
kv: map[string][]byte{
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/exports"),
string(filer.InodeIndexKey(202)): testIndexRecord(t, 202, 3, "/exports/existing.txt"),
},
entries: map[util.FullPath]*filer_pb.Entry{
"/exports": testEntry("exports", true, 101, uint32(0755), nil),
"/exports/existing.txt": testEntry("existing.txt", false, 202, uint32(0644), []byte("seed")),
},
}
server := newTestServer(t, "/exports", client)
server.option.ReadOnly = true
target, cleanup := mountTestTarget(t, server)
defer cleanup()
defer target.Close()
_, err := target.OpenFile("/created.txt", 0o644)
require.Error(t, err)
nfsErr, ok := err.(*nfsclient.Error)
require.True(t, ok)
assert.Equal(t, uint32(nfsclient.NFS3ErrROFS), nfsErr.ErrorNum)
file, err := target.Open("/existing.txt")
require.NoError(t, err)
_, err = file.Write([]byte("mutate"))
require.Error(t, err)
nfsErr, ok = err.(*nfsclient.Error)
require.True(t, ok)
assert.Equal(t, uint32(nfsclient.NFS3ErrROFS), nfsErr.ErrorNum)
_ = file.Close()
readFile, err := target.Open("/existing.txt")
require.NoError(t, err)
defer readFile.Close()
data, err := io.ReadAll(readFile)
require.NoError(t, err)
assert.Equal(t, []byte("seed"), data)
}
func TestSeaweedNFSServesSymlinkRoundTripOverRPC(t *testing.T) {
client := &fakeNFSFilerClient{
kv: map[string][]byte{
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/exports"),
},
entries: map[util.FullPath]*filer_pb.Entry{
"/exports": testEntry("exports", true, 101, uint32(0755), nil),
},
}
server := newTestServer(t, "/exports", client)
target, cleanup := mountTestTarget(t, server)
defer cleanup()
defer target.Close()
file, err := target.OpenFile("/target.txt", 0o644)
require.NoError(t, err)
_, err = file.Write([]byte("payload"))
require.NoError(t, err)
require.NoError(t, file.Close())
require.NoError(t, target.Symlink("target.txt", "/target.link"))
info, _, err := target.Lookup("/target.link")
require.NoError(t, err)
attr, ok := info.(*nfsclient.Fattr)
require.True(t, ok)
assert.Equal(t, uint32(nfsclient.NF3Lnk), attr.Type)
linkFile, err := target.Open("/target.link")
require.NoError(t, err)
defer linkFile.Close()
linkTarget, err := linkFile.Readlink()
require.NoError(t, err)
assert.Equal(t, "target.txt", linkTarget)
entry := client.entries["/exports/target.link"]
require.NotNil(t, entry)
assert.Equal(t, "target.txt", entry.GetAttributes().GetSymlinkTarget())
}
func TestSeaweedNFSServesHardLinkRoundTripOverRPC(t *testing.T) {
client := &fakeNFSFilerClient{
kv: map[string][]byte{
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/exports"),
},
entries: map[util.FullPath]*filer_pb.Entry{
"/exports": testEntry("exports", true, 101, uint32(0755), nil),
},
}
server := newTestServer(t, "/exports", client)
target, cleanup := mountTestTarget(t, server)
defer cleanup()
defer target.Close()
file, err := target.OpenFile("/source.txt", 0o644)
require.NoError(t, err)
payload := []byte("shared content")
_, err = file.Write(payload)
require.NoError(t, err)
require.NoError(t, file.Close())
_, sourceHandle, err := target.Lookup("/source.txt")
require.NoError(t, err)
require.NoError(t, nfsLink(target, sourceHandle, "/linked.txt"))
sourceInfo, sourceHandle, err := target.Lookup("/source.txt")
require.NoError(t, err)
linkedInfo, linkedHandle, err := target.Lookup("/linked.txt")
require.NoError(t, err)
sourceAttr, ok := sourceInfo.(*nfsclient.Fattr)
require.True(t, ok)
linkAttr, ok := linkedInfo.(*nfsclient.Fattr)
require.True(t, ok)
assert.Equal(t, sourceHandle, linkedHandle)
assert.Equal(t, sourceAttr.Fileid, linkAttr.Fileid)
assert.Equal(t, uint32(2), sourceAttr.Nlink)
assert.Equal(t, uint32(2), linkAttr.Nlink)
linkedFile, err := target.Open("/linked.txt")
require.NoError(t, err)
defer linkedFile.Close()
data, err := io.ReadAll(linkedFile)
require.NoError(t, err)
assert.Equal(t, payload, data)
sourceEntry := client.entries["/exports/source.txt"]
linkedEntry := client.entries["/exports/linked.txt"]
require.NotNil(t, sourceEntry)
require.NotNil(t, linkedEntry)
assert.Equal(t, sourceEntry.GetHardLinkId(), linkedEntry.GetHardLinkId())
assert.Equal(t, int32(2), sourceEntry.GetHardLinkCounter())
assert.Equal(t, int32(2), linkedEntry.GetHardLinkCounter())
require.NoError(t, target.Remove("/source.txt"))
remainingAttr, err := target.GetAttr(sourceHandle)
require.NoError(t, err)
assert.Equal(t, uint32(1), remainingAttr.Nlink)
_, _, err = target.Lookup("/source.txt")
require.Error(t, err)
linkedFile, err = target.Open("/linked.txt")
require.NoError(t, err)
data, err = io.ReadAll(linkedFile)
require.NoError(t, err)
require.NoError(t, linkedFile.Close())
assert.Equal(t, payload, data)
require.NoError(t, target.Remove("/linked.txt"))
_, err = target.GetAttr(linkedHandle)
require.Error(t, err)
nfsErr, ok := err.(*nfsclient.Error)
require.True(t, ok)
assert.Equal(t, uint32(nfsclient.NFS3ErrStale), nfsErr.ErrorNum)
}
func TestSeaweedNFSServesLargeChunkRoundTripOverRPC(t *testing.T) {
initIntegrationHTTPClient.Do(util_http.InitGlobalHttpClient)
client := &fakeNFSFilerClient{
kv: map[string][]byte{
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/exports"),
},
entries: map[util.FullPath]*filer_pb.Entry{
"/exports": testEntry("exports", true, 101, uint32(0755), nil),
},
}
volumeServer := newFakeVolumeServer(t)
controlPlane := &fakeVolumeControlPlane{host: volumeServer.host()}
controlPlaneAddr := startFakeVolumeControlPlane(t, controlPlane)
_, grpcPortString, err := net.SplitHostPort(controlPlaneAddr)
require.NoError(t, err)
grpcPort, err := strconv.Atoi(grpcPortString)
require.NoError(t, err)
server := newTestServer(t, "/exports", client)
server.option.Filer = pb.NewServerAddressWithGrpcPort(controlPlaneAddr, grpcPort)
server.option.GrpcDialOption = grpc.WithTransportCredentials(insecure.NewCredentials())
if server.filerClient != nil {
server.filerClient.Close()
}
server.filerClient = wdclient.NewFilerClient([]pb.ServerAddress{server.option.Filer}, server.option.GrpcDialOption, "")
server.withFilerClient = func(_ bool, fn func(filer_pb.SeaweedFilerClient) error) error {
conn, err := grpc.NewClient(controlPlaneAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil {
return err
}
defer conn.Close()
return fn(filer_pb.NewSeaweedFilerClient(conn))
}
target, cleanup := mountTestTarget(t, server)
defer cleanup()
defer target.Close()
payload := make([]byte, maxInlineWriteSize+4096)
_, err = rand.New(rand.NewSource(1)).Read(payload)
require.NoError(t, err)
file, err := target.OpenFile("/big.bin", 0o644)
require.NoError(t, err)
_, err = file.Write(payload)
require.NoError(t, err)
require.NoError(t, file.Close())
entry := client.entries["/exports/big.bin"]
require.NotNil(t, entry)
require.Len(t, entry.GetChunks(), 1)
assert.Nil(t, entry.Content)
assert.Equal(t, uint64(len(payload)), entry.GetAttributes().GetFileSize())
readFile, err := target.Open("/big.bin")
require.NoError(t, err)
defer readFile.Close()
data, err := io.ReadAll(readFile)
require.NoError(t, err)
assert.Equal(t, payload, data)
controlPlane.mu.Lock()
defer controlPlane.mu.Unlock()
require.Len(t, controlPlane.assigns, 1)
assert.Equal(t, "/exports/big.bin", controlPlane.assigns[0].GetPath())
assert.NotEmpty(t, controlPlane.lookups)
}
func TestSeaweedNFSRejectsStaleHandleAfterDeleteRecreate(t *testing.T) {
client := &fakeNFSFilerClient{
kv: map[string][]byte{
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/exports"),
},
entries: map[util.FullPath]*filer_pb.Entry{
"/exports": testEntry("exports", true, 101, uint32(0755), nil),
},
}
server := newTestServer(t, "/exports", client)
target, cleanup := mountTestTarget(t, server)
defer cleanup()
defer target.Close()
file, err := target.OpenFile("/stale.txt", 0o644)
require.NoError(t, err)
_, err = file.Write([]byte("old"))
require.NoError(t, err)
require.NoError(t, file.Close())
_, oldHandle, err := target.Lookup("/stale.txt")
require.NoError(t, err)
require.NoError(t, target.Remove("/stale.txt"))
file, err = target.OpenFile("/stale.txt", 0o644)
require.NoError(t, err)
_, err = file.Write([]byte("new"))
require.NoError(t, err)
require.NoError(t, file.Close())
_, err = target.GetAttr(oldHandle)
require.Error(t, err)
nfsErr, ok := err.(*nfsclient.Error)
require.True(t, ok)
assert.Equal(t, uint32(nfsclient.NFS3ErrStale), nfsErr.ErrorNum)
_, newHandle, err := target.Lookup("/stale.txt")
require.NoError(t, err)
_, err = target.GetAttr(newHandle)
require.NoError(t, err)
}
func TestSeaweedNFSFileHandleSurvivesServerRestart(t *testing.T) {
client := &fakeNFSFilerClient{
kv: map[string][]byte{
string(filer.InodeIndexKey(101)): testIndexRecord(t, 101, 1, "/exports"),
},
entries: map[util.FullPath]*filer_pb.Entry{
"/exports": testEntry("exports", true, 101, uint32(0755), nil),
},
}
server := newTestServer(t, "/exports", client)
target, cleanup := mountTestTarget(t, server)
file, err := target.OpenFile("/restart.txt", 0o644)
require.NoError(t, err)
payload := []byte("survives restart")
_, err = file.Write(payload)
require.NoError(t, err)
require.NoError(t, file.Close())
_, handle, err := target.Lookup("/restart.txt")
require.NoError(t, err)
target.Close()
cleanup()
restartedServer := newTestServer(t, "/exports", client)
restartedTarget, restartedCleanup := mountTestTarget(t, restartedServer)
defer restartedCleanup()
defer restartedTarget.Close()
attr, err := restartedTarget.GetAttr(handle)
require.NoError(t, err)
assert.Equal(t, uint64(client.entries["/exports/restart.txt"].GetAttributes().GetInode()), attr.Fileid)
_, restartedHandle, err := restartedTarget.Lookup("/restart.txt")
require.NoError(t, err)
assert.Equal(t, handle, restartedHandle)
readFile, err := restartedTarget.Open("/restart.txt")
require.NoError(t, err)
defer readFile.Close()
data, err := io.ReadAll(readFile)
require.NoError(t, err)
assert.Equal(t, payload, data)
}
Reference in New Issue View Git Blame Copy Permalink