mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-16 20:26:45 +00:00
Chris LuandGitHub
8aa57bef78
* mount: readdir enters a child in the inode table only when it takes a reference Only readdirplus into the kernel takes a reference on the children it reports, and only that reference brings a FORGET later to take the entry back out. Every other listing was inserting all its children anyway. On WinFsp that meant a listing looked each child up, took a reference, and immediately gave it back, so a walk of a wide directory paid three write-lock acquisitions per entry to leave the table exactly as it found it. On a plain kernel readdir nothing gives the entry back at all, so listing a directory of 200k files grew both maps by 200k entries that were never reclaimed. A dirent's inode number is informational either way: the kernel must LOOKUP before it can use a nodeid, and the WinFsp adapter re-resolves every operation by path. So report the number and let the mapping be built when something actually looks the entry up. * mount: take the readdirplus reference without a second full lookup The entry has just been resolved a few lines above, so redoing the whole lookup only rebuilds the child path and walks both maps again to reach a counter. Bump it directly, falling back to the full lookup if a Forget removed the entry in between. * mount: benchmark a readdir over a 200k directory Drives doReadDirectory against a meta cache holding 200k entries, one round of 4096 at a time, for the three front ends that behave differently: a plain kernel readdir, kernel readdirplus, and a WinFsp listing that gets attributes but never returns a reference. Reports what each leaves behind in the inode table alongside the usual metrics. The sink declares TakesLookupRef as an ordinary method rather than through the interface, so the same file runs unchanged against an older tree for comparison. * mount: stamp an inode on the benchmark's entries The filer stores one on every entry it writes, so a real listing arrives with an inode and never derives its own. Leaving it zero made every child in the benchmark fall through to the MD5 in AsInode, work no filer-backed mount does, and charged it to both sides of the comparison.
44 lines
1.5 KiB
Go
44 lines
1.5 KiB
Go
package mount
|
|
|
|
import (
|
|
"github.com/seaweedfs/go-fuse/v2/fuse"
|
|
)
|
|
|
|
// DirEntrySink receives the entries a readdir produces. The FUSE server packs
|
|
// them straight into the kernel's reply buffer; a front end that is not the
|
|
// kernel reads them out instead of re-parsing that wire format.
|
|
type DirEntrySink interface {
|
|
// AddEntry reports one entry, returning false once the sink is full. A
|
|
// full sink ends the batch; the client resumes from the entry's Off.
|
|
AddEntry(entry fuse.DirEntry) bool
|
|
|
|
// AddEntryPlus is AddEntry for readdirplus, returning the attribute block
|
|
// to fill in, or nil once the sink is full.
|
|
AddEntryPlus(entry fuse.DirEntry) *fuse.EntryOut
|
|
|
|
// TakesLookupRef reports whether AddEntryPlus hands the sink a reference the
|
|
// mount must hold until a FORGET returns it.
|
|
TakesLookupRef() bool
|
|
}
|
|
|
|
// fuseDirEntryList adapts the kernel reply buffer to DirEntrySink.
|
|
type fuseDirEntryList struct {
|
|
*fuse.DirEntryList
|
|
}
|
|
|
|
func (l fuseDirEntryList) AddEntry(entry fuse.DirEntry) bool {
|
|
return l.AddDirEntry(entry)
|
|
}
|
|
|
|
func (l fuseDirEntryList) AddEntryPlus(entry fuse.DirEntry) *fuse.EntryOut {
|
|
return l.AddDirLookupEntry(entry)
|
|
}
|
|
|
|
func (l fuseDirEntryList) TakesLookupRef() bool { return true }
|
|
|
|
// ReadDirectoryInto runs a readdir against sink. ReadDir and ReadDirPlus are
|
|
// this with the kernel reply buffer as the sink.
|
|
func (wfs *WFS) ReadDirectoryInto(input *fuse.ReadIn, sink DirEntrySink, isPlusMode bool) fuse.Status {
|
|
return wfs.doReadDirectory(input, sink, isPlusMode)
|
|
}
|