mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-18 13:17:08 +00:00
refactor(command): expand "~" in all path-style CLI flags (#9306)
* refactor(command): expand "~" in all path-style CLI flags Many of weed's path-bearing flags (-s3.config, -s3.iam.config, -admin.dataDir, -webdav.cacheDir, -volume.dir.idx, TLS cert/key files, profile output paths, mount cache dirs, sftp key files, ...) were never run through util.ResolvePath, so a value like "~/iam.json" was used literally. Tilde only worked when the shell expanded it, which silently fails for the common -flag=~/path form (bash leaves the tilde literal in --opt=~/path). - Extend util.ResolvePath to also handle "~user" / "~user/rest", matching shell tilde expansion. Add unit tests. - Apply util.ResolvePath at the top of each shared start* function (s3, webdav, sftp) so mini/server/filer/standalone callers all inherit it; resolve at the few one-off use sites (mount cache dirs, volume idx folder, mini admin.dataDir, profile paths). - Drop the duplicate expandHomeDir helper from admin.go in favor of the now-equivalent util.ResolvePath. * fixup: handle comma-separated -dir flags for tilde expansion `weed mini -dir`, `weed server -dir`, and `weed volume -dir` accept comma-separated paths (`dir[,dir]...`). Calling util.ResolvePath on the whole string mishandled multi-folder values with tilde, e.g. "~/d1,~/d2" would resolve as if "d1,~/d2" were a single subpath. - Add util.ResolveCommaSeparatedPaths: split on ",", run each entry through ResolvePath, rejoin. Short-circuits when no "~" present. - Use it for *miniDataFolders (mini.go), *volumeDataFolders (server.go), and resolve each entry of v.folders in-place (volume.go) so all downstream consumers see resolved paths. - Add 7-case TestResolveCommaSeparatedPaths covering empty, single, multiple, and mixed inputs. * address PR review: metaFolder + Windows backslash - master.go: resolve *m.metaFolder at the top of runMaster so util.FullPath(*m.metaFolder) on the next line sees an expanded path. Drop the now-redundant ResolvePath in TestFolderWritable. - server.go: same treatment for *masterOptions.metaFolder, paired with the existing cpu/mem profile resolves. Drop the redundant inner ResolvePath at TestFolderWritable. - file_util.go: ResolvePath now accepts filepath.Separator as a separator after the tilde, so "~\\data" works on Windows. Other platforms keep current behaviour (backslash stays literal because it is a valid filename character in usernames and paths). - file_util_test.go: add two cases using filepath.Separator that exercise the new code path on Windows and remain a no-op on Unix. * address PR review: resolve "~" in remaining command path flags Comprehensive sweep of path-bearing flags across every weed subcommand, applying util.ResolvePath in-place at the top of each run* function so all downstream consumers see expanded paths. - webdav.go: resolve *wo.cacheDir at the top of startWebDav so mini/server/filer/standalone callers all inherit it. - mount_std.go: cpu/mem profile paths. - filer_sync.go: cpu/mem profile paths. - mq_broker.go: cpu/mem profile paths. - benchmark.go: cpuprofile output path. - backup.go: -dir resolved once at runBackup; drop the duplicated inline ResolvePath in NewVolume calls. - compact.go: -dir resolved at runCompact; drop inline ResolvePath. - export.go: -dir and -o resolved at runExport; drop inline ResolvePath in LoadFromIdx and ScanVolumeFile. - download.go: -dir resolved at runDownload; drop inline. - update.go: -dir resolved at runUpdate so filepath.Join uses the expanded path; drop inline ResolvePath in TestFolderWritable. - scaffold.go: -output expanded before filepath.Join. - worker.go: -workingDir expanded before being passed to runtime. * address PR review: resolve option-struct paths at run* entry points server.go:381 propagates s3Options.config to filerOptions.s3ConfigFile *before* startS3Server runs, which meant the filer-side code saw the unresolved tilde-prefixed pointer. Same pattern for webdavOptions and sftpOptions (and equivalent in mini.go / filer.go). The fix: hoist resolution from the shared start* functions up to the run* entry points, where every shared pointer is set up before any propagation happens. - s3.go, webdav.go, sftp.go: extract a resolvePaths() method on each Options struct that runs every path field through util.ResolvePath in-place. Idempotent. - runS3, runWebDav, runSftp: call the standalone struct's resolvePaths before starting metrics / loading security config. - runServer, runMini, runFiler: call resolvePaths on every embedded options struct, plus resolve loose flags (serverIamConfig, miniS3Config, miniIamConfig, miniMasterOptions.metaFolder, and filer's defaultLevelDbDirectory) so they're expanded before any pointer copy or use. - Drop the now-redundant inline ResolvePath at filer's defaultLevelDbDirectory composition. * address PR review: re-resolve mini -dir post-config, cover misc paths - mini.go: applyConfigFileOptions can overwrite -dir with a literal ~/data from mini.options. Re-resolve *miniDataFolders after the config-file apply, alongside the other path resolves, so the mini filer no longer ends up with a literal ~/data/filerldb2. - benchmark.go: resolve *b.idListFile (-list). - filer_sync.go: resolve *syncOptions.aSecurity / .bSecurity (-a.security / -b.security) before LoadClientTLSFromFile. - filer_cat.go: resolve *filerCat.output (-o) before os.OpenFile. - admin.go: drop trailing blank line at EOF (git diff --check). * address PR review: resolve -a.security/-b.security/-config before use Three follow-up fixes: - filer_sync.go: the -a.security / -b.security resolves were placed *after* LoadClientTLSFromFile / LoadHTTPClientFromFile were called, so weed filer.sync -a.security=~/a.toml still passed the literal tilde path. Hoist the resolves above the security-loading block so TLS clients see expanded paths. - filer_sync_verify.go: same flag pair was never resolved at all in the verify command; resolve at the top of runFilerSyncVerify. - filer_meta_backup.go: -config (the backup_filer.toml path) was passed directly to viper. Resolve at the top of runFilerMetaBackup. - mini.go: master.dir defaulted to the entire comma-joined miniDataFolders. With weed mini -dir=~/d1,~/d2 (or any multi-dir setup), TestFolderWritable then stat'd the joined string instead of a single directory. Default to the first entry via StringSplit to mirror the disk-space calculation a few lines below, and drop the now-redundant ResolvePath in TestFolderWritable.
This commit is contained in:
+3
-50
@@ -11,7 +11,6 @@ import (
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"os/user"
|
||||
"path/filepath"
|
||||
"runtime/debug"
|
||||
"strings"
|
||||
@@ -173,6 +172,8 @@ func runAdmin(cmd *Command, args []string) bool {
|
||||
grace.StartDebugServer(*a.debugPort)
|
||||
}
|
||||
|
||||
*a.cpuProfile = util.ResolvePath(*a.cpuProfile)
|
||||
*a.memProfile = util.ResolvePath(*a.memProfile)
|
||||
grace.SetupProfiling(*a.cpuProfile, *a.memProfile)
|
||||
|
||||
// Load security configuration
|
||||
@@ -309,18 +310,10 @@ func startAdminServer(ctx context.Context, options AdminOptions, enableUI bool,
|
||||
// Create data directory first if specified (needed for session key storage)
|
||||
var dataDir string
|
||||
if *options.dataDir != "" {
|
||||
// Expand tilde (~) to home directory
|
||||
expandedDir, err := expandHomeDir(*options.dataDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to expand dataDir path %s: %v", *options.dataDir, err)
|
||||
}
|
||||
dataDir = expandedDir
|
||||
|
||||
// Show path expansion if it occurred
|
||||
dataDir = util.ResolvePath(*options.dataDir)
|
||||
if dataDir != *options.dataDir {
|
||||
fmt.Printf("Expanded dataDir: %s -> %s\n", *options.dataDir, dataDir)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(dataDir, 0755); err != nil {
|
||||
return fmt.Errorf("failed to create data directory %s: %v", dataDir, err)
|
||||
}
|
||||
@@ -645,43 +638,3 @@ func applyViperFallback(cmd *Command, flagPtr *string, flagName, viperKey string
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// expandHomeDir expands the tilde (~) in a path to the user's home directory
|
||||
func expandHomeDir(path string) (string, error) {
|
||||
if path == "" {
|
||||
return path, nil
|
||||
}
|
||||
|
||||
if !strings.HasPrefix(path, "~") {
|
||||
return path, nil
|
||||
}
|
||||
|
||||
// Get current user
|
||||
currentUser, err := user.Current()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to get current user: %w", err)
|
||||
}
|
||||
|
||||
// Handle different tilde patterns
|
||||
if path == "~" {
|
||||
return currentUser.HomeDir, nil
|
||||
}
|
||||
|
||||
if strings.HasPrefix(path, "~/") {
|
||||
return filepath.Join(currentUser.HomeDir, path[2:]), nil
|
||||
}
|
||||
|
||||
// Handle ~username/ patterns
|
||||
parts := strings.SplitN(path[1:], "/", 2)
|
||||
username := parts[0]
|
||||
|
||||
targetUser, err := user.Lookup(username)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("user %s not found: %v", username, err)
|
||||
}
|
||||
|
||||
if len(parts) == 1 {
|
||||
return targetUser.HomeDir, nil
|
||||
}
|
||||
return filepath.Join(targetUser.HomeDir, parts[1]), nil
|
||||
}
|
||||
|
||||
@@ -130,7 +130,7 @@ func backupFromLocation(volumeServer pb.ServerAddress, grpcDialOption grpc.DialO
|
||||
ver := needle.Version(stats.Version)
|
||||
|
||||
// Create or load the volume
|
||||
v, err := storage.NewVolume(util.ResolvePath(*s.dir), util.ResolvePath(*s.dir), *s.collection, vid, storage.NeedleMapInMemory, replication, ttl, 0, ver, 0, 0)
|
||||
v, err := storage.NewVolume(*s.dir, *s.dir, *s.collection, vid, storage.NeedleMapInMemory, replication, ttl, 0, ver, 0, 0)
|
||||
if err != nil {
|
||||
return fmt.Errorf("creating or reading volume: %w", err), false
|
||||
}
|
||||
@@ -162,7 +162,7 @@ func backupFromLocation(volumeServer pb.ServerAddress, grpcDialOption grpc.DialO
|
||||
}
|
||||
v.Close() // Close the destroyed volume
|
||||
// recreate an empty volume
|
||||
v, err = storage.NewVolume(util.ResolvePath(*s.dir), util.ResolvePath(*s.dir), *s.collection, vid, storage.NeedleMapInMemory, replication, ttl, 0, ver, 0, 0)
|
||||
v, err = storage.NewVolume(*s.dir, *s.dir, *s.collection, vid, storage.NeedleMapInMemory, replication, ttl, 0, ver, 0, 0)
|
||||
if err != nil {
|
||||
return fmt.Errorf("recreating volume: %w", err), false
|
||||
}
|
||||
@@ -180,6 +180,7 @@ func backupFromLocation(volumeServer pb.ServerAddress, grpcDialOption grpc.DialO
|
||||
|
||||
func runBackup(cmd *Command, args []string) bool {
|
||||
|
||||
*s.dir = util.ResolvePath(*s.dir)
|
||||
util.LoadSecurityConfiguration()
|
||||
grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.client")
|
||||
|
||||
|
||||
@@ -123,6 +123,8 @@ func runBenchmark(cmd *Command, args []string) bool {
|
||||
*b.maxCpu = runtime.NumCPU()
|
||||
}
|
||||
runtime.GOMAXPROCS(*b.maxCpu)
|
||||
*b.cpuprofile = util.ResolvePath(*b.cpuprofile)
|
||||
*b.idListFile = util.ResolvePath(*b.idListFile)
|
||||
if *b.cpuprofile != "" {
|
||||
f, err := os.Create(*b.cpuprofile)
|
||||
if err != nil {
|
||||
|
||||
@@ -41,10 +41,11 @@ func runCompact(cmd *Command, args []string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
*compactVolumePath = util.ResolvePath(*compactVolumePath)
|
||||
preallocateBytes := *compactVolumePreallocate * (1 << 20)
|
||||
|
||||
vid := needle.VolumeId(*compactVolumeId)
|
||||
v, err := storage.NewVolume(util.ResolvePath(*compactVolumePath), util.ResolvePath(*compactVolumePath), *compactVolumeCollection, vid, storage.NeedleMapInMemory, nil, nil, preallocateBytes, needle.GetCurrentVersion(), 0, 0)
|
||||
v, err := storage.NewVolume(*compactVolumePath, *compactVolumePath, *compactVolumeCollection, vid, storage.NeedleMapInMemory, nil, nil, preallocateBytes, needle.GetCurrentVersion(), 0, 0)
|
||||
if err != nil {
|
||||
glog.Fatalf("Load Volume [ERROR] %s\n", err)
|
||||
}
|
||||
|
||||
@@ -59,8 +59,9 @@ func runDownload(cmd *Command, args []string) bool {
|
||||
masterServer = *d.server
|
||||
}
|
||||
|
||||
*d.dir = util.ResolvePath(*d.dir)
|
||||
for _, fid := range args {
|
||||
if e := downloadToFile(func(_ context.Context) pb.ServerAddress { return pb.ServerAddress(masterServer) }, grpcDialOption, fid, util.ResolvePath(*d.dir)); e != nil {
|
||||
if e := downloadToFile(func(_ context.Context) pb.ServerAddress { return pb.ServerAddress(masterServer) }, grpcDialOption, fid, *d.dir); e != nil {
|
||||
fmt.Println("Download Error: ", fid, e)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -148,6 +148,9 @@ func (scanner *VolumeFileScanner4Export) VisitNeedle(n *needle.Needle, offset in
|
||||
|
||||
func runExport(cmd *Command, args []string) bool {
|
||||
|
||||
*export.dir = util.ResolvePath(*export.dir)
|
||||
*output = util.ResolvePath(*output)
|
||||
|
||||
var err error
|
||||
|
||||
if *newer != "" {
|
||||
@@ -200,7 +203,7 @@ func runExport(cmd *Command, args []string) bool {
|
||||
needleMap := needle_map.NewMemDb()
|
||||
defer needleMap.Close()
|
||||
|
||||
if err := needleMap.LoadFromIdx(path.Join(util.ResolvePath(*export.dir), fileName+".idx")); err != nil {
|
||||
if err := needleMap.LoadFromIdx(path.Join(*export.dir, fileName+".idx")); err != nil {
|
||||
glog.Fatalf("cannot load needle map from %s.idx: %s", fileName, err)
|
||||
}
|
||||
|
||||
@@ -213,7 +216,7 @@ func runExport(cmd *Command, args []string) bool {
|
||||
fmt.Printf("key\tname\tsize\tgzip\tmime\tmodified\tttl\tdeleted\tstart\tstop\n")
|
||||
}
|
||||
|
||||
err = storage.ScanVolumeFile(util.ResolvePath(*export.dir), *export.collection, vid, storage.NeedleMapInMemory, volumeFileScanner)
|
||||
err = storage.ScanVolumeFile(*export.dir, *export.collection, vid, storage.NeedleMapInMemory, volumeFileScanner)
|
||||
if err != nil && err != io.EOF {
|
||||
glog.Errorf("Export Volume File [ERROR] %s\n", err)
|
||||
}
|
||||
|
||||
@@ -230,6 +230,10 @@ func runFiler(cmd *Command, args []string) bool {
|
||||
go http.ListenAndServe(fmt.Sprintf(":%d", *f.debugPort), nil)
|
||||
}
|
||||
|
||||
*f.defaultLevelDbDirectory = util.ResolvePath(*f.defaultLevelDbDirectory)
|
||||
filerS3Options.resolvePaths()
|
||||
filerWebDavOptions.resolvePaths()
|
||||
filerSftpOptions.resolvePaths()
|
||||
util.LoadSecurityConfiguration()
|
||||
|
||||
switch {
|
||||
@@ -328,7 +332,7 @@ func (fo *FilerOptions) startFiler() {
|
||||
*fo.allowedOrigins = "*"
|
||||
}
|
||||
|
||||
defaultLevelDbDirectory := util.ResolvePath(*fo.defaultLevelDbDirectory + "/filerldb2")
|
||||
defaultLevelDbDirectory := *fo.defaultLevelDbDirectory + "/filerldb2"
|
||||
|
||||
filerAddress := pb.NewServerAddress(*fo.ip, *fo.port, *fo.portGrpc)
|
||||
|
||||
|
||||
@@ -60,6 +60,7 @@ var cmdFilerCat = &Command{
|
||||
|
||||
func runFilerCat(cmd *Command, args []string) bool {
|
||||
|
||||
*filerCat.output = util.ResolvePath(*filerCat.output)
|
||||
util.LoadSecurityConfiguration()
|
||||
|
||||
if len(args) == 0 {
|
||||
|
||||
@@ -65,6 +65,8 @@ When both match, the deeper prefix wins.
|
||||
|
||||
func runFilerMetaBackup(cmd *Command, args []string) bool {
|
||||
|
||||
*metaBackup.backupFilerConfig = util.ResolvePath(*metaBackup.backupFilerConfig)
|
||||
|
||||
util.LoadSecurityConfiguration()
|
||||
metaBackup.grpcDialOption = security.LoadClientTLS(util.GetViper(), "grpc.client")
|
||||
|
||||
|
||||
@@ -147,6 +147,11 @@ func runFilerSynchronize(cmd *Command, args []string) bool {
|
||||
grace.StartDebugServer(*syncOptions.debugPort)
|
||||
}
|
||||
|
||||
*syncCpuProfile = util.ResolvePath(*syncCpuProfile)
|
||||
*syncMemProfile = util.ResolvePath(*syncMemProfile)
|
||||
*syncOptions.aSecurity = util.ResolvePath(*syncOptions.aSecurity)
|
||||
*syncOptions.bSecurity = util.ResolvePath(*syncOptions.bSecurity)
|
||||
|
||||
util.LoadSecurityConfiguration()
|
||||
grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.client")
|
||||
|
||||
|
||||
@@ -63,6 +63,8 @@ var cmdFilerSyncVerify = &Command{
|
||||
}
|
||||
|
||||
func runFilerSyncVerify(cmd *Command, args []string) bool {
|
||||
*syncVerifyOptions.aSecurity = util.ResolvePath(*syncVerifyOptions.aSecurity)
|
||||
*syncVerifyOptions.bSecurity = util.ResolvePath(*syncVerifyOptions.bSecurity)
|
||||
util.LoadSecurityConfiguration()
|
||||
grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.client")
|
||||
|
||||
|
||||
@@ -144,6 +144,9 @@ func runMaster(cmd *Command, args []string) bool {
|
||||
*m.metaFolder = v
|
||||
}
|
||||
|
||||
*m.metaFolder = util.ResolvePath(*m.metaFolder)
|
||||
*masterCpuProfile = util.ResolvePath(*masterCpuProfile)
|
||||
*masterMemProfile = util.ResolvePath(*masterMemProfile)
|
||||
grace.SetupProfiling(*masterCpuProfile, *masterMemProfile)
|
||||
|
||||
parent, _ := util.FullPath(*m.metaFolder).DirAndName()
|
||||
@@ -152,7 +155,7 @@ func runMaster(cmd *Command, args []string) bool {
|
||||
glog.Fatalf("Could not create Meta Folder %s: %v", *m.metaFolder, err)
|
||||
}
|
||||
}
|
||||
if err := util.TestFolderWritable(util.ResolvePath(*m.metaFolder)); err != nil {
|
||||
if err := util.TestFolderWritable(*m.metaFolder); err != nil {
|
||||
glog.Fatalf("Check Meta Folder (-mdir) Writable %s : %s", *m.metaFolder, err)
|
||||
}
|
||||
|
||||
|
||||
+21
-6
@@ -875,7 +875,7 @@ func saveMiniConfiguration(dataFolder string) error {
|
||||
}
|
||||
|
||||
func runMini(cmd *Command, args []string) bool {
|
||||
*miniDataFolders = util.ResolvePath(*miniDataFolders)
|
||||
*miniDataFolders = util.ResolveCommaSeparatedPaths(*miniDataFolders)
|
||||
|
||||
// Capture which port flags were explicitly passed on CLI BEFORE config file is applied
|
||||
// This is necessary to distinguish user-specified ports from defaults or config file options
|
||||
@@ -900,6 +900,17 @@ func runMini(cmd *Command, args []string) bool {
|
||||
util.LoadSecurityConfiguration()
|
||||
util.LoadConfiguration("master", false)
|
||||
|
||||
// applyConfigFileOptions above may have overwritten -dir from the
|
||||
// mini.options file, so re-resolve it here alongside the other paths.
|
||||
*miniDataFolders = util.ResolveCommaSeparatedPaths(*miniDataFolders)
|
||||
*miniOptions.cpuprofile = util.ResolvePath(*miniOptions.cpuprofile)
|
||||
*miniOptions.memprofile = util.ResolvePath(*miniOptions.memprofile)
|
||||
*miniS3Config = util.ResolvePath(*miniS3Config)
|
||||
*miniIamConfig = util.ResolvePath(*miniIamConfig)
|
||||
*miniMasterOptions.metaFolder = util.ResolvePath(*miniMasterOptions.metaFolder)
|
||||
*miniAdminOptions.dataDir = util.ResolvePath(*miniAdminOptions.dataDir)
|
||||
miniS3Options.resolvePaths()
|
||||
miniWebDavOptions.resolvePaths()
|
||||
grace.SetupProfiling(*miniOptions.cpuprofile, *miniOptions.memprofile)
|
||||
|
||||
// Determine bind IP
|
||||
@@ -976,9 +987,13 @@ func runMini(cmd *Command, args []string) bool {
|
||||
}
|
||||
|
||||
if *miniMasterOptions.metaFolder == "" {
|
||||
*miniMasterOptions.metaFolder = *miniDataFolders
|
||||
// -dir may be comma-separated (dir[,dir]...); the master expects a
|
||||
// single directory, so default to the first entry. Both miniDataFolders
|
||||
// and miniMasterOptions.metaFolder were already tilde-resolved at the
|
||||
// top of runMini.
|
||||
*miniMasterOptions.metaFolder = util.StringSplit(*miniDataFolders, ",")[0]
|
||||
}
|
||||
if err := util.TestFolderWritable(util.ResolvePath(*miniMasterOptions.metaFolder)); err != nil {
|
||||
if err := util.TestFolderWritable(*miniMasterOptions.metaFolder); err != nil {
|
||||
glog.Fatalf("Check Meta Folder (-dir=\"%s\") Writable: %s", *miniMasterOptions.metaFolder, err)
|
||||
}
|
||||
miniFilerOptions.defaultLevelDbDirectory = miniMasterOptions.metaFolder
|
||||
@@ -987,9 +1002,9 @@ func runMini(cmd *Command, args []string) bool {
|
||||
// Only auto-calculate if user didn't explicitly specify a value via -master.volumeSizeLimitMB
|
||||
if !isFlagPassed("master.volumeSizeLimitMB") {
|
||||
// User didn't override, use auto-calculated value
|
||||
// The -dir flag can accept comma-separated directories; use the first one for disk space calculation
|
||||
resolvedDataFolder := util.ResolvePath(util.StringSplit(*miniDataFolders, ",")[0])
|
||||
optimalVolumeSizeMB := calculateOptimalVolumeSizeMB(resolvedDataFolder)
|
||||
// The -dir flag can accept comma-separated directories; use the first one for disk space calculation.
|
||||
// miniDataFolders was already tilde-resolved at the top of runMini.
|
||||
optimalVolumeSizeMB := calculateOptimalVolumeSizeMB(util.StringSplit(*miniDataFolders, ",")[0])
|
||||
miniMasterOptions.volumeSizeLimitMB = &optimalVolumeSizeMB
|
||||
glog.Infof("Mini started with auto-calculated optimal volume size limit: %dMB", optimalVolumeSizeMB)
|
||||
} else {
|
||||
|
||||
@@ -41,6 +41,8 @@ func runMount(cmd *Command, args []string) bool {
|
||||
go http.ListenAndServe(fmt.Sprintf(":%d", *mountOptions.debugPort), nil)
|
||||
}
|
||||
|
||||
*mountCpuProfile = util.ResolvePath(*mountCpuProfile)
|
||||
*mountMemProfile = util.ResolvePath(*mountMemProfile)
|
||||
grace.SetupProfiling(*mountCpuProfile, *mountMemProfile)
|
||||
if *mountReadRetryTime < time.Second {
|
||||
*mountReadRetryTime = time.Second
|
||||
@@ -320,9 +322,10 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {
|
||||
mountRoot = mountRoot[0 : len(mountRoot)-1]
|
||||
}
|
||||
|
||||
cacheDirForWrite := *option.cacheDirForWrite
|
||||
cacheDirForRead := util.ResolvePath(*option.cacheDirForRead)
|
||||
cacheDirForWrite := util.ResolvePath(*option.cacheDirForWrite)
|
||||
if cacheDirForWrite == "" {
|
||||
cacheDirForWrite = *option.cacheDirForRead
|
||||
cacheDirForWrite = cacheDirForRead
|
||||
}
|
||||
|
||||
seaweedFileSystem := mount.NewSeaweedFileSystem(&mount.Option{
|
||||
@@ -339,7 +342,7 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {
|
||||
ChunkSizeLimit: int64(chunkSizeLimitMB) * 1024 * 1024,
|
||||
ConcurrentWriters: *option.concurrentWriters,
|
||||
ConcurrentReaders: *option.concurrentReaders,
|
||||
CacheDirForRead: *option.cacheDirForRead,
|
||||
CacheDirForRead: cacheDirForRead,
|
||||
CacheSizeMBForRead: *option.cacheSizeMBForRead,
|
||||
CacheDirForWrite: cacheDirForWrite,
|
||||
WriteBufferSizeMB: *option.writeBufferSizeMB,
|
||||
|
||||
@@ -73,6 +73,8 @@ func runMqBroker(cmd *Command, args []string) bool {
|
||||
|
||||
func (mqBrokerOpt *MessageQueueBrokerOptions) startQueueServer() bool {
|
||||
|
||||
*mqBrokerStandaloneOptions.cpuprofile = util.ResolvePath(*mqBrokerStandaloneOptions.cpuprofile)
|
||||
*mqBrokerStandaloneOptions.memprofile = util.ResolvePath(*mqBrokerStandaloneOptions.memprofile)
|
||||
grace.SetupProfiling(*mqBrokerStandaloneOptions.cpuprofile, *mqBrokerStandaloneOptions.memprofile)
|
||||
|
||||
grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.msg_broker")
|
||||
|
||||
@@ -211,6 +211,7 @@ func runS3(cmd *Command, args []string) bool {
|
||||
grace.StartDebugServer(*s3StandaloneOptions.debugPort)
|
||||
}
|
||||
|
||||
s3StandaloneOptions.resolvePaths()
|
||||
util.LoadSecurityConfiguration()
|
||||
|
||||
switch {
|
||||
@@ -244,6 +245,19 @@ func (s3opt *S3Options) parseDefaultFileMode() (uint32, error) {
|
||||
return uint32(mode), nil
|
||||
}
|
||||
|
||||
// resolvePaths expands "~" in every user-supplied path flag so callers
|
||||
// that share these pointers (e.g. server.go propagating s3Options.config
|
||||
// to filerOptions.s3ConfigFile before startS3Server runs) see resolved
|
||||
// values. Idempotent — safe to call from any entry point.
|
||||
func (s3opt *S3Options) resolvePaths() {
|
||||
*s3opt.config = util.ResolvePath(*s3opt.config)
|
||||
*s3opt.iamConfig = util.ResolvePath(*s3opt.iamConfig)
|
||||
*s3opt.tlsCertificate = util.ResolvePath(*s3opt.tlsCertificate)
|
||||
*s3opt.tlsPrivateKey = util.ResolvePath(*s3opt.tlsPrivateKey)
|
||||
*s3opt.tlsCACertificate = util.ResolvePath(*s3opt.tlsCACertificate)
|
||||
*s3opt.auditLogConfig = util.ResolvePath(*s3opt.auditLogConfig)
|
||||
}
|
||||
|
||||
func (s3opt *S3Options) startS3Server() bool {
|
||||
|
||||
filerAddresses := pb.ServerAddresses(*s3opt.filer).ToAddresses()
|
||||
|
||||
@@ -59,7 +59,7 @@ func runScaffold(cmd *Command, args []string) bool {
|
||||
}
|
||||
|
||||
if *outputPath != "" {
|
||||
util.WriteFile(filepath.Join(*outputPath, *config+".toml"), []byte(content), 0644)
|
||||
util.WriteFile(filepath.Join(util.ResolvePath(*outputPath), *config+".toml"), []byte(content), 0644)
|
||||
} else {
|
||||
fmt.Println(content)
|
||||
}
|
||||
|
||||
@@ -224,6 +224,13 @@ func runServer(cmd *Command, args []string) bool {
|
||||
util.LoadSecurityConfiguration()
|
||||
util.LoadConfiguration("master", false)
|
||||
|
||||
*serverOptions.cpuprofile = util.ResolvePath(*serverOptions.cpuprofile)
|
||||
*serverOptions.memprofile = util.ResolvePath(*serverOptions.memprofile)
|
||||
*serverIamConfig = util.ResolvePath(*serverIamConfig)
|
||||
*masterOptions.metaFolder = util.ResolvePath(*masterOptions.metaFolder)
|
||||
s3Options.resolvePaths()
|
||||
webdavOptions.resolvePaths()
|
||||
sftpOptions.resolvePaths()
|
||||
grace.SetupProfiling(*serverOptions.cpuprofile, *serverOptions.memprofile)
|
||||
|
||||
if *isStartingS3 {
|
||||
@@ -318,6 +325,7 @@ func runServer(cmd *Command, args []string) bool {
|
||||
|
||||
go stats_collect.StartMetricsServer(*serverMetricsHttpIp, *serverMetricsHttpPort)
|
||||
|
||||
*volumeDataFolders = util.ResolveCommaSeparatedPaths(*volumeDataFolders)
|
||||
folders := strings.Split(*volumeDataFolders, ",")
|
||||
|
||||
if *masterOptions.volumeSizeLimitMB > util.VolumeSizeLimitGB*1000 {
|
||||
@@ -327,7 +335,7 @@ func runServer(cmd *Command, args []string) bool {
|
||||
if *masterOptions.metaFolder == "" {
|
||||
*masterOptions.metaFolder = folders[0]
|
||||
}
|
||||
if err := util.TestFolderWritable(util.ResolvePath(*masterOptions.metaFolder)); err != nil {
|
||||
if err := util.TestFolderWritable(*masterOptions.metaFolder); err != nil {
|
||||
glog.Fatalf("Check Meta Folder (-mdir=\"%s\") Writable: %s", *masterOptions.metaFolder, err)
|
||||
}
|
||||
filerOptions.defaultLevelDbDirectory = masterOptions.metaFolder
|
||||
|
||||
@@ -79,6 +79,7 @@ func init() {
|
||||
|
||||
// runSftp is the command entry point.
|
||||
func runSftp(cmd *Command, args []string) bool {
|
||||
sftpOptionsStandalone.resolvePaths()
|
||||
// Load security configuration as done in other SeaweedFS services.
|
||||
util.LoadSecurityConfiguration()
|
||||
|
||||
@@ -94,6 +95,14 @@ func runSftp(cmd *Command, args []string) bool {
|
||||
return sftpOptionsStandalone.startSftpServer()
|
||||
}
|
||||
|
||||
// resolvePaths expands "~" in every user-supplied path flag.
|
||||
// Idempotent — safe to call from any entry point.
|
||||
func (sftpOpt *SftpOptions) resolvePaths() {
|
||||
*sftpOpt.sshPrivateKey = util.ResolvePath(*sftpOpt.sshPrivateKey)
|
||||
*sftpOpt.hostKeysFolder = util.ResolvePath(*sftpOpt.hostKeysFolder)
|
||||
*sftpOpt.userStoreFile = util.ResolvePath(*sftpOpt.userStoreFile)
|
||||
}
|
||||
|
||||
func (sftpOpt *SftpOptions) startSftpServer() bool {
|
||||
if *sftpOpt.bindIp == "" {
|
||||
*sftpOpt.bindIp = "0.0.0.0"
|
||||
|
||||
@@ -85,8 +85,9 @@ func runUpdate(cmd *Command, args []string) bool {
|
||||
path, _ := os.Executable()
|
||||
_, name := filepath.Split(path)
|
||||
|
||||
*updateOpt.dir = util.ResolvePath(*updateOpt.dir)
|
||||
if *updateOpt.dir != "" {
|
||||
if err := util.TestFolderWritable(util.ResolvePath(*updateOpt.dir)); err != nil {
|
||||
if err := util.TestFolderWritable(*updateOpt.dir); err != nil {
|
||||
glog.Fatalf("Check Folder(-dir) Writable %s : %s", *updateOpt.dir, err)
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -150,6 +150,8 @@ func runVolume(cmd *Command, args []string) bool {
|
||||
// If --pprof is set we assume the caller wants to be able to collect
|
||||
// cpu and memory profiles via go tool pprof
|
||||
if !*v.pprof {
|
||||
*v.cpuProfile = util.ResolvePath(*v.cpuProfile)
|
||||
*v.memProfile = util.ResolvePath(*v.memProfile)
|
||||
grace.SetupProfiling(*v.cpuProfile, *v.memProfile)
|
||||
}
|
||||
|
||||
@@ -179,9 +181,10 @@ func (v VolumeServerOptions) startVolumeServer(volumeFolders, maxVolumeCounts, v
|
||||
|
||||
// Set multiple folders and each folder's max volume count limit'
|
||||
v.folders = strings.Split(volumeFolders, ",")
|
||||
for _, folder := range v.folders {
|
||||
if err := util.TestFolderWritable(util.ResolvePath(folder)); err != nil {
|
||||
glog.Fatalf("Check Data Folder(-dir) Writable %s : %s", folder, err)
|
||||
for i, folder := range v.folders {
|
||||
v.folders[i] = util.ResolvePath(folder)
|
||||
if err := util.TestFolderWritable(v.folders[i]); err != nil {
|
||||
glog.Fatalf("Check Data Folder(-dir) Writable %s : %s", v.folders[i], err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -284,7 +287,7 @@ func (v VolumeServerOptions) startVolumeServer(volumeFolders, maxVolumeCounts, v
|
||||
volumeServer := weed_server.NewVolumeServer(volumeMux, publicVolumeMux,
|
||||
*v.ip, *v.port, *v.portGrpc, *v.publicUrl, volumeServerId,
|
||||
v.folders, v.folderMaxLimits, minFreeSpaces, diskTypes, folderTags,
|
||||
*v.idxFolder,
|
||||
util.ResolvePath(*v.idxFolder),
|
||||
volumeNeedleMapKind,
|
||||
v.masters, constants.VolumePulsePeriod, *v.dataCenter, *v.rack,
|
||||
v.whiteList,
|
||||
|
||||
+10
-1
@@ -69,6 +69,7 @@ var cmdWebDav = &Command{
|
||||
|
||||
func runWebDav(cmd *Command, args []string) bool {
|
||||
|
||||
webDavStandaloneOptions.resolvePaths()
|
||||
util.LoadSecurityConfiguration()
|
||||
|
||||
listenAddress := fmt.Sprintf("%s:%d", *webDavStandaloneOptions.ipBind, *webDavStandaloneOptions.port)
|
||||
@@ -78,6 +79,14 @@ func runWebDav(cmd *Command, args []string) bool {
|
||||
|
||||
}
|
||||
|
||||
// resolvePaths expands "~" in every user-supplied path flag.
|
||||
// Idempotent — safe to call from any entry point.
|
||||
func (wo *WebDavOption) resolvePaths() {
|
||||
*wo.cacheDir = util.ResolvePath(*wo.cacheDir)
|
||||
*wo.tlsCertificate = util.ResolvePath(*wo.tlsCertificate)
|
||||
*wo.tlsPrivateKey = util.ResolvePath(*wo.tlsPrivateKey)
|
||||
}
|
||||
|
||||
func (wo *WebDavOption) startWebDav() bool {
|
||||
|
||||
// detect current user
|
||||
@@ -126,7 +135,7 @@ func (wo *WebDavOption) startWebDav() bool {
|
||||
Uid: uid,
|
||||
Gid: gid,
|
||||
Cipher: cipher,
|
||||
CacheDir: util.ResolvePath(*wo.cacheDir),
|
||||
CacheDir: *wo.cacheDir,
|
||||
CacheSizeMB: *wo.cacheSizeMB,
|
||||
MaxMB: *wo.maxMB,
|
||||
})
|
||||
|
||||
@@ -3,6 +3,7 @@ package command
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/seaweedfs/seaweedfs/weed/util"
|
||||
"github.com/seaweedfs/seaweedfs/weed/util/grace"
|
||||
)
|
||||
|
||||
@@ -62,6 +63,7 @@ func runWorker(cmd *Command, args []string) bool {
|
||||
grace.StartDebugServer(*workerDebugPort)
|
||||
}
|
||||
|
||||
*workerWorkingDir = util.ResolvePath(*workerWorkingDir)
|
||||
return runPluginWorkerWithOptions(pluginWorkerRunOptions{
|
||||
AdminServer: *workerAdminServer,
|
||||
WorkerID: *workerID,
|
||||
|
||||
+60
-11
@@ -82,25 +82,74 @@ func CheckFile(filename string) (exists, canRead, canWrite bool, modTime time.Ti
|
||||
return
|
||||
}
|
||||
|
||||
// ResolvePath expands a leading "~", "~/", or "~username/" in path to the
|
||||
// corresponding home directory, mirroring shell tilde expansion. A path
|
||||
// without a leading "~" or whose tilde cannot be resolved is returned
|
||||
// unchanged so callers can pass any user-supplied path through this helper
|
||||
// without worrying about non-tilde inputs or lookup failures.
|
||||
//
|
||||
// Forward slashes are always recognised as separators after the tilde;
|
||||
// the platform-native separator is also accepted so "~\\data" works on
|
||||
// Windows. Backslashes are deliberately not treated as separators on
|
||||
// other platforms because they are legal characters in usernames and
|
||||
// path segments there.
|
||||
func ResolvePath(path string) string {
|
||||
|
||||
if !strings.Contains(path, "~") {
|
||||
if !strings.HasPrefix(path, "~") {
|
||||
return path
|
||||
}
|
||||
|
||||
usr, _ := user.Current()
|
||||
dir := usr.HomeDir
|
||||
|
||||
if path == "~" {
|
||||
// In case of "~", which won't be caught by the "else if"
|
||||
path = dir
|
||||
} else if strings.HasPrefix(path, "~/") {
|
||||
// Use strings.HasPrefix so we don't match paths like
|
||||
// "/something/~/something/"
|
||||
path = filepath.Join(dir, path[2:])
|
||||
if usr, err := user.Current(); err == nil {
|
||||
return usr.HomeDir
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
return path
|
||||
isSep := func(b byte) bool {
|
||||
return b == '/' || b == byte(filepath.Separator)
|
||||
}
|
||||
|
||||
if isSep(path[1]) {
|
||||
if usr, err := user.Current(); err == nil {
|
||||
return filepath.Join(usr.HomeDir, path[2:])
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
// "~username" or "~username<sep>rest"
|
||||
name := path[1:]
|
||||
rest := ""
|
||||
for i := 0; i < len(name); i++ {
|
||||
if isSep(name[i]) {
|
||||
rest = name[i+1:]
|
||||
name = name[:i]
|
||||
break
|
||||
}
|
||||
}
|
||||
usr, err := user.Lookup(name)
|
||||
if err != nil {
|
||||
return path
|
||||
}
|
||||
if rest == "" {
|
||||
return usr.HomeDir
|
||||
}
|
||||
return filepath.Join(usr.HomeDir, rest)
|
||||
}
|
||||
|
||||
// ResolveCommaSeparatedPaths splits paths on "," and runs each entry through
|
||||
// ResolvePath, then rejoins them. This lets flags like `weed mini -dir` or
|
||||
// `weed volume -dir` accept tilde-prefixed entries (e.g. "~/d1,~/d2") even
|
||||
// in the comma-separated form the help text advertises.
|
||||
func ResolveCommaSeparatedPaths(paths string) string {
|
||||
if !strings.Contains(paths, "~") {
|
||||
return paths
|
||||
}
|
||||
parts := strings.Split(paths, ",")
|
||||
for i, p := range parts {
|
||||
parts[i] = ResolvePath(p)
|
||||
}
|
||||
return strings.Join(parts, ",")
|
||||
}
|
||||
|
||||
func FileNameBase(filename string) string {
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
package util
|
||||
|
||||
import (
|
||||
"os/user"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@@ -20,3 +22,67 @@ func TestToShortFileName(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolvePath(t *testing.T) {
|
||||
usr, err := user.Current()
|
||||
if err != nil {
|
||||
t.Fatalf("user.Current: %v", err)
|
||||
}
|
||||
home := usr.HomeDir
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
in string
|
||||
want string
|
||||
}{
|
||||
{"empty", "", ""},
|
||||
{"absolute", "/var/data", "/var/data"},
|
||||
{"relative", "data", "data"},
|
||||
{"tilde mid path is literal", "/foo/~/bar", "/foo/~/bar"},
|
||||
{"bare tilde", "~", home},
|
||||
{"tilde slash", "~/", home},
|
||||
{"tilde with subpath", "~/data", filepath.Join(home, "data")},
|
||||
{"tilde with current user", "~" + usr.Username, home},
|
||||
{"tilde with current user and subpath", "~" + usr.Username + "/data", filepath.Join(home, "data")},
|
||||
{"tilde unknown user falls back to literal", "~no-such-user-xyz/data", "~no-such-user-xyz/data"},
|
||||
// Native separator: identical to "~/data" on Unix; exercises backslash on Windows.
|
||||
{"tilde with native separator", "~" + string(filepath.Separator) + "data", filepath.Join(home, "data")},
|
||||
{"tilde user with native separator", "~" + usr.Username + string(filepath.Separator) + "data", filepath.Join(home, "data")},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
if got := ResolvePath(c.in); got != c.want {
|
||||
t.Errorf("ResolvePath(%q) = %q; want %q", c.in, got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveCommaSeparatedPaths(t *testing.T) {
|
||||
usr, err := user.Current()
|
||||
if err != nil {
|
||||
t.Fatalf("user.Current: %v", err)
|
||||
}
|
||||
home := usr.HomeDir
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
in string
|
||||
want string
|
||||
}{
|
||||
{"empty", "", ""},
|
||||
{"single absolute", "/a", "/a"},
|
||||
{"single tilde", "~/a", filepath.Join(home, "a")},
|
||||
{"two absolutes", "/a,/b", "/a,/b"},
|
||||
{"two tildes", "~/a,~/b", filepath.Join(home, "a") + "," + filepath.Join(home, "b")},
|
||||
{"mixed", "/a,~/b,/c", "/a," + filepath.Join(home, "b") + ",/c"},
|
||||
{"no tilde fast path", "/a,/b,/c", "/a,/b,/c"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
if got := ResolveCommaSeparatedPaths(c.in); got != c.want {
|
||||
t.Errorf("ResolveCommaSeparatedPaths(%q) = %q; want %q", c.in, got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user