refactor(command): expand "~" in all path-style CLI flags (#9306)

* refactor(command): expand "~" in all path-style CLI flags

Many of weed's path-bearing flags (-s3.config, -s3.iam.config,
-admin.dataDir, -webdav.cacheDir, -volume.dir.idx, TLS cert/key
files, profile output paths, mount cache dirs, sftp key files, ...)
were never run through util.ResolvePath, so a value like "~/iam.json"
was used literally. Tilde only worked when the shell expanded it,
which silently fails for the common -flag=~/path form (bash leaves
the tilde literal in --opt=~/path).

- Extend util.ResolvePath to also handle "~user" / "~user/rest",
  matching shell tilde expansion. Add unit tests.
- Apply util.ResolvePath at the top of each shared start* function
  (s3, webdav, sftp) so mini/server/filer/standalone callers all
  inherit it; resolve at the few one-off use sites (mount cache
  dirs, volume idx folder, mini admin.dataDir, profile paths).
- Drop the duplicate expandHomeDir helper from admin.go in favor of
  the now-equivalent util.ResolvePath.

* fixup: handle comma-separated -dir flags for tilde expansion

`weed mini -dir`, `weed server -dir`, and `weed volume -dir` accept
comma-separated paths (`dir[,dir]...`). Calling util.ResolvePath on
the whole string mishandled multi-folder values with tilde, e.g.
"~/d1,~/d2" would resolve as if "d1,~/d2" were a single subpath.

- Add util.ResolveCommaSeparatedPaths: split on ",", run each entry
  through ResolvePath, rejoin. Short-circuits when no "~" present.
- Use it for *miniDataFolders (mini.go), *volumeDataFolders (server.go),
  and resolve each entry of v.folders in-place (volume.go) so all
  downstream consumers see resolved paths.
- Add 7-case TestResolveCommaSeparatedPaths covering empty, single,
  multiple, and mixed inputs.

* address PR review: metaFolder + Windows backslash

- master.go: resolve *m.metaFolder at the top of runMaster so
  util.FullPath(*m.metaFolder) on the next line sees an expanded
  path. Drop the now-redundant ResolvePath in TestFolderWritable.
- server.go: same treatment for *masterOptions.metaFolder, paired
  with the existing cpu/mem profile resolves. Drop the redundant
  inner ResolvePath at TestFolderWritable.
- file_util.go: ResolvePath now accepts filepath.Separator as a
  separator after the tilde, so "~\\data" works on Windows. Other
  platforms keep current behaviour (backslash stays literal because
  it is a valid filename character in usernames and paths).
- file_util_test.go: add two cases using filepath.Separator that
  exercise the new code path on Windows and remain a no-op on Unix.

* address PR review: resolve "~" in remaining command path flags

Comprehensive sweep of path-bearing flags across every weed
subcommand, applying util.ResolvePath in-place at the top of each
run* function so all downstream consumers see expanded paths.

- webdav.go: resolve *wo.cacheDir at the top of startWebDav so
  mini/server/filer/standalone callers all inherit it.
- mount_std.go: cpu/mem profile paths.
- filer_sync.go: cpu/mem profile paths.
- mq_broker.go: cpu/mem profile paths.
- benchmark.go: cpuprofile output path.
- backup.go: -dir resolved once at runBackup; drop the duplicated
  inline ResolvePath in NewVolume calls.
- compact.go: -dir resolved at runCompact; drop inline ResolvePath.
- export.go: -dir and -o resolved at runExport; drop inline
  ResolvePath in LoadFromIdx and ScanVolumeFile.
- download.go: -dir resolved at runDownload; drop inline.
- update.go: -dir resolved at runUpdate so filepath.Join uses the
  expanded path; drop inline ResolvePath in TestFolderWritable.
- scaffold.go: -output expanded before filepath.Join.
- worker.go: -workingDir expanded before being passed to runtime.

* address PR review: resolve option-struct paths at run* entry points

server.go:381 propagates s3Options.config to filerOptions.s3ConfigFile
*before* startS3Server runs, which meant the filer-side code saw the
unresolved tilde-prefixed pointer. Same pattern for webdavOptions and
sftpOptions (and equivalent in mini.go / filer.go).

The fix: hoist resolution from the shared start* functions up to the
run* entry points, where every shared pointer is set up before any
propagation happens.

- s3.go, webdav.go, sftp.go: extract a resolvePaths() method on each
  Options struct that runs every path field through util.ResolvePath
  in-place. Idempotent.
- runS3, runWebDav, runSftp: call the standalone struct's resolvePaths
  before starting metrics / loading security config.
- runServer, runMini, runFiler: call resolvePaths on every embedded
  options struct, plus resolve loose flags (serverIamConfig,
  miniS3Config, miniIamConfig, miniMasterOptions.metaFolder, and
  filer's defaultLevelDbDirectory) so they're expanded before any
  pointer copy or use.
- Drop the now-redundant inline ResolvePath at filer's
  defaultLevelDbDirectory composition.

* address PR review: re-resolve mini -dir post-config, cover misc paths

- mini.go: applyConfigFileOptions can overwrite -dir with a literal
  ~/data from mini.options. Re-resolve *miniDataFolders after the
  config-file apply, alongside the other path resolves, so the mini
  filer no longer ends up with a literal ~/data/filerldb2.
- benchmark.go: resolve *b.idListFile (-list).
- filer_sync.go: resolve *syncOptions.aSecurity / .bSecurity
  (-a.security / -b.security) before LoadClientTLSFromFile.
- filer_cat.go: resolve *filerCat.output (-o) before os.OpenFile.
- admin.go: drop trailing blank line at EOF (git diff --check).

* address PR review: resolve -a.security/-b.security/-config before use

Three follow-up fixes:

- filer_sync.go: the -a.security / -b.security resolves were placed
  *after* LoadClientTLSFromFile / LoadHTTPClientFromFile were called,
  so weed filer.sync -a.security=~/a.toml still passed the literal
  tilde path. Hoist the resolves above the security-loading block so
  TLS clients see expanded paths.
- filer_sync_verify.go: same flag pair was never resolved at all in
  the verify command; resolve at the top of runFilerSyncVerify.
- filer_meta_backup.go: -config (the backup_filer.toml path) was
  passed directly to viper. Resolve at the top of runFilerMetaBackup.
- mini.go: master.dir defaulted to the entire comma-joined
  miniDataFolders. With weed mini -dir=~/d1,~/d2 (or any multi-dir
  setup), TestFolderWritable then stat'd the joined string instead
  of a single directory. Default to the first entry via StringSplit
  to mirror the disk-space calculation a few lines below, and drop
  the now-redundant ResolvePath in TestFolderWritable.
This commit is contained in:
Chris Lu
2026-05-03 21:46:21 -07:00
committed by GitHub
parent 87bb0a4115
commit d605feb403
25 changed files with 245 additions and 86 deletions
+3 -50
View File
@@ -11,7 +11,6 @@ import (
"net/http"
"os"
"os/signal"
"os/user"
"path/filepath"
"runtime/debug"
"strings"
@@ -173,6 +172,8 @@ func runAdmin(cmd *Command, args []string) bool {
grace.StartDebugServer(*a.debugPort)
}
*a.cpuProfile = util.ResolvePath(*a.cpuProfile)
*a.memProfile = util.ResolvePath(*a.memProfile)
grace.SetupProfiling(*a.cpuProfile, *a.memProfile)
// Load security configuration
@@ -309,18 +310,10 @@ func startAdminServer(ctx context.Context, options AdminOptions, enableUI bool,
// Create data directory first if specified (needed for session key storage)
var dataDir string
if *options.dataDir != "" {
// Expand tilde (~) to home directory
expandedDir, err := expandHomeDir(*options.dataDir)
if err != nil {
return fmt.Errorf("failed to expand dataDir path %s: %v", *options.dataDir, err)
}
dataDir = expandedDir
// Show path expansion if it occurred
dataDir = util.ResolvePath(*options.dataDir)
if dataDir != *options.dataDir {
fmt.Printf("Expanded dataDir: %s -> %s\n", *options.dataDir, dataDir)
}
if err := os.MkdirAll(dataDir, 0755); err != nil {
return fmt.Errorf("failed to create data directory %s: %v", dataDir, err)
}
@@ -645,43 +638,3 @@ func applyViperFallback(cmd *Command, flagPtr *string, flagName, viperKey string
}
}
}
// expandHomeDir expands the tilde (~) in a path to the user's home directory
func expandHomeDir(path string) (string, error) {
if path == "" {
return path, nil
}
if !strings.HasPrefix(path, "~") {
return path, nil
}
// Get current user
currentUser, err := user.Current()
if err != nil {
return "", fmt.Errorf("failed to get current user: %w", err)
}
// Handle different tilde patterns
if path == "~" {
return currentUser.HomeDir, nil
}
if strings.HasPrefix(path, "~/") {
return filepath.Join(currentUser.HomeDir, path[2:]), nil
}
// Handle ~username/ patterns
parts := strings.SplitN(path[1:], "/", 2)
username := parts[0]
targetUser, err := user.Lookup(username)
if err != nil {
return "", fmt.Errorf("user %s not found: %v", username, err)
}
if len(parts) == 1 {
return targetUser.HomeDir, nil
}
return filepath.Join(targetUser.HomeDir, parts[1]), nil
}
+3 -2
View File
@@ -130,7 +130,7 @@ func backupFromLocation(volumeServer pb.ServerAddress, grpcDialOption grpc.DialO
ver := needle.Version(stats.Version)
// Create or load the volume
v, err := storage.NewVolume(util.ResolvePath(*s.dir), util.ResolvePath(*s.dir), *s.collection, vid, storage.NeedleMapInMemory, replication, ttl, 0, ver, 0, 0)
v, err := storage.NewVolume(*s.dir, *s.dir, *s.collection, vid, storage.NeedleMapInMemory, replication, ttl, 0, ver, 0, 0)
if err != nil {
return fmt.Errorf("creating or reading volume: %w", err), false
}
@@ -162,7 +162,7 @@ func backupFromLocation(volumeServer pb.ServerAddress, grpcDialOption grpc.DialO
}
v.Close() // Close the destroyed volume
// recreate an empty volume
v, err = storage.NewVolume(util.ResolvePath(*s.dir), util.ResolvePath(*s.dir), *s.collection, vid, storage.NeedleMapInMemory, replication, ttl, 0, ver, 0, 0)
v, err = storage.NewVolume(*s.dir, *s.dir, *s.collection, vid, storage.NeedleMapInMemory, replication, ttl, 0, ver, 0, 0)
if err != nil {
return fmt.Errorf("recreating volume: %w", err), false
}
@@ -180,6 +180,7 @@ func backupFromLocation(volumeServer pb.ServerAddress, grpcDialOption grpc.DialO
func runBackup(cmd *Command, args []string) bool {
*s.dir = util.ResolvePath(*s.dir)
util.LoadSecurityConfiguration()
grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.client")
+2
View File
@@ -123,6 +123,8 @@ func runBenchmark(cmd *Command, args []string) bool {
*b.maxCpu = runtime.NumCPU()
}
runtime.GOMAXPROCS(*b.maxCpu)
*b.cpuprofile = util.ResolvePath(*b.cpuprofile)
*b.idListFile = util.ResolvePath(*b.idListFile)
if *b.cpuprofile != "" {
f, err := os.Create(*b.cpuprofile)
if err != nil {
+2 -1
View File
@@ -41,10 +41,11 @@ func runCompact(cmd *Command, args []string) bool {
return false
}
*compactVolumePath = util.ResolvePath(*compactVolumePath)
preallocateBytes := *compactVolumePreallocate * (1 << 20)
vid := needle.VolumeId(*compactVolumeId)
v, err := storage.NewVolume(util.ResolvePath(*compactVolumePath), util.ResolvePath(*compactVolumePath), *compactVolumeCollection, vid, storage.NeedleMapInMemory, nil, nil, preallocateBytes, needle.GetCurrentVersion(), 0, 0)
v, err := storage.NewVolume(*compactVolumePath, *compactVolumePath, *compactVolumeCollection, vid, storage.NeedleMapInMemory, nil, nil, preallocateBytes, needle.GetCurrentVersion(), 0, 0)
if err != nil {
glog.Fatalf("Load Volume [ERROR] %s\n", err)
}
+2 -1
View File
@@ -59,8 +59,9 @@ func runDownload(cmd *Command, args []string) bool {
masterServer = *d.server
}
*d.dir = util.ResolvePath(*d.dir)
for _, fid := range args {
if e := downloadToFile(func(_ context.Context) pb.ServerAddress { return pb.ServerAddress(masterServer) }, grpcDialOption, fid, util.ResolvePath(*d.dir)); e != nil {
if e := downloadToFile(func(_ context.Context) pb.ServerAddress { return pb.ServerAddress(masterServer) }, grpcDialOption, fid, *d.dir); e != nil {
fmt.Println("Download Error: ", fid, e)
}
}
+5 -2
View File
@@ -148,6 +148,9 @@ func (scanner *VolumeFileScanner4Export) VisitNeedle(n *needle.Needle, offset in
func runExport(cmd *Command, args []string) bool {
*export.dir = util.ResolvePath(*export.dir)
*output = util.ResolvePath(*output)
var err error
if *newer != "" {
@@ -200,7 +203,7 @@ func runExport(cmd *Command, args []string) bool {
needleMap := needle_map.NewMemDb()
defer needleMap.Close()
if err := needleMap.LoadFromIdx(path.Join(util.ResolvePath(*export.dir), fileName+".idx")); err != nil {
if err := needleMap.LoadFromIdx(path.Join(*export.dir, fileName+".idx")); err != nil {
glog.Fatalf("cannot load needle map from %s.idx: %s", fileName, err)
}
@@ -213,7 +216,7 @@ func runExport(cmd *Command, args []string) bool {
fmt.Printf("key\tname\tsize\tgzip\tmime\tmodified\tttl\tdeleted\tstart\tstop\n")
}
err = storage.ScanVolumeFile(util.ResolvePath(*export.dir), *export.collection, vid, storage.NeedleMapInMemory, volumeFileScanner)
err = storage.ScanVolumeFile(*export.dir, *export.collection, vid, storage.NeedleMapInMemory, volumeFileScanner)
if err != nil && err != io.EOF {
glog.Errorf("Export Volume File [ERROR] %s\n", err)
}
+5 -1
View File
@@ -230,6 +230,10 @@ func runFiler(cmd *Command, args []string) bool {
go http.ListenAndServe(fmt.Sprintf(":%d", *f.debugPort), nil)
}
*f.defaultLevelDbDirectory = util.ResolvePath(*f.defaultLevelDbDirectory)
filerS3Options.resolvePaths()
filerWebDavOptions.resolvePaths()
filerSftpOptions.resolvePaths()
util.LoadSecurityConfiguration()
switch {
@@ -328,7 +332,7 @@ func (fo *FilerOptions) startFiler() {
*fo.allowedOrigins = "*"
}
defaultLevelDbDirectory := util.ResolvePath(*fo.defaultLevelDbDirectory + "/filerldb2")
defaultLevelDbDirectory := *fo.defaultLevelDbDirectory + "/filerldb2"
filerAddress := pb.NewServerAddress(*fo.ip, *fo.port, *fo.portGrpc)
+1
View File
@@ -60,6 +60,7 @@ var cmdFilerCat = &Command{
func runFilerCat(cmd *Command, args []string) bool {
*filerCat.output = util.ResolvePath(*filerCat.output)
util.LoadSecurityConfiguration()
if len(args) == 0 {
+2
View File
@@ -65,6 +65,8 @@ When both match, the deeper prefix wins.
func runFilerMetaBackup(cmd *Command, args []string) bool {
*metaBackup.backupFilerConfig = util.ResolvePath(*metaBackup.backupFilerConfig)
util.LoadSecurityConfiguration()
metaBackup.grpcDialOption = security.LoadClientTLS(util.GetViper(), "grpc.client")
+5
View File
@@ -147,6 +147,11 @@ func runFilerSynchronize(cmd *Command, args []string) bool {
grace.StartDebugServer(*syncOptions.debugPort)
}
*syncCpuProfile = util.ResolvePath(*syncCpuProfile)
*syncMemProfile = util.ResolvePath(*syncMemProfile)
*syncOptions.aSecurity = util.ResolvePath(*syncOptions.aSecurity)
*syncOptions.bSecurity = util.ResolvePath(*syncOptions.bSecurity)
util.LoadSecurityConfiguration()
grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.client")
+2
View File
@@ -63,6 +63,8 @@ var cmdFilerSyncVerify = &Command{
}
func runFilerSyncVerify(cmd *Command, args []string) bool {
*syncVerifyOptions.aSecurity = util.ResolvePath(*syncVerifyOptions.aSecurity)
*syncVerifyOptions.bSecurity = util.ResolvePath(*syncVerifyOptions.bSecurity)
util.LoadSecurityConfiguration()
grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.client")
+4 -1
View File
@@ -144,6 +144,9 @@ func runMaster(cmd *Command, args []string) bool {
*m.metaFolder = v
}
*m.metaFolder = util.ResolvePath(*m.metaFolder)
*masterCpuProfile = util.ResolvePath(*masterCpuProfile)
*masterMemProfile = util.ResolvePath(*masterMemProfile)
grace.SetupProfiling(*masterCpuProfile, *masterMemProfile)
parent, _ := util.FullPath(*m.metaFolder).DirAndName()
@@ -152,7 +155,7 @@ func runMaster(cmd *Command, args []string) bool {
glog.Fatalf("Could not create Meta Folder %s: %v", *m.metaFolder, err)
}
}
if err := util.TestFolderWritable(util.ResolvePath(*m.metaFolder)); err != nil {
if err := util.TestFolderWritable(*m.metaFolder); err != nil {
glog.Fatalf("Check Meta Folder (-mdir) Writable %s : %s", *m.metaFolder, err)
}
+21 -6
View File
@@ -875,7 +875,7 @@ func saveMiniConfiguration(dataFolder string) error {
}
func runMini(cmd *Command, args []string) bool {
*miniDataFolders = util.ResolvePath(*miniDataFolders)
*miniDataFolders = util.ResolveCommaSeparatedPaths(*miniDataFolders)
// Capture which port flags were explicitly passed on CLI BEFORE config file is applied
// This is necessary to distinguish user-specified ports from defaults or config file options
@@ -900,6 +900,17 @@ func runMini(cmd *Command, args []string) bool {
util.LoadSecurityConfiguration()
util.LoadConfiguration("master", false)
// applyConfigFileOptions above may have overwritten -dir from the
// mini.options file, so re-resolve it here alongside the other paths.
*miniDataFolders = util.ResolveCommaSeparatedPaths(*miniDataFolders)
*miniOptions.cpuprofile = util.ResolvePath(*miniOptions.cpuprofile)
*miniOptions.memprofile = util.ResolvePath(*miniOptions.memprofile)
*miniS3Config = util.ResolvePath(*miniS3Config)
*miniIamConfig = util.ResolvePath(*miniIamConfig)
*miniMasterOptions.metaFolder = util.ResolvePath(*miniMasterOptions.metaFolder)
*miniAdminOptions.dataDir = util.ResolvePath(*miniAdminOptions.dataDir)
miniS3Options.resolvePaths()
miniWebDavOptions.resolvePaths()
grace.SetupProfiling(*miniOptions.cpuprofile, *miniOptions.memprofile)
// Determine bind IP
@@ -976,9 +987,13 @@ func runMini(cmd *Command, args []string) bool {
}
if *miniMasterOptions.metaFolder == "" {
*miniMasterOptions.metaFolder = *miniDataFolders
// -dir may be comma-separated (dir[,dir]...); the master expects a
// single directory, so default to the first entry. Both miniDataFolders
// and miniMasterOptions.metaFolder were already tilde-resolved at the
// top of runMini.
*miniMasterOptions.metaFolder = util.StringSplit(*miniDataFolders, ",")[0]
}
if err := util.TestFolderWritable(util.ResolvePath(*miniMasterOptions.metaFolder)); err != nil {
if err := util.TestFolderWritable(*miniMasterOptions.metaFolder); err != nil {
glog.Fatalf("Check Meta Folder (-dir=\"%s\") Writable: %s", *miniMasterOptions.metaFolder, err)
}
miniFilerOptions.defaultLevelDbDirectory = miniMasterOptions.metaFolder
@@ -987,9 +1002,9 @@ func runMini(cmd *Command, args []string) bool {
// Only auto-calculate if user didn't explicitly specify a value via -master.volumeSizeLimitMB
if !isFlagPassed("master.volumeSizeLimitMB") {
// User didn't override, use auto-calculated value
// The -dir flag can accept comma-separated directories; use the first one for disk space calculation
resolvedDataFolder := util.ResolvePath(util.StringSplit(*miniDataFolders, ",")[0])
optimalVolumeSizeMB := calculateOptimalVolumeSizeMB(resolvedDataFolder)
// The -dir flag can accept comma-separated directories; use the first one for disk space calculation.
// miniDataFolders was already tilde-resolved at the top of runMini.
optimalVolumeSizeMB := calculateOptimalVolumeSizeMB(util.StringSplit(*miniDataFolders, ",")[0])
miniMasterOptions.volumeSizeLimitMB = &optimalVolumeSizeMB
glog.Infof("Mini started with auto-calculated optimal volume size limit: %dMB", optimalVolumeSizeMB)
} else {
+6 -3
View File
@@ -41,6 +41,8 @@ func runMount(cmd *Command, args []string) bool {
go http.ListenAndServe(fmt.Sprintf(":%d", *mountOptions.debugPort), nil)
}
*mountCpuProfile = util.ResolvePath(*mountCpuProfile)
*mountMemProfile = util.ResolvePath(*mountMemProfile)
grace.SetupProfiling(*mountCpuProfile, *mountMemProfile)
if *mountReadRetryTime < time.Second {
*mountReadRetryTime = time.Second
@@ -320,9 +322,10 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {
mountRoot = mountRoot[0 : len(mountRoot)-1]
}
cacheDirForWrite := *option.cacheDirForWrite
cacheDirForRead := util.ResolvePath(*option.cacheDirForRead)
cacheDirForWrite := util.ResolvePath(*option.cacheDirForWrite)
if cacheDirForWrite == "" {
cacheDirForWrite = *option.cacheDirForRead
cacheDirForWrite = cacheDirForRead
}
seaweedFileSystem := mount.NewSeaweedFileSystem(&mount.Option{
@@ -339,7 +342,7 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {
ChunkSizeLimit: int64(chunkSizeLimitMB) * 1024 * 1024,
ConcurrentWriters: *option.concurrentWriters,
ConcurrentReaders: *option.concurrentReaders,
CacheDirForRead: *option.cacheDirForRead,
CacheDirForRead: cacheDirForRead,
CacheSizeMBForRead: *option.cacheSizeMBForRead,
CacheDirForWrite: cacheDirForWrite,
WriteBufferSizeMB: *option.writeBufferSizeMB,
+2
View File
@@ -73,6 +73,8 @@ func runMqBroker(cmd *Command, args []string) bool {
func (mqBrokerOpt *MessageQueueBrokerOptions) startQueueServer() bool {
*mqBrokerStandaloneOptions.cpuprofile = util.ResolvePath(*mqBrokerStandaloneOptions.cpuprofile)
*mqBrokerStandaloneOptions.memprofile = util.ResolvePath(*mqBrokerStandaloneOptions.memprofile)
grace.SetupProfiling(*mqBrokerStandaloneOptions.cpuprofile, *mqBrokerStandaloneOptions.memprofile)
grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.msg_broker")
+14
View File
@@ -211,6 +211,7 @@ func runS3(cmd *Command, args []string) bool {
grace.StartDebugServer(*s3StandaloneOptions.debugPort)
}
s3StandaloneOptions.resolvePaths()
util.LoadSecurityConfiguration()
switch {
@@ -244,6 +245,19 @@ func (s3opt *S3Options) parseDefaultFileMode() (uint32, error) {
return uint32(mode), nil
}
// resolvePaths expands "~" in every user-supplied path flag so callers
// that share these pointers (e.g. server.go propagating s3Options.config
// to filerOptions.s3ConfigFile before startS3Server runs) see resolved
// values. Idempotent — safe to call from any entry point.
func (s3opt *S3Options) resolvePaths() {
*s3opt.config = util.ResolvePath(*s3opt.config)
*s3opt.iamConfig = util.ResolvePath(*s3opt.iamConfig)
*s3opt.tlsCertificate = util.ResolvePath(*s3opt.tlsCertificate)
*s3opt.tlsPrivateKey = util.ResolvePath(*s3opt.tlsPrivateKey)
*s3opt.tlsCACertificate = util.ResolvePath(*s3opt.tlsCACertificate)
*s3opt.auditLogConfig = util.ResolvePath(*s3opt.auditLogConfig)
}
func (s3opt *S3Options) startS3Server() bool {
filerAddresses := pb.ServerAddresses(*s3opt.filer).ToAddresses()
+1 -1
View File
@@ -59,7 +59,7 @@ func runScaffold(cmd *Command, args []string) bool {
}
if *outputPath != "" {
util.WriteFile(filepath.Join(*outputPath, *config+".toml"), []byte(content), 0644)
util.WriteFile(filepath.Join(util.ResolvePath(*outputPath), *config+".toml"), []byte(content), 0644)
} else {
fmt.Println(content)
}
+9 -1
View File
@@ -224,6 +224,13 @@ func runServer(cmd *Command, args []string) bool {
util.LoadSecurityConfiguration()
util.LoadConfiguration("master", false)
*serverOptions.cpuprofile = util.ResolvePath(*serverOptions.cpuprofile)
*serverOptions.memprofile = util.ResolvePath(*serverOptions.memprofile)
*serverIamConfig = util.ResolvePath(*serverIamConfig)
*masterOptions.metaFolder = util.ResolvePath(*masterOptions.metaFolder)
s3Options.resolvePaths()
webdavOptions.resolvePaths()
sftpOptions.resolvePaths()
grace.SetupProfiling(*serverOptions.cpuprofile, *serverOptions.memprofile)
if *isStartingS3 {
@@ -318,6 +325,7 @@ func runServer(cmd *Command, args []string) bool {
go stats_collect.StartMetricsServer(*serverMetricsHttpIp, *serverMetricsHttpPort)
*volumeDataFolders = util.ResolveCommaSeparatedPaths(*volumeDataFolders)
folders := strings.Split(*volumeDataFolders, ",")
if *masterOptions.volumeSizeLimitMB > util.VolumeSizeLimitGB*1000 {
@@ -327,7 +335,7 @@ func runServer(cmd *Command, args []string) bool {
if *masterOptions.metaFolder == "" {
*masterOptions.metaFolder = folders[0]
}
if err := util.TestFolderWritable(util.ResolvePath(*masterOptions.metaFolder)); err != nil {
if err := util.TestFolderWritable(*masterOptions.metaFolder); err != nil {
glog.Fatalf("Check Meta Folder (-mdir=\"%s\") Writable: %s", *masterOptions.metaFolder, err)
}
filerOptions.defaultLevelDbDirectory = masterOptions.metaFolder
+9
View File
@@ -79,6 +79,7 @@ func init() {
// runSftp is the command entry point.
func runSftp(cmd *Command, args []string) bool {
sftpOptionsStandalone.resolvePaths()
// Load security configuration as done in other SeaweedFS services.
util.LoadSecurityConfiguration()
@@ -94,6 +95,14 @@ func runSftp(cmd *Command, args []string) bool {
return sftpOptionsStandalone.startSftpServer()
}
// resolvePaths expands "~" in every user-supplied path flag.
// Idempotent — safe to call from any entry point.
func (sftpOpt *SftpOptions) resolvePaths() {
*sftpOpt.sshPrivateKey = util.ResolvePath(*sftpOpt.sshPrivateKey)
*sftpOpt.hostKeysFolder = util.ResolvePath(*sftpOpt.hostKeysFolder)
*sftpOpt.userStoreFile = util.ResolvePath(*sftpOpt.userStoreFile)
}
func (sftpOpt *SftpOptions) startSftpServer() bool {
if *sftpOpt.bindIp == "" {
*sftpOpt.bindIp = "0.0.0.0"
+2 -1
View File
@@ -85,8 +85,9 @@ func runUpdate(cmd *Command, args []string) bool {
path, _ := os.Executable()
_, name := filepath.Split(path)
*updateOpt.dir = util.ResolvePath(*updateOpt.dir)
if *updateOpt.dir != "" {
if err := util.TestFolderWritable(util.ResolvePath(*updateOpt.dir)); err != nil {
if err := util.TestFolderWritable(*updateOpt.dir); err != nil {
glog.Fatalf("Check Folder(-dir) Writable %s : %s", *updateOpt.dir, err)
return false
}
+7 -4
View File
@@ -150,6 +150,8 @@ func runVolume(cmd *Command, args []string) bool {
// If --pprof is set we assume the caller wants to be able to collect
// cpu and memory profiles via go tool pprof
if !*v.pprof {
*v.cpuProfile = util.ResolvePath(*v.cpuProfile)
*v.memProfile = util.ResolvePath(*v.memProfile)
grace.SetupProfiling(*v.cpuProfile, *v.memProfile)
}
@@ -179,9 +181,10 @@ func (v VolumeServerOptions) startVolumeServer(volumeFolders, maxVolumeCounts, v
// Set multiple folders and each folder's max volume count limit'
v.folders = strings.Split(volumeFolders, ",")
for _, folder := range v.folders {
if err := util.TestFolderWritable(util.ResolvePath(folder)); err != nil {
glog.Fatalf("Check Data Folder(-dir) Writable %s : %s", folder, err)
for i, folder := range v.folders {
v.folders[i] = util.ResolvePath(folder)
if err := util.TestFolderWritable(v.folders[i]); err != nil {
glog.Fatalf("Check Data Folder(-dir) Writable %s : %s", v.folders[i], err)
}
}
@@ -284,7 +287,7 @@ func (v VolumeServerOptions) startVolumeServer(volumeFolders, maxVolumeCounts, v
volumeServer := weed_server.NewVolumeServer(volumeMux, publicVolumeMux,
*v.ip, *v.port, *v.portGrpc, *v.publicUrl, volumeServerId,
v.folders, v.folderMaxLimits, minFreeSpaces, diskTypes, folderTags,
*v.idxFolder,
util.ResolvePath(*v.idxFolder),
volumeNeedleMapKind,
v.masters, constants.VolumePulsePeriod, *v.dataCenter, *v.rack,
v.whiteList,
+10 -1
View File
@@ -69,6 +69,7 @@ var cmdWebDav = &Command{
func runWebDav(cmd *Command, args []string) bool {
webDavStandaloneOptions.resolvePaths()
util.LoadSecurityConfiguration()
listenAddress := fmt.Sprintf("%s:%d", *webDavStandaloneOptions.ipBind, *webDavStandaloneOptions.port)
@@ -78,6 +79,14 @@ func runWebDav(cmd *Command, args []string) bool {
}
// resolvePaths expands "~" in every user-supplied path flag.
// Idempotent — safe to call from any entry point.
func (wo *WebDavOption) resolvePaths() {
*wo.cacheDir = util.ResolvePath(*wo.cacheDir)
*wo.tlsCertificate = util.ResolvePath(*wo.tlsCertificate)
*wo.tlsPrivateKey = util.ResolvePath(*wo.tlsPrivateKey)
}
func (wo *WebDavOption) startWebDav() bool {
// detect current user
@@ -126,7 +135,7 @@ func (wo *WebDavOption) startWebDav() bool {
Uid: uid,
Gid: gid,
Cipher: cipher,
CacheDir: util.ResolvePath(*wo.cacheDir),
CacheDir: *wo.cacheDir,
CacheSizeMB: *wo.cacheSizeMB,
MaxMB: *wo.maxMB,
})
+2
View File
@@ -3,6 +3,7 @@ package command
import (
"time"
"github.com/seaweedfs/seaweedfs/weed/util"
"github.com/seaweedfs/seaweedfs/weed/util/grace"
)
@@ -62,6 +63,7 @@ func runWorker(cmd *Command, args []string) bool {
grace.StartDebugServer(*workerDebugPort)
}
*workerWorkingDir = util.ResolvePath(*workerWorkingDir)
return runPluginWorkerWithOptions(pluginWorkerRunOptions{
AdminServer: *workerAdminServer,
WorkerID: *workerID,
+60 -11
View File
@@ -82,25 +82,74 @@ func CheckFile(filename string) (exists, canRead, canWrite bool, modTime time.Ti
return
}
// ResolvePath expands a leading "~", "~/", or "~username/" in path to the
// corresponding home directory, mirroring shell tilde expansion. A path
// without a leading "~" or whose tilde cannot be resolved is returned
// unchanged so callers can pass any user-supplied path through this helper
// without worrying about non-tilde inputs or lookup failures.
//
// Forward slashes are always recognised as separators after the tilde;
// the platform-native separator is also accepted so "~\\data" works on
// Windows. Backslashes are deliberately not treated as separators on
// other platforms because they are legal characters in usernames and
// path segments there.
func ResolvePath(path string) string {
if !strings.Contains(path, "~") {
if !strings.HasPrefix(path, "~") {
return path
}
usr, _ := user.Current()
dir := usr.HomeDir
if path == "~" {
// In case of "~", which won't be caught by the "else if"
path = dir
} else if strings.HasPrefix(path, "~/") {
// Use strings.HasPrefix so we don't match paths like
// "/something/~/something/"
path = filepath.Join(dir, path[2:])
if usr, err := user.Current(); err == nil {
return usr.HomeDir
}
return path
}
return path
isSep := func(b byte) bool {
return b == '/' || b == byte(filepath.Separator)
}
if isSep(path[1]) {
if usr, err := user.Current(); err == nil {
return filepath.Join(usr.HomeDir, path[2:])
}
return path
}
// "~username" or "~username<sep>rest"
name := path[1:]
rest := ""
for i := 0; i < len(name); i++ {
if isSep(name[i]) {
rest = name[i+1:]
name = name[:i]
break
}
}
usr, err := user.Lookup(name)
if err != nil {
return path
}
if rest == "" {
return usr.HomeDir
}
return filepath.Join(usr.HomeDir, rest)
}
// ResolveCommaSeparatedPaths splits paths on "," and runs each entry through
// ResolvePath, then rejoins them. This lets flags like `weed mini -dir` or
// `weed volume -dir` accept tilde-prefixed entries (e.g. "~/d1,~/d2") even
// in the comma-separated form the help text advertises.
func ResolveCommaSeparatedPaths(paths string) string {
if !strings.Contains(paths, "~") {
return paths
}
parts := strings.Split(paths, ",")
for i, p := range parts {
parts[i] = ResolvePath(p)
}
return strings.Join(parts, ",")
}
func FileNameBase(filename string) string {
+66
View File
@@ -1,6 +1,8 @@
package util
import (
"os/user"
"path/filepath"
"testing"
)
@@ -20,3 +22,67 @@ func TestToShortFileName(t *testing.T) {
}
}
}
func TestResolvePath(t *testing.T) {
usr, err := user.Current()
if err != nil {
t.Fatalf("user.Current: %v", err)
}
home := usr.HomeDir
cases := []struct {
name string
in string
want string
}{
{"empty", "", ""},
{"absolute", "/var/data", "/var/data"},
{"relative", "data", "data"},
{"tilde mid path is literal", "/foo/~/bar", "/foo/~/bar"},
{"bare tilde", "~", home},
{"tilde slash", "~/", home},
{"tilde with subpath", "~/data", filepath.Join(home, "data")},
{"tilde with current user", "~" + usr.Username, home},
{"tilde with current user and subpath", "~" + usr.Username + "/data", filepath.Join(home, "data")},
{"tilde unknown user falls back to literal", "~no-such-user-xyz/data", "~no-such-user-xyz/data"},
// Native separator: identical to "~/data" on Unix; exercises backslash on Windows.
{"tilde with native separator", "~" + string(filepath.Separator) + "data", filepath.Join(home, "data")},
{"tilde user with native separator", "~" + usr.Username + string(filepath.Separator) + "data", filepath.Join(home, "data")},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
if got := ResolvePath(c.in); got != c.want {
t.Errorf("ResolvePath(%q) = %q; want %q", c.in, got, c.want)
}
})
}
}
func TestResolveCommaSeparatedPaths(t *testing.T) {
usr, err := user.Current()
if err != nil {
t.Fatalf("user.Current: %v", err)
}
home := usr.HomeDir
cases := []struct {
name string
in string
want string
}{
{"empty", "", ""},
{"single absolute", "/a", "/a"},
{"single tilde", "~/a", filepath.Join(home, "a")},
{"two absolutes", "/a,/b", "/a,/b"},
{"two tildes", "~/a,~/b", filepath.Join(home, "a") + "," + filepath.Join(home, "b")},
{"mixed", "/a,~/b,/c", "/a," + filepath.Join(home, "b") + ",/c"},
{"no tilde fast path", "/a,/b,/c", "/a,/b,/c"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
if got := ResolveCommaSeparatedPaths(c.in); got != c.want {
t.Errorf("ResolveCommaSeparatedPaths(%q) = %q; want %q", c.in, got, c.want)
}
})
}
}