Commit Graph
22 Commits
Author SHA1 Message Date
e64d01825f feat(shell): add -delete option to remote.copy.local (#10228)
* shell: add -delete option to remote.copy.local

Add a -delete flag to remote.copy.local that removes files and
directories from remote storage when they no longer exist locally,
similar to rsync --delete. This makes the command usable for
scheduled one-shot backups that also propagate local deletions.

- -include/-exclude patterns also limit which remote files are deleted
- size/age filters only apply to copying, since remote entries have no
  local attributes to filter on
- orphaned remote directories are removed after their contents,
  deepest first, and only when no name filter is set (a recursive
  RemoveDirectory could otherwise remove intentionally kept files)
- deletion is skipped entirely if any copy failed
- -dryRun shows DELETE lines for review before committing to anything

Fixes seaweedfs/seaweedfs#8609

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* shell: fix remote.copy.local -delete deleting files outside -dir

Traverse lists remote objects by key prefix with no delimiter, so a -dir
pointing at a subdirectory also matches siblings that merely share its
name prefix (foo -> foobar). Those are not under the local traversal
root, so -delete treated them as extraneous and removed them. Scope
deletion candidates to paths under dirToCopy.

Also drop the directory-removal path: RemoveDirectory is a no-op on every
backend and Traverse never emits directory entries, so it only ever
printed success for work it never did.

---------

Co-authored-by: Jason Lin <jason@jtx.com.tw>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-07-04 10:50:32 -07:00
Chris LuandGitHub 3306abae10 shell: add minCacheAge flag to remote.uncache command (#8225)
* add minCacheAge flag to remote.uncache command #8221

* address code review feedback: add nil check and improve test isolation

* address code review feedback: use consistent timestamp in FileFilter
2026-02-05 12:57:27 -08:00
Chris LuandGitHub 69553e5ba6 convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
chrislu ec30a504ba refactor 2024-09-29 10:38:22 -07:00
chrislu 701abbb9df add IsResourceHeavy() to command interface 2024-09-28 20:23:01 -07:00
chrislu e825dd7496 fix opposite logic
fix https://github.com/seaweedfs/seaweedfs/issues/5871
2024-08-08 23:16:38 -07:00
chrislu e069009f09 [remote.cache, remote.uncache] fix min/max age
fix https://github.com/seaweedfs/seaweedfs/issues/5871
2024-08-08 21:25:52 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu 9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu f3a334965d Update command_remote_uncache.go
minor
2021-11-28 23:10:32 -08:00
Chris Lu 57a95887d2 remote.cache remote.uncache supports all mounted directories 2021-09-05 14:47:06 -07:00
Chris Lu d983aa4c7d correct filtering 2021-09-04 13:58:14 -07:00
Chris Lu 49952ba905 fix cache/uncache filters 2021-09-04 13:45:39 -07:00
Chris Lu 97b7948cd9 adjust error message 2021-08-31 01:07:34 -07:00
Chris Lu 05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
Chris Lu 2158d4fe4d adjust help message 2021-08-21 02:17:10 -07:00
Chris Lu c34747c79d rename, fix wrong logic. 2021-08-14 21:46:34 -07:00
Chris Lu cb53802752 adjust help message 2021-08-14 15:55:53 -07:00
Chris Lu 708debca14 remote.cache and uncache: more flexible options to select files to cache or uncache 2021-08-14 15:11:55 -07:00
Chris Lu 02f728cb54 fix bug with remote.uncache 2021-08-09 15:24:21 -07:00
Chris Lu 402315f117 go fmt 2021-08-09 14:37:34 -07:00
Chris Lu 713c035a6e shell: remote.cache remote.uncache 2021-08-09 14:35:18 -07:00