Commit Graph
5 Commits
Author SHA1 Message Date
Chris LuandGitHub 298ab35fd7 shell: default batch size for ec.encode and ec.decode (#10308)
* shell: ec.encode batches 10 volumes by default

Encoding a whole collection as one batch means a late failure leaves
everything half-converted. Default -batchSize to 10 so each batch is
encoded, rebalanced, verified, and its originals deleted before the
next starts. -batchSize=0 keeps the all-at-once behavior. Batch
progress messages now print only when there is more than one batch,
so small runs read as before.

* shell: ec.decode decodes in batches, refreshing topology in between

ec.decode walked every volume off the single topology snapshot taken
at startup, which goes stale as earlier decodes move shards around and
create volumes. Decode 10 volumes per batch by default, re-collecting
the topology and rebuilding the free-space accounting between batches.
-batchSize=0 keeps the single-snapshot behavior.
2026-07-10 16:44:46 -07:00
Chris LuandGitHub 0a46577700 Fix #8040: Support '_default' keyword in collectionPattern to match default collection (#8046)
* Fix #8040: Support 'default' keyword in collectionPattern to match default collection

The default collection in SeaweedFS is represented as an empty string internally.
Previously, it was impossible to specifically target only the default collection
because:
- Empty collectionPattern matched ALL collections (filter was skipped)
- Using collectionPattern="default" tried to match the literal string "default"

This commit adds special handling for the keyword "default" in collectionPattern
across multiple shell commands:
- volume.tier.move
- volume.list
- volume.fix.replication
- volume.configure.replication

Now users can use -collectionPattern="default" to specifically target volumes
in the default collection (empty collection name), while maintaining backward
compatibility where empty pattern matches all collections.

Updated help text to document this feature.

* Update compileCollectionPattern to support 'default' keyword

This extends the fix to all commands that use regex-based collection
pattern matching:
- ec.encode
- ec.decode
- volume.tier.download
- volume.balance

The compileCollectionPattern function now treats "default" as a special
keyword that compiles to the regex "^$" (matching empty strings), making
it consistent with the other commands that use filepath.Match.

* Use CollectionDefault constant instead of hardcoded "default" string

Refactored the collection pattern matching logic to use a central constant
CollectionDefault defined in weed/shell/common.go. This improves maintainability
and ensures consistency across all shell commands.

* Address PR review feedback: simplify logic and use '_default' keyword

Changes:
1. Changed CollectionDefault from "default" to "_default" to avoid collision
   with literal collection names
2. Simplified pattern matching logic to reduce code duplication across all
   affected commands
3. Fixed error handling in command_volume_tier_move.go to properly propagate
   filepath.Match errors instead of swallowing them
4. Updated documentation to clarify how to match a literal "default"
   collection using regex patterns like "^default$"

This addresses all feedback from PR review comments.

* Remove unnecessary documentation about matching literal 'default'

Since we changed the keyword to '_default', users can now simply use
'default' to match a literal collection named "default". The previous
documentation about using regex patterns was confusing and no longer needed.

* Fix error propagation and empty pattern handling

1. command_volume_tier_move.go: Added early termination check after
   eachDataNode callback to stop processing remaining nodes if a pattern
   matching error occurred, improving efficiency

2. command_volume_configure_replication.go: Fixed empty pattern handling
   to match all collections (collectionMatched = true when pattern is empty),
   mirroring the behavior in other commands

These changes address the remaining PR review feedback.
2026-01-16 12:31:48 -08:00
Lisandro PinandGitHub ee775825bc Parallelize read-only volume check pass for volume.check.disk. (#7602) 2025-12-02 09:29:27 -08:00
Lisandro PinandGitHub 0be020b0fa Nit: unify the default --maxParallelization value for weed shell commands supporting this option (#6788) 2025-05-13 07:59:26 -07:00
Lisandro PinandGitHub ba1d82db90 Move shell.ErrorWaitGroup into a common file, to cleanly reuse across weed shell commands. (#6780)
Move `shell.ErrorWaitGroup` into a dedicated common file, to cleanly reuse across `weed shell` commands.
2025-05-12 14:38:55 -07:00