Files
seaweedfs/weed/shell
qzhello 4f9393889c feat(shell): Support batched EC encode and multi-volume selection in ec.encode (#10030)
* fix(shell): correct volume.list -writable filter unit and comparison

* fix(shell): correct volume.list -writable filter unit and comparison

* chore(shell): fix typo in EC shard helper param names

* fix(shell): use exact match for volume.balance -racks/-nodes filter

The old strings.Contains-based filter quietly included any id that was a
  substring of the user-supplied flag value (e.g. -racks=rack10 also matched
  rack1). Replace it with an exact-match set parsed from the comma-separated
  flag value, and add regression tests for both -racks and -nodes paths.

  Also fix a small typo in the "remote storage" error returned by
  maybeMoveOneVolume.

* fix(shell): use exact match for volume.balance -racks/-nodes filter

The old strings.Contains-based filter quietly included any id that was a
  substring of the user-supplied flag value (e.g. -racks=rack10 also matched
  rack1). Replace it with an exact-match set parsed from the comma-separated
  flag value, and add regression tests for both -racks and -nodes paths.

  Also fix a small typo in the "remote storage" error returned by
  maybeMoveOneVolume.

* refactor(shell): drop nil sentinel in splitCSVSet, use len() in callers

* feat(shell): support batched EC encode and multi-volume selection

Add -volumeIds (comma-separated) and -batchSize flags to ec.encode.

When -batchSize > 0, volumes are processed in independent batches, each
committed separately: encode -> rebalance -> verify -> delete originals.
This bounds the working set and lets source volumes be reclaimed without
waiting for the entire set to finish, at the cost of per-batch rebalancing.
Because each batch deletes its originals, a failure in a later batch is
unrecoverable for already-completed batches.

To let the single-volume, multi-volume, and collection paths share one
per-batch routine, the re-balance scope is now always derived from the
volumes actually selected for encoding (collectCollectionsForVolumeIds),
rather than every collection matching the -collection regex. Practical
effect: with -collection, a collection that matches the pattern but
contributes no encodable volumes is no longer re-balanced as a side effect.
The -volumeId path is unchanged; -batchSize=0 (default) preserves the
original single-pass flow.

The per-batch routine reuses the existing assertEncodableRegularVolumes
guard, doEcEncode skipped-node handling, and verifyEcShardsBeforeDelete
retry loop. The capacity pre-flight check takes the already-fetched
topology instead of issuing another VolumeList to the master per batch.

Also clarify the -collection flag description to note it accepts a regex
pattern, matching the existing command help.

-volumeId and -volumeIds are mutually exclusive; ids in -volumeIds are
validated and de-duplicated.
2026-06-22 01:22:20 -07:00
..
2026-06-19 09:09:01 -07:00
2026-06-19 09:09:01 -07:00
2024-09-29 10:38:22 -07:00
2024-09-29 10:38:22 -07:00
2024-09-29 10:38:22 -07:00