Chris LuandGitHub 944d967502 refactor: extract EC orchestration into a shared weed/ec package (#10760) ...

* shell: move ErrorWaitGroup to weed/util

* shell: remove unused CandidateEcNode and EcRack types

* ec: extract EC orchestration logic from weed/shell into weed/ec

Move the EC node/topology model, balance engine, encode pipeline, decode
pipeline, and rebuild engine into a new weed/ec package so shell commands
and maintenance workers can share the logic. Shell commands keep flag
parsing and delegate through a small ec.Env (dial option, topology fetch,
volume locations, lock check). Tests move along with the code.

* shell: remove unused proportional-rebalance type stubs

* ec: move scrub, replication check, and shard unmount engines into weed/ec

* worker: share the EC generation-aware shard counter from weed/ec

* ec: gofmt

* shell: drop EC aliases with no remaining callers

* ec: guard a missing topology hook and nil disk entries in topology helpers

* ec: drop trailing newlines from decode error strings

* ec: re-check the shell lock before applying shard unmounts

* shell: trim -node entries in ec.scrub

2026-08-14 13:54:12 -07:00

..

command_cluster_check.go

grpc: don't tear down the shared master connection on a caller's own timeout (#9775)

2026-06-01 15:11:02 -07:00

command_cluster_ps.go

shell: cluster.ps lists s3 servers (#10359)

2026-07-18 13:56:47 -07:00

command_cluster_raft_add.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_remove.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_status_test.go

shell: show who holds the cluster lock (#10353)

2026-07-17 12:30:42 -07:00

command_cluster_status.go

shell: show who holds the cluster lock (#10353)

2026-07-17 12:30:42 -07:00

command_collection_delete.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_collection_list_test.go

fix(s3/shell): factor EC volumes into bucket size metrics and collection.list (#9182)

2026-04-21 20:17:42 -07:00

command_collection_list.go

s3: enforce bucket quota on logical size and surface read-only state in Admin UI (#10224)

2026-07-03 12:39:45 -07:00

command_ec_balance.go

feat: throughput limits for replicate, EC shard, and worker-driven moves (#10749)

2026-08-13 13:22:58 -07:00

command_ec_check_replication.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

command_ec_common.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

command_ec_decode.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

command_ec_encode.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

command_ec_rebuild.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

command_ec_scrub.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

command_ec_shard_unmount.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

command_fs_cat.go

…

command_fs_cd.go

…

command_fs_configure.go

filer: let a nested path rule turn worm off (#10503)

2026-07-31 00:34:22 -07:00

command_fs_distribute_chunks_test.go

feat(shell): add fs.distributeChunks command for even chunk distribution (#9117)

2026-04-17 21:09:36 -07:00

command_fs_distribute_chunks.go

feat: throughput limits for replicate, EC shard, and worker-driven moves (#10749)

2026-08-13 13:22:58 -07:00

command_fs_du.go

…

command_fs_log.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_fs_ls.go

…

command_fs_merge_volumes_test.go

shell: multi-target fs.mergeVolumes and volume.mark -readonlyCanDelete (#10706)

2026-08-10 16:31:26 -07:00

command_fs_merge_volumes.go

shell: send read jwt when downloading chunks in fs.mergeVolumes and fs.distributeChunks (#10717)

2026-08-11 11:57:53 -07:00

command_fs_meta_cat.go

…

command_fs_meta_change_volume_id.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_fs_meta_load.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_fs_meta_notify.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_fs_meta_save.go

mount: version open file handles by filer log position (#10403)

2026-07-23 17:44:02 -07:00

command_fs_mkdir.go

…

command_fs_mv_test.go

fix(shell): move files into existing destination directories (#9887)

2026-06-08 23:42:13 -07:00

command_fs_mv.go

fix(shell): move files into existing destination directories (#9887)

2026-06-08 23:42:13 -07:00

command_fs_pwd.go

…

command_fs_rm.go

…

command_fs_tree.go

…

command_fs_verify.go

Logs typos (#10018)

2026-06-19 09:09:01 -07:00

command_lock_unlock.go

shell: show who holds the cluster lock (#10353)

2026-07-17 12:30:42 -07:00

command_mount_configure.go

…

command_mq_balance.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_compact.go

…

command_mq_topic_configure.go

shell: expose retention flags on mq.topic.configure (#9416)

2026-05-12 12:37:09 -07:00

command_mq_topic_desc.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_list.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_truncate.go

…

command_remote_cache.go

preserve Content-Encoding for remote-mounted objects (#10340)

2026-07-15 19:20:00 -07:00

command_remote_configure.go

azure: let the blob endpoint be configured (#10460)

2026-07-27 16:41:13 -07:00

command_remote_copy_local_test.go

feat(shell): add -delete option to remote.copy.local (#10228)

2026-07-04 10:50:32 -07:00

command_remote_copy_local.go

feat(shell): add -delete option to remote.copy.local (#10228)

2026-07-04 10:50:32 -07:00

command_remote_meta_sync.go

preserve Content-Encoding for remote-mounted objects (#10340)

2026-07-15 19:20:00 -07:00

command_remote_mount_buckets.go

preserve Content-Encoding for remote-mounted objects (#10340)

2026-07-15 19:20:00 -07:00

command_remote_mount.go

preserve Content-Encoding for remote-mounted objects (#10340)

2026-07-15 19:20:00 -07:00

command_remote_uncache_test.go

shell: add minCacheAge flag to remote.uncache command (#8225)

2026-02-05 12:57:27 -08:00

command_remote_uncache.go

Add codespell support with configuration and typo fixes (#10393)

2026-07-22 14:38:06 -07:00

command_remote_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_s3_accesskey_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_rotate.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_get.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_set.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_bucket_access_test.go

shell: add s3.bucket.access command for anonymous access policy (#8774)

2026-03-25 23:09:53 -07:00

command_s3_bucket_access.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_bucket_create.go

feat(shell): add Object Lock management commands (#8141)

2026-01-27 10:50:16 -08:00

command_s3_bucket_delete.go

fix: admin UI bucket delete now properly deletes collection and checks Object Lock (#7734)

2025-12-13 13:41:25 -08:00

command_s3_bucket_lifecycle_fastpath.go

s3: make lifecycle TTL fast path per-bucket opt-in (#9825)

2026-06-06 11:20:15 -07:00

command_s3_bucket_list.go

s3.bucket.list: measure quota usage by logical size (#10274)

2026-07-08 14:32:04 -07:00

command_s3_bucket_lock.go

feat(shell): add Object Lock management commands (#8141)

2026-01-27 10:50:16 -08:00

command_s3_bucket_owner.go

s3: honor a bucket owner recorded as an identity (#10567)

2026-08-04 16:00:04 -07:00

command_s3_bucket_quota_check.go

s3: enforce bucket quota on logical size and surface read-only state in Admin UI (#10224)

2026-07-03 12:39:45 -07:00

command_s3_bucket_quota.go

filer: allow clearing a bucket read-only flag stuck after quota removal (#10310)

2026-07-10 22:09:39 -07:00

command_s3_bucket_versioning.go

Chart createBuckets config #8368: Add TTL, Object Lock, and Versioning support (#8375)

2026-02-26 11:56:10 -08:00

command_s3_circuitbreaker_test.go

…

command_s3_circuitbreaker.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_s3_clean_uploads.go

…

command_s3_config_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_configure.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_add_user.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_remove_user.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_iam_client.go

security: add BearerPrefix constant for Authorization headers (#10101)

2026-06-24 19:36:42 -07:00

command_s3_iam_export.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_iam_import.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_lifecycle_run_shard_test.go

fix(shell): restore s3.lifecycle.run-shard for CI/integration-test compatibility (#9475)

2026-05-12 18:29:18 -07:00

command_s3_lifecycle_run_shard.go

feat: throughput limits for replicate, EC shard, and worker-driven moves (#10749)

2026-08-13 13:22:58 -07:00

command_s3_policy_attach.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_policy_detach.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_policy.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_disable.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_enable.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_provision.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_versions_audit.go

s3: tear down the emptied .versions directory on last-version delete; drain existing residue (#10278)

2026-07-08 18:50:54 -07:00

command_s3tables_bucket.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_s3tables_namespace.go

Add s3tables shell and admin UI (#8172)

2026-01-30 22:57:05 -08:00

command_s3tables_table.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_s3tables_tag.go

Add s3tables shell and admin UI (#8172)

2026-01-30 22:57:05 -08:00

command_sleep.go

add admin script worker (#8491)

2026-03-03 15:10:40 -08:00

command_volume_balance_test.go

feat(shell): parallelize volume balance moves (#10737)

2026-08-13 19:11:38 -07:00

command_volume_balance.go

feat(shell): parallelize volume balance moves (#10737)

2026-08-13 19:11:38 -07:00

command_volume_check_disk_test.go

shell: volume.check.disk -resurrectMissingNeedles for never-vacuumed replicas (#10316)

2026-07-12 00:14:16 -07:00

command_volume_check_disk.go

shell: volume.check.disk -resurrectMissingNeedles for never-vacuumed replicas (#10316)

2026-07-12 00:14:16 -07:00

command_volume_configure_replication.go

Fix #8040: Support '_default' keyword in collectionPattern to match default collection (#8046)

2026-01-16 12:31:48 -08:00

command_volume_copy.go

refactor: share volume and EC shard move logic between shell and workers (#10727)

2026-08-12 12:29:40 -07:00

command_volume_delete_empty_test.go

feat: add collection pattern to delete empty volumes (#10129)

2026-06-29 12:43:19 -07:00

command_volume_delete_empty.go

shell: accept a context in the volume move helpers (#10415)

2026-07-24 01:29:41 -07:00

command_volume_delete.go

shell: volume.delete and volume.move accept a -timeout (#10701)

2026-08-10 11:10:29 -07:00

command_volume_fix_replication_collection_test.go

fix: apply collectionPattern during detection in volume.fix.replication (#10115)

2026-06-26 00:48:29 -07:00

command_volume_fix_replication_parallel_test.go

volume.fix.replication: parallelize under-replicated volume copies (#10275)

2026-07-08 20:03:01 -07:00

command_volume_fix_replication_test.go

volume.fix.replication: add a well-placed replica before deleting a misplaced one (#10364)

2026-07-21 12:43:38 -07:00

command_volume_fix_replication.go

feat: throughput limits for replicate, EC shard, and worker-driven moves (#10749)

2026-08-13 13:22:58 -07:00

command_volume_fsck.go

shell: accept a context in the volume move helpers (#10415)

2026-07-24 01:29:41 -07:00

command_volume_grow.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_list_test.go

fix: avoid duplicate volume.list parent headers (#10126)

2026-06-29 11:31:45 -07:00

command_volume_list.go

feat: add collection pattern to delete empty volumes (#10129)

2026-06-29 12:43:19 -07:00

command_volume_mark_test.go

feat: support marking volumes by collection (#9585)

2026-06-23 11:27:43 -07:00

command_volume_mark.go

shell: multi-target fs.mergeVolumes and volume.mark -readonlyCanDelete (#10706)

2026-08-10 16:31:26 -07:00

command_volume_merge_test.go

Fix volume.merge corrupting every needle it copies (#10565)

2026-08-04 16:58:25 -07:00

command_volume_merge.go

refactor: share volume and EC shard move logic between shell and workers (#10727)

2026-08-12 12:29:40 -07:00

command_volume_mount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_move.go

feat: throughput limits for replicate, EC shard, and worker-driven moves (#10749)

2026-08-13 13:22:58 -07:00

command_volume_scrub.go

volume.scrub and ec.scrub shell commands: make the display of scrub details optional. (#9911)

2026-06-10 13:29:07 -07:00

command_volume_server_evacuate_test.go

…

command_volume_server_evacuate.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

command_volume_server_leave.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_volume_server_state.go

Add codespell support with configuration and typo fixes (#10393)

2026-07-22 14:38:06 -07:00

command_volume_tier_compact_test.go

feat(shell): add volume.tier.compact command to reclaim cloud storage space (#8715)

2026-03-20 23:52:12 -07:00

command_volume_tier_compact.go

storage: stop keeping the remote storage key on the master (#10672)

2026-08-09 12:43:31 -07:00

command_volume_tier_download.go

storage: stop keeping the remote storage key on the master (#10672)

2026-08-09 12:43:31 -07:00

command_volume_tier_move_test.go

shell: volume.tier.move can move volumes between data centers (#9925)

2026-06-11 10:46:34 -07:00

command_volume_tier_move.go

feat: throughput limits for replicate, EC shard, and worker-driven moves (#10749)

2026-08-13 13:22:58 -07:00

command_volume_tier_upload_test.go

shell: volume.tier.upload keeps volume replicas (#10314)

2026-07-11 13:49:36 -07:00

command_volume_tier_upload.go

feat: throughput limits for replicate, EC shard, and worker-driven moves (#10749)

2026-08-13 13:22:58 -07:00

command_volume_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_vacuum_disable.go

…

command_volume_vacuum_enable.go

…

command_volume_vacuum.go

fix(shell): error on missing volume id in fsck, mergeVolumes, vacuum (#9158)

2026-04-20 15:32:31 -07:00

command.go

shell: add s3.iam.*, s3.config.show, s3.user.provision; hide legacy commands (#8956)

2026-04-07 14:10:15 -07:00

commands.go

shell: show who holds the cluster lock (#10353)

2026-07-17 12:30:42 -07:00

common_test.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

common.go

refactor: extract EC orchestration into a shared weed/ec package (#10760)

2026-08-14 13:54:12 -07:00

s3tables_helpers.go

s3tables: scope management authorization to the caller's identity (#9961)

2026-06-14 13:55:36 -07:00

shell_liner.go

fix(weed/shell): suppress prompt when piped (#8990)

2026-04-08 13:07:15 -07:00

volume.ecshards.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list2.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00