Chris LuandGitHub 78e428758b volume.fix.replication: parallelize under-replicated volume copies (#10275) ...

* volume.fix.replication: parallelize under-replicated volume copies

Fan out fixOneUnderReplicatedVolume up to -maxParallelization at a time.
Destination selection and free-slot accounting stay atomic behind a
scheduler mutex, so concurrent fixes see each other's reservations; a
failed copy returns its reserved slot. A per-server in-flight cap
(-maxParallelizationPerServer, default 1) keeps many simultaneous copies
from swamping a single destination server; when every eligible
destination is at the cap the fix waits for a slot instead of failing.

* volume.fix.replication: isolate the scheduler's location ordering

Clone allLocations before the per-volume fan-out so the scheduler's
re-sorting cannot alias the slice shared with the concurrent delete
phases, and make the test's per-iteration location copy explicit.

2026-07-08 20:03:01 -07:00

..

command_cluster_check.go

grpc: don't tear down the shared master connection on a caller's own timeout (#9775)

2026-06-01 15:11:02 -07:00

command_cluster_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_add.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_remove.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_status_test.go

fix(ec): correct multi-disk disk counting and EC balance shard attribution (#9594)

2026-05-20 23:31:21 -07:00

command_cluster_status.go

fix(ec): correct multi-disk disk counting and EC balance shard attribution (#9594)

2026-05-20 23:31:21 -07:00

command_collection_delete.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_collection_list_test.go

fix(s3/shell): factor EC volumes into bucket size metrics and collection.list (#9182)

2026-04-21 20:17:42 -07:00

command_collection_list.go

s3: enforce bucket quota on logical size and surface read-only state in Admin UI (#10224)

2026-07-03 12:39:45 -07:00

command_ec_balance.go

fix(ec): don't mix EC shards from different encode runs (#9880)

2026-06-10 22:31:18 -07:00

command_ec_common_test.go

ec: skip physically near-full disks when placing EC shards (#10167)

2026-06-30 20:01:55 -07:00

command_ec_common.go

ec: skip physically near-full disks when placing EC shards (#10167)

2026-06-30 20:01:55 -07:00

command_ec_decode.go

fix(ec): pass per-volume data-shard count to the parity-shard split (#9781)

2026-06-01 19:25:15 -07:00

command_ec_destructive_guards_test.go

fix(shell): exclude failed EC shard copies from rebuild recoverability gate (#10043)

2026-06-22 11:23:23 -07:00

command_ec_encode_test.go

feat(shell): Support batched EC encode and multi-volume selection in ec.encode (#10030)

2026-06-22 01:22:20 -07:00

command_ec_encode.go

feat(shell): Support batched EC encode and multi-volume selection in ec.encode (#10030)

2026-06-22 01:22:20 -07:00

command_ec_rebuild_test.go

Weed shell ec.rebuild: Allow targeting rebuild to specific volume IDs. (#10087)

2026-06-24 08:40:29 -07:00

command_ec_rebuild.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

command_ec_scrub.go

ec: expose force_deleted_needles_check in ScrubEcVolume RPC and shell (#10176)

2026-06-30 23:20:50 -07:00

command_ec_shard_unmount_test.go

weed shell: add ec.shard.unmount command (#10204)

2026-07-02 11:58:08 -07:00

command_ec_shard_unmount.go

fix(shell): pass collection in ec.shard.unmount --delete request (#10219)

2026-07-03 10:47:14 -07:00

command_ec_test.go

fix(topology): keep physical disk 0 distinct in SplitByPhysicalDisk (#10161)

2026-06-30 15:35:27 -07:00

command_fs_cat.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_cd.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_configure.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_fs_distribute_chunks_test.go

feat(shell): add fs.distributeChunks command for even chunk distribution (#9117)

2026-04-17 21:09:36 -07:00

command_fs_distribute_chunks.go

feat(shell): add fs.distributeChunks command for even chunk distribution (#9117)

2026-04-17 21:09:36 -07:00

command_fs_du.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_log.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_fs_ls.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_merge_volumes_test.go

fix(shell): honor explicit fs.mergeVolumes from/to direction (#10159)

2026-06-30 13:28:53 -07:00

command_fs_merge_volumes.go

fix(shell): honor explicit fs.mergeVolumes from/to direction (#10159)

2026-06-30 13:28:53 -07:00

command_fs_meta_cat.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_meta_change_volume_id.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_fs_meta_load.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_fs_meta_notify.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_fs_meta_save.go

shell: volume.fsck must not skip the system-log subtree (#9764)

2026-06-01 09:54:22 -07:00

command_fs_mkdir.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_mv_test.go

fix(shell): move files into existing destination directories (#9887)

2026-06-08 23:42:13 -07:00

command_fs_mv.go

fix(shell): move files into existing destination directories (#9887)

2026-06-08 23:42:13 -07:00

command_fs_pwd.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_rm.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_tree.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_verify.go

Logs typos (#10018)

2026-06-19 09:09:01 -07:00

command_lock_unlock.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mount_configure.go

Clean up logs and deprecated functions (#7339)

2025-10-17 22:11:50 -07:00

command_mq_balance.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_compact.go

Add Kafka Gateway (#7231)

2025-10-13 18:05:17 -07:00

command_mq_topic_configure.go

shell: expose retention flags on mq.topic.configure (#9416)

2026-05-12 12:37:09 -07:00

command_mq_topic_desc.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_list.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_truncate.go

Message Queue: Add sql querying (#7185)

2025-09-09 01:01:03 -07:00

command_remote_cache.go

fix(remote): correct content and permissions when syncing/caching remote objects (#9879)

2026-06-08 13:55:53 -07:00

command_remote_configure.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_remote_copy_local_test.go

feat(shell): add -delete option to remote.copy.local (#10228)

2026-07-04 10:50:32 -07:00

command_remote_copy_local.go

feat(shell): add -delete option to remote.copy.local (#10228)

2026-07-04 10:50:32 -07:00

command_remote_meta_sync.go

remote.meta.sync: sync directories and remove files deleted from remote (#10184)

2026-07-01 12:14:19 -07:00

command_remote_mount_buckets.go

remote.meta.sync: sync directories and remove files deleted from remote (#10184)

2026-07-01 12:14:19 -07:00

command_remote_mount.go

remote.meta.sync: sync directories and remove files deleted from remote (#10184)

2026-07-01 12:14:19 -07:00

command_remote_uncache_test.go

shell: add minCacheAge flag to remote.uncache command (#8225)

2026-02-05 12:57:27 -08:00

command_remote_uncache.go

feat(shell): add -delete option to remote.copy.local (#10228)

2026-07-04 10:50:32 -07:00

command_remote_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_s3_accesskey_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_rotate.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_get.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_set.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_bucket_access_test.go

shell: add s3.bucket.access command for anonymous access policy (#8774)

2026-03-25 23:09:53 -07:00

command_s3_bucket_access.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_bucket_create.go

feat(shell): add Object Lock management commands (#8141)

2026-01-27 10:50:16 -08:00

command_s3_bucket_delete.go

fix: admin UI bucket delete now properly deletes collection and checks Object Lock (#7734)

2025-12-13 13:41:25 -08:00

command_s3_bucket_lifecycle_fastpath.go

s3: make lifecycle TTL fast path per-bucket opt-in (#9825)

2026-06-06 11:20:15 -07:00

command_s3_bucket_list.go

s3.bucket.list: measure quota usage by logical size (#10274)

2026-07-08 14:32:04 -07:00

command_s3_bucket_lock.go

feat(shell): add Object Lock management commands (#8141)

2026-01-27 10:50:16 -08:00

command_s3_bucket_owner.go

shell: add -owner flag to s3.bucket.create command (#7728)

2025-12-12 18:06:13 -08:00

command_s3_bucket_quota_check.go

s3: enforce bucket quota on logical size and surface read-only state in Admin UI (#10224)

2026-07-03 12:39:45 -07:00

command_s3_bucket_quota.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

command_s3_bucket_versioning.go

Chart createBuckets config #8368: Add TTL, Object Lock, and Versioning support (#8375)

2026-02-26 11:56:10 -08:00

command_s3_circuitbreaker_test.go

refactor(shell): readability improvements (#3704)

2022-09-16 02:43:49 -07:00

command_s3_circuitbreaker.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_s3_clean_uploads.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

command_s3_config_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_configure.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_add_user.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_remove_user.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_iam_client.go

security: add BearerPrefix constant for Authorization headers (#10101)

2026-06-24 19:36:42 -07:00

command_s3_iam_export.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_iam_import.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_lifecycle_run_shard_test.go

fix(shell): restore s3.lifecycle.run-shard for CI/integration-test compatibility (#9475)

2026-05-12 18:29:18 -07:00

command_s3_lifecycle_run_shard.go

fix(s3/lifecycle): walker dispatch uses entry.Path for ABORT_MPU (#9477)

2026-05-13 00:19:05 -07:00

command_s3_policy_attach.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_policy_detach.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_policy.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_disable.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_enable.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_provision.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_versions_audit.go

s3: tear down the emptied .versions directory on last-version delete; drain existing residue (#10278)

2026-07-08 18:50:54 -07:00

command_s3tables_bucket.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_s3tables_namespace.go

Add s3tables shell and admin UI (#8172)

2026-01-30 22:57:05 -08:00

command_s3tables_table.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_s3tables_tag.go

Add s3tables shell and admin UI (#8172)

2026-01-30 22:57:05 -08:00

command_sleep.go

add admin script worker (#8491)

2026-03-03 15:10:40 -08:00

command_volume_balance_test.go

volume.balance: rank by physical disk usage (#10271)

2026-07-08 09:29:59 -07:00

command_volume_balance.go

volume.balance: rank by physical disk usage (#10271)

2026-07-08 09:29:59 -07:00

command_volume_check_disk_test.go

fix(replication): verify-before-destroy in VolumeCopy, check.disk, and over-replication trim (#9943)

2026-06-13 20:05:33 -07:00

command_volume_check_disk.go

fix(replication): verify-before-destroy in VolumeCopy, check.disk, and over-replication trim (#9943)

2026-06-13 20:05:33 -07:00

command_volume_configure_replication.go

Fix #8040: Support '_default' keyword in collectionPattern to match default collection (#8046)

2026-01-16 12:31:48 -08:00

command_volume_copy.go

Fix live volume move tail timestamp (#8440)

2026-02-24 20:07:26 -08:00

command_volume_delete_empty_test.go

feat: add collection pattern to delete empty volumes (#10129)

2026-06-29 12:43:19 -07:00

command_volume_delete_empty.go

feat: add collection pattern to delete empty volumes (#10129)

2026-06-29 12:43:19 -07:00

command_volume_delete.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_fix_replication_collection_test.go

fix: apply collectionPattern during detection in volume.fix.replication (#10115)

2026-06-26 00:48:29 -07:00

command_volume_fix_replication_parallel_test.go

volume.fix.replication: parallelize under-replicated volume copies (#10275)

2026-07-08 20:03:01 -07:00

command_volume_fix_replication_test.go

fix(replication): verify-before-destroy in VolumeCopy, check.disk, and over-replication trim (#9943)

2026-06-13 20:05:33 -07:00

command_volume_fix_replication.go

volume.fix.replication: parallelize under-replicated volume copies (#10275)

2026-07-08 20:03:01 -07:00

command_volume_fsck.go

security: add BearerPrefix constant for Authorization headers (#10101)

2026-06-24 19:36:42 -07:00

command_volume_grow.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_list_test.go

fix: avoid duplicate volume.list parent headers (#10126)

2026-06-29 11:31:45 -07:00

command_volume_list.go

feat: add collection pattern to delete empty volumes (#10129)

2026-06-29 12:43:19 -07:00

command_volume_mark_test.go

feat: support marking volumes by collection (#9585)

2026-06-23 11:27:43 -07:00

command_volume_mark.go

feat: support marking volumes by collection (#9585)

2026-06-23 11:27:43 -07:00

command_volume_merge_test.go

fix(shell): verify volume.merge output before overwriting replicas (#9731)

2026-05-28 19:29:25 -07:00

command_volume_merge.go

fix(shell): verify volume.merge output before overwriting replicas (#9731)

2026-05-28 19:29:25 -07:00

command_volume_mount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_move.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_scrub.go

volume.scrub and ec.scrub shell commands: make the display of scrub details optional. (#9911)

2026-06-10 13:29:07 -07:00

command_volume_server_evacuate_test.go

fix tests

2023-03-13 10:00:27 -07:00

command_volume_server_evacuate.go

fix(shell): volumeServer.evacuate no longer panics on a nil volume (#9587)

2026-05-20 10:27:00 -07:00

command_volume_server_leave.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_volume_server_state.go

Add weed shell command volumeServer.state to query/update volume server state settings. (#8271)

2026-02-11 11:02:37 -08:00

command_volume_tier_compact_test.go

feat(shell): add volume.tier.compact command to reclaim cloud storage space (#8715)

2026-03-20 23:52:12 -07:00

command_volume_tier_compact.go

fix(tiering): never delete a shared remote object while replicas still reference it (#9942)

2026-06-13 20:09:00 -07:00

command_volume_tier_download.go

fix(tiering): never delete a shared remote object while replicas still reference it (#9942)

2026-06-13 20:09:00 -07:00

command_volume_tier_move_test.go

shell: volume.tier.move can move volumes between data centers (#9925)

2026-06-11 10:46:34 -07:00

command_volume_tier_move.go

fix(tiering): never delete a shared remote object while replicas still reference it (#9942)

2026-06-13 20:09:00 -07:00

command_volume_tier_upload.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_vacuum_disable.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_vacuum_enable.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_vacuum.go

fix(shell): error on missing volume id in fsck, mergeVolumes, vacuum (#9158)

2026-04-20 15:32:31 -07:00

command.go

shell: add s3.iam.*, s3.config.show, s3.user.provision; hide legacy commands (#8956)

2026-04-07 14:10:15 -07:00

commands.go

fix(ec): bring ec.encode worker and EC/volume helpers to parity with shell (#9599)

2026-05-21 02:16:28 -07:00

common_test.go

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

common.go

Fix #8040: Support '_default' keyword in collectionPattern to match default collection (#8046)

2026-01-16 12:31:48 -08:00

ec_proportional_rebalance.go

chore: remove ~50k lines of unreachable dead code (#8913)

2026-04-03 16:04:27 -07:00

ec_rebalance_slots_test.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974)

2026-01-06 00:09:52 -08:00

s3tables_helpers.go

s3tables: scope management authorization to the caller's identity (#9961)

2026-06-14 13:55:36 -07:00

shell_liner.go

fix(weed/shell): suppress prompt when piped (#8990)

2026-04-08 13:07:15 -07:00

volume.ecshards.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list2.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00