Chris LuandGitHub 298ab35fd7 shell: default batch size for ec.encode and ec.decode (#10308) ...

* shell: ec.encode batches 10 volumes by default

Encoding a whole collection as one batch means a late failure leaves
everything half-converted. Default -batchSize to 10 so each batch is
encoded, rebalanced, verified, and its originals deleted before the
next starts. -batchSize=0 keeps the all-at-once behavior. Batch
progress messages now print only when there is more than one batch,
so small runs read as before.

* shell: ec.decode decodes in batches, refreshing topology in between

ec.decode walked every volume off the single topology snapshot taken
at startup, which goes stale as earlier decodes move shards around and
create volumes. Decode 10 volumes per batch by default, re-collecting
the topology and rebuilding the free-space accounting between batches.
-batchSize=0 keeps the single-snapshot behavior.

2026-07-10 16:44:46 -07:00

..

command_cluster_check.go

grpc: don't tear down the shared master connection on a caller's own timeout (#9775)

2026-06-01 15:11:02 -07:00

command_cluster_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_add.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_leader_transfer.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_ps.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_raft_remove.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_cluster_status_test.go

fix(ec): correct multi-disk disk counting and EC balance shard attribution (#9594)

2026-05-20 23:31:21 -07:00

command_cluster_status.go

fix(ec): correct multi-disk disk counting and EC balance shard attribution (#9594)

2026-05-20 23:31:21 -07:00

command_collection_delete.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_collection_list_test.go

fix(s3/shell): factor EC volumes into bucket size metrics and collection.list (#9182)

2026-04-21 20:17:42 -07:00

command_collection_list.go

s3: enforce bucket quota on logical size and surface read-only state in Admin UI (#10224)

2026-07-03 12:39:45 -07:00

command_ec_balance.go

fix(ec): don't mix EC shards from different encode runs (#9880)

2026-06-10 22:31:18 -07:00

command_ec_common_test.go

ec: skip physically near-full disks when placing EC shards (#10167)

2026-06-30 20:01:55 -07:00

command_ec_common.go

ec: skip physically near-full disks when placing EC shards (#10167)

2026-06-30 20:01:55 -07:00

command_ec_decode.go

shell: default batch size for ec.encode and ec.decode (#10308)

2026-07-10 16:44:46 -07:00

command_ec_destructive_guards_test.go

fix(shell): exclude failed EC shard copies from rebuild recoverability gate (#10043)

2026-06-22 11:23:23 -07:00

command_ec_encode_test.go

shell: default batch size for ec.encode and ec.decode (#10308)

2026-07-10 16:44:46 -07:00

command_ec_encode.go

shell: default batch size for ec.encode and ec.decode (#10308)

2026-07-10 16:44:46 -07:00

command_ec_rebuild_test.go

Weed shell ec.rebuild: Allow targeting rebuild to specific volume IDs. (#10087)

2026-06-24 08:40:29 -07:00

command_ec_rebuild.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

command_ec_scrub.go

ec: expose force_deleted_needles_check in ScrubEcVolume RPC and shell (#10176)

2026-06-30 23:20:50 -07:00

command_ec_shard_unmount_test.go

weed shell: add ec.shard.unmount command (#10204)

2026-07-02 11:58:08 -07:00

command_ec_shard_unmount.go

fix(shell): pass collection in ec.shard.unmount --delete request (#10219)

2026-07-03 10:47:14 -07:00

command_ec_test.go

fix(topology): keep physical disk 0 distinct in SplitByPhysicalDisk (#10161)

2026-06-30 15:35:27 -07:00

command_fs_cat.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_cd.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_configure.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_fs_distribute_chunks_test.go

feat(shell): add fs.distributeChunks command for even chunk distribution (#9117)

2026-04-17 21:09:36 -07:00

command_fs_distribute_chunks.go

feat(shell): add fs.distributeChunks command for even chunk distribution (#9117)

2026-04-17 21:09:36 -07:00

command_fs_du.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_log.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_fs_ls.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_merge_volumes_test.go

fix(shell): honor explicit fs.mergeVolumes from/to direction (#10159)

2026-06-30 13:28:53 -07:00

command_fs_merge_volumes.go

fix(shell): honor explicit fs.mergeVolumes from/to direction (#10159)

2026-06-30 13:28:53 -07:00

command_fs_meta_cat.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_meta_change_volume_id.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_fs_meta_load.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_fs_meta_notify.go

Fix volume.fsck -forcePurging -reallyDeleteFromVolume to fail fast on filer traversal errors (#8015)

2026-01-14 21:37:50 -08:00

command_fs_meta_save.go

shell: volume.fsck must not skip the system-log subtree (#9764)

2026-06-01 09:54:22 -07:00

command_fs_mkdir.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_mv_test.go

fix(shell): move files into existing destination directories (#9887)

2026-06-08 23:42:13 -07:00

command_fs_mv.go

fix(shell): move files into existing destination directories (#9887)

2026-06-08 23:42:13 -07:00

command_fs_pwd.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_rm.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_tree.go

Shell: Added a helper function isHelpRequest() (#7380)

2025-10-24 20:21:35 -07:00

command_fs_verify.go

Logs typos (#10018)

2026-06-19 09:09:01 -07:00

command_lock_unlock.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mount_configure.go

Clean up logs and deprecated functions (#7339)

2025-10-17 22:11:50 -07:00

command_mq_balance.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_compact.go

Add Kafka Gateway (#7231)

2025-10-13 18:05:17 -07:00

command_mq_topic_configure.go

shell: expose retention flags on mq.topic.configure (#9416)

2026-05-12 12:37:09 -07:00

command_mq_topic_desc.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_list.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_mq_topic_truncate.go

Message Queue: Add sql querying (#7185)

2025-09-09 01:01:03 -07:00

command_remote_cache.go

fix(remote): correct content and permissions when syncing/caching remote objects (#9879)

2026-06-08 13:55:53 -07:00

command_remote_configure.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_remote_copy_local_test.go

feat(shell): add -delete option to remote.copy.local (#10228)

2026-07-04 10:50:32 -07:00

command_remote_copy_local.go

feat(shell): add -delete option to remote.copy.local (#10228)

2026-07-04 10:50:32 -07:00

command_remote_meta_sync.go

remote.meta.sync: sync directories and remove files deleted from remote (#10184)

2026-07-01 12:14:19 -07:00

command_remote_mount_buckets.go

remote.meta.sync: sync directories and remove files deleted from remote (#10184)

2026-07-01 12:14:19 -07:00

command_remote_mount.go

remote.meta.sync: sync directories and remove files deleted from remote (#10184)

2026-07-01 12:14:19 -07:00

command_remote_uncache_test.go

shell: add minCacheAge flag to remote.uncache command (#8225)

2026-02-05 12:57:27 -08:00

command_remote_uncache.go

feat(shell): add -delete option to remote.copy.local (#10228)

2026-07-04 10:50:32 -07:00

command_remote_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_s3_accesskey_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_accesskey_rotate.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_get.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_anonymous_set.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_bucket_access_test.go

shell: add s3.bucket.access command for anonymous access policy (#8774)

2026-03-25 23:09:53 -07:00

command_s3_bucket_access.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_bucket_create.go

feat(shell): add Object Lock management commands (#8141)

2026-01-27 10:50:16 -08:00

command_s3_bucket_delete.go

fix: admin UI bucket delete now properly deletes collection and checks Object Lock (#7734)

2025-12-13 13:41:25 -08:00

command_s3_bucket_lifecycle_fastpath.go

s3: make lifecycle TTL fast path per-bucket opt-in (#9825)

2026-06-06 11:20:15 -07:00

command_s3_bucket_list.go

s3.bucket.list: measure quota usage by logical size (#10274)

2026-07-08 14:32:04 -07:00

command_s3_bucket_lock.go

feat(shell): add Object Lock management commands (#8141)

2026-01-27 10:50:16 -08:00

command_s3_bucket_owner.go

shell: add -owner flag to s3.bucket.create command (#7728)

2025-12-12 18:06:13 -08:00

command_s3_bucket_quota_check.go

s3: enforce bucket quota on logical size and surface read-only state in Admin UI (#10224)

2026-07-03 12:39:45 -07:00

command_s3_bucket_quota.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

command_s3_bucket_versioning.go

Chart createBuckets config #8368: Add TTL, Object Lock, and Versioning support (#8375)

2026-02-26 11:56:10 -08:00

command_s3_circuitbreaker_test.go

…

command_s3_circuitbreaker.go

feat(s3/lifecycle): filer-backed cursor Persister + drop BlockerStore (#9358)

2026-05-07 17:45:04 -07:00

command_s3_clean_uploads.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

command_s3_config_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_configure.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_add_user.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_remove_user.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_group_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_iam_client.go

security: add BearerPrefix constant for Authorization headers (#10101)

2026-06-24 19:36:42 -07:00

command_s3_iam_export.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_iam_import.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_lifecycle_run_shard_test.go

fix(shell): restore s3.lifecycle.run-shard for CI/integration-test compatibility (#9475)

2026-05-12 18:29:18 -07:00

command_s3_lifecycle_run_shard.go

fix(s3/lifecycle): walker dispatch uses entry.Path for ABORT_MPU (#9477)

2026-05-13 00:19:05 -07:00

command_s3_policy_attach.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_policy_detach.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_policy.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_serviceaccount_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_create.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_delete.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_disable.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_enable.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_list.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_provision.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_user_show.go

fix(shell): attach admin JWT for filer IAM gRPC calls (#9536)

2026-05-18 13:42:32 -07:00

command_s3_versions_audit.go

s3: tear down the emptied .versions directory on last-version delete; drain existing residue (#10278)

2026-07-08 18:50:54 -07:00

command_s3tables_bucket.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_s3tables_namespace.go

Add s3tables shell and admin UI (#8172)

2026-01-30 22:57:05 -08:00

command_s3tables_table.go

shell: expand ~ in local file path arguments (#9265)

2026-04-28 12:30:13 -07:00

command_s3tables_tag.go

Add s3tables shell and admin UI (#8172)

2026-01-30 22:57:05 -08:00

command_sleep.go

add admin script worker (#8491)

2026-03-03 15:10:40 -08:00

command_volume_balance_test.go

volume.balance: rank by physical disk usage (#10271)

2026-07-08 09:29:59 -07:00

command_volume_balance.go

volume.balance: rank by physical disk usage (#10271)

2026-07-08 09:29:59 -07:00

command_volume_check_disk_test.go

fix(replication): verify-before-destroy in VolumeCopy, check.disk, and over-replication trim (#9943)

2026-06-13 20:05:33 -07:00

command_volume_check_disk.go

fix(replication): verify-before-destroy in VolumeCopy, check.disk, and over-replication trim (#9943)

2026-06-13 20:05:33 -07:00

command_volume_configure_replication.go

Fix #8040: Support '_default' keyword in collectionPattern to match default collection (#8046)

2026-01-16 12:31:48 -08:00

command_volume_copy.go

Fix live volume move tail timestamp (#8440)

2026-02-24 20:07:26 -08:00

command_volume_delete_empty_test.go

feat: add collection pattern to delete empty volumes (#10129)

2026-06-29 12:43:19 -07:00

command_volume_delete_empty.go

feat: add collection pattern to delete empty volumes (#10129)

2026-06-29 12:43:19 -07:00

command_volume_delete.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_fix_replication_collection_test.go

fix: apply collectionPattern during detection in volume.fix.replication (#10115)

2026-06-26 00:48:29 -07:00

command_volume_fix_replication_parallel_test.go

volume.fix.replication: parallelize under-replicated volume copies (#10275)

2026-07-08 20:03:01 -07:00

command_volume_fix_replication_test.go

fix(replication): verify-before-destroy in VolumeCopy, check.disk, and over-replication trim (#9943)

2026-06-13 20:05:33 -07:00

command_volume_fix_replication.go

topology/balancer: share replica selection between shell and workers (#10276)

2026-07-08 20:06:15 -07:00

command_volume_fsck.go

security: add BearerPrefix constant for Authorization headers (#10101)

2026-06-24 19:36:42 -07:00

command_volume_grow.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_list_test.go

fix: avoid duplicate volume.list parent headers (#10126)

2026-06-29 11:31:45 -07:00

command_volume_list.go

feat: add collection pattern to delete empty volumes (#10129)

2026-06-29 12:43:19 -07:00

command_volume_mark_test.go

feat: support marking volumes by collection (#9585)

2026-06-23 11:27:43 -07:00

command_volume_mark.go

feat: support marking volumes by collection (#9585)

2026-06-23 11:27:43 -07:00

command_volume_merge_test.go

fix(shell): verify volume.merge output before overwriting replicas (#9731)

2026-05-28 19:29:25 -07:00

command_volume_merge.go

fix(shell): verify volume.merge output before overwriting replicas (#9731)

2026-05-28 19:29:25 -07:00

command_volume_mount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_move.go

fix(shell): print markVolumeWritable/markVolumeReadonly based on the writable flag (#10283)

2026-07-09 00:30:21 -07:00

command_volume_scrub.go

volume.scrub and ec.scrub shell commands: make the display of scrub details optional. (#9911)

2026-06-10 13:29:07 -07:00

command_volume_server_evacuate_test.go

fix tests

2023-03-13 10:00:27 -07:00

command_volume_server_evacuate.go

fix(shell): volumeServer.evacuate no longer panics on a nil volume (#9587)

2026-05-20 10:27:00 -07:00

command_volume_server_leave.go

Unify the parameter to disable dry-run on weed shell commands to -apply (instead of -force). (#7450)

2025-11-09 19:58:38 -08:00

command_volume_server_state.go

Add weed shell command volumeServer.state to query/update volume server state settings. (#8271)

2026-02-11 11:02:37 -08:00

command_volume_tier_compact_test.go

feat(shell): add volume.tier.compact command to reclaim cloud storage space (#8715)

2026-03-20 23:52:12 -07:00

command_volume_tier_compact.go

fix(tiering): never delete a shared remote object while replicas still reference it (#9942)

2026-06-13 20:09:00 -07:00

command_volume_tier_download.go

fix(tiering): never delete a shared remote object while replicas still reference it (#9942)

2026-06-13 20:09:00 -07:00

command_volume_tier_move_test.go

shell: volume.tier.move can move volumes between data centers (#9925)

2026-06-11 10:46:34 -07:00

command_volume_tier_move.go

fix(tiering): never delete a shared remote object while replicas still reference it (#9942)

2026-06-13 20:09:00 -07:00

command_volume_tier_upload.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

command_volume_unmount.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

command_volume_vacuum_disable.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_vacuum_enable.go

refactor

2024-09-29 10:38:22 -07:00

command_volume_vacuum.go

fix(shell): error on missing volume id in fsck, mergeVolumes, vacuum (#9158)

2026-04-20 15:32:31 -07:00

command.go

shell: add s3.iam.*, s3.config.show, s3.user.provision; hide legacy commands (#8956)

2026-04-07 14:10:15 -07:00

commands.go

fix(ec): bring ec.encode worker and EC/volume helpers to parity with shell (#9599)

2026-05-21 02:16:28 -07:00

common_test.go

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

common.go

shell: default batch size for ec.encode and ec.decode (#10308)

2026-07-10 16:44:46 -07:00

ec_proportional_rebalance.go

chore: remove ~50k lines of unreachable dead code (#8913)

2026-04-03 16:04:27 -07:00

ec_rebalance_slots_test.go

opt: reduce ShardsInfo memory usage with bitmap and sorted slice (#7974)

2026-01-06 00:09:52 -08:00

s3tables_helpers.go

s3tables: scope management authorization to the caller's identity (#9961)

2026-06-14 13:55:36 -07:00

shell_liner.go

fix(weed/shell): suppress prompt when piped (#8990)

2026-04-08 13:07:15 -07:00

volume.ecshards.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list2.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00

volume.list.txt

Humanize output for weed.server by default (#7758)

2025-12-15 11:18:45 -08:00