Lisandro Pin cac83bb4a8 Fix scrubbing of deleted needles on EC volumes. (#10130) ...

EC volumes do not propagate deletions to all shard indexes, so it is possible
to run scrubbing on a volume where a deleted needle is still present in the
index, or a needle deleted from the index is still present on the volume.
On either scenario, scrubbing will fail due to size mismatch errors.

This PR reworks the scrubbing logic so needle size mismatches are
ignored in such scenarios.

Scrubbing can still be forced to check deleted needles (f.ex. to discover
index inconsistencies); this option will be exposed in RPCs and `weed shell`
on a follow-up PR.

2026-06-30 02:05:14 -07:00

..

backend

storage: register tier backends at the binary composition root (#9989)

2026-06-16 11:47:32 -07:00

erasure_coding

fix(ec): suppress deleted-needle size mismatch in EC LOCAL scrub (#10147)

2026-06-30 01:58:57 -07:00

idx

fix(scrub): don't flag offset-0 logical tombstones in volume scrub (#10148)

2026-06-30 02:01:14 -07:00

needle

[volume] preserve compression state during replication (#9946)

2026-06-13 21:52:59 -07:00

needle_map

fix(vacuum): crash-safe compaction commit with a durable .cpc marker, fsync-before-rename, and a reload fence (#9944)

2026-06-13 20:06:24 -07:00

super_block

storage: nil-safe ReplicaPlacement.String()

2026-06-16 10:42:43 -07:00

test_files

Have volume scrubs account for zero-sized volumes. (#9609)

2026-05-21 09:42:07 -07:00

types

fix(master): register EC shards per physical disk on full heartbeat sync (#9212) (#9219)

2026-04-24 14:01:09 -07:00

volume_info

storage: register tier backends at the binary composition root (#9989)

2026-06-16 11:47:32 -07:00

volume_replica

fix(ec): bring ec.encode worker and EC/volume helpers to parity with shell (#9599)

2026-05-21 02:16:28 -07:00

disk_location_ec_realworld_test.go

EC bitrot detection: per-shard checksum sidecars (#9761)

2026-05-31 18:52:44 -07:00

disk_location_ec_shard_size_test.go

fix(volume_server): load orphan EC shards across disks on startup (#9212) (#9244)

2026-04-27 16:01:10 -07:00

disk_location_ec_test.go

fix(ec): never delete recoverable EC shards on startup/reconcile (the non-empty-.dat sibling of the stub bug) (#9941)

2026-06-12 23:51:29 -07:00

disk_location_ec.go

fix(ec): fence stale-worker EC shard cleanup by encode generation (#9953)

2026-06-14 01:54:04 -07:00

disk_location_test.go

Implement RPC skeleton for regular/EC volumes scrubbing. (#8187)

2026-02-02 17:55:04 -08:00

disk_location.go

fix(vacuum): crash-safe compaction commit with a durable .cpc marker, fsync-before-rename, and a reload fence (#9944)

2026-06-13 20:06:24 -07:00

ec_startup_shard_safety_test.go

fix(ec): never delete recoverable EC shards on startup/reconcile (the non-empty-.dat sibling of the stub bug) (#9941)

2026-06-12 23:51:29 -07:00

needle_map_leveldb_test.go

fix(vacuum): crash-safe compaction commit with a durable .cpc marker, fsync-before-rename, and a reload fence (#9944)

2026-06-13 20:06:24 -07:00

needle_map_leveldb.go

fix(vacuum): crash-safe compaction commit with a durable .cpc marker, fsync-before-rename, and a reload fence (#9944)

2026-06-13 20:06:24 -07:00

needle_map_memory.go

fix(volume): keep vacuum running past dangling .idx entries (#9115)

2026-04-16 22:01:34 -07:00

needle_map_metric_test.go

fix(volume): keep vacuum running past dangling .idx entries (#9115)

2026-04-16 22:01:34 -07:00

needle_map_metric.go

fix(volume): keep vacuum running past dangling .idx entries (#9115)

2026-04-16 22:01:34 -07:00

needle_map_sorted_file_test.go

fix(volume): seed indexFileOffset in SortedFileNeedleMap so Delete appends (#9483)

2026-05-13 10:22:01 -07:00

needle_map_sorted_file.go

fix(volume): seed indexFileOffset in SortedFileNeedleMap so Delete appends (#9483)

2026-05-13 10:22:01 -07:00

needle_map.go

fix(volume): keep vacuum running past dangling .idx entries (#9115)

2026-04-16 22:01:34 -07:00

remote_tier_integration_test.go

fix(seaweed-volume): stop EC shard deletion from phantom .dat on restart (#9874)

2026-06-08 22:10:16 -07:00

store_disk_space_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_duplicate_vid_test.go

fix(storage): delete/unmount every copy of a duplicate volume id (#9954)

2026-06-14 06:36:47 -07:00

store_ec_9plus3_reboot_repro_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_delete.go

admin: report file and delete counts for EC volumes (#9060)

2026-04-13 21:10:36 -07:00

store_ec_disk_type_test.go

batch drain delta heartbeat messages (#9914)

2026-06-10 13:33:45 -07:00

store_ec_hybrid_repro_test.go

fix(ec): never delete recoverable EC shards on startup/reconcile (the non-empty-.dat sibling of the stub bug) (#9941)

2026-06-12 23:51:29 -07:00

store_ec_mirror_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_mirror.go

fix(ec): mirror EC sidecars onto every shard-bearing disk at startup (#9525)

2026-05-17 19:55:15 -07:00

store_ec_mount_cross_disk_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_orphan_shard_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_phantom_dat_test.go

fix(storage): never let an empty .dat delete healthy distributed EC shards (#9930)

2026-06-11 20:26:20 -07:00

store_ec_reconcile.go

fix(ec): never delete recoverable EC shards on startup/reconcile (the non-empty-.dat sibling of the stub bug) (#9941)

2026-06-12 23:51:29 -07:00

store_ec_recover_test.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

store_ec_recover.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

store_ec_recovery_test.go

Update store_ec_recovery_test.go

2026-01-23 16:38:36 -08:00

store_ec_scrub.go

Fix scrubbing of deleted needles on EC volumes. (#10130)

2026-06-30 02:05:14 -07:00

store_ec_target_location_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec.go

fix(ec): recover EC shards with the volume's own ratio, not the build default (#9958)

2026-06-14 07:32:36 -07:00

store_load_balancing_simple_test.go

create new volumes on less occupied disk locations (#7349)

2025-10-20 16:11:29 -07:00

store_load_balancing_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_maxvolume_deadzone_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_state.go

chore: remove ~50k lines of unreachable dead code (#8913)

2026-04-03 16:04:27 -07:00

store_vacuum_test.go

fix tests

2025-06-30 13:57:28 -07:00

store_vacuum.go

Process .ecj deletions during EC decode and vacuum decoded volume (#8863)

2026-04-01 01:15:26 -07:00

store.go

fix(storage): delete/unmount every copy of a duplicate volume id (#9954)

2026-06-14 06:36:47 -07:00

volume_backup_test.go

…

volume_backup.go

add version to volume proto

2025-06-16 22:05:06 -07:00

volume_checking_test.go

fix(scrub): don't flag offset-0 logical tombstones in volume scrub (#10148)

2026-06-30 02:01:14 -07:00

volume_checking.go

fix(scrub): don't flag offset-0 logical tombstones in volume scrub (#10148)

2026-06-30 02:01:14 -07:00

volume_destroy_ec_vif_test.go

fix(storage): keep EC .vif when deleting a coexisting regular volume (#9723)

2026-05-28 15:39:31 -07:00

volume_info_test.go

…

volume_info.go

shell: show remote storage name/key in volume.list output (#9987)

2026-06-16 10:35:06 -07:00

volume_io_error_test.go

fix(volume): sticky EIO quarantine; track streamed reads (#9384)

2026-05-09 09:55:02 -07:00

volume_loading_corrupt_idx_test.go

fix(volume): avoid nil-deref when needle map loader errors (#9694) (#9697)

2026-05-26 16:56:49 -07:00

volume_loading.go

fix(volume): avoid nil-deref when needle map loader errors (#9694) (#9697)

2026-05-26 16:56:49 -07:00

volume_mark_writable_test.go

fix(volume): reopen .idx writable after MarkVolumeWritable (fixes #9515) (#9526)

2026-05-18 20:51:04 -07:00

volume_read_all.go

…

volume_read_test.go

fix(volume): don't panic on read when needle map is nil (#9342)

2026-05-06 18:23:06 -07:00

volume_read.go

fix(volume): sticky EIO quarantine; track streamed reads (#9384)

2026-05-09 09:55:02 -07:00

volume_super_block.go

Fix: Fail fast on unsupported volume versions (#8047)

2026-01-16 19:19:18 -08:00

volume_tier.go

storage: register tier backends at the binary composition root (#9989)

2026-06-16 11:47:32 -07:00

volume_vacuum_crash_safe_test.go

fix(vacuum): crash-safe compaction commit with a durable .cpc marker, fsync-before-rename, and a reload fence (#9944)

2026-06-13 20:06:24 -07:00

volume_vacuum_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

volume_vacuum.go

fix(vacuum): crash-safe compaction commit with a durable .cpc marker, fsync-before-rename, and a reload fence (#9944)

2026-06-13 20:06:24 -07:00

volume_write_test.go

fix(balance): don't move remote-tiered volumes; don't fatal on missing .idx (#9335)

2026-05-06 15:19:43 -07:00

volume_write.go

fix(vacuum): crash-safe compaction commit with a durable .cpc marker, fsync-before-rename, and a reload fence (#9944)

2026-06-13 20:06:24 -07:00

volume.go

volume: detect phantom volumes held open as deleted FDs (#10011)

2026-06-19 09:24:04 -07:00