mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-22 06:22:38 +00:00
ab738bfd33
* rust volume: repair an unindexed .dat tail at load Mirror Go's repairUnindexedDatTail (#11870): a crash between the .dat\nappend and the .idx append leaves a .dat tail no index entry covers,\nwhich sent the volume read-only on the next load and left acknowledged\nneedles unreadable. Before the index loads, re-index the complete\nneedles in the tail (a size 0 record as the tombstone or empty write\nresolveEmptyTailRecords reads it as) and cut only an incomplete last\nrecord. A tail an interrupted append does not explain is left as it is\nand the integrity check quarantines the volume read-only as before. Generated with [Devin](https://devin.ai)\n\nCo-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: tests for the unindexed .dat tail repair Reload volumes through the real load path after dropping .idx entries\nand cutting or damaging the .dat, mirroring Go's\nvolume_checking_tail_repair_test.go: complete unindexed needles are\nre-indexed (deletes as tombstones, empty writes as empty needles), an\nincomplete last record is cut, and a tail an interrupted append does\nnot explain is left untouched with the volume read-only. Generated with [Devin](https://devin.ai)\n\nCo-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: re-index a torn size 0 record as a live empty needle A delete and a write with no data append the identical size 0 record, so a recovered one cannot be told apart from the prior index state alone: the write path accepts an empty overwrite of a live key and indexes it with size 0, while a delete of the same key publishes a tombstone. Inferring a tombstone from prior live state turned a lost index row after an empty overwrite into a deletion -- the object read back not-found although the write was answered. Every re-indexed record is now indexed at its on-disk size, the entry publishWriteRequest writes for an empty write. The other misread is bounded: a recovered delete reads its key back as an empty object rather than not-found, and the size 0 record carries no data to resurrect. Review feedback on the Rust mirror (#11878): the same bug exists here; fixing both sides identically. Generated with [Devin](https://devin.ai)\n\nCo-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: re-index a torn size 0 record as a live empty needle Mirrors the Go fix in this pull request. A delete and a write with no data append the identical size 0 record, so resolve_empty_tail_records could not tell them apart from the prior index state: it treated a zero-size record of a live id as a tombstone, although the write path indexes the same bytes as a live size 0 entry. A crash after an empty overwrite of a live key therefore recovered the object as deleted. Every re-indexed record is now indexed at its on-disk size, the entry publish_write writes for an empty write. A recovered delete reads its key back as an empty object rather than not-found; the size 0 record carries no data to resurrect. Review: greptile-apps[bot] and devin-ai-integration[bot] on #11878. Generated with [Devin](https://devin.ai)\n\nCo-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: repair a .dat tail left past an empty .idx The writable index loader creates the .idx before the first needle append, so a crash after that append leaves a volume whose data nothing indexes. CheckVolumeDataIntegrity skipped an empty index outright, so the volume loaded writable and the surviving first needle stayed invisible while later writes appended past it. A zero-size index now runs the same repair with tailEntryPos -1: the walk starts at the first record past the superblock with no anchor needle and no append timestamp to compare against, gets the same complete-record validation and incomplete-tail policy, and re-verifies through the normal tail check once it wrote entries. A tail the repair cannot explain fails the check, so the volume loads read-only instead of writable on an unaccounted-for .dat. Review feedback on the Rust mirror (#11878): the same bug exists here; fixing both sides identically. Generated with [Devin](https://devin.ai)\n\nCo-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: repair a .dat tail left past an empty .idx Mirrors the Go fix in this pull request. The writable index loader creates the .idx before the first needle append, so a crash after that append leaves a volume whose data nothing indexes. Both the repair and the integrity check skipped an empty index, so the volume loaded writable and the surviving first needle stayed invisible while later writes appended past it. An empty .idx now runs the same repair with no anchor: the walk starts at the first record past the superblock, and a tail it cannot explain fails the load like any other repair failure, since every later check skips an empty index. Review: devin-ai-integration[bot] on #11878. Generated with [Devin](https://devin.ai)\n\nCo-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: sync the .dat before publishing re-indexed .idx rows A power loss after the repaired .idx rows become durable but before the .dat tail records they name are flushed leaves the index pointing past the end of the data file; the next load then keeps the volume read-only until an operator steps in. Sync the .dat first so the rows never outlive the records they reference. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: sync the .dat before publishing re-indexed .idx rows Mirrors the Go fix: a power loss after the repaired .idx rows become durable but before the .dat tail records they name are flushed leaves the index pointing past the end of the data file. Sync the .dat first. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>