mirrors/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-08-21 06:36:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

EC bitrot follow-ups: protect destination sidecar on optional copy; cap sidecar block_size (#9763)

Browse Source 
* fix(ec_bitrot): cap sidecar block_size in ValidateBitrotManifest

A sidecar loaded from disk (or supplied via a backfill/peer RPC) could carry a
huge power-of-two block_size that passed validation, then force a multi-GiB
scratch-buffer allocation in scrub/verify. Add a shared MaxBitrotBlockSize
(64 MiB) constant, enforce it as an upper bound in isPow2MultipleOf1MiB, and
derive the volume flag cap from the same constant so they cannot drift.

* fix(ec_bitrot): don't destroy a valid destination sidecar on an optional copy

writeToFile opened the destination with O_TRUNC before knowing whether the
source had the file, so an optional copy (ignoreSourceFileNotFound) from a source
that lacks the .ecsum truncated and then removed a valid pre-existing destination
sidecar. Stage the optional copy into a temp sibling and commit it with an atomic
rename only when the source actually delivered the file; a missing source is now
a no-op. Mandatory copies keep their in-place behavior.
This commit is contained in:
Chris Lu
2026-05-31 23:42:33 -07:00
committed by GitHub
parent 9658f309d2
commit 80dd3b2621
5 changed files with 113 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
weed/command/volume.go
+4 -6
View File
@@ -177,12 +177,10 @@ func runVolume(cmd *Command, args []string) bool {
// Apply EC bitrot checksum settings.
erasure_coding.BitrotProtectionEnabled = *ecBitrotChecksum
// Validate the block size before multiplying so an absurd MiB value cannot
// overflow int64 and slip a bogus size past the power-of-two check. The
// block size also becomes the per-shard scratch buffer the scrub/backfill
// path allocates, so the upper bound caps that allocation (64 MiB per
// concurrent scrub worker) and keeps a typo from taking the server down.
const maxBitrotBlockSizeMB = 64
// Bound-check before the multiply so a huge value cannot overflow int64 past
// the power-of-two check. Cap = shared MaxBitrotBlockSize, kept in sync with
// ValidateBitrotManifest.
const maxBitrotBlockSizeMB = erasure_coding.MaxBitrotBlockSize / (1024 * 1024)
if mb := *ecBitrotBlockSizeMB; mb >= 1 && mb <= maxBitrotBlockSizeMB {
if blockSize := int64(mb) * 1024 * 1024; blockSize&(blockSize-1) == 0 {
erasure_coding.BitrotBlockSize = blockSize