mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-26 18:04:33 +00:00
Chris LuandGitHub
e264e9883e
* fix(seaweed-volume): bound request body and stored-content expansion to prevent OOM The Rust volume server buffered the entire upload body with to_bytes(usize::MAX) and only checked the file-size limit afterward, so a single large upload — or many concurrent uploads, since the in-flight byte throttle defaults to 0 (unlimited) — could exhaust memory and get the process OOM-killed under load. The read path had two more single-request OOM vectors: `vec![0u8; manifest.size]` allocated from an attacker-controlled chunk-manifest size, and gzip decompression was unbounded (gzip bomb). - Bound the upload body read by file_size_limit_bytes (plus a margin for multipart framing), mirroring Go's io.LimitReader(sizeLimit+1), and reject oversize before the whole body is buffered. - Validate manifest.size (reject negative / oversized) before allocating. - Cap gzip output in maybe_decompress_gzip and route the inline GzDecoder sites through it. * fix(seaweed-volume): address review - chunk offset, 32-bit cast, decompress errors - Validate chunk.offset before indexing in chunk-manifest expansion: a negative offset wrapped to a huge usize and underflowed `end - offset` (panic from a crafted manifest). Reject negative, skip out-of-range, use saturating math. - Use usize::try_from for the upload body limit instead of `as usize`, so a >usize::MAX file_size_limit on 32-bit caps at usize::MAX rather than silently truncating to a tiny value. - maybe_decompress_gzip now returns Result<_, GunzipError> distinguishing a decode failure (callers fall back to raw bytes, as before) from hitting the size cap (TooLarge), which now returns 413 instead of silently serving the still-compressed bytes. * fix(seaweed-volume): inflate manifest chunks into the result window to cap peak memory The chunk-manifest expansion still doubled memory: `result` was already allocated at manifest.size (<=2 GiB) and each compressed chunk was inflated into a separate Vec (also up to 2 GiB), so a single request could peak near 4 GiB. Decompress compressed chunks directly into their result[offset..] window (bounded by the remaining space) so a chunk never allocates a second large buffer; peak stays at ~manifest.size. Bytes past the window are dropped (matching the prior truncation), and a fully-undecodable chunk still falls back to its raw bytes. * fix(seaweed-volume): fall back to raw chunk bytes on any decode failure Per review: the gzip fallback must run on any decode error, not only when no bytes were decoded. Clear the partially-written output and copy the chunk's raw bytes (truncated to the window), restoring the prior decode-failure behavior.