989 Commits

Include renames

Author	SHA1	Message	Date
github-actions[bot]	de34a1a87c	4.41	2026-08-06 07:39:20 +00:00
Chris LuandGitHub	f46b2a1925	Stop the filer test helpers from pinning gigabytes of log buffers (#10560) ... * log buffer: wake the interval loop on shutdown instead of sleeping through it loopInterval parked in time.Sleep(flushInterval) and only re-checked IsStopping when it woke, so a buffer shut down early kept both loop goroutines - and the PreviousBufferCount+1 slabs of BufferSize they reach - alive for up to a full interval afterwards. Select on shutdownCh against a ticker instead, and give the loops a WaitGroup so a test can observe that they exit. * test: release the filers the server tests build Every helper here left its filer's meta log buffer running, so each test pinned PreviousBufferCount+1 buffers of BufferSize for the rest of the run: ~3.5GB of live heap across the package, which overruns the address space on linux/386 and kills the 32-bit job with an out-of-memory throw. Thread the test through the helpers so the buffer is shut down on cleanup, and shut the subscribe harness's filer down outright - its deletion loop keeps the whole filer reachable otherwise. That harness quiesces its flush path first, since Filer.Shutdown closes the store a flush still in flight would write through.	2026-08-04 11:38:38 -07:00
4 people	813a4b0711	fix(filer): stop skipping recent unflushed events on metadata subscription gaps (#10501) ... * fix(filer): don't skip unflushed events on metadata subscription gaps A subscriber that falls behind the in-memory log ring during a write burst could have its read position jumped past events that were evicted but not yet flushed to disk — silently lost for filer.backup/filer.sync/ mount subscribers. Route all three gap-skip sites through resolveDiskGapResume: only skip past windows older than a settled horizon (2*LogFlushInterval); recent gaps wait for the flush and re-read disk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(filer): harden metadata gap-skip guard Address review findings on the settled-horizon guard: - local subscriptions: gate the skip on the buffer's flush watermark observed before the disk read (resolveLocalGapResume) — a disk miss is then proof the gap is empty, with no wall-clock assumptions - aggregated subscriptions: cap skips strictly below the horizon boundary (persisted reads exclude ts <= cursor) and pace capped advances, so the sliding horizon cannot cause disk-probe spinning - replace unbounded sync.Cond waits with a bounded select on the buffer's subscriber channel + retry timer + ctx cancellation, eliminating the lost-wakeup stall Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(filer): close remaining gap-skip loss paths from re-review - log_buffer: a sentinel-offset (time-based) read below the earliest in-memory entry silently started at earliest, skipping a window that may hold evicted-but-unflushed events. Track the ring's eviction watermark (lastEvictedTsNs) and keep the inclusive fast path only when nothing at/after the position was ever evicted; otherwise return ResumeFromDiskError so the subscription gap guard decides. - capped horizon advances stay on the disk-probe path (never expose a mid-gap position to the memory read) and keep pacing - gap jumps land just below earliest: positions are exclusive, so the earliest entry itself is still delivered - subscriber notification keys include clientId/epoch so a replacement stream never inherits a channel the old stream's cleanup closes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(log_buffer): generalize the eviction-watermark read gate Third-review findings: - epoch/zero-time reads bypassed the eviction watermark: gate them the same way, so a SinceNs=0 subscriber cannot silently start at the earliest retained entry after an unflushed window was evicted - apply the watermark gate regardless of the cursor's batch offset (batch offsets carry no meaning for time-based reads); this also serves adjacent cursors (earliest == current+1) from memory instead of stalling them in the gap loop - LoopProcessLogData reader names include clientId/epoch, since they are registered as subscriber keys internally (same collision as the outer notification keys) - test: pin the gate (below/at watermark, epoch-after-eviction) and update the slow-consumer test to the sharper contract — complete in-memory history is served from memory; disk only once evicted Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(log_buffer): watermark equality is unsafe for inclusive sentinel cursors Sentinel (Offset <= 0) time-based cursors search from ts-1ns, i.e. they read inclusively of their own timestamp — and the evicted window may end exactly at that timestamp. Allow watermark equality only for exclusive (positive-offset) cursors; sentinel cursors must be strictly above it. Also shut down the test buffer and pin the equality cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(filer): wait on an empty aggregated buffer instead of falling through When a disk read finds nothing for a ResumeFromDiskError gap and the aggregated buffer has no readable entries (zero earliest time), resolveDiskGapResume declines to advance and control fell through to the in-memory read — which returns ResumeFromDiskError again immediately, spinning through the probe cycle without any wait. Fold the case into the existing recent-gap branch so it waits (notification, cancellation, or the retry interval) before re-probing, matching the local subscription path, which already waits unconditionally. * fix(filer): serve the exact eviction-boundary entry without another flush wait When the flush watermark has passed the earliest in-memory entry but that entry sits exactly one nanosecond above the cursor, the exclusive resume target collapses onto the cursor and resolveLocalGapResume declines to advance — while a sentinel cursor at the eviction watermark keeps deferring the in-memory read to disk. Progress then depends on the next flush cycle. Re-arm the cursor with a positive (exclusive) offset instead: ReadFromBuffer explicitly allows positive-offset cursors at the watermark, so the boundary entry is served from memory immediately. Also promote the aggregated path's horizon-capped gap skip to a warning: that skip may pass events a stalled flush lands later (the aggregated ring has no flush watermark to gate on), so operators should see when a flush stall outlasts the settled horizon. * fix(filer): resume a gap skip with an exclusive cursor The resume position landed one nanosecond below the earliest in-memory entry but kept the inclusive sentinel offset. When that timestamp is also the eviction watermark, the read gate answers ResumeFromDiskError for an inclusive cursor, the disk read finds nothing, and the resume target collapses onto the cursor, so neither helper can advance: the subscriber parks on timed waits forever. A skip is only taken once the gap is proven empty, so nothing remains to deliver at the resume timestamp. Resume exclusively instead, and let an inclusive cursor already sitting on the target count as progress. * fix(filer): gate aggregated gap skips on eviction, not wall clock Wall-clock age never proved persistence. The aggregated ring has no flush watermark because peers persist their own local logs, so a horizon of 2 * LogFlushInterval was standing in for one. While the disk stayed empty the horizon kept sliding forward and walked the cursor past evicted-but-unflushed events one window at a time - the volume-outage stall this change set exists to survive. The ring does carry a real proof: its eviction watermark. If nothing at or after the cursor was ever dropped, memory still holds every entry after it and the gap is provably empty. Below the watermark entries were dropped and only the producing peer's flush can supply them, so wait for that flush instead of advancing. The skip that remains is the one the ring can prove, which keeps the infinite-loop guard for a genuinely empty gap. * perf(filer): stop re-probing disk on every metadata append A subscriber parked on a gap woke on the log buffer's subscriber channel, which fires on every append. On the aggregated path each wake re-ran a full ReadPersistedLogBuffer - a ListDirectoryEntries plus a readahead goroutine - so one parked subscriber turned every cluster-wide metadata event into a store query, exactly while the cluster is already struggling with the flush stall that parked it. An append also cannot settle the gap: what this waits on is a peer persisting its own log, which nothing local signals. Wait on the timer alone there. The local buffer does signal its flush on that channel, so keep it, but drain the stale token first: otherwise the appends riding the same channel spin the wait at write rate. The retry interval covers the notification the drain discards. * fix(filer): surface a metadata subscriber parked on a gap Refusing to skip an unresolved gap trades silent loss for a silent stall, and a stall is no easier to diagnose: filer.sync and mount followers just stop advancing, with no error on either end. The only trace was a V(3) line nobody runs with. Warn on entry and once a minute after, and carry a subscribe_gap_stalled gauge for the duration, so a flush that never lands shows up as a stalled subscriber rather than a consumer that mysteriously went quiet. * refactor(filer): drop the metadata listener cond with no waiters left Both listenersCond.Wait() sites are gone, so listenersWaits never leaves zero, the guard in the filer's notify callback never fires, and the two Broadcast calls around client registration wake nobody. Remove the cond, its counter and its lock, and pass a nil notify func: the log buffer already skips a nil one, and the subscriber channels carry these wakeups now. * test(log_buffer): drive the eviction watermark through a real seal Both eviction tests wrote lastEvictedTsNs directly, leaving the one line that sets it uncovered: copyToFlushInternal has to read slot 0 before SealBuffer shifts it out, and moving that read one statement later still passed every test. Append past the ring instead, and check the watermark appears only on the seal that drops a window, matches that window's stop, and then advances. The buffer under test never flushes, matching the aggregated meta ring where the watermark is the only emptiness proof. * refactor(filer): build the subscriber reader name once per stream It was rebuilt on every loop iteration from values that cannot change for the life of the stream. Hoist it next to the notification key it mirrors. * docs(filer): tighten the gap-handling comments Several ran to eight lines restating the same reasoning at each site. Keep the non-obvious why, drop the retelling. * fix(filer): mark a confirmed disk position as an exclusive cursor A disk read returns the timestamp of the last entry it handed to the subscriber, but the cursor built from it stayed inclusive. Land that cursor exactly on the eviction watermark and the read gate sends it back to disk for an entry the disk just delivered; the re-read finds nothing, neither emptiness proof holds for an inclusive cursor there, and the subscriber parks - until some later flush, or forever while one stays stalled. Everything after the watermark was sitting in memory the whole time. Carry the offset instead, so it also covers the ring evicting onto the cursor after the read rather than before it. The constant is no longer gap-specific, so it is now named for what it asserts. * perf(filer): re-park a gap wait woken by an ordinary append Draining one stale token did not bound anything: the subscriber channel carries an append per metadata write, so under continuous writes the next one satisfies the wait immediately. During the write burst these stalls come from, each parked subscriber ran the recovery loop at write rate rather than the intended two-second cadence. Only a flush can settle a gap, so check the flush watermark on wake-up and go back to waiting if an append is all that arrived. The retry timer is created once, so re-parking does not extend the interval. * fix(filer): keep a disk-derived cursor inclusive Marking every disk position exclusive assumed the entry at that timestamp was the only one there. On the aggregated stream it is not: disk can deliver one filer's persisted event at T while another filer's event at the same T is still unflushed in the aggregate ring. The exclusive cursor skipped it, and since the persisted reader also excludes ts <= its start, nothing would ever bring it back. Take the weaker guarantee instead. The reason the exclusive cursor was introduced - an inclusive one landing on the eviction watermark parks forever - is better answered in the resolvers: at the watermark memory holds nothing (retained windows start strictly after it) and the persisted reader cannot return that entry at any later time either, so refusing the gap buys nothing and never ends. Skip it whatever the cursor's inclusivity. That proof does not depend on a flush, so the local resolver takes it as a second, independent disjunct alongside its flush watermark. * perf(filer): wake a parked gap wait on flushes only Re-parking on an append bounded the work but not the wake-ups: the subscriber channel carries one per metadata write, so a parked subscriber still took a scheduling round-trip per write. Worse, draining it kept the channel empty, so every writer's non-blocking notify succeeded instead of falling through - the burst paid for the wake-ups too. Give the log buffer a flush-only subscriber list, notified from loopFlush, and park on that. The append channel now fills once and stays full, which is exactly the state the non-blocking send is designed for. * fix(filer): stop the gap-stall gauge from leaking a series per connection clientName is req.ClientName + "@" + peer address, so it carries the client's ephemeral source port and changes on every reconnect. Labelling the gauge with it minted a new series per connection, and clearing it only ever Set(0), so nothing was ever released - a client in a reconnect loop grows the filer's metric map and /metrics payload without bound. Key it on the stable client-supplied name, as the neighbouring subscribe gauge already does, and delete the series on teardown. Two logging fixes ride along, both in the same reporter: the resume warning was unpaced while the park warning throttles to one a minute, so a burst that parks and resumes every couple of seconds warned on every cycle; and clear() doubled as the teardown path, announcing "resumed after 14m0s parked" for a client that actually gave up and disconnected still behind. * fix(filer): give a parked gap wait the exits the read loop has A park never re-enters the read loop, so every exit that loop relies on stopped working while a subscriber was parked. It kept scanning the filer store every 2s for a client that a higher-epoch reconnect had already superseded, until the TCP connection finally died - hours, on a half-open one. It never reached the only code that honors UntilNs, so bounded callers like `weed shell fs.verify` and `filer.meta.tail -until=` hung instead of exiting. And a notification channel closed out from under it turned the bounded wait into a spin, since a receive on a closed channel returns instantly. Check all three where the subscriber actually waits. Bound the wait too: waiting is productive while the window is still queued for flush, but a peer that never returns, or whose filer store this filer cannot read at all, makes it permanent - and a subscriber that silently stops delivering is no better than one that silently skips. Fail the stream after that instead of hanging, loudly enough to say which gap and for how long. * fix(log_buffer): keep the eviction gate out of the shared read path The gate belonged to the filer's subscribe loops but was installed in ReadFromBuffer, which the message queue shares and which has no gap handling of its own. Three MQ paths broke on it: GetUnflushedMessages asks for everything in memory past the flush watermark and got ResumeFromDiskError instead, so SQL results silently dropped unflushed rows; a RESET_TO_EARLIEST consumer's epoch cursor was sent to disk, and the MQ disk reader resets an empty read back to epoch, so the whole partition replayed on every pass; and a disk cursor landing on the watermark carried offset -2, which the gate refused and the disk reader's ts <= start filter also skips, so neither side could ever serve it. The rewrite also dropped the old Offset <= 0 requirement, letting a stale positive-offset cursor jump to memory over on-disk history, and refused any negative-timestamp cursor even with nothing evicted. Restore the read path exactly as it was and keep only lastEvictedTsNs, which is the useful primitive. The filer loops now consult the watermark themselves before reading memory, which is where the gap handling that makes the refusal actionable already lives - and they check it every pass, not just when the disk came up empty, since a disk read can leave the cursor short of the watermark too. * fix(filer): read the log file whose window spans past its own name A log file is named for the start of the window it holds, but a window runs up to a flush interval longer, so "12-30" can hold entries through 12:31:58. File selection compared the cursor's minute against that name and skipped anything sorting earlier, so a subscriber resuming at 12:31:10 never saw the rest of that file: the read reported nothing on disk while the entries sat in it. That was survivable when a miss only meant "wait and retry", but the gap resolvers now read a miss as proof the range is empty and move the cursor past it, which turns those entries into silent loss. Start the file scan a flush interval early; entries are still filtered against the exact cursor, so this only opens one more file and never re-delivers. * fix(filer): resume a gap with a cursor the memory read will serve Reverting the eviction gate put ReadFromBuffer back to refusing every positive-offset cursor below the in-memory window, but the resolvers still handed back one - earliest-1 marked exclusive. So the resume bounced straight to ResumeFromDiskError, the disk had nothing, and the resolver saw its own cursor as no progress and parked: a subscriber stalled, and after the new bound failed outright, with the whole gap sitting in the ring the entire time. The exclusive marking only existed to dodge a park the gate itself caused, and the gate is gone. Resume at earliest with the sentinel offset, which is the position master used and which case 2.1 reads inclusively. That makes the resume unconditionally ahead of the cursor, so the progress check it needed goes away with it. * fix(filer): stop dropping a log file whose window outruns its name Listing the earlier file was not enough: the iterator then decided whether to read it by comparing the *following* file's name against the cursor, which treats that name as an upper bound on this file's contents. It is not one. A file is named for the start of the window it holds, minute-truncated, and the window runs up to a flush interval longer, so "12-30" can hold an event at 12:31:20 while "12-31" sits right after it - and a cursor at 12:31:10 skipped straight past the event. Bound the decision on the file itself: skip it only when its name plus the minute truncation plus a flush interval still lands at or before the cursor. That also covers the last file in the queue, which the old check never skipped because it had no successor to compare against. * fix(filer): stop a chunk-ref read from rewinding the subscriber CollectLogFileRefs reports the minute-level name of the last file it shipped, and the caller assigns that straight to the read position. Since the scan now reaches back a flush interval to catch a spanning file, a request at 12:31:10 that picks up the 12-30 ref moved the cursor to 12:30:00 - so the memory read that followed replayed events older than the client's own SinceNs and re-sent what the chunk reader had already been handed. The same rewind was reachable before, within a minute, whenever the last file's name sorted behind the request. Clamp the reported position to the one that was asked for. It still under-advances by design, since the server never reads the entries it ships refs for and cannot know where they end. * fix(filer): resume below earliest so a one-entry window is not skipped Moving the resume onto earliest itself was wrong for the smallest window there is. A sealed window holding a single entry has startTime == stopTime, and the sealed-buffer lookup only enters a window whose stopTime is strictly after the cursor, so a cursor sitting exactly on earliest walks past it and its sole event is never delivered. Low-volume metadata windows are routinely one entry, which is precisely when losing it is hardest to notice. Go back to one nanosecond below, which takes the startTime.After branch and returns the whole window, and keep the sentinel offset the memory read requires. The earlier test used an active multi-entry buffer, where both cursors happen to work; the new one seals single-entry windows and compares which entry comes back. * fix(filer): count a persisted read as progress only when it moves A chunk-ref read reports the minute-level name of the last file it shipped, now clamped so it never rewinds, so it comes back non-zero even when it names the position the subscriber already held. The loops read non-zero as progress: they cleared the stall timer, then found the cursor still short of the eviction watermark and parked again. Every retry re-shipped the same refs and reset the timer, so the bound that is supposed to end an unrecoverable stall was never reached - and a chunk-capable client buffers those refs waiting for an event that never comes, so it just accumulates duplicates. Require the reported position to be strictly ahead of the cursor. * fix(filer): count the evicted ranges the aggregated stream cannot prove The eviction watermark belongs to the merged ring, but the disk it gets checked against is the union of every peer's own log and each peer flushes on its own schedule. A read that lifts the cursor from below the watermark to above it may have done so entirely on a peer that is already ahead, while a lagging peer still holds unflushed events inside the range just crossed; when it flushes them they sit behind the cursor and are never delivered. One aggregate maximum is not proof that every peer persisted the range. Nothing available locally separates that from the ordinary case where every peer had in fact persisted it: the aggregator tracks peers by address while log files carry a random per-filer id, so "has this peer flushed through T" cannot be answered here at all. Deciding it needs the source filer's own flush watermark carried on the subscribe stream, which is a wire change this does not make. Count and log the crossing so the window is at least measurable instead of invisible. * fix(filer): send log file refs through the pipelined sender sendLogFileRefs wrote on the raw gRPC stream while pipelinedSender's goroutine concurrently calls Send for queued memory events - two senders on one stream, which gRPC forbids. The window used to open once per fall-behind; the gap retry loop now reopens it every pass. Routing refs through the sender also restores ordering: refs used to overtake up to 1024 queued events, and the client treats any non-ref message as the signal to process buffered refs, so overtaken refs were applied against the wrong position. The reason refs bypassed the sender was the batcher: their TsNs of 0 reads as far behind, and the client recognizes refs by the top-level field alone - a refs envelope would drop its Events tail, and refs inside Events would be applied as an empty event. Teach the batcher instead: refs messages always go solo, and one drained mid-batch is sent solo right after that batch. * fix(filer): count parked subscribers instead of flagging them by name The per-client gauge series could not work. Its label was rebuilt from the peer address at first, which leaks a series per reconnect; keyed on the client-supplied name instead, it collides - every mount registers as "mount" - so one stream's teardown deleted a parked sibling's live series, and the sibling never re-created it because its own park state said the gauge was already set. Either way the alert this gauge exists to drive goes dark. A count needs no identity: Inc on park, Dec on resume or teardown, scope as the only label. Client details stay in the logs. Also start warning only once a stall has outlived the warn interval. park() warned immediately on every first park, so catch-up churn that parks and resumes every couple of seconds logged a warning pair per cycle - exactly the flood the pacing was supposed to prevent, burying the long-stall warnings that matter. * fix(filer): one gap resolver, and re-arm an unservable adjacent cursor The two resolvers were the same function - the aggregated one is the local one with a flush watermark of zero, since its ring never flushes - duplicated down to the comment justifying the resume target. A fix applied to one and not the other is how the two streams drift; merge them. The merge carries the one behavioral fix both copies needed. Timestamp collision bumps make adjacent entries exactly 1ns apart, so an entry ending an evicted window leaves the cursor exactly one below earliest with a positive batch offset. The resume target then equals the cursor and both copies refused it as no progress - but that cursor cannot be served (ReadFromBuffer refuses positive offsets below the window) while the sentinel resume at the same timestamp is, and both deliver exactly the entries after it. Refusing parked a subscriber whose data was entirely in memory: until the next flush locally, and through a 15-minute stall failure on the aggregated path. Advance on equal target when the held cursor is exclusive; a sentinel there is already served, so it still refuses. * fix(filer): a bounded subscription parked exactly on UntilNs is finished The park's UntilNs check was strict while the bound is inclusive and cursors are exclusive: a disk read whose last entry sits exactly on UntilNs leaves the cursor there with everything up to the bound already delivered. If the next range was an unprovable gap, the completed subscription parked anyway and eventually failed - fs.verify hanging and then erroring on a healthy cluster. * fix(filer): re-derive the subscribe loops as one state machine The loops had grown three generations of gap handling - a post-disk-read branch, a post-memory-read branch, and an every-pass guard bolted on in front of the memory read - each consulting state the others mutated. The worst interaction wedged the aggregated stream permanently: diskExhausted compared the disk result against the cursor that result had just updated, and paired it with a ResumeFromDiskError latch that only a resolver advance cleared, so one fall-behind sent every later pass into the gap block and LoopProcessLogData never ran again. Mounts kept a healthy-looking stream and applied nothing. Both loops now run the same derived sequence. One disk pass; progress is the pre-update cursor against the result, freshly each pass. One gap decision before the memory read: a cursor the ring evicted past either keeps draining the disk (it just advanced), resolves forward (the gap is proven empty), or parks - and a cursor memory refused with nothing evicted after it re-arms onto the retained window. The stale-latch branch is gone: the error is only consulted on a pass whose own disk read came up empty. All four parks go through one parkOnGap helper, so the exits live in one place. The eviction watermark is read after the disk read and the same value feeds both the guard and the unproven-crossing report, which previously compared against a snapshot taken before a potentially minutes-long backlog read and missed evictions landing during it. The next-day jump now clears the stall reporter - it used to leave a stale park epoch that could kill the next brief park instantly at the 15-minute bound - and counts its own watermark crossing. Chunk-ref reads get two rules the old loops lacked. Refs are sent once per position: retries re-sent identical batches every two seconds into a client that only drains them on a non-ref message, growing an unbounded pending list. And when refs cannot advance the cursor - they report file start minutes, which sit below the content the client was actually given, pinning the cursor under the watermark forever during bursts - the pass falls through to entry reads, which move the cursor by real timestamps and double as the client's drain signal. * fix(filer): give up on an unprovable gap instead of failing the stream Failing after maxGapStall assumed the client could do something better, but every consumer just reconnects at the same SinceNs and hits the same wall, so an unprovable gap - a dead peer, or a peer whose filer store this filer cannot read at all - turned into a permanent 15-minute fail/reconnect loop delivering nothing. Master handled the same state by skipping instantly and silently. Take the middle: wait the full bound, then abandon the gap and resume at the eviction watermark, where everything retained starts strictly after, so the loss is exactly the range that could not be proven. The skip shares the unproven-crossing counter and logs at error level - loss is bounded, recorded, and the stream keeps working. A stall with nothing evicted past the cursor loses nothing by waiting, so it restarts the clock and keeps parking rather than skipping. * test(filer): pin the file-skip bound through the production predicate The spanning-file test asserted against its own copy of the arithmetic, so a regression in the iterator - restoring the next-file-name comparison or dropping the flush-interval term - would keep CI green while re-introducing the silent loss the fix closed. Extract the bound into logFileMayContainAfter, call it from the iterator, and point the test at it; breaking the production expression now fails the test. * test(log_buffer): pin the flush-subscriber contract The registry the filer's gap parks wait on had no test at all. Cover the observable contract: an append never wakes a flush subscriber, a flush does with the watermark already stored, unregistering closes the channel so an abandoned waiter unblocks, and double or unknown unregisters are harmless. The store-before-notify ordering in loopFlush is what makes the parks' wake-up re-check sound, and it is not black-box testable - reordering leaves a same-goroutine window of nanoseconds that hundreds of tight round-trips never catch. Mark it load-bearing at the site instead; a reorder now at least has to argue with the comment it deletes. * fix(filer): a -1 SinceNs is a position, not the refs-gate sentinel The once-per-position refs gate used -1 as "never sent", but a client may legally subscribe with SinceNs=-1, whose cursor timestamp is exactly -1: the very first pass then believed refs were already sent there and fell back to streaming the whole persisted history entry by entry - the bootstrap load chunks mode exists to avoid. Use MinInt64, which no cursor can carry. * refactor(filer): drop the aggregator's listener cond with no waiters left Same shape as the FilerServer cond already removed: nothing increments ListenersWaits and nothing ever calls Wait, so the three Broadcasts wake nobody and the notify callback's guard is always false. Aggregated subscribers wake through the buffer's subscriber channels now. * fix(filer): make the gap metrics say what they count The crossing counter's help text described only the aggregated peer case, but give-ups increment it for local stalls too - a wedged local flush - which sends an operator chasing peer replication when the problem is the local store. Label it by scope and say both. The stalled gauge counted every park, including waits with nothing evicted and nothing at risk, while its help text promised evicted-but-unpersisted events; describe it as what it is, a count of subscribers parked on a gap. * test(filer): make the flush and stall tests assert what they claim The flush-subscriber rounds were vacuous: the probe entry sat above the round timestamps, so every round entry was collision-bumped and the stored watermark exceeded the local value each assertion compared against - the same bump mistake this test suite already made once. Put the probe below the rounds and guard each round against bumping, so a vacuous setup fails instead of passing. The stall-outcome test wrote the reporter's park epoch directly, bypassing the gauge Inc that gaveUp() later Decs - leaving the shared process gauge at -1 for every test that runs after it. Park through the real path, age the park by hand, release what the test holds, and assert the gauge lands back where it started. * fix(filer): finish the stream checks before marking it parked parkOnGap stamped the reporter before waitOnGap ran its instant done exits, so a bounded subscription completing inside a gap state was marked parked for the microsecond before done fired - a phantom gauge blip and a false "disconnected still behind" warning on every healthy completion. Fold waitOnGap into parkOnGap so the done exits run first and the park mark only ever covers a stream that actually waits. While the park owns its timer, back the retry off as the stall ages - 2s probes growing toward one a minute - since every retry re-reads the persisted log, and probing the store each 2s for 15 minutes per parked subscriber during the very outage that parked them makes the bad time worse. The subtest still named for the old fail-the-stream stall behavior goes with the merge. * fix(log_buffer): gate filer cursors against eviction under the read lock The subscribe loops checked the eviction watermark and then read memory, but a seal can land between the two: the read then served a sentinel cursor from the earliest retained window, silently skipping the window just evicted - the loss class this PR exists to make loud, surviving as a race. The only place the check is atomic with the serve decision is inside ReadFromBuffer, under the lock seals take to evict. Rather than put the policy back into the shared read path - which broke four message-queue readers last time - add a new sentinel offset that opts into it: EvictionGatedOffset reads inclusively exactly like -2, except below the watermark it is refused to disk. The filer loops stamp it on every cursor they hand the memory read; a refusal lands in the same gap machinery the loop-side check feeds, so the race collapses into the handled path. MQ cursors never carry it and keep master behavior byte-for-byte. * fix(filer): gate the aggregated gap on received, not bumped, timestamps The aggregated ring rewrites an out-of-order arrival to its head plus a nanosecond, so after any bump-heavy interval - a peer history replay following a restart is enough - its eviction watermark lives above every timestamp that exists on any peer's disk. Comparing a disk cursor against it parked subscribers that had in fact drained every peer's log: a 15-minute delivery freeze ending in a give-up skip and a false loss alarm, on a healthy cluster where master resumed instantly. Track a second watermark in the received timestamp space - the highest pre-bump timestamp among evicted entries - and gate the aggregated loop on that. Disk cursors and received timestamps are the same space, so the comparison means what it says: at or past it, every evicted entry's original was at or below the cursor, and everything flushed of them was already delivered. The bumped watermark keeps guarding the in-ring read gate, whose cursors live in ring space. The local buffer is untouched: it flushes its own bumped timestamps, so there the two spaces are one. * fix(filer): ship each log chunk once and stop echoing ref'd files inline Chunk mode duplicated data through two doors. Consecutive ref collections overlap by design - the scan backs off a flush interval to catch a spanning file, and a filer appends chunks to its newest file - so the same file was shipped again on every pass that re-listed it: the client re-downloaded its chunks, and a duplicated file mid-batch rewinds timestamps inside the client's per-filer merge, which reads each stream as sorted - transiently resurrecting deleted entries during catch-up. Track per subscription how many chunks of each file were shipped and send only the unsent suffix; state prunes with the scan window, so it holds a few files per filer. The second door was the entry fallback: when refs cannot advance the minute-named cursor, the pass streamed the ref'd file's tail inline, and the client applies inline events unfiltered - the same tail it already applied from chunks. Entry passes for chunk clients now advance the cursor without delivering; everything they skip is covered by the refs already sent or the deltas the next collection ships. * refactor(filer): one gap decision shared by both subscribe loops The post-disk gap tree - guard, drain, resolve, two parks, the re-arm - existed twice, differing only in buffer, watermark space, flush getter, park channels, and reason strings. Four rounds of review fixes have shown the copies drift the moment one is edited alone. gapPass now carries the five differences and the tree lives once; the loops shrink to a three-way switch between reading memory, restarting the pass, and ending the stream. * docs(filer): trim the gap-machinery comments to the why Several blocks had grown to ten-plus lines restating what the tests already pin or retelling one rationale at multiple sites. Keep the non-obvious why - the load-bearing flush ordering, the two timestamp spaces, the refusal-at-equality argument - in a few lines each. * fix(log_buffer): credit an entry's received timestamp to its own window The received-ts capture ran before the rollover check, so an append that sealed the previous window stamped its timestamp onto that window and then lost it in the reset of the new one. The eviction watermark this feeds broke both ways: the sealed window's value was inflated by an entry it does not contain - parking aggregated subscribers on gaps that were drained - and the entry's real window was deflated, proving gaps empty that still held its event on some peer's unflushed path. Credit the timestamp only after the entry lands, when its window is known. * fix(filer): rebase a shipped chunk suffix to logical offset zero A grown file's delta kept the chunks' original file offsets, but the client's chunk reader starts at logical zero and a list opening higher reads as instant EOF - a successfully empty replay, and since chunk clients no longer receive disk entries inline, the appended events were silently dropped. Clone the suffix chunks with offsets rebased to zero; the cut is record-aligned because each append is one uploaded chunk of whole entries, so the suffix decodes as a file of its own. * fix(filer): finish every chunk refs batch with a transition the client acts on Both chunk consumers buffer refs until a non-ref message arrives, so a source with historical logs and a quiet ring - a mount reconnecting after a filer restart is the common case - shipped its backlog and then went silent: the client sat on the refs until the next metadata mutation anywhere in the cluster. The disk step now ends every batch with the empty-notification marker the client already treats as a resume-cursor advance. The same step closes the inline replay: the cursor used to stay at the last file's minute name, so the memory read re-delivered the retained tail of a file the client had just read via chunks - T1..Tn applied twice. The advance-only entry read now runs on every chunk pass, moving the cursor to the true disk content end before memory is consulted. Ordering inside the pass is load-bearing: the entry read can outrun the shipped refs by a chunk appended between collection and read, and the transition timestamp becomes the client's refs filter - stamping it past unshipped content would silently drop that chunk's events on the next delta. The pass therefore re-ships the delta after the entry read, so the transition never exceeds shipped content. Bump-displaced aggregated entries can still arrive inline above the cursor with originals below it; that duplication is bounded and stays within the documented at-least-once residual. * fix(filer): prune ref state at the minute the scan actually stops at The collector compares file names at minute granularity while the prune used the exact-nanosecond scan bound, so for a cursor at 12:31:20 the 12-30 file was still collected but its sent state was already deleted - the next pass reshipped the whole file, re-creating the duplicate-refs class the state exists to prevent. Truncate the bound to the minute the file names live in. * fix(filer): derive the chunk cursor from the shipped refs themselves The advance-only entry read left the three positions that must agree in each other's blind spots. Its snapshot could trail the second delta's, so a chunk appended between them shipped events newer than the cursor and the memory pass sent them again. And it made the filer decode the tail range on every pass, serialized ahead of the client's own reads by the transition marker - re-introducing a slice of the replay work chunk mode exists to offload. Compute the cursor from the shipped set instead: the final entry timestamp of each filer's last shipped chunk, decoded once through the shared chunk cache. Refs coverage, transition marker, and memory start are then the same number by construction - nothing is decoded twice, nothing is dropped, and the per-pass server cost falls to one cached chunk decode per filer. The second delta and the once-per-position refs gate existed to patch the entry read's snapshot races, so both go with it; the range read survives only as a fallback for legacy chunks that do not decode standalone. * fix(filer): keep the chunk-cursor probe inside the shipped snapshot Three holes in the tail probe, all variations of stepping outside what was shipped. A permanently missing chunk failed the stream before the transition marker, so the client discarded its pending refs and reconnected to the same failure forever - blocking all later metadata behind one dead volume, where every other replay path (including the client's own reader) skips such chunks; the probe now walks back to the last readable chunk, and a filer with nothing readable simply contributes no cursor. The legacy fallback re-listed the logs after the refs were collected, so a concurrent append could push the range end over an unshipped chunk and the marker past events the client never received; it now streams the shipped chunk list itself, so no snapshot other than the shipped one is ever consulted. And a file selected before UntilNs can hold entries past it, which the client filters while still adopting the marker as its checkpoint - a later bounded request then skipped them; the marker is clamped to the bound. * fix(filer): make the cursor probe an exact mirror of the client's reader The probe answered from the server's view of the chunks; the marker's correctness depends on the client's. Its backward walk found the last readable chunk, but the client reads forward and stops at the first unreadable one, never resuming within a file - for readable, missing, readable the marker claimed the suffix the client never applied, losing those entries permanently. Keeping only each filer's final file ref discarded the progress of earlier readable files when that file was wholly missing, rewinding the marker to the start cursor. And a torn trailing size prefix - what a crashed writer leaves - failed the probe where the client reads a clean end, blocking the marker forever on data the client accepts. The probe is now shaped like the reader it answers for: per file the readable prefix, per filer the newest file with content, and no condition escapes as an error - understating the marker only re-ships, overstating loses events, and a probe failure must never block the transition the client is waiting on. Each rule is pinned by a test that fails against the previous shape. * fix(filer): judge chunk readability at the volumes, not the decode cache Two ways the probe's answer could drift from what the client experiences. A chunk this server decoded earlier stays warm in the shared cache after its volume dies, so the probe sailed past a chunk the direct-reading client stops at - marker beyond the unread suffix, entries lost. Every chunk now passes a volume lookup before the cache is consulted; the lookup rides the master client's in-memory map, so the probe stays cheap. And a probe stop was treated as harmless understatement, but the delta had already marked the whole ref sent: a transient server-side failure left the cursor stranded behind shipped content for the life of the connection, parking aggregated streams below the watermark for data the client already holds. The pass now rolls back the sent state of every ref above the file that answered the probe, so unreached refs re-ship and re-probe until the cursor gets there. Re-shipped entries at or below the client's checkpoint are filtered client-side, and batches are marker-separated, so a re-shipped file cannot rewind a merge mid-batch. * test(filer): end-to-end subscribe-loop harness and wire-contract tests Every escaped bug across this change's review rounds lived in an interaction the unit tests could not see: the loop state machine, the disk/memory handoff, or the server/client contract. The harness runs the real SubscribeLocalMetadata loop against a real leveldb-backed filer, faking only the volume layer behind the existing test hooks, and asserts the delivered stream itself. Eight scenarios, each pinning a class this change was reviewed for: the headline evicted-unflushed gap parks and then delivers in full; a ring that evicted nothing serves memory promptly; a backlog-to-live handoff with 1ms-adjacent timestamps across every boundary delivers exactly once; a flush-proven gap over vacuumed log files skips to the retained ring including a single-entry window; a bounded subscription terminates at its bound; a permanently wedged flush ends in the give-up skip with the stream still alive; and chunk mode is checked against the real client code - pb.ReadLogFileRefs applied to the shipped refs must cover everything the transition marker claims, with and without a dead volume in the middle. Validated by re-introducing three fixed bugs: the missing eviction guard delivers during the unproven gap, a 2ms cursor error at the handoff drops exactly one event, and resuming at rather than below the earliest retained window loses a single-entry window's sole event - each caught by the scenario built for it. The gap timing knobs become vars so parks run at test speed, a small filer hook swaps the volume-touching read functions, and a sender test pins the refs wire rules the client depends on: never batched, never an envelope, everything in order. * fix(filer): re-ship a partially read answering file, pin the probe's limits The sent-state rollback stopped at files newer than the one that answered the probe. When the answering file itself was only prefix-readable - a dead or transient chunk mid-file - its unread suffix stayed marked sent, and the next append advanced the cursor past it for good. The probe now reports whether the answering file was read through to its end, and a prefix-limited answer re-ships that file too; a torn tail counts as complete, since the client's read ends there as well. The rollback rules live in one predicate with a table test - files below a complete answer stay sent, because the client has moved past them and re-shipping cannot rewind its filter. Two test honesty fixes ride along. The loop harness derived its timestamp base from time.Now() per call, so expectations recomputed across a second boundary drifted by exactly one second; the base is now fixed per harness. And the probe's liveness boundary is pinned as a test instead of a comment: a volume lookup cannot see a dead needle or a stale location inside a resolvable volume, so a warm cache can answer past a chunk the client fails on - accepted because metadata log chunks die volume-at-a-time and the alternative is a real read per probe, which is what the probe exists to avoid. The test states the boundary so changing it is a decision, not an accident. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chris.lu@gmail.com>	2026-08-02 00:06:22 -07:00
Chris LuandGitHub	d867b6e739	log_buffer: bound the flush queue in bytes, not in copies (#10433) ... The queue holds sixteen sealed windows, which is a memory bound only while a window is BufferSize. An entry larger than that grows its window to fit, and the depth then multiplies straight through: sixteen queued copies of a 100 MB window is 1.6 GB of flush data alone. Account the queued bytes and make producers wait once they pass the ceiling the depth was chosen for. What is charged is the pooled slab rather than the window length, since mem.Allocate rounds up to a size class and the queue holds the whole slab. A window larger than the whole budget still goes through on its own, so an oversized entry is never stuck. Windows are admitted in the order they were sealed. A producer can now park here for seconds, and letting a later window overtake an earlier one would persist them out of order and walk lastFlushedOffset and lastFlushTsNs backwards. A window is copied into its slab under the write lock, before the reservation is taken, so a burst of concurrent oversized writers would each hold a full copy in hand while queueing up -- memory the budget never sees. Large writers wait for queue headroom before they take the lock, which throttles the burst; it does not bound it, since a writer that passes the check still seals unconditionally. Take any room in the queue before the shutdown escape, too: the window is already sealed by then, so dropping it loses records the caller was told were accepted. A shutdown that races a full queue can still drop one -- that predates this change and needs the flush loop's lifetime reworked. Size a grown window to the entry rather than to twice it: the extra room only bought space for a second oversized record in the same window, which doubles the flush copy and the snapshot taken of it. The overflow guard halved its bound for that doubled allocation, so raise it to match what is now allocated and what maxBufferSize documents.	2026-07-25 10:20:53 -07:00
Chris LuandGitHub	7b3462be6a	filer: stop an oversized metadata log flush from wedging the change feed (#10430) ... * filer: write the metadata log in pieces a volume server will accept A single oversized metadata event grows the log buffer past the volume server's fileSizeLimitMB, and the flush of that buffer is then rejected forever: the retry loop has no exit, so the blob at the head of the queue blocks every later flush and the metadata feed stalls until restart. Split the flushed buffer into BufferSize pieces, on record boundaries where possible so each piece still decodes on its own, and retry each piece separately so a partial success is not replayed. Log files are already read as a chunk stream, with a whole-file fallback when a chunk does not decode standalone, so a record may cross a piece boundary. * log_buffer: let go of a window array grown for an oversized entry An entry larger than BufferSize grows the window array to 2*size+4, and window arrays cycle through SealBuffer rather than being freed. One such entry therefore leaves every later window carrying its size, and currentSnapshotView allocates a snapshot as wide as the array on each window, so a few KB of metadata keeps paying for it. Drop the array when SealBuffer hands it back. Growth is on demand, so the next oversized entry just reallocates. * iceberg maintenance: store merged data files as chunks, not inline saveFilerFile had no size threshold, so compaction wrote whole merged parquet files -- hundreds of MB -- as Entry.Content. That puts the parquet bytes verbatim in the filer store and sends them through the metadata change log again as one event. Keep manifests and metadata JSON inline, upload anything larger to volume servers in chunks, assigning through the filer so the path's storage rules apply. * filer: follow the file size limit the volume servers report The starting piece size is a constant, so a cluster whose -fileSizeLimitMB is set below it would reject every piece and wedge just the same. The rejection names the limit, so take it from there and re-cut the rest of the flush to fit. Piece the buffer one at a time rather than up front, since the size can change partway through a flush.	2026-07-24 17:59:20 -07:00
Chris LuandGitHub	19ce7c0b6f	consolidate the duplicated transient-error classifiers onto util.IsTransientError (#10429) ... * util: match transient error messages case-insensitively, expose the message form The same condition reaches different layers capitalized differently -- a volume server relays its idle timeout as "I/O timeout" inside a JSON string -- and the callers that grew their own substring lists all lower-case first. Also split out IsTransientErrorMessage for the paths that carry only the text, such as the per-file status strings in a batch delete response, and pick up "no route to host" and "network is unreachable" from the gRPC classifier. * filersink: classify transient network errors through util.IsTransientError The local list caught i/o timeout, connection reset, and broken pipe but not connection refused, no such host, unexpected EOF, the syscall errnos, or the gRPC and S3 overload codes. Keep only the bare io.EOF case, which is transient here -- a truncated chunk read -- but a clean stream end elsewhere. * filer deletion: reuse util.IsTransientErrorMessage for the network patterns Six of the sixteen patterns were already covered. Keep the ones specific to this pipeline -- read-only volumes, lookup failures, backpressure -- and note why context cancellation stays retryable here: it decides whether to requeue the deletion, not whether to retry a call. * wdclient: fold the shared classifier into the volume lookup retry check The string tail duplicated the shared list and missed the syscall errnos and net.Error timeouts. Keep "connection" and "timeout", which are broader than the shared classifier on purpose: a volume lookup is a cheap read-only call.	2026-07-24 11:08:18 -07:00
Chris LuandGitHub	652273301e	filer sync: do not advance the sync offset past a failed event (#10424) ... * util: retry transient errors, not just the ones containing "transport" util.Retry only retried when the error string contained "transport", so a plain "read: connection reset by peer" from S3 got zero retries. Classify the error instead: net timeouts, connection resets, and the throttling and overload replies S3 and gRPC return are all worth another attempt, while a cancelled or expired context is not. * filer sync: hold the sync offset behind a failed event A sync job that returned an error was logged and forgotten, and the watermark advanced past it anyway. The offset is the durable resume point, so the event was never replayed: for filer.remote.sync that left the file present locally, absent on the remote, with no RemoteEntry and nothing to retry it. Pin the watermark at the oldest failed event. Later events keep flowing, but the persisted offset stays behind the failure, so a restart replays it.	2026-07-24 10:32:14 -07:00
3 people	490379bff3	Add codespell support with configuration and typo fixes (#10393) ... * Add GitHub Actions workflow for codespell on master * Add rudimentary codespell config * Tune codespell config: skip generated code, ignore camelCase, whitelist domain terms Add camelCase/PascalCase regex to ignore common Go/Rust/JS identifiers like allLocations, publishErr, ReadInside, FlushInterval. Also skip templ-generated *_templ.go files, and whitelist a handful of short/domain-specific words (visibles, fo, te, ser, bject, unparseable, keep-alives, tread, anc, ue) that show up as false positives across the tree. Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix ambiguous typos and protect false positives Fixes typos that codespell reports with multiple candidate suggestions (so `codespell -w` cannot auto-apply them), plus one inline pragma and one config entry to protect legitimate identifiers. Manual fixes (single correct answer chosen from context): - pattens -> patterns (5x) in filer/upload/shell flag help strings - finded -> found (2x) in tarantool storage.lua comment - spacify -> specify (2x) in helm chart values.yaml comment - wether -> whether in skiplist.go docstring - simpe -> simple in mq schema test case name False-positive protection: - Add `//codespell:ignore` next to `source GET's` (possessive of HTTP verb) in s3api_object_handlers_copy_stream.go - Whitelist `auther` in .codespellrc — it's a local variable meaning "authenticator" in weed/security/tls.go, not a typo of "author". Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Extend codespell ignore list: .git-meta path and thirdparty groupId Also skip `.git-meta` (scratch dir for commit messages that may contain typo words verbatim) and whitelist `thirdparty` — it appears as the literal Maven groupId `org.apache.hadoop.thirdparty` in hdfs3 poms and cannot be renamed. Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [DATALAD RUNCMD] Fix non-ambiguous typos with codespell -w Auto-applied fixes to the 44 remaining single-suggestion typos across docs, comments, log messages, tests, config, and one Java pom. === Do not change lines below === { "chain": [], "cmd": "uvx codespell -w", "exit": 0, "extra_inputs": [], "inputs": [], "outputs": [], "pwd": "." } ^^^ Do not change lines above ^^^ * Revert breaking codespell fixes; whitelist unknwon and atleast Two of the auto-applied `codespell -w` fixes were false positives that would break the build/tests: - go.mod: `github.com/unknwon/goconfig` is a real Go module path — the upstream author's GitHub handle is literally `unknwon`. Renaming to `unknown` would fail dependency resolution. - test/benchmark/fuse_db/bin/{sqlite_verify.py,run_mysql.sh,run_sqlite.sh}: `atleast` is a literal CLI mode value (a string constant compared and passed as a positional argument). Rewriting to `at least` splits it into two arguments and breaks the mode check. Reverted those files and whitelisted both words in .codespellrc so future runs won't re-suggest the same broken fixes. Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-07-22 14:38:06 -07:00
github-actions[bot]	875cd1f67e	4.40	2026-07-20 03:57:03 +00:00
Chris LuandGitHub	c015cc3939	generate vtproto marshalers for filer_pb and use them on the metadata log path (#10337) ... * generate vtproto marshalers for filer_pb and use them on the metadata log path Reflection-based proto.Unmarshal allocates a fresh message tree through reflect.New on every call. On the metadata subscription fan-out the same event is decoded once per subscriber, so reflect.New tops the decode churn under many mounts. Generate MarshalVT/UnmarshalVT/SizeVT for filer.proto (a separate filer_vtproto.pb.go, filer.pb.go untouched) and call them on the log entry marshal and the subscribe/replay decode paths. UnmarshalVT allocates message structs directly and copies byte and string fields, so it stays wire-compatible with proto.Unmarshal and preserves the non-aliasing the persisted-log cache depends on. For SubscribeMetadataResponse this cuts decode allocations 69 -> 50 and ~4.5us -> ~2.1us per event; the win scales with subscriber overlap. * marshal log entries directly into the buffer SizeVT is allocation-free and MarshalToSizedBufferVT writes into a pre-sized slice, so the log entry can be marshaled straight into logBuffer.buf. This drops the per-entry MarshalVT allocation and the follow-up copy on the write path. * expand vtproto benchmarks: marshal, decode, and marshal-into-buffer by chunk count Parametrize by nested-message count (chunks per event) and add encode + zero-alloc marshal-into-buffer benchmarks alongside the decode one, so the write-path win from MarshalToSizedBufferVT is measurable too. * keep proto.Unmarshal for metadata events to preserve UTF-8 validation UnmarshalVT skips proto3's UTF-8 validation of string fields, so a SubscribeMetadataResponse with an invalid-UTF-8 string (e.g. Directory "\xff") that proto.Unmarshal rejects would decode and reach path filtering and subscribers. Decode events with proto.Unmarshal again; UnmarshalVT stays on the log entry paths, whose only variable-length fields are bytes and so carry no UTF-8 constraint. Tests cover the codec difference and that a malformed event is skipped before delivery.	2026-07-15 02:32:05 -07:00
Chris LuandGitHub	8f29d0a91e	route log buffer flush copies through the shared slab pool (#10336) ... * route log buffer flush copies through the shared slab pool Each flush copied the sealed window into a bytes.Buffer drawn from a package-local sync.Pool. GC drains that pool, so once collections run often (e.g. under GOMEMLIMIT) most flushes miss and Write grows a fresh window-sized array, the dominant bytes.growSlice source under write load. Copy into a size-classed slab from weed/util/mem instead. Slabs are reused process-wide and returned to their exact size class after the flush, so variable-sized flushes across many partitions stop churning mismatched buffers. * guard nil slab in flush releaseMemory mem.Free(nil) resolves to the smallest slot pool and stores a zero-cap slice, so a later mem.Allocate could return nil and panic. The current call sites never pass nil, but the guard keeps a double release harmless.	2026-07-15 00:51:41 -07:00
Chris LuandGitHub	76ec1d8f0f	s3: accept raw semicolons in query strings (#10305) ... * s3: accept raw semicolons in query strings Go's url.ParseQuery drops any key=value pair containing a raw ';'. A presigned PUT that signs content-type carries X-Amz-SignedHeaders=content-type%3Bhost; when a client or proxy decodes the %3B, the parameter vanished and the upload failed with MissingFields, while AWS accepts the raw ';' as query data. Re-encode it before routing so the pair survives parsing and signature verification. * iam, iceberg: recover raw-semicolon query pairs on the other listeners The standalone IAM API verifies SigV4 with a canonical query recomputed from the parsed query, and Iceberg REST warehouse/parent values may legally contain ';'. Move the normalization middleware to util/http and attach it to both routers.	2026-07-10 12:17:35 -07:00
github-actions[bot]	db42bb4975	4.39	2026-07-10 04:08:14 +00:00
Chris LuandGitHub	fd5a6ff427	log_buffer: cap the shared-snapshot race test's heap (#10281) ... TestSharedSnapshotConcurrentIntegrity writes as fast as one core can marshal for two seconds, so its transient heap scales with host speed. On linux/386 a fast runner walks the heap into the 4GB address-space ceiling and the run dies with out of memory. Set a 1GB memory limit for the duration of the test so the GC paces the writer instead of the address space; the seal-and-recycle race being tested is unaffected.	2026-07-09 00:29:17 -07:00
Chris LuandGitHub	e873e671b6	filer: share one log-buffer window snapshot across all subscriber reads (#10267) ... * log_buffer: share one window snapshot across all subscriber reads Every in-memory read handed each subscriber a private pooled copy of the window it wanted, so N subscribers reading the same data cost N copies of up to 8MB each -- and slow consumers (grpc send backpressure) held those copies live for their whole iteration. With hundreds of mount subscribers that multiplied into gigabytes of live heap on the filer. Share the bytes instead of copying per reader: - Sealed windows get a lazily created GC-owned snapshot, made once by the first reader and handed out zero-copy to the rest. The snapshot travels with its window when SealBuffer shifts slots, so recycling the sealed array never invalidates it. - The current window keeps a shared snapshot of its append-only prefix buf[:pos], extended on demand; each byte is copied once per window (writer-rate-bound) instead of once per reader. At seal a fully extended prefix becomes the sealed window's snapshot. ReadFromBuffer now reports whether the returned buffer is a pooled copy (flush path) or a shared view that must not be released; the read loops only recycle pooled buffers. With 200 subscribers consuming at grpc pace over sealed and current windows, peak live heap drops from 5.2GB to 178MB. * log_buffer: clear released read buffer so a panic cannot double-free it The read loops release the previous iteration's pooled buffer and then call ReadFromBuffer. If that call panicked before reassigning bytesBuf, the deferred cleanup would put the same buffer into the pool a second time, letting two future readers share one backing array. Nil the pointer at the release site so the defer sees nothing to free.	2026-07-08 01:50:51 -07:00
Chris LuandGitHub	6413b774df	filer: cut metadata-subscription allocation churn (issue #10253) (#10260) ... * log_buffer: read only ts_ns during metadata-read binary search readTs ran a full proto.Unmarshal of each LogEntry just to compare timestamps during ReadFromBuffer's binary search, allocating fresh slices for the data and key byte fields on every probe. Under metadata-subscription fan-out this decode churn dominated allocations and drove heavy GC, inflating filer RSS well past the live heap. Scan the wire format for ts_ns (field 1) instead and skip the data/key payloads without copying them. readTs is now zero-alloc; the binary search no longer touches the event payloads at all. * log_buffer: alias event payloads instead of copying them on delivery The subscribe read loops decoded each LogEntry with a full proto.Unmarshal, allocating fresh slices for the data and key byte fields on every entry (protobuf consumeBytesNoZero). When many metadata subscribers each re-read the in-memory log window under a reconnect storm, that per-entry copy was a large share of allocation churn -- and for entries a subscriber filters out by prefix, the copied payload was never even looked at. Decode with a wire scan that points data/key at sub-slices of the source buffer instead of copying. The aliased slices are valid only for the duration of the eachLogEntryFn callback; every current subscriber either re-decodes the event into its own struct or hands it to a synchronous grpc Send, so none retain them. Per delivered entry the loop decode drops from 176 B / 2 allocs to zero.	2026-07-08 00:05:20 -07:00
Chris LuandGitHub	e521a2a7b9	mount: re-assign to a live volume when a write can't land (#10239) ... With replication 001 and one node down, ~2/3 of volumes have their replica on the dead node. The primary write lands locally but the replica forward fails, so the volume returns 500 "failed to write to replicas". #9744 made that upload fail fast so the client re-assigns, but the client retry never fired: the reassign gate matched a fixed list of error substrings that didn't include this one. The mount surfaced I/O error and dropped the chunk, leaving missing lines and null-byte gaps in an append workload while a node rebooted. Decide reassignment by HTTP status instead of matching message text. On the write path a volume server only 5xxs on a ReplicatedWrite failure (local disk, replica peer down, under-replication) — all of which a different volume dodges — so any 5xx reassigns; a no-response transport failure (the assigned target itself is down) reassigns too; a 4xx is a genuine client error and is surfaced. doUploadData tags its errors with the response status via uploadStatusError, and the gate moves from util.MultiRetry(errList) to util.RetryOnError(predicate). This drops the fragile substring list (and the message-prefix constants and guard test it needed); store_replicate.go is untouched.	2026-07-06 11:41:26 -07:00
github-actions[bot]	65dff4a492	4.38	2026-07-06 05:48:51 +00:00
Chris LuandGitHub	843210790e	volume: bound intra-cluster HTTP so an unresponsive peer can't hang reads and writes (#10229) ... A read or a replicated write to a volume server that is TCP-reachable but not answering -- one still loading its volumes after a restart, or reached over a stale keep-alive to a container that came back on a new IP -- blocked forever: the shared HTTP transport had a dial timeout but no response timeout. Add ResponseHeaderTimeout so a chunk read fails over to another replica and a replicated write fails fast for the client to retry, and IdleConnTimeout so pooled connections to a departed server are evicted instead of reused.	2026-07-04 10:44:38 -07:00
Chris LuandGitHub	39961ce5d7	util: don't let the activity timeout clobber externally-set conn deadlines (#10212) ... * util: don't let the activity timeout clobber externally-set conn deadlines util.Conn extended the connection deadline at the start of every Read/Write. net/http's server also sets deadlines directly on the same conn - abortPendingRead sets one in the past to interrupt the pending background read after each response. The activity extension raced with and silently overwrote that interrupt, leaving the read blocked (and the server's conn.serve goroutine stuck in abortPendingRead) until the full -idleTimeout (default 30s) expired. Wedged connections count as active, so the volume server's graceful HTTP drain waited out its whole 30s StopTimeout on shutdown - observed as weed mini taking ~30s to exit in the FUSE integration tests after any filer->volume traffic. Track externally-set deadlines, suspend the activity extension while one is in force, and serialize deadline updates with a mutex. Activity still extends both directions at once: a long write-only response must keep the read deadline alive too, or the server's background read would time out and cancel the in-flight request. * util: extend read/write deadlines independently when one side is external A server-configured WriteTimeout keeps an external write deadline in force for the whole request, which previously suspended the activity extension entirely - leaving the read deadline stale from before the request and letting net/http's background read time out mid-response. Extend each direction independently instead.	2026-07-02 17:03:00 -07:00
github-actions[bot]	c06a2dca87	4.37	2026-06-29 06:45:55 +00:00
github-actions[bot]	d0b90d29eb	4.36	2026-06-25 05:09:40 +00:00
Chris LuandGitHub	95427b5573	security: add BearerPrefix constant for Authorization headers (#10101) ... Introduce security.BearerPrefix ("Bearer ", RFC 6750) and use it everywhere an "Authorization: Bearer <token>" header is constructed, replacing the scattered "BEARER "/"Bearer " string literals. SeaweedFS matches the scheme case-insensitively when parsing (security.GetJwt), so behavior is unchanged; this removes the magic string and settles the casing on the standard form. The parser's upper-case comparison stays as is on purpose.	2026-06-24 19:36:42 -07:00
patrickandGitHub	3e2c637858	util: trim minFreeSpace values before parsing (#10083)	2026-06-24 09:03:38 -07:00
4 people	faa8c3963b	fix(chunk_cache): close data/index files on initialization error (#10057) ... * fix(chunk_cache): close data/index files on initialization error * chunk_cache: assign outer err on the .dat open path The error-path defer keys off the function-level err, but the .dat OpenFile used := and shadowed it, so that path relied on nothing being open yet rather than the cleanup invariant. Assign the outer err so every error return is uniform. * chunk_cache: verify descriptor closure on POSIX, not just Windows os.Remove succeeds on open files on Linux/macOS, so the removal check only proved closure on Windows. Compare the open-fd count before and after the failed load; gate the removal check to Windows. --------- Co-authored-by: Contributor <contributor@example.com> Co-authored-by: Chris Lu <chris.lu@gmail.com>	2026-06-23 01:49:35 -07:00
5 people	11b7b7247f	util: support IPv6 host port parsing (#10046) ... * util: support IPv6 host port parsing * Update weed/util/parse.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2026-06-22 20:32:43 -07:00
Chris LuandGitHub	16ba8af0b7	util/http: lazily init the global HTTP client to fix admin metrics nil panic (#10044) ... util/http: lazily init the global HTTP client GetGlobalHttpClient returned a nil client until InitGlobalHttpClient ran, which only happens in weed.go's main. Anything that starts a command in-process bypasses that: the admin server's metrics goroutine seeds a dashboard sample on startup, reaching fetchPublicUrlMap -> GetGlobalHttpClient().Do, and nil-derefs the receiver in GetHttpScheme. Init the client on first Get via sync.Once so it is never nil regardless of the startup path. InitGlobalHttpClient keeps its eager-init role through the same Once.	2026-06-22 10:20:02 -07:00
Chris Lu	eb93976166	4.35	2026-06-22 00:24:50 -07:00
4 people	18868e5204	fix(mount): run entry invalidations off the meta-cache apply loop (#10002) ... * fix(mount): run entry invalidations off the meta-cache apply loop The apply loop ran invalidateFunc inline, which acquires the open file handle's lock in fhLockTable. Meanwhile flushMetadataToFiler holds that same fh lock and then waits on the apply loop (applyLocalMetadataEvent). When both target the same open file concurrently, the loop blocks on the fh lock while the lock holder blocks on the loop: an ABBA deadlock that backs up every later readdir/flush and hangs the mount. Fix: dispatch entry invalidations to a dedicated FIFO worker goroutine so the apply loop never blocks on locks held by goroutines waiting on it. Adds a regression test reproducing the interleaving. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * perf(mount): update invalidate counter once per batch Run the batch's invalidateFunc calls without re-taking invalidateMu per item, then bump invalidateProcessed and broadcast once after the loop. WaitForEntryInvalidations only needs the count to reach its target and a batch always completes together, so the per-item lock + broadcast was wasted work. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * mount: extract the invalidate worker into util.AsyncBatchWorker The apply loop's off-thread entry-invalidation queue was a one-off mutex + cond + slice + counters living inside MetaCache. Pull it out as a generic unbounded FIFO worker so the deadlock-avoidance contract (never block the producer, drain on shutdown, wait-for-quiesce) lives in one place. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Chris Lu <chris.lu@gmail.com>	2026-06-19 09:19:35 -07:00
Chris Lu	c6cf5a5bd7	4.34	2026-06-14 22:41:44 -07:00
Chris LuandGitHub	1e858d8af0	fix(ec): make ec.decode write-path crash-safe and atomic (#9949) ... * fix(ec): check decode .idx writes and fsync decoded .dat/.idx WriteIdxFileFromEcIndex silently dropped io.Copy and Write errors, so a short or failed write of the reconstructed .idx went unnoticed and the caller proceeded to delete the source EC shards. Propagate those errors. Also fsync the decoded .dat and .idx before returning, so the bytes are durable before the shards that produced them are removed cluster-wide. Mirror the .idx fsync into the Rust volume server (its .dat already syncs and its writes already propagate errors). * fix(ec): publish decoded .dat/.idx atomically via temp file and rename WriteDatFile and WriteIdxFileFromEcIndex wrote in place at the final name with O_TRUNC. A crash mid-write left a truncated .dat/.idx at the final name beside the still-present EC shards; on restart that partial file could be mounted as the live volume even though the shards held the real data. Write to a .tmp file, fsync it, then rename into place and fsync the directory, so the final name is only ever absent or complete. A failed decode removes its own temp file rather than leaking it. Add util.FsyncDir as the shared directory-fsync primitive and reuse the Rust volume server's fsync_dir for the mirrored change. * fix(ec): propagate .ecj read errors in the Rust decoder Path::exists returned false for any error (permission denied, transient IO), silently skipping the deletion journal and resurrecting deleted needles as live. Read the journal directly and treat only NotFound as absent, propagating other errors. The Go decoder already behaves this way (FileExists returns false only for IsNotExist, then the open surfaces other errors). * fix(ec): remove rename destination on Windows in the Rust decoder publish std::fs::rename does not replace an existing file on every Windows version. Remove the destination first under a Windows guard before the atomic publish rename, matching the compaction commit path.	2026-06-13 21:26:07 -07:00
5 people	5468707289	fix(util): ignore comment only sql input (#9933) ... * fix(util): ignore comment only sql input Problem: sqlutil.SplitStatements strips SQL comments while scanning, but when no statements remain it falls back to returning the original query. Inputs that contain only comments are therefore reported as executable SQL statements. Root cause: The no-statements fallback did not distinguish a real single statement from input that had been fully removed by comment filtering. Fix: Remove the original-query fallback and return an explicit empty slice when scanning produces no statements. Reproduction: env GOCACHE=/private/tmp/seaweedfs-go-cache go test ./weed/util/sqlutil -run TestSplitStatements -count=1 failed before the fix because comment-only inputs returned the comment text as a statement. Validation: gofmt -w weed/util/sqlutil/splitter.go weed/util/sqlutil/splitter_test.go; env GOCACHE=/private/tmp/seaweedfs-go-cache go test ./weed/util/sqlutil -run TestSplitStatements -count=1; env GOCACHE=/private/tmp/seaweedfs-go-cache go test ./weed/util/sqlutil -count=1; git diff --check; git diff --cached --check. Duplicate check: Searched /private/tmp/seaweedfs-codex0610-old-branch-index.tsv and existing tests for sqlutil, SplitStatements, comments, and comment-only. Old PostgreSQL query branches cover malformed wire frames and SQL engine numeric parsing, not comment-only statement splitting. Co-authored-by: Codex <noreply@openai.com> * Update weed/util/sqlutil/splitter.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2026-06-12 10:10:27 -07:00
Chris Lu	55010be19b	4.33	2026-06-11 00:52:31 -07:00
Bruce ZouandGitHub	1dd292fb84	batch drain delta heartbeat messages (#9914)	2026-06-10 13:33:45 -07:00
Chris LuandGitHub	c2271d59bb	log_buffer: stop dumping the whole log entry on callback errors (#9919) ... The eachLogDataFn error path printed the full LogEntry proto. For an entry carrying a large chunk manifest that is hundreds of KB of escaped bytes in a single log line, burying the actual error -- often just a subscriber disconnect -- at the very end. Log the key, timestamp, offset and data size instead.	2026-06-10 12:47:35 -07:00
Chris LuandGitHub	7bf2dfc9ab	Bound the metadata-log flush queue (#9907) ... * Bound the metadata-log flush queue A stalled flush, e.g. slow volume servers under a reconnect storm, let up to 256 queued 8MB buffer copies pin two gigabytes per log buffer while producers kept filling the queue. Cap the queue at 16 so a sustained stall backpressures writers instead of growing the heap. The flush goroutine never feeds back into the buffer (system-log paths skip event notification), so blocked producers cannot deadlock the consumer. * Don't drop a force-flushed buffer on a full queue ForceFlush enqueued with a two-second timeout, but by then the live buffer was already sealed and reset, so a timed-out send silently lost the copy. Block until the flush is queued; the wait for completion stays bounded since the data is durable once the flush loop drains it. * Never close the flush channel ShutdownLogBuffer closed flushChan while producers could still be blocked sending into it, which panics. Terminate loopFlush with a nil sentinel instead, so the channel is never closed, and give every producer-side send a shutdown escape so none parks forever once the flush loop exits. Everything queued before the sentinel still drains, preserving IsAllFlushed semantics. * Copy the shutdown flush under the buffer lock Every other copyToFlush call site holds the lock; the shutdown path read the live buffer unlocked while producers could still be appending.	2026-06-10 10:57:30 -07:00
5 people	a9e4995d76	fix(http): accept no content delete responses (#9893) ... * fix(http): accept no content delete responses Problem: util/http.Delete reports an error for a successful HTTP 204 No Content response. Root cause: Delete only treats 200 OK, 202 Accepted, and 404 Not Found as non-error responses, omitting the standard 204 status commonly returned by DELETE endpoints. Fix: Include http.StatusNoContent in the Delete success status set. Reproduction: go test ./weed/util/http -run TestDeleteTreatsNoContentAsSuccess -count=1 fails before the fix with an empty error for a 204 response. Validation: go test ./weed/util/http -run TestDeleteTreatsNoContentAsSuccess -count=1; go test ./weed/util/http -count=1; git diff --check; git diff --cached --check Co-authored-by: Codex <noreply@openai.com> * Update weed/util/http/http_global_client_util_test.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2026-06-09 11:45:14 -07:00
7y-9andGitHub	7bbd28634a	fix(util): return full uint64 randomness (#9864) ... Problem: RandomUint64 generated eight random bytes but returned int32, truncating the value before mount file and directory handles converted it to uint64. This reduced handle entropy to 32 bits and produced sign-extended handle values.\n\nRoot cause: the helper cast BytesToUint64 to int32 and exposed int32 as its return type.\n\nFix: make RandomUint64 return uint64 and return the full BytesToUint64 result.\n\nReproduction: go test ./weed/util -run TestRandomUint64ReturnsUint64 -count=1 failed before the fix because RandomUint64() had kind int32.\n\nValidation: gofmt -w weed/util/bytes.go weed/util/bytes_test.go; git diff --check; go test ./weed/util -run TestRandomUint64ReturnsUint64 -count=1; go test ./weed/util -count=1; go test ./weed/mount -count=1; git diff --cached --check	2026-06-08 22:07:24 -07:00
Chris Lu	78da9572ae	4.32	2026-06-07 23:37:57 -07:00
Chris LuandGitHub	be7f417a03	ip.bind: bind outbound connections to the configured address (#9834) ... * ip.bind: bind outbound connections to the configured address -ip.bind only governed listeners; outbound gRPC and HTTP connections let the OS pick the source IP, which may not even be able to reach the target. Mirror the bind address into a process-global source address and apply it to outbound TCP dials: the gRPC context dialer, the per-client HTTP transports, and the default transport. Loopback targets and unix sockets keep the OS-chosen source so same-host traffic still works. * ip.bind: first-write-wins source IP, skip on address-family mismatch Make SetOutboundLocalIP first-write-wins so a `weed server` component's own bind setting (run in its goroutine) can't clobber the process-wide source address the top-level -ip.bind already established for the other components. Skip source binding when the target is a literal IP of a different family than the bind address, since forcing a mismatched source fails the dial.	2026-06-05 12:44:21 -07:00
Chris LuandGitHub	ab7be7867d	security: hot-reload JWT signing keys on SIGHUP (#9826) ... * security: reload JWT signing keys on SIGHUP Signing keys were read once in the server constructors and never refreshed. After a key rotation (Secret update, divergent reads) the in-memory key stayed stale and every request kept failing "wrong jwt" until the affected process was restarted. Add Guard.UpdateSigningKeys and call it from the master, volume and filer reload paths and the s3 reload hook, next to the existing whitelist refresh. Make the global chunk-read JWT cache reloadable via an atomic swap, and register the master's Reload with grace.OnReload -- it was never wired, so the master ignored SIGHUP entirely. Mirror the same refresh in the Rust volume server's SIGHUP handler. * security: swap signing keys behind an atomic pointer Addresses review feedback on the in-place key swap: SigningKey is a []byte, so reassigning the Guard fields while a request handler reads them is a data race that can tear the multi-word slice header and read out of bounds. Hold the four signing-key fields in an immutable signingConfig snapshot behind atomic.Pointer; UpdateSigningKeys swaps the whole pointer, so a reader sees either the old keys or the new ones. Reads go through new SigningKey/ExpiresAfterSec/ReadSigningKey/ReadExpiresAfterSec accessors. The Rust guard is already safe: every read and the SIGHUP write go through the shared RwLock<Guard>. * security: fold whitelist + auth state into the atomic snapshot Review follow-up. UpdateSigningKeys still wrote isWriteActive while the request path read it (and the whitelist maps) unsynchronized, so a SIGHUP under load could expose an inconsistent mix of activation bits and whitelist contents. Move all hot-reloadable Guard state -- keys, expirations, whitelist, and the activation flags -- into a single immutable guardState swapped behind one atomic.Pointer. The Update* methods take a small mutex to serialize the read-modify-write; readers stay lock-free. The concurrency test now also rotates the whitelist and probes IsWhiteListed under -race. Also read each signing key once per branch in the volume/filer JWT auth checks, so a reload landing mid-check can't take the allow-fast-path after auth was enabled or verify against a different key than the branch saw.	2026-06-04 22:26:08 -07:00
Chris LuandGitHub	f711868fb6	fix(log_buffer): re-check buffer before bailing with ResumeFromDiskError (#9804) ... ReadFromBuffer and HasData() take the read lock separately, so a write that lands between them can make a subscriber which just read a momentarily empty buffer return ResumeFromDiskError even though the data is now servable from memory. Re-read under a fresh lock and only bail when the position is genuinely behind the in-memory window (flushed to disk); otherwise loop back and read it.	2026-06-02 21:37:15 -07:00
7y-9andGitHub	38a47d1dd3	fix(http): check delete request errors before auth (#9784) ... Explain: - problem: Delete and DeleteProxied could panic on malformed URLs when a JWT was provided. - root cause: maybeAddAuth was called before checking the error returned by http.NewRequest, so req could be nil. - fix: return the request construction error before adding the Authorization header. - validation: go test ./weed/util/http -run 'TestDelete(ReturnsInvalidRequestErrorBeforeAddingAuth|ProxiedReturnsInvalidRequestErrorBeforeAddingAuth)' -count=1; git diff --check	2026-06-02 00:41:17 -07:00
Chris Lu	2a46d457ac	4.31	2026-06-01 23:32:04 -07:00
7y-9andGitHub	5ea75dcc67	fix(http): handle invalid gzip stream errors (#9767) ... * fix(http): handle invalid gzip stream errors Explain: - problem: ReadUrlAsStream could panic when a response claimed gzip encoding but the body was not a valid gzip stream. - root cause: the gzip reader error was ignored and a nil reader was deferred and read from. - fix: return the gzip.NewReader error before registering Close or reading. - validation: go test ./weed/util/http -run TestReadUrlAsStreamReturnsGzipReaderError -count=1; git diff --check. * test: avoid closing shared global HTTP client in unit test	2026-06-01 12:21:19 -07:00
Chris Lu	34be9170f0	4.30	2026-05-30 10:52:32 -07:00
Chris LuandGitHub	65d557cbb0	fix(util): guard BytesToUint{16,32,64} against short input (#9713) ... * fix(util): guard BytesToUint{16,32,64} against short input length is computed as uint, so length-1 on an empty slice underflows to MaxUint and the loop indexes b[0] on a zero-length slice. BytesToUint16 also indexed b[0]/b[1] with no length check. All call sites today gate the slice length explicitly, so this hardens the API for new callers rather than fixing a live crash. Return 0 on short input, matching the existing variable-length contract. * BytesToUint16: match variable-length contract of the 32/64 helpers A 1-byte slice should return uint16(b[0]) rather than 0, matching how BytesToUint32 and BytesToUint64 treat short input.	2026-05-27 11:29:01 -07:00
Chris Lu	1355c7a102	4.29	2026-05-25 22:41:25 -07:00
Chris Lu	adfd731bb8	4.28	2026-05-21 17:16:32 -07:00
Chris LuandGitHub	2c2b2d4d3e	chore(skiplist): remove unused NameList/NameBatch implementation (#9603) ... NameList, NameBatch and their serde were an earlier in-memory directory batch implementation. The redis3 filer store uses its own ItemList backed by Redis sorted sets, so these types had no production callers (NameList only via its own test, LoadNameList none at all). Drop them and the now-orphaned NameBatchData proto message, regenerating skiplist.pb.go with the repo-standard protoc-gen-go v1.36.6.	2026-05-21 02:30:49 -07:00