mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-17 04:36:50 +00:00
chrislu
c5634470ed
This commit implements async disk I/O fallback to handle cases where: 1. Data is flushed from memory before consumers can read it (CI issue) 2. Consumers request historical offsets not in memory 3. Small LogBuffer retention in resource-constrained environments Changes: - Add readHistoricalDataFromDisk() helper function - Update ReadMessagesAtOffset() to call ReadFromDiskFn when offset < bufferStartOffset - Properly handle maxMessages and maxBytes limits during disk reads - Return appropriate nextOffset after disk reads - Log disk read operations at V(2) and V(3) levels Benefits: - Fixes CI test failures where data is flushed before consumption - Enables consumers to catch up even if they fall behind memory retention - No blocking on hot path (disk read only for historical data) - Respects existing ReadFromDiskFn timeout handling How it works: 1. Try in-memory read first (fast path) 2. If offset too old and ReadFromDiskFn configured, read from disk 3. Return disk data with proper nextOffset 4. Consumer continues reading seamlessly This fixes the 'offset 0 too old (earliest in-memory: 5)' error in TestOffsetManagement where messages were flushed before consumer started.