mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-21 14:46:58 +00:00
chrislu
2bc29f93d3
Phase 6: Root Cause Debugging - Broker Disk Read Path Added extensive logging to trace disk read failures: - FetchMessage: Logs every read attempt with full details - ReadMessagesAtOffset: Tracks which code path (memory/disk) - readHistoricalDataFromDisk: Logs cache hits/misses - extractMessagesFromCache: Traces extraction logic Changes: - broker_grpc_fetch.go: Added CRITICAL detection for empty reads - log_read_stateless.go: Comprehensive PATH and state logging Test Results: - 87.9% delivery (consistent) - FOUND THE BUG: Cache hit but extraction returns empty! Root Cause Identified: [DiskCache] Cache HIT: cachedMessages=572 [StatelessRead] WARNING: Disk read returned 0 messages The Problem: - Request offset 1572 - Chunk start: 1000 - Position in chunk: 572 - Chunk has messages 0-571 (572 total) - Check: positionInChunk (572) >= len(chunkMessages) (572) → TRUE - Returns empty! This is an OFF-BY-ONE ERROR in extractMessagesFromCache: The chunk contains offsets 1000-1571, but request for 1572 is out of range. The real issue: chunk was only read up to 1571, but HWM says 1572+ exist. Next: Fix the chunk reading logic or offset calculation