Files
seaweedfs/weed/command
Chris Lu eaf561e86c perf(s3): add optional shared in-memory chunk cache for GET (#9069)
Adds the -s3.cacheCapacityMB flag (default 0, disabled) that attaches
an in-memory chunk_cache.ChunkCacheInMemory to the server-wide
ReaderCache introduced in the previous commit. When enabled,
completed chunks are deposited into the shared cache as they are
downloaded, so concurrent and repeat GETs of the same object hit
memory instead of re-fetching chunks from volume servers.

When 0 (the default) the shared ReaderCache still runs — it just
attaches a nil chunk cache, so behaviour matches the previous commit
exactly. No behaviour change for clusters that don't opt in.

Disk-backed TieredChunkCache was evaluated and rejected: its
synchronous SetChunk writes regressed cold reads ~12x on loopback
because the chunk fetchers block on local disk I/O that is *slower*
than the TCP volume-server fetch it is supposed to accelerate.
Memory-only avoids that.

Flag registered in all four S3 flag sites (s3.go, server.go,
filer.go, mini.go) per the comment on command.S3Options. The chunk
size used to convert CacheSizeMB → entry count is encapsulated in
the s3ChunkCacheChunkSizeMB constant so it's easy to grep and
revisit if the filer default chunk size changes.

Measured on weed mini + 1 GiB random object over loopback, single
curl on a presigned URL:

    cacheCapacityMB=0 (off):  cold ~2900, warm ~2900 MB/s
    cacheCapacityMB=4096:     cold ~2790, warm ~5050 MB/s (+70%)
2026-04-14 09:24:35 -07:00
..
2026-02-20 18:42:00 -08:00
2026-02-20 18:42:00 -08:00
2026-04-10 17:31:14 -07:00
2026-04-10 17:31:14 -07:00
2026-02-20 18:42:00 -08:00
2026-04-10 17:31:14 -07:00
2025-10-13 18:05:17 -07:00
2026-02-20 18:42:00 -08:00
2025-12-14 16:02:06 -08:00
2026-03-15 09:44:14 -07:00
2026-02-20 18:42:00 -08:00
2026-02-20 18:42:00 -08:00
2025-10-13 18:05:17 -07:00
2019-11-28 18:44:27 -08:00
2025-07-02 18:03:17 -07:00
2026-04-10 17:31:14 -07:00
2026-02-20 18:42:00 -08:00