So far commitlog was using contiguous buffers for storing the data that is about to be written to disk. It was able to coalesce small writes so that multiple small mutations would use the same buffer, but if a muation was large the commitlog would attempt to allocate a single, appropriately large buffer. This excessively stresses the memory allocator and may cause memory fragmentation to become an issue. The solution is to use fixed-size buffers of 128 kB, which is the standard buffer size in Scylla and keep large values fragmented.
2.1 KiB
2.1 KiB