mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-23 10:00:35 +00:00
fdefee696e
" Currently we keep the entries in a circular_buffer, which uses a contiguous storage. For large partitions with many promoted index entries this can cause OOM and sstable compaction failure. A similar problem exists for the offset vector built in write_promoted_index(). This change solves the problem by serializing promoted index entries and the offset vector on the fly directly into a bytes_ostream, which uses fragmented storage. The serialization of the first entry is deferred, so that serialization is avoided if there will be less than 2 entries. Promoted index is not added for such partitions. There still remains a problem that large-enough promoted index can cause OOM. Refs #4217 Tests: - unit (release) - scylla-bench write Branches: 3.0 " * tag 'fix-large-alloc-for-promoted-index-v3' of github.com:tgrabiec/scylla: sstables: mc: writer: Avoid large allocations for maintaining promoted index sstables: mc: writer: Avoid double-serialization of the promoted index