Add a segment_set member to replica::compaction_group that manages the
logstor segments that belong to the compaction group, similarly to how
it manages sstables. Add also a separator buffer in each compaction
group.
When writing a mutation to a compaction group, the mutation is written
to the active segment and to the separator buffer of the compaction
group, and when the separator buffer is flushed the segment is added to
the compaction_group's segment set.
add a write gate to write_buffer. when writing a record to the write
buffer, the gate is held and passed back to the caller, and the caller
holds the gate until the write operation is complete, including
follow-up operations such as updating the index after the write.
in particular, when writing a mutation in logstor::write, the write
buffer is held open until the write is completed and updated in the
index.
when writing the write buffer to the active segment, we write the buffer
and then wait for the write buffer gate to close, i.e. we wait for all
index updates to complete before proceeding. the segment is held open
until all the write operations and index updates are complete.
this property is useful for correctness: when a segment is closed we
know that all the writes to it are updated in the index. this is needed
in compaction for example, where we take closed segments and check
which records in them are alive by looking them up in the index. if the
index is not updated yet then it will be wrong.
initial implementation of the separator. it replaces "mixed" segments -
segments that have records from different groups, to segments by group.
every write is written to the active segment and to a buffer in the
active separator. the active separator has in-memory buffers by group.
at some threshold number of segments we switch the active segment and
separator atomically, and start flushing the separator.
the separator is flushed by writing the buffers into new non-mixed
segments, adding them to a compaction group, and frees the mixed
segments.
add segment generation number that is incremented when the segment is
reused, and it's written to every buffer that is written to the segment.
this is useful for recovery.
initial implementation of the logstor storage engine for key-value
tables that supports writes, reads and basic compaction.
main components:
* logstor: this is the main interface to users that supports writing and
reading back mutations, and manages the internal components.
* index: the primary index in-memory that maps a key to a location on
disk.
* write buffer: writes go initially to a write buffer. it accumulates
multiple records in a buffer and writes them to the segment manager in
4k sized blocks.
* segment manager: manages the storage - files, segments, compaction. it
manages file and segment allocation, and writes 4k aligned buffers to
the active segment sequentially. it tracks the used space in each
segment. the compaction finds segment with low space usage and writes
them to new segments, and frees the old segments.
