It is used when a stream_transfer_task sends all the mutations inside
the messages::outgoing_file_message's it contains to notify the remote
peer this stream_transfer_task is done.
Instead of playing the game of casting between stream_event and derived
class. We overload handle_stream_event with derived stream_event class.
virtual void handle_stream_event(session_complete_event event) {}
virtual void handle_stream_event(progress_event event) {}
virtual void handle_stream_event(session_prepared_event event) {}
Also, make the virtual function non pure virtual, so user can override
the interested event only without defining all of the three.
It is used by IncomingFileMessage's deserialize to read from network and
write int sstable. In urchin, we use messaging service, incoming
mutation is handler within the STREAM_MUTATION handler. No need for
stream_reader.
In Origin, it is used by OutgoingFileMessage's serialize function to
write given section of the SSTable to network. In urchin, we send
mutaion directly in stream_transfer_task::start(). We can kill
stream_write now.
This patch saves almost 20 GB (!!) of disk space in Urchin's build
directory, as well as a lot of memory during the link phase of the
build (which can be noticable on low-memory machines which leads to
slow swapping).
Because of C++'s extremely lengthy mangled names, and extremely numerous
functions, the debugging information generated for Seastar code is absurdly
large, and added to every single executable we generate. This is most
noticable in tests - we currently have over 30 tests (with hopefully much
more to come), each compiled into a separate excutable with its own copy of
all this debug information. Many of these executables are half a gigabyte,
each!
So this patch creates all test executables - whether debug or release mode -
stripped. When a user encounters a failing test he wants debug information
for (for gdb or the sanitizer), he can trivially relink it unstripped,
with a command like:
ninja build/release/tests/urchin/sstable_test_g
note the added "_g". This links the already existing object files (which
still have their debug information, which takes just a fraction of a second.
On my machine, this patch reduces the Urchin built tests from about
27 GB to 8.1 GB. The build/release/tests directory drops from 10 GB to
just 0.6 GB! The build/debug/tests directory is still huge (7.5 GB),
although still smaller than what it was (17 GB). This remaining hugeness
is not because of the debug information, but because of the undefined-
behavior sanitizer (-fsanitize=undefined), which unfortunately adds a
huge data segment to each executable and I still don't know how to improve
on that. Nevertheless, it's still a significant reduction in space and will
be even more important as we write more tests.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
That's helpful for the purpose of testing, and leveled compaction may
also end up using size-tiered compaction strategy for selecting
candidates.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
If old average is equals to new average, then we would remove
new average entry. That's totally wrong.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Booting a bootstrapped node had a race condition between setting and
advertising the state as boot strapped (call to
storage_service::bootstrap) and setting and advertising the
state as normal (call to storage_service::set_tokens) - as such a node
could get into a state in which it was "stuck" in bootstrap mode.
Following this patch you must wait for 5 seconds to have the cluster in
a stable state.
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
Size-tired strategy basically consists of creating buckets with sstables
of nearly the same size.
Afterwards, it will find the most interesting bucket, which size must be
between min threshold and max threshold. Bucket with the smallest average
size is the most interesting one.
Bucket hotness is also considered when finding the most interesting bucket,
but we don't support this yet.
We are also missing some code that discards sstable based on its coldness,
i.e. hardly read.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
compact_all_sstables is about selecting all available sstables
for compaction and executing a compaction code on them.
This compaction code was moved to a more generic function called
compact_sstables, which will compact a list of given sstables.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Returns the sum of the size of all sstable components.
It will be used by size-tiered strategy.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
