scylladb

mirror of https://github.com/scylladb/scylladb.git synced 2026-05-12 19:02:12 +00:00

Files

Asias He 9b5585ebd5 range_streamer: Stream 10% of ranges instead of 10 ranges per time

If there are a lot of ranges, e.g., num_tokens=2048, 10 ranges per
stream plan will cause tons of stream plan to be created to stream data,
each having very few data. This cause each stream plan has low transfer
bandwidth, so that the total time to complete the streaming increases.

It makes more sense to send a percentage of the total ranges per stream
plan than a fixed ranges.

Here is an example to stream a keyspace with 513 ranges in
total, 10 ranges v.s. 10% ranges:

Before:
[shard 0] range_streamer - Bootstrap with 127.0.0.1 for
keyspace=system_traces, 510 out of 513 ranges: ranges = 51
[shard 0] range_streamer - Bootstrap with ks for keyspace=127.0.0.1
succeeded, took 107 seconds

After:
[shard 0] range_streamer - Bootstrap with 127.0.0.1 for
keyspace=system_traces, 510 out of 513 ranges: ranges = 10
[shard 0] range_streamer - Bootstrap with ks for keyspace=127.0.0.1
succeeded, took 22 seconds

Message-Id: <a890b84fbac0f3c3cc4021e30dbf4cdf135b93ea.1520992228.git.asias@scylladb.com>

2018-03-14 10:12:12 +02:00

boot_strapper.cc

storage_service: Use the new range_streamer interface for bootstrap

2017-08-07 16:31:47 +08:00

boot_strapper.hh

Fix pre-ScyllaDB copyright statements

2016-04-08 08:12:47 +03:00

byte_ordered_partitioner.cc

Use abort() instead of assert + throw in unreachable code

2017-09-03 11:07:27 +03:00

byte_ordered_partitioner.hh

dht: extend i_partitioner::next_token_for_shard()