Aside from being the obviously correct thing to do, not having this will force us
to manually adjust num_tokens when running our sstables into Cassandra.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
It needs to access the non-existent "DatabaseDescriptor". Do as we have been doing,
and just pass the database object instead.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Error reported by debug mode when running sstable test
Solution is to use unaligned cast.
dht/murmur3_partitioner.cc:67:25: runtime error: load of misaligned
address 0x6030000478fc for type 'const long int', which requires 8
byte alignment
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
We need a container which can be used with compacting
allocators. "bytes" can't be used with compacting allocator because it
can't handle its external storage being moved.
Make sharding partitioner-specific, since different partitioners interpret
the byte content differently.
Implement it by extracting the shard from the most significant bits, which
can be used to minimize cross shard traffic for range queries, and reduces
sstable sharing.
range::is_wrap_around() and range::contains() rely on total ordering
on values to work properly. Current ring_position_comparator was only
imposing a weak ordering (token positions equal to all key positions
with that token).
range::before() and range::after() can't work for weak ordering. If
the bound is exclusive, we don't know if user-provided token position
is inside or outside.
Also, is_wrap_around() can't properly detect wrap around in all
cases. Consider this case:
(1) ]A; B]
(2) [A; B]
For A = (tok1) and B = (tok1, key1), (1) is a wrap around and (2) is
not. Without total ordering between A and B, range::is_wrap_around() can't
tell that.
I think the simplest soution is to define a total ordering on
ring_position by making token positions positioned either before or
after all keys with that token.
Two ring_positions are equal if tokens and keys are equal or tokens are
equal and one or both of them do not specify key. So ring_positions
without a key is a wildcard that equals any ring_positions with the same
token.
Some of the tests in DTEST take advantage of the fact that
ByteOrderedPartitioner guarantees certain ordering of partition keys.
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
We need to be able to do it so we can, among other things, create CQL
statements that include the current state of the tokens.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
It is a better fit for things that are names, not blobs. We have a user that expects
a bytes parameter, but that is for no other reason than the fact that the field used
to be of bytes type.
Let's fix that, and future users will be able to use sstrings
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
midpoint(l, r) where l > r needs to wrap around the end of the ring. Adjust
the midpoint() function to do this.
Note this is still broken for the murmur3 partitioner, since it doesn't treat
tokens as unsigned.
In sstables the paritioner name is store for validation. To allow Origin
to process our files we need to comply with Origin's paritioner name or
else Origin's SSTableReader::open fails on paritioner comparison.
We do not care about the order of the tokens.
Also, in token_metadata, we use unordered_set for tokens as well, e.g.
update_normal_tokens. Unify the usage.
Validation metadata stores partitioner name and bloom filter chance.
Cassandra gets the partitioner name by getting a object of the class
itself and getting its canonical name.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Use decorated_key in partition maps, from Tomasz:
"Partitions should be ordered using Origin's ordering, which is the natural
ordering of decorated_key. This is achieved by switching column_family's
partition map to use decorated_key instead of a bare partition_key.
This also includes some cleanups."
[avi] trivial adjustments to sstables/keys.cc
It has been determined that we will store partition_keys in the decorated_keys.
That is totally fine, but the token needs to be generatable from an sstable::key
as well.
Since both types convert well to a bytes_view - and the first thing get_token() does
is precisely to generate that view, let's generate the token from a bytes_view instead.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
