Use decorated_key in partition maps, from Tomasz:
"Partitions should be ordered using Origin's ordering, which is the natural
ordering of decorated_key. This is achieved by switching column_family's
partition map to use decorated_key instead of a bare partition_key.
This also includes some cleanups."
[avi] trivial adjustments to sstables/keys.cc
It has been determined that we will store partition_keys in the decorated_keys.
That is totally fine, but the token needs to be generatable from an sstable::key
as well.
Since both types convert well to a bytes_view - and the first thing get_token() does
is precisely to generate that view, let's generate the token from a bytes_view instead.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
bytes and sstring are distinct types, since their internal buffers are of
different length, but bytes_view is an alias of sstring_view, which makes
it possible of objects of different types to leak across the abstraction
boundary.
Fix this by making bytes a basic_sstring<int8_t, ...> instead of using char.
int8_t is a 'signed char', which is a distinct type from char, so now
bytes_view is a distinct type from sstring_view.
uint8_t would have been an even better choice, but that diverges from Origin
and would have required an audit.
Inspired in Gleb's previous patch, this patch adds a hash and comparison
operator for dht::token.
The previous patch, however, had a number of problems. Comparisons were failing
in tokens that were verified (by me) to be equal to the ones Origin was
generating.
The main reasons for that, was that the byte-comparison loop must be unsigned,
not signed.
With the above change, the comparison function would always succeed *except*
when the integer version of _data was that of a signed one.
Looking at Origin, one verifies that the Murmur3Partitioner class overrides the
comparison functions, and just does a Long comparison with the token.
This patch implements a similar mechanism. With that, a list of tokens
generated by origin in ascending order is verified by us to also be in
ascending order.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Holding keys and their prefixes as "bytes" is error prone. It's easy
to mix them up (or use wrong types). This change adds wrappers for
keys with accessors which are meant to make misuses as difficult as
possible.
Prefix and full keys are now distinguished. Places which assumed that
the representation is the same (it currently is) were changed not to
do so. This will allow us to introduce more compact storage for non-prefix
keys.
C++ doesn't define overflow on signed types, so use unsigned types instead.
Luckily all right shifts were unsigned anyway.
Some signed extension was happening (handling remainders after processing
8-byte chunks) but should still be there.
Caught by debug build.
We don't follow origin precisely in normalizing the token (converting a
zero to something else). We probably should, to allow direct import of
a database.
Rather than converting to unsigned longs for the fractional computations,
do them it bytes. The overhead of allocating longs will be larger than
the computation, given that tokens are usually short (8 bytes), and
our bytes type stores them inline.
Origin uses abstract types for Token; for two reasons:
1. To create a distinction between tokens for keys and tokens
that represent the end of the range
2. To use different implementations for tokens belonging to different
partitioners.
Using abstract types carries a penalty of indirection, more complex
memory management, and performance. We can eliminate it by using
a concrete type, and defer any differences in the implementation
to the partitioner. End-of-range token representation is folded into
the token class.
