We will invoke the schema builder from schema_tables.cc, and at that point, the
information about compact storage no longer exists anywhere. If we just call it
like this, it will be the same as calling it with compact_storage::no, which
will trigger a (wrong) recomputation for compact_storage::yes CFs
The best way to solve that, is make the compact_storage parameter mandatory
every time we create a new table - instead of defaulting to no. This will
ensure that the correct dense and compound calculation are always done when
calling the builder with a parameter, and not done at all when we call it
without a parameter.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
When a schema is available, we use it. However, we have, by now, way too many
tests. Some of them use tables for which we don't even know the schema. It would
have been a massive amount of work to require a schema for all of them - so I am
keeping both constructors around.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Add a test case that triggers the heap overflow fixed in previous commit
("bytes_ostream: Fix current_space_left()").
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
This reverts commit 6b2be41df00bc42331eccd423b7031b345cf979d; tests should
work without a data directory, so let's find why they don't and fix it
instead.
It is needed for db.get_version(). I really hated to pass &db everywhere
If we had a global helper function like get_local_db(), life will be much
easier.
From Pawel:
This series fixes SELECT DISTINC statements. Previously, we relied on the
existance of static row to get proper results. That obviously doesn't work
when there is no static row in the partition. The solution for that is
to introduce new option to partition_slice: distinct which informs that
the only important information is static row and whether the partition
exists.
In preparation for adding listener state to migration manager, use
sharded<> for migration manager.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
This patch introduce init.cc file which hosts all the initialization
code. The benefits are 1) we can share initialization code with tests
code. 2) all the service startup dependency / order code is in one
single place instead of everywhere.
"range::is_wrap_around() will not work with current ring_position, because it
relies on total ordering. Same for range::contains(). Currently ring_position
is weakly ordered. This series fixes this problem by making ring_position
totally ordered.
Another problem fixed by this series is handling of wrap-around ranges. In
Origin, ]x; x] is treated as a wrap around range covering whole ring."
If we revert the type of the clustering key, which is what would happen if we
defined the table as with clustering order by (cl desc), we expect the
clustering keys to be in descending order on disk.
There is no work needed for sstables for that to happen. But we should still
verify that this is indeed the case.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
range::is_wrap_around() and range::contains() rely on total ordering
on values to work properly. Current ring_position_comparator was only
imposing a weak ordering (token positions equal to all key positions
with that token).
range::before() and range::after() can't work for weak ordering. If
the bound is exclusive, we don't know if user-provided token position
is inside or outside.
Also, is_wrap_around() can't properly detect wrap around in all
cases. Consider this case:
(1) ]A; B]
(2) [A; B]
For A = (tok1) and B = (tok1, key1), (1) is a wrap around and (2) is
not. Without total ordering between A and B, range::is_wrap_around() can't
tell that.
I think the simplest soution is to define a total ordering on
ring_position by making token positions positioned either before or
after all keys with that token.
"This is my current proposal for Compact Storage tables - plus
the needed infrastructure.
Getting rid of the CellName abstraction allows us to simplify
things by quite a lot: now all we need is to mark whether or
not a table is composite, and provide functions to play the
role of the comparator when dealing with the strings."
After commit 67f4b55b16 "gms/gossiper: Fix is_gossip_only_member() logic",
storage_service is needed by gossip.
To fix, start storage_service in the test. Also, improve the
indentation.
"These patches fix more problems related to ORDER BY clauses.
Firstly, mutation_partition::query() can now return rows in reveresed
order which makes it easy for select statements to ask for data from
single partition with clustering keys in reversed order even if limit
of rows is set.
That alone is not sufficient, though, if the request contains IN clause
on partition keys and number of returned rows is limited. The information
needed to determine which rows need to be in the reply isn't available
before post-query sort is done, so select statement asks for more rows
than the limit and trims the output later."
Since the read tests are validated using Origin-generated tables, our
write test will just write to the tables and make sure we can read them
back ok.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
I would like to use them from sstable_datafile_test.cc to make sure that
the tables we write are really correct.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Test the read of a table that is not compound NOR dense.
Dense tables will be handled later.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Technically, this is not needed for the existing schemas: the builder is only
necessary when we are setting properties. For consistency, however, let's
convert them all.
Soon we will have some schemas that will set properties. In particular, compact
storage.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This solitary schema definition should go live with the rest.
We give it a more descriptive name now that it is going.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
It is now necessary to close() a file before destroying it, otherwise a big
ugly warning message is printed by the reactor. Our sstable read path was
especially careless about closing the countless files it opens, and the
sstable test generated as many as 400 (!) of these warning messages, despite
running correctly. This patch adds the missing close() calls.
After this patch, the sstable test still shows 3 warning messages.
Those are unavoidable: They happen while broken sstables are being
tested, and an exception is thrown in the middle of the sstable processing,
causing us to destroy a file object without calling close() on it first.
This, in my opinion, proves that requiring close() in the read path is not
a good thing, it is un-RAII-like and not exception-safe. But it is benign
except the warning message, so whatever. 3 scary warning messages from the
test are better than 400...
If these 3 remaining messages really bother us, I guess we can fix it by
catching the exceptions in the sstable code, closing the file and rethrowing
the exception, but it will be quite ugly...
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
