"This is my current proposal for Compact Storage tables - plus
the needed infrastructure.
Getting rid of the CellName abstraction allows us to simplify
things by quite a lot: now all we need is to mark whether or
not a table is composite, and provide functions to play the
role of the comparator when dealing with the strings."
After commit 67f4b55b16 "gms/gossiper: Fix is_gossip_only_member() logic",
storage_service is needed by gossip.
To fix, start storage_service in the test. Also, improve the
indentation.
"These patches fix more problems related to ORDER BY clauses.
Firstly, mutation_partition::query() can now return rows in reveresed
order which makes it easy for select statements to ask for data from
single partition with clustering keys in reversed order even if limit
of rows is set.
That alone is not sufficient, though, if the request contains IN clause
on partition keys and number of returned rows is limited. The information
needed to determine which rows need to be in the reply isn't available
before post-query sort is done, so select statement asks for more rows
than the limit and trims the output later."
Since the read tests are validated using Origin-generated tables, our
write test will just write to the tables and make sure we can read them
back ok.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
I would like to use them from sstable_datafile_test.cc to make sure that
the tables we write are really correct.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Test the read of a table that is not compound NOR dense.
Dense tables will be handled later.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Technically, this is not needed for the existing schemas: the builder is only
necessary when we are setting properties. For consistency, however, let's
convert them all.
Soon we will have some schemas that will set properties. In particular, compact
storage.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This solitary schema definition should go live with the rest.
We give it a more descriptive name now that it is going.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
It is now necessary to close() a file before destroying it, otherwise a big
ugly warning message is printed by the reactor. Our sstable read path was
especially careless about closing the countless files it opens, and the
sstable test generated as many as 400 (!) of these warning messages, despite
running correctly. This patch adds the missing close() calls.
After this patch, the sstable test still shows 3 warning messages.
Those are unavoidable: They happen while broken sstables are being
tested, and an exception is thrown in the middle of the sstable processing,
causing us to destroy a file object without calling close() on it first.
This, in my opinion, proves that requiring close() in the read path is not
a good thing, it is un-RAII-like and not exception-safe. But it is benign
except the warning message, so whatever. 3 scary warning messages from the
test are better than 400...
If these 3 remaining messages really bother us, I guess we can fix it by
catching the exceptions in the sstable code, closing the file and rethrowing
the exception, but it will be quite ugly...
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
The idea is to reuse the same testing code on any mutation_source, for
example on memtable.
The range query test cases are now part of a generic mutation_source
test suite.
Make storage proxy a singleton and add helpers to look up a reference.
This makes it easier to convert code from Origin in areas such as
storage service.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
recently, "file" started to use a shared_ptr internally, and is already
copy-able and reference counted, and there is no reason to use
lw_shared_ptr<file>. This patch cleans up a few remaining places where
lw_shared_ptr<file> was used.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
By using free functions that take the serializer as a parameter (rather than
'this'), we can add serialization functions without changing the serializer
class.
rpc currently allows serializers and deserializers to defer, because
the input and output stream may not be ready. They may not, however,
defer on behalf of the object being serialized or deserialized (i.e.
you cannot serialize to disk or deserialize from disk) because that
causes the tcp connection to block until serialization/deserialization is
complete. So in practice messages must be small enough to fit in memory,
and there is nothing gained by the complexity.
To simplify things, switch to non-deferring serialization. Add a frame
header to messages that specifies the buffer size, which allows rpc to
use a read_exacly() to consume the message, and thereafter deserialize it
immediately.
The result is significantly simpler, which should help with compile time.
config.hh changes rapidly, so don't force lots of recompiles by including it.
Need to place seed_provider_type in namespace scope, so we can forward
declare it for that.
It should not be called directly: externall callers should be calling flush()
instead.
To be sure it doesn't happen again, make seal_active_memtable private.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
When we create a column family, we can pass as an extra parameter, the
commitlog - or lack thereof. Because the commitlog is optional to begin with -
it won't exist if we don't call init_commitlog, we can have this to be empty
meaning no commit log.
The creation of a column family should be always done through
add_column_family. And if that is the case, we have the database's commitlog
right there and can get the pointer through the db. Only tests are not creating
the column family this way, and for them, it is fine.
We want to do that, because some column family operations will use the commit log.
Right now, they are forcing us to add parameters to APIs that would be much cleaner
without it. So while separation is good, this level of coupling is a net win as it
allows us to clean up some visible APIs.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
"Fixes for ORDER BY clauses" from Pawel.
"The patches fix several issues in CQL3 frontend related to ORDER BY
clauses. Also, column component indexes are now handled properly and it
is possible to create tables with more than one column in clustering key."
