This is my turn to make the same mistake as Nadav recently made: I have
written down the expected position from the test output itself, instead
of checking the file.
A position of 0x400c0000000000 is obviously ridiculous. The file itself is not
that big. My recent patch to fix the summary broke the test, and so we need to
fix it.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Since that whole entries array is memory mapped, we should not call
read_integer - which will byteswap it.
We should instead just fetch the position directly.
This was not spotted before because our test sstables tend to be relatively
small.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Currently if you run more than one instance on the same machine, they
all bind to the same local address and port (we enable an option that
makes this non error) and cql/thrift requests are distributed randomly
among them.
Some urchin parameters has default value of localhost and need to be
resolved into IP address. Add stub resolver that do just that: if it
sees "localhost" string it translates it to a loopback address,
otherwise assume that string is IP address already.
Java uses long, so we should use int64_t. Using uint64_t causes the wrong
indexes to be calculated, and therefore, the filter to respond incorrectly
to a given key.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
I actually wrote this code before I learned that we could just return a local
vector by copy without major hassles.
Never too late for a cleanup.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This patch is built on the previous one, where access to members of sstable
class was made possible.
Index file is being properly generated, but it's important mentioning that
promoted indexes aren't supported yet.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
The function write_datafile was renamed to write_components and made a member
of sstable class because write of components requires access to private
members. This change is an important step towards the generation of components
other than data file.
The respective testcases were adapted to the changes.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
This function is likely to be duplicated over time, so let's make
it available as a static method of sstable class.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
file_writer wraps output_stream<char> around.
file_writer also adds the method offset(), intended to return the current
offset of the stream.
It's also a small step towards compression support where a specialized output
stream is required.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
A long running query may use memtables and sstables that are being removed
by memtable flush and sstable compaction paths. Use copy-on-write to
prevent use-after-free.
If keyspace already exists, throw a already_exists_exception like Origin
does. Spotted while reading the code.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
When thrift sees an exception that was not declared as part of the interface,
it wraps it using std::exception::what() for the exception text. This is
often cryptic, so add an "Internal server error" prefix.
The test tests/memcached/test_ascii_parser hung after the change to
consume(). The problem was that consume() notified the consumer of an
EOF by sending it an empty buffer, and then it expected to get back a
message that it shouldn't read more (by setting the unconsumed buffer),
if it didn't, it continued to send empty buffers in a never-ending loops.
So this patch changes consume() to send one empty buffer to the consumer
on the end-of-file, and then stop (regardless of what the consumer returns).
It would have probably made sense to *not* send an empty buffer to the
consumer after the data is done - not even once - but if we change this
behavior, it will break the existing tests.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Tested-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
We cannot capture keyspace_metadata by reference because it can be
allocated on the stack. Fixes SIGSEGV while running cassandra-stress.
The bug was introduced in commit commit cd35617 ("database: Use
keyspace_metadata for creation functions").
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
The CQL server is really noisy during cassandra-stress run because it
prints out STARTUP message options. Fix that up.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Glauber says:
"This is the basic bloom filter implementation. We can read an existing
filter file, but we won't construct one (pending on Index creation)
TODO:
- add the tracker bloom filter (to track its statistics)
- write the filter file"
From Avi:
"This patchset prepares for adding sstables to the read path. Because sstables
involve I/O, their APIs return futures, which means that APIs that may call
those sstable APIs also need to return futures.
This patchset uses the two-space indent + do_with + reference aliases trick
to make patches more readable. Cleanup patches will follow once it is merged."
Pekka says:
"Clean up keyspace creation functions by using keyspace_metadata. While
at it, simplify creation by moving replication strategy initialization
to database.cc. This paves the way for adding keyspace metadata lookup
functionality to database that's needed by migration manager."
Our input_stream::consume() mechanism allows feeding data from an input
stream into a consumer, piece by piece, until the consumer doesn't want
any more. It currently assumed the input can block (when reading from disk),
but the consumption is assumed to be immediate. This patch adds support for
blocking in the consumption function: The consumer now returns a future
which it promises to fulfill after consuming the given buffer.
This patch goes further by somewhat simplifying (?) the interface of the
consumer. Instead of receiving a mysterious "done" lambda the consumer
is supposed to call when done (doesn't want any more input), the consumer
now returns a future<optional<temporary_buffer<char>>, which means:
1. The future is fulfilled when the consumer is done with this buffer
and either wants more - or wants to stop.
2. If the consumer wants to stop, it returns the *remaining* part of the
buffer it didn't want to process (this will be pushed back into the
input stream).
3. If the consumer is not done, and wants to consume more, it returns an
unset optional.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Use the provided filter instead of always returning true. For existing tables,
this arrives from the bloom filter file. We don't yet fully write a bloom
filter file.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
