We usually don't read the whole file into memory, so the probing interface will
also allow for the specification of boundaries that we should be use for
reading.
The sstable needs to be informed - usually by the schema - of how many columns
the partition key is composed of - 1 for simple keys, more than one, for
composites.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Because we're expected to go to those files many times, it doesn't make sense
to keep opening them. Upon sstable load, we will open those files and then move
the reference to the sstable main structure. From this point on, we can just seek
to the position we want.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
So it represents all kinds of mismatches. Not only buf < expected.
It will be useful in the index parsing code.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Right now we will always move the file to create a shared pointer from it.
However, there are situations in which we don't really want to move it, because
it will be still used elsewhere.
One example is the index and data readers, where we will store a file object to
avoid opening the file all the time. In such situation, we can pass a shared
pointer that is already constructed to the file_input_stream.
The alternative to that would have been to store the file_input_stream itself.
That would, however, require us to export that in some header. It's best to
keep it private. Since we will already deal with a shared pointer anyway, it is
best to provide this option.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Includes adaptation by Nadav for the removal of file_input_stream:
sstables.cc used file_input_stream, which we replaced by the new
make_file_input_stream. We also couldn't make sstables.cc read either
a file_input_stream or the planned compressed_file_input_stream.
So in this patch we implement an API similar to the old "file_input_stream"
based on the new make_file_input_stream. file_input_stream now has a parent
class "random_access_reader", preparing for a future patch to support both
file_input_stream and compressed_file_input_stream in the same code - by making
all the parsers take a random_access_reader reference instead of file_input_stream.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
The file_input_stream interface was messy: it was not fiber safe (e.g., code
doing seek() in the middle of an ongoing read_exactly()), and went against
the PIMPL philosophy.
So this patch removes the file_input_stream class, and replaces it with a
completely different design:
We now have in fstream.hh a global function:
input_stream<char>
make_file_input_stream(
lw_shared_ptr<file> file, uint64_t offset = 0,
uint64_t buffer_size = 8192);
In other words, instead of "seeking" in an input stream, we just open a new
input stream object at a particular offset of the given file. Multiple input
streams might be concurrently active on the same file.
Note how make_file_input_stream now returns a regular "input_stream", not a
subtype, and it can be used just like any normal input_stream to read the stream
starting at the given position.
This patch makes "input_stream" a "final" type: we no longer subclass it in our
code, and we shouldn't in the future because it goes against the PIMPL design
(the subclass should be of the inner workings, like the data_source_impl, not
of input_stream).
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
We had an include loop, which can cause problems in some cases:
1. iostream.hh #includes scattered_message.hh #includes reactor.hh
2. reactor.hh #includes iostream.hh
This patch fixes the loop be removing an unnecessary include.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
reactor.hh includes forward declarations for input_stream, output_stream
template classes. These are unnecessary, because it #include's iostream.hh,
which contains the full definition of these classes.
These unnecessary forward declarations are harmless in the current code, but
they will become harmful if we change the definitions in iostream.hh (e.g., I
wanted to make input_stream a "final" class).
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
core/reactor.hh had two #include lines in the middle of the file, which
duplicate previous #include lines in the top of the file. So remove them.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
make_shared() has a special case for detecting a created class deriving
from enable_shared_from_this<>, so it can point the refcount pointer into
the object's data area instead of creating a shared_ptr_count_base for it.
The code, however, fails to detect a creating class deriving indirectly
from enable_shared_from_this:
struct base : enable_shared_from_this<base> {};
struct derived : base {};
make_shared<derived>(); // <- allocates independent refcount
The result is that the object reference counter lives in two locations.
Fix by detecting the derived class case as well.
