Since types can be embedded at any position in memory we cannot assume
alignment.
Side note: It seems that on x86 access to the variable via packed<>
does not result in any extra instructions.
Sstable support from Glauber:
"These are the new sstable parsers, based on recursive templates.
They currently read the following files successfully:
- Filter
- Statistics
- Compression
- Summary (sans the indices, which are schema dependent)
The TOC file is also parsed, but slightly different because it contains
a newline separated list of strings. The CRC and Digest files are not
yet read, but they are only used for consistency checking, which can wait.
Aside from that, only missing the Data and Index files themselves."
This tests the sstable parsing code. The tables provided as exemple are real
tables, that I was using to test my code manually.
The corrupt tables, however, are corrupted by hand.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This is a parser for the sstable files based on template recursion.
With this, one can easily extend it to parse any complex data structure
by writing
parse(in, struct.a, struct.b ... );
This patch contains the most basic types used during parsing as building
blocks.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This represents an sstable with its multiple in-disk file. We don't yet
read any of them, though.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
boost::join() provided by boost/algorithm/string.hpp conflicts with
boost::join() from boost/range/join.hpp. It looks like a boost issue
but let's not pollute the namespace unnecesssarily.
Regarding the change in configure.py, it looks like scollectd.cc is
part of the 'core' package, but it needs 'net/api.hh', so I added
'net/net.cc' to core.
It is sometimes frustrating to use open_file_dma, because it has the hardcoded
behavior of always assuming O_CREAT. Sometimes this is not desirable, and it
would be nice to have the option not to do so.
Note that, by design, I am only including in the open_flags enum things that we
want the user of the API to control. Stuff like O_DIRECT should not be
optional, and therefore is not included in the visible interface.
Because of that I am changing the function signature to include a paramater
that specifies whether or not we should create the file if it does not exist.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Only partially translated. I had to comment out some "static"
specifications to avoid compiler warnings because these are not used yet.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Add a performance test case for CQL statement parsing to better
understand its performance impact. We also include ANTLR tokenizer and
parser setup as that's what we do in query_processor for each request.
Running the test on my Haswell machine yields the following results:
[penberg@nero urchin]$ build/release/tests/perf/perf_cql_parser
Timing CQL statement parsing...
108090.10 tps
125366.11 tps
124400.64 tps
124274.75 tps
124850.85 tps
That means that CQL parsing alone sets an upper limit of 120k requests
per second for Urchin for a single core.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
* Item structure was changed to work with slab, where its last field
is used to access key, value, and ascii prefix, respectively.
* Item structured was worked to have a smaller footprint. It was
reduced from 104 bytes to 72 bytes.
* Reclaimer was temporarily disabled.
* Global LRU was removed from the cache. Now LRU eviction is done
on a per-slab-class basis, whenever the slab allocator runs out of
slab pages.
* Fragmentation issue is naturally solved with the slab algorithm,
where slab classes have chunks of the same size.
Expiration time is a 32-bit value as specified by memcached protocol.
Thus, no need to store it as a 64-bit value. When the value is needed,
convert it to the target type. Change intended to save space from
item structure.
In addition, there is no need to insert an entry into _alive when
expiration time is zero, meaning item will never expire.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
* Slab allocator resembles the one used by stock memcached, where
it's composed of slab classes and slab classes are composed of
chunks of the same size.
* Per-slab-class LRU is also available.
* Slab allocator exports stats to collectd.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Flashcached integration was done relying on certain similarities between
flashcached and memcached. Old assumptions no longer hold.
As a result, the current code is terrible to integrate the slab allocator
into it, while keeping flashcached alive.
This patch reverts flashcached from memcached. It should be re-integrated
in the future, but definitely in a better way.
In v2: rebase, and add a more descriptive exception if trying to deserialize
an ipv6 address.
Add to list of types the Internet address type, known as INET in CQL
or as InetAddressType in the Cassandra code.
I based this code on stuff I found in InetAddressType and
InetAddressSerializer in the Cassandra code.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
star-dev into db
Long type improvements, from Pekka:
"There's unimplemented methods in long_type_impl that are already
implemented in int32_type_impl. Generify the latter and use that for
long_type_impl as well."
If request processing throws an exception, print it to stdout rather
than silently ignoring it. Eventually we need to respond with ERROR
message to the client.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Switch to sizeof in preparation for converting int32_type_impl into a
generic integer_type_impl.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
The ".or_terminate()" continuation is needed when one needs to exit the
application on an unhandled exception, instead of just letting the event
loop continue to spin forever.
or_terminate() currently calls std::terminate() which only incidentally
(as a gcc-specific implementation) prints out the exception's type; It
then calls abort(), which results in a painfully slow core dump. This
abort() is NOT helpful, because at that point the debugger can only find
where abort() was called, not where the original exception was thrown (see
issue #32).
So instead of calling std::terminate(), this patch switches to calling
engine().exit(), to cleanly shut down the application, without dumping
core. It also prints, like gcc's std::terminate(), the exception's type.
This printing requires non-standard gcc extensions.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
