Commit Graph

53948 Commits

Author SHA1 Message Date
Nadav Har'El
e7a59652b5 Add comments to schemas in system_keyspace
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-02 15:18:41 +01:00
Nadav Har'El
8265a13dbd schema: add "comment" string
Add a comment string to a schema, which may be set but is currently
not further used.

The originals Cassandra code has a comment for each of the builtin
schemas, and it's a shame not to remember them.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-02 15:18:23 +01:00
Nadav Har'El
4d9cf99cd9 start converting LegacySchemaTables.java to C++
Convert the schemas from LegacySchemaTables.java to C++.
Some FIXMEs still left because of types we don't yet support, or
schema features we don't yet support - this will be fixed later.

Note that I put legacy_schema_tables in the "db" namespace, not in the
"schema" namespace where Origin had it. This was the *only* class in
the "schema" package in Origin, and unfortunately in Urchin we can't use
the name "schema" as a namespace, as we already have it as a class.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-02 15:18:19 +01:00
Nadav Har'El
161de485f7 LegacySchemaTables.java from origin
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-02 15:18:16 +01:00
Asias He
55a43eb5f9 tcp: Improve retransmit
Do not store the tcp header in unacked queue. When partial ack of a
segment happens, trim off the acked part of a segment. When retransmits,
recalculate the tcp header and retransmit only the unacked part.
2015-03-02 12:36:47 +02:00
Nadav Har'El
6e94cfb3ac system_keyspace - translate more tables to C++
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-02 10:59:38 +01:00
Pekka Enberg
99a09020e8 types: Fix bytes_type_impl string conversion
Tomek points out that:

  Origin calls org.apache.cassandra.utils.Hex#hexToBytes here, which is
  not what to_bytes() does. BytesType.getSerializer().toString() calls
  ByteBufferUtil.bytesToHex(value), so you should call to_hex() here.

Fix that up.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-02 10:38:27 +01:00
Avi Kivity
6379ce5cc3 Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-03-01 18:47:54 +02:00
Avi Kivity
3aaac55a7a tcp: fix bad checksum on RST packets
We neglected to set offload_info::needs_csum on reset packets, resuling in
them being ignored by the recipient.  Symptoms include connection attempts
to closed ports (seastar = passive end) hanging instead the active end
understanding the port is closed.
2015-03-01 18:45:35 +02:00
Avi Kivity
1bf214b7df thrift: stop reading on short buffer
Fixes buffer overrun.  Should perhaps be part of read_exactly().
2015-03-01 17:14:25 +02:00
Avi Kivity
1d02b8bc6d Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-03-01 16:52:23 +02:00
Avi Kivity
27913013b1 virtio: tighten rx packets for debug mode
Allocate exactly the available fragment size in order to catch buffer
overflows.

We get similar behaviour in dpdk, since without huge pages, it must copy
the packet into a newly allocated buffer.
2015-03-01 16:42:07 +02:00
Gleb Natapov
590d8da4f1 core: provide "void" promise specialization
Makes it easier to write generic code.
2015-03-01 15:58:36 +02:00
Gleb Natapov
e1aa9f85a1 fix xenstore compilation 2015-03-01 15:29:13 +02:00
Avi Kivity
cefe6a9b3e tcp: fix option parsing
Two bugs:
  1. get_header<type>(offset) was used with size given as the offset
  2. opt_end failed to account for the mandatory tcp header, and thus was
     20 bytes to large, resulting in overflow.
2015-03-01 15:26:46 +02:00
Avi Kivity
f039904d75 Merge branch 'master' into db
Updated usages of std::hash<some_enum_type> to accomodate 25168fc73d.
2015-03-01 15:24:47 +02:00
Avi Kivity
25168fc73d enum: fix std::hash<> non-specialization for enum types
SFINAE only works for substituted template parameters, not any complication
error (or it would be called CEINAE); therefore hash<T> for enums will fail
to compile, given a non-enum, rather than being ignored.

It's not possible to specialize hash<> for enums, since the primary template
does not have en extra Enable template argument for use with enable_if.  We
therefore rename it to enum_hash<> and require users to explicitly define
hash<MyEnum> as inheriting from it.
2015-03-01 13:36:30 +02:00
Avi Kivity
1e5df06f9a shared_ptr: provide default std::hash<> specializations
Folloing std::shared_ptr.
2015-03-01 13:36:17 +02:00
Glauber Costa
eb384236fd iostream: extend eof condition to read_exactly users as well
We currently only signal eof for consume() users. If one is calling
read_exactly, eof will never be signalled.

This might make some sense in continuous streams, but specially when
dealing with files, eof is a natural part of line that can and will
happen all the time. Every "read-until-finish" file-loop will at some
point rely on eof.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-03-01 09:48:51 +02:00
Avi Kivity
81ca06fa36 sstable: fix TOC size check
Make sure we detect a too-large TOC correctly.
2015-02-28 23:50:36 +02:00
Avi Kivity
7441ce5b51 sstable: fix buffer overflow in TOC
boost::split() expects either a NUL terminated string or a proper container.
We give it neither.

Fix by wrapping the buffer in a string_view, which tells split() what size
the string is.
2015-02-28 23:49:07 +02:00
Avi Kivity
0b29e4bdaa Merge branch 'tgrabiec/db' of github.com:cloudius-systems/seastar-dev into urchin
Miscellaneous db changes, from Tomasz.
2015-02-28 22:33:06 +02:00
Tomasz Grabiec
74295a9759 db: Use opaque bytes for cell values instead of boost::any
Storing cells as boost::any objects makes us use expensive
boost::any_cast to access the data. This change replaces boost::any
with bytes object which holds the value in serialized form (the same
as will be used for on-wire format).

If the cell type is atomic, you use fields accessors defined in
atomic_cell class, eg like this:

if (column.type.is_atomic()) {
   if (atomic_cell::is_live(c) {
      auto timestamp = atomic_cell::timestamp(c);
      ...
   }
}

Eventually we could switch to a more officient semi-serialized form
with native byte order but I don't want to introduce it just yet for
simplicity.
2015-02-27 10:59:43 +01:00
Tomasz Grabiec
91176f469e schema: Add schema::all_columns_in_select_order()
After org.apache.cassandra.config.CFMetaData#allColumnsInSelectOrder
2015-02-27 10:48:56 +01:00
Tomasz Grabiec
1a0ffdfb99 schema: Encapsulate column sets 2015-02-27 10:48:56 +01:00
Tomasz Grabiec
a61d9ee18e schema: Add static columns to schema 2015-02-27 10:48:56 +01:00
Tomasz Grabiec
8b9078c86a schema: Make column_kind an enum class 2015-02-27 10:48:56 +01:00
Tomasz Grabiec
b77367dabe cql3: Simplify primary key membership checks 2015-02-27 10:48:56 +01:00
Tomasz Grabiec
609e893055 unimplemented: Separate subject from behavior
You can now do:

  fail(unimplemented::cause::PAGING);

and:

  warn(unimplemented::cause::PAGING);
2015-02-27 10:48:56 +01:00
Tomasz Grabiec
1f9d22f7e3 types: Fix read_simple*() variants to work for unaligned positions
Since types can be embedded at any position in memory we cannot assume
alignment.

Side note: It seems that on x86 access to the variable via packed<>
does not result in any extra instructions.
2015-02-27 10:48:56 +01:00
Tomasz Grabiec
00984609cb types: Add abstract_tyep::is_value_compatible_with() method 2015-02-27 10:48:56 +01:00
Tomasz Grabiec
f21d85cfc5 tests: Stop distributed db when test ends
Fixes:

cql_query_test: ./core/distributed.hh:150: distributed<Service>::~distributed() [with Service = database]: Assertion `_instances.empty()' failed.
2015-02-27 10:48:55 +01:00
Tomasz Grabiec
1dfc7267db cql3: Fix formatting in relation.hh 2015-02-27 10:46:05 +01:00
Avi Kivity
5bc8d7cfad build: speed up linking on large machines 2015-02-26 19:26:32 +02:00
Avi Kivity
a7132359a8 Merge branch 'glommer/sstables-v2' of github.com:cloudius-systems/seastar-dev into db
Sstable support from Glauber:

"These are the new sstable parsers, based on recursive templates.
They currently read the following files successfully:
 - Filter
 - Statistics
 - Compression
 - Summary (sans the indices, which are schema dependent)

The TOC file is also parsed, but slightly different because it contains
a newline separated list of strings. The CRC and Digest files are not
yet read, but they are only used for consistency checking, which can wait.

Aside from that, only missing the Data and Index files themselves."
2015-02-26 19:17:52 +02:00
Glauber Costa
8015b1c1bb sstable: unit tests
This tests the sstable parsing code. The tables provided as exemple are real
tables, that I was using to test my code manually.

The corrupt tables, however, are corrupted by hand.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-02-26 12:14:20 -05:00
Glauber Costa
fb3682cb4f sstable statistics file
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-02-26 12:14:19 -05:00
Glauber Costa
0d98caf885 summary file
TODO: read in the actual index. This is schema-dependent.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-02-26 12:14:19 -05:00
Glauber Costa
1b75a5bccb bloom filter
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-02-26 12:14:19 -05:00
Glauber Costa
d810f03bb7 compression file
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-02-26 12:14:19 -05:00
Glauber Costa
3e4ab6848b read toc
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-02-26 12:14:19 -05:00
Glauber Costa
cc4f8f09e5 parser: disk_hash type
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-02-26 12:06:25 -05:00
Glauber Costa
110facf479 parser: more basic types
bool, enum and double.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-02-26 12:06:25 -05:00
Glauber Costa
001606209a sstable parser
This is a parser for the sstable files based on template recursion.
With this, one can easily extend it to parse any complex data structure
by writing

    parse(in, struct.a, struct.b ... );

This patch contains the most basic types used during parsing as building
blocks.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-02-26 12:06:25 -05:00
Glauber Costa
b7281bd463 sstables: initial file
This represents an sstable with its multiple in-disk file. We don't yet
read any of them, though.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-02-26 12:06:25 -05:00
Glauber Costa
cb5c2de06c README: add documentation about then_wrapped
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-02-26 18:51:40 +02:00
Avi Kivity
ede9a5b634 build: enable native network stack for urchin 2015-02-26 18:49:28 +02:00
Avi Kivity
d39c844ea3 Merge branch 'master' of github.com:cloudius-systems/seastar into db
Conflicts:
	configure.py
	database.cc (added missing include)
	utils/serialize.hh (added missing inlines)
2015-02-26 17:57:18 +02:00
Tomasz Grabiec
1fadd7d608 net: Move ipv4_addr constructor to the source file
boost::join() provided by boost/algorithm/string.hpp conflicts with
boost::join() from boost/range/join.hpp. It looks like a boost issue
but let's not pollute the namespace unnecesssarily.

Regarding the change in configure.py, it looks like scollectd.cc is
part of the 'core' package, but it needs 'net/api.hh', so I added
'net/net.cc' to core.
2015-02-26 17:34:27 +02:00
Glauber Costa
6b77cf7d08 linecount: change rw to ro
Avi says it is enough.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-02-26 17:30:02 +02:00