Commit Graph

53948 Commits

Author SHA1 Message Date
Avi Kivity
22f24cc5f6 cql3: implement sets::setter tombstone
Kill existing cells before adding new ones.
2015-03-27 16:09:41 +01:00
Avi Kivity
2602032b5e tests: add tests for setting entire maps
UPDATE tab set my_map = { key: val, ... };

also test a random syntax error.
2015-03-26 18:14:35 +01:00
Avi Kivity
17b929d739 db: implement as_cql3_type() for collections 2015-03-26 18:14:35 +01:00
Avi Kivity
6c343057ea cql3: implement maps setter correctly
Now that we have collection tombstones, we can kill all existing cells
in maps::setter.
2015-03-26 18:14:35 +01:00
Avi Kivity
579a794d14 cql3: de-virtualize cql3_type
cql3_type is a wrapper around data_type, so there's no need for a class
hierarchy -- anything that depends on the actual type can be forwarded
to abstract_type_impl.

We can now use this as a value type (dropping shared_ptr<cql3_type>), but
that is left for later.
2015-03-26 16:04:55 +01:00
Avi Kivity
d92d5b9217 Merge branch 'tgrabiec/static_col' of github.com:cloudius-systems/seastar-dev into db
Static column queries, from Tomasz.
2015-03-26 16:24:29 +02:00
Tomasz Grabiec
2452df8617 tests: cql: Add test for static column query 2015-03-26 14:58:36 +01:00
Tomasz Grabiec
b8063cd76e cql3: Support for querying of static columns 2015-03-26 14:58:36 +01:00
Tomasz Grabiec
fcb4036408 cql3: Fix reading of query limit parameter 2015-03-26 14:57:20 +01:00
Tomasz Grabiec
35b4199374 Merge remote-tracking branch 'dev/penberg/create-keyspace/v4'
From Pekka:

This series adds support for creating keyspaces. We already have the CQL
front-end implemented so all that remains is converging mutations in
legacy_schema_tables.cc as well as parts of migration_manager.hh and
wiring that up to the CQL execution path.
2015-03-26 14:25:54 +01:00
Avi Kivity
1c1c4f923a db: fix collection_type_impl::deserialize_mutation_form() types
It accepts a bytes_view instead of the type-safe wrapper.
2015-03-26 14:31:01 +02:00
Avi Kivity
e664c1640f Merge branch 'tgrabiec/fixes' of github.com:cloudius-systems/seastar-dev into db 2015-03-26 13:47:12 +02:00
Pekka Enberg
d0fef3e31b db: Apply mutations in legacy_schema_tables::merge_schema()
Pass a reference to storage_proxy and apply mutations in
legacy_schema_tables::merge_schema().

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 13:33:48 +02:00
Pekka Enberg
5bab83b9e0 cql3: Switch to futurized migration_manager
Migration manager announce functions now return a future. Switch to the
new API in modification statements.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 13:33:48 +02:00
Pekka Enberg
5dfe06d5f1 service: Convert MigrationManager to C++, take 2
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 13:33:48 +02:00
Pekka Enberg
f0cb6a4f6e db: Convert LegacySchemaTables to C++, take 2
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 13:33:48 +02:00
Pekka Enberg
d20d835331 service: Add mutate_local() to storage proxy
Schema mutations are applied locally so add a mutate_local() function to
storage proxy.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 13:33:48 +02:00
Shlomi Livne
514bead08b tests: Fix output bug in test.py
The change to add a timeout to test.py introduced a bug that caused
thestdout,stderr not to be printed in case of an error. This patch fixes
this bug.

Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-03-26 13:27:49 +02:00
Tomasz Grabiec
c434b058bd Merge tag 'avi/collections-query/v1' 2015-03-26 12:16:17 +01:00
Pekka Enberg
3150bb5b78 database: Initialize system keyspace in database constructor
System keyspace is used for things like keyspace and table metadata.
Initialize it in database constructor so that they're always available.
Needed for CQL create keyspace test case, for example.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 12:41:00 +02:00
Pekka Enberg
fd8e92ab07 database: Add mutation::set_cell() variant
This adds a set_cell() variant which accepts a column name and a
boost::any value and does column definition lookup and decomposition
under the hood. Simplifies code that manipulates system tables directly
in db/legacy_schema_tables.cc.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 12:40:58 +02:00
Pekka Enberg
0117906af1 db_clock: Add now_in_usecs() helper function
Add a helper function similar to FBUtilities.timestampMicros() in origin.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 12:25:00 +02:00
Pekka Enberg
ad0deebbb4 config: Convert KSMetaData to C++, take 2
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-26 12:25:00 +02:00
Avi Kivity
37633a2cdd tests: tests list setter_by_index 2015-03-26 12:14:01 +02:00
Avi Kivity
b3f3c76dd8 cql3: fix overzealous move in modification_statement::get_mutations()
'keys' and 'prefix' are used twice in the same expression, and as the language
does not guarantee any ordering in this case, any moves are illegal.

Get rid of them.
2015-03-26 12:14:01 +02:00
Avi Kivity
b650383d67 cql3: implement read_required_rows()
Some modification statements require reading rows before modifying them;
implement it.
2015-03-26 12:14:01 +02:00
Avi Kivity
bfa37eb2f8 db: implement get_partition_slice() for collections 2015-03-26 12:14:01 +02:00
Avi Kivity
6a8b682c0e db: add collection_type_impl::mutation_view::materialize()
Converts a mutation_view to a mutation, so it can be modified.
2015-03-26 12:14:01 +02:00
Tomasz Grabiec
61c4d5150c cql3: Fix error message
single_column_primary_key_restrictions are used for both partition key
and clustering key restrictions.
2015-03-26 10:46:53 +01:00
Tomasz Grabiec
cd7e607a07 cql3: Optimize bounds() calculation for all EQ restrictions case 2015-03-26 10:46:53 +01:00
Tomasz Grabiec
38c9cd4400 cql3: Optimize iteration over column restrictions 2015-03-26 10:46:53 +01:00
Tomasz Grabiec
283f52549b test: cql: Add test for results order with variable length keys 2015-03-26 10:46:53 +01:00
Tomasz Grabiec
2bc1c7b534 tests: Make rows_assertions::with_rows() verify order 2015-03-26 10:46:53 +01:00
Tomasz Grabiec
ef52bb6048 type: Remove unused code 2015-03-26 10:46:53 +01:00
Tomasz Grabiec
17825a8e81 tests: Test is_prefix_of() on keys rather than tuples
tuple_type<>::is_prefix_of() is not used and will be removed.
2015-03-26 10:46:53 +01:00
Tomasz Grabiec
ec867fb93d type: Fix _is_byte_order_comparable calculation
Because the length preceeds component contents, serialized tuples are
not byte order comparable. It breaks lexi ordering. It should be that
"abc" < "b", but with length prefix "<3>abc" is greater than
"<1>b". Only single element tuples can be byte order capared because
they have no length component.

Spotted during code review.
2015-03-26 10:22:13 +01:00
Tomasz Grabiec
7a6dd463dd types: Optimize serialization of single-element tuples
There are two sides of this optimization:

 1) We don't store the length of the last component, so the
    representation is now shorter.

 2) A single-element tuple is serialized exactly as the component it
    holds, which allows us to optimize conversions for such keys.
2015-03-26 10:21:48 +01:00
Avi Kivity
30c3348702 db: add ostream support to consistency_level 2015-03-26 09:34:49 +02:00
Marek Waszkiewicz
bf564b463a README: add xen-devel boost-devel for fedora 21 build instructions
Signed-off-by: Marek Waszkiewicz <marek.waszkiewicz77@gmail.com>
2015-03-26 09:33:03 +02:00
Avi Kivity
467ede53b7 Merge branch 'tgrabiec/query-v2' of github.com:cloudius-systems/seastar-dev into db
Query tests and cleanups, from Tomasz.
2015-03-25 15:45:14 +02:00
Nadav Har'El
d305e1f95c sstables: add FIXME
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-25 15:45:00 +02:00
Tomasz Grabiec
121fa4ff46 test: Introduce test for in-memory CQL query performance
The schema and queries resemble that used by cassandra-stress.

Results on my laptop:

$ build/release/tests/perf/perf_simple_query -c1
Concurrency = 100 x 1
Creating 1000 partitions...
Timing single key selects...
452146.50 tps
449365.24 tps
457650.58 tps
460334.78 tps
458281.51 tps
Timing random key selects...
439181.03 tps
449899.67 tps
405146.90 tps
440228.84 tps
440889.50 tps

$ build/release/tests/perf/perf_simple_query -c3
Concurrency = 100 x 3
Creating 1000 partitions...
Timing single key selects...
302615.08 tps
301471.02 tps
303040.67 tps
302114.77 tps
302465.13 tps
Timing random key selects...
627516.46 tps
628978.04 tps
623664.15 tps
624098.48 tps
614549.85 tps
2015-03-25 12:59:25 +01:00
Tomasz Grabiec
8f73f7df47 tests: Add time_parallel() utility 2015-03-25 12:59:25 +01:00
Raphael S. Carvalho
d81cbbabf7 tests: add testcase for sstables compression info
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-03-25 13:24:48 +02:00
Raphael S. Carvalho
92b75413cb sstables: add function to set generation
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-03-25 13:23:24 +02:00
Raphael S. Carvalho
daaa1a6dcb sstables: extend it to support write of components
By the time being, compression info is the unique component being
written by store(). Changes introduced by this patch are generic,
so as to make it easier writing other components as well.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-03-25 13:22:42 +02:00
Nadav Har'El
b4e805e811 sstable: test reading from data file
Test sstable::data_read(pos, len), for both compressed and uncompressed
data files. We already have compressed and uncompressed sstables in our
test tree with identical data, we read from them in a particular position
and expect to find the string "gustaf" there.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-25 12:35:33 +02:00
Nadav Har'El
891763d7fd sstables: implement low-level data file random-access reading
This patch implements sstable::data_read(pos, len) to do random-access
read of a specific byte range from the data file. Later we'll determine
the byte range needed to read a specific row, using the summary and index
files.

This function works for either a compressed or uncompressed data file.
To support the compressed data file, we need to determine when opening
the sstable also the data file's size, and then make a compression_metadata
object using the data we read from the compression file and the data
file's size.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-25 12:35:32 +02:00
Nadav Har'El
4d57c8fd28 sstables: fix LZ4 decompression
It turns out that Cassandra's LZ4Compressor doesn't use the LZ4
compressor directly - instead it prepends the uncompressed length,
in 4-byte little-endian (!) encoding, to the compressed chunk.
We don't need this extra information - we already know the expected
uncompressed chunk length, so we need to just skip it.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-03-25 12:35:32 +02:00
Tomasz Grabiec
b829062985 tests: Make time_it() template static 2015-03-25 10:36:19 +01:00