Commit Graph

186 Commits

Author SHA1 Message Date
Pekka Enberg
4dc488afb2 database: Store metadata in 'struct keyspace'
Store a lw_shared_ptr<keyspace_metadata> in struct keyspace so callers
in migration manager, for example, can look it up.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-25 09:12:29 +02:00
Avi Kivity
bbc8dbbab5 db: abstract memtable empty test 2015-05-21 15:48:51 +03:00
Avi Kivity
068a3190d1 db: add written sstable to read set 2015-05-21 14:44:04 +03:00
Avi Kivity
21f7975b57 db: don't flush empty memtables
You can't have an empty sstable, so don't flush an empty memtable.
2015-05-21 13:28:14 +03:00
Avi Kivity
3961e0391a db: fix sstable use-after-free in seal_active_memtable()
We write the new sstable, but we don't wait for the write to complete,
so do_with() kills it.
2015-05-21 13:18:35 +03:00
Avi Kivity
b483d506eb Merge "generation of sstable files"
More sstable component generation, from Raphael.
2015-05-21 11:11:55 +03:00
Raphael S. Carvalho
b00bb80c18 sstables: improve approach to generation of sstables
The function write_datafile was renamed to write_components and made a member
of sstable class because write of components requires access to private
members. This change is an important step towards the generation of components
other than data file.
The respective testcases were adapted to the changes.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-20 14:10:24 -03:00
Avi Kivity
32c274f69e db: wire up sstable read path for single partition lookups
Untested, as we never add sstables to the read set yet.
2015-05-20 17:01:28 +02:00
Avi Kivity
ff42d58881 db: use CoW to modify the memtable list in column_family
Allow memtables to be removed from a column_family while a running query
continues to use them.
2015-05-20 16:00:00 +03:00
Avi Kivity
1342553fed db: remove column_family::testonly_all_memtables()
Unused and gets in the way.
2015-05-20 15:28:53 +03:00
Avi Kivity
f8f6e979ef db: use CoW to modify the sstable table in column_family
Allow sstables to be removed from a column_family while a running query
continues to use them.
2015-05-20 15:17:35 +03:00
Pekka Enberg
f34f195860 database: Fix capture by reference in create_keyspace()
We cannot capture keyspace_metadata by reference because it can be
allocated on the stack. Fixes SIGSEGV while running cassandra-stress.
The bug was introduced in commit commit cd35617 ("database: Use
keyspace_metadata for creation functions").

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-20 12:35:55 +03:00
Avi Kivity
42a587a954 db: reindent column_family::query() 2015-05-19 23:47:26 +03:00
Avi Kivity
6867ad8fd7 db: reindent column_family::for_all_partitions 2015-05-19 23:46:11 +03:00
Tomasz Grabiec
137b3beb2f Merge tag 'avi/readpath-prep/v1' from seastar-dev.git
From Avi:

"This patchset prepares for adding sstables to the read path.  Because sstables
involve I/O, their APIs return futures, which means that APIs that may call
those sstable APIs also need to return futures.

This patchset uses the two-space indent + do_with + reference aliases trick
to make patches more readable.  Cleanup patches will follow once it is merged."
2015-05-19 20:39:36 +02:00
Pekka Enberg
56d6fdacfe database: Simplify replication strategy initialization
Initialize replication strategy when keyspace is being created now that
we have access to keyspace_metadata.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 15:27:47 +03:00
Pekka Enberg
cd35617855 database: Use keyspace_metadata for creation functions
Use the keyspace_metadata type for keyspace creation functions. This is
needed to be able to have a mapping from keyspace name to keyspace
metadata for various call-sites.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 15:27:47 +03:00
Avi Kivity
db04bba208 db: futurize the single partition query path
Prepare for disk reads.
2015-05-19 15:13:09 +03:00
Avi Kivity
738be63b28 db: define column_family move constructor in .cc
Allows using it from files that do not include sstable.hh.
2015-05-19 15:13:09 +03:00
Avi Kivity
72d721ea56 db: futurize column_family::query() outer loop
In preparation for reading from sstables, allow the outer loop of
column_family::query() (iterating over partition ranges) do defer.
2015-05-19 15:13:06 +03:00
Avi Kivity
d6754823b8 db: conform to sstable naming convention wrt generation 2015-05-19 15:01:29 +03:00
Avi Kivity
104557b3a0 Merge "Keyspace metadata cleanup"
Pekka says:

"We are going to keep the ks_meta_data class around and use it in core
code like the migration manager. Therefore, clean up the class and move
it to the database.hh where user_types_metadata also is defined in. As a
bonus, this also fixes the circular dependency between ks_meta_data.hh
and database.hh."
2015-05-19 11:39:03 +03:00
Avi Kivity
06a9eb2f31 db: fix bad print() in seal_active_memtable()
Noticed by Raphael.
2015-05-19 11:29:52 +03:00
Pekka Enberg
8380df84b4 database: Rename ks_meta_data to keyspace_metadata
Follow the naming convention set by user_types_metadata and rename
ks_meta_data to keyspace_metadata.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 11:24:06 +03:00
Pekka Enberg
032af4d53b database: Move ks_meta_data definition to database.hh
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 11:03:28 +03:00
Pekka Enberg
e80fe254d2 config: Encapsulate ks_meta_data member variables
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 10:51:39 +03:00
Avi Kivity
886928962b db: fix query of multiple pkeys ("pkey IN (k1, k2, k3)") failing on missing key
Need to skip on missing partition, not stop the whole thing.
2015-05-18 17:55:28 +02:00
Avi Kivity
07d7f410f3 Merge branch 'memtable' into db
Conflicts:
	database.hh
	memtable changes moved to memtable.hh
2015-05-18 15:50:24 +03:00
Avi Kivity
875148dae6 db: create keyspace/column_family directory structure
This is slightly awkwards, since the directory structure is not sharded.
This requires some processing to occur outside the shard, while the rest
is sharded.
2015-05-18 15:34:41 +03:00
Avi Kivity
20775b9d5c db: store a column_family's memtables in a list instead of a vector
A vector can cause memtables to be move()d around, which breaks any
code that captures a memtable's this pointer.

Fix by using a linked list.
2015-05-18 15:34:25 +03:00
Avi Kivity
394e0d3a8c db: make database::add_keyspace() return void
Returning a reference to the keyspace is dangerous in that the keyspace can
be moved away, when we start futurizing the add_keyspace() process.  Make
it return void and look up the keyspace at the point of use.
2015-05-18 15:34:25 +03:00
Avi Kivity
0eb842dc5b db: write memtable after sealing it
Still missing handling after write completes.
2015-05-18 15:00:33 +03:00
Avi Kivity
ca49d73f97 db: allow configuring a column family to be memory-only
Useful for tests.
2015-05-18 15:00:33 +03:00
Avi Kivity
dda5cbfd0d db: make column_family and keyspace configurable
Currently used for the data directory.
2015-05-18 15:00:31 +03:00
Avi Kivity
7842113cb6 db: prune some unused column_familiy methods
Made redundant by switching tests to using memtable directly.
2015-05-18 14:59:02 +03:00
Avi Kivity
d9d16a37b3 db: avoid storing frozen_mutation serialized size in commit log
The commit log already stores the record size, so we can get away with just
storing the data.

Improves 7630fe332d.
2015-05-18 09:03:21 +02:00
Avi Kivity
40c2d91cd8 db: add memtable::find_or_create_row_slow()
Useful for tests that do not need a column_family.
2015-05-17 10:31:22 +03:00
Tomasz Grabiec
f656ae8ed4 db: Encapsulate deletable_row fields 2015-05-13 08:56:54 +02:00
Tomasz Grabiec
dbc40dfb09 db: Encapsulate the "row" class
Reduces coupling. User's should not rely on the fact that it's an
std::map<>.  It also allows us to extend row's interface with
domain-specific methods, which are a lot easier to discover than free
functions.
2015-05-13 08:56:54 +02:00
Tomasz Grabiec
56bea440a7 mutation_partition: Pass schema by const& where applicable
If method doesn't want to share schema ownership it doesn't have to
take it by shared pointer. The benefit is that it's slightly cheaper
and those methods may now be called from places which don't own
schema.
2015-05-13 08:56:54 +02:00
Avi Kivity
7630fe332d db: pass correct mutation size to commitlog
Use serialized_size() instead of reprentation().size(), to account for the
size header.
2015-05-11 19:19:23 +03:00
Tomasz Grabiec
eaceb61801 db: Add atomic_cell::deletion_time()
Deleted cells store deletion time not expiry time. This change makes
expiry() valid only for live cells with TTL and adds deletion_time(),
which is inteded to be used with deleted cells.
2015-05-10 12:03:26 +03:00
Tomasz Grabiec
f7abbda156 db: Apply frozen_mutation directly
We don't convert it back to mutation before applying.

mutation_partition has now apply() which works on
mutation_partition_view.
2015-05-08 09:19:02 +02:00
Tomasz Grabiec
bdcd11efe9 db: Use operator<< for partition printing 2015-05-08 09:19:02 +02:00
Tomasz Grabiec
4ab66de0ae db: Introduce frozen_mutation
The immediate motivation for introducing frozen_mutation is inability
to deserialize current "mutation" object, which needs schema reference
at the time it's constructed. It needs schema to initialize its
internal maps with proper key comparators, which depend on schema.

frozen_mutation is an immutable, compact form of a mutation. It
doesn't use complex in-memory strucutres, data is stored in a linear
buffer. In case of frozen_mutation schema needs to be supplied only at
the time mutation partition is visited. Therefore it can be trivially
deserialized without schema.
2015-05-08 09:19:01 +02:00
Tomasz Grabiec
f43836eb68 db: Handle expired cells in compare_atomic_cell_for_merge()
While at it, clarify some comments.
2015-05-06 18:31:21 +02:00
Tomasz Grabiec
5ba1486ae7 db: Rename "ttl" to "expiry" when it's used as time point
To avoid confusion with "ttl" the duration.
2015-05-06 17:27:22 +02:00
Tomasz Grabiec
36ad6c9aa8 Merge tag 'avi/memtables/v3' from seastar-dev.git
Multiple memtable support from Avi.
2015-05-06 15:02:42 +02:00
Avi Kivity
ef5c661d11 db: add variant of column_family::for_all_partitions() for unit tests
Since it's for tests, we can pass a slower std::function<>.
2015-05-06 15:43:06 +03:00
Avi Kivity
1d6ac071c0 db: add API to seal current active memtable 2015-05-06 15:39:31 +03:00