Commit Graph

3186 Commits

Author SHA1 Message Date
Avi Kivity
7bc465ae8a Merge branch 'tgrabiec/store-ttl-in-atomic-cell' of github.com:cloudius-systems/seastar-dev into db
Store both ttl and expiry time in atomic_cell, from Tomasz.
2015-05-07 10:23:52 +03:00
Tomasz Grabiec
b1e45e4401 db: Store ttl in atomic_cell
Origin does that, so should we. Both ttl and expiry time are stored in
sstables. The value of ttl seems to be used to calculate the read
digest (expiry is not used for that).

The API for creating atomic_cells changed a bit.

To create a non-expiring cell:

  atomic_cell::make_live(timestamp, value);

To create an expiring cell:

  atomic_cell::make_live(timestamp, value, expiry, ttl);

or:

  // Expiry is calculated based on current clock reading
  atomic_cell::make_live(timestamp, value, ttl_optional);
2015-05-06 19:42:38 +02:00
Tomasz Grabiec
784f841b8b tests: Add test for cell ordering 2015-05-06 18:32:17 +02:00
Tomasz Grabiec
f43836eb68 db: Handle expired cells in compare_atomic_cell_for_merge()
While at it, clarify some comments.
2015-05-06 18:31:21 +02:00
Avi Kivity
195a90a175 Merge branch 'tgrabiec/frozen-mutation-prep-v2' of github.com:cloudius-systems/seastar-dev into db
Preparatory changes before introducing frozen_mutation, from Tomasz.
2015-05-06 18:39:01 +03:00
Tomasz Grabiec
5ba1486ae7 db: Rename "ttl" to "expiry" when it's used as time point
To avoid confusion with "ttl" the duration.
2015-05-06 17:27:22 +02:00
Tomasz Grabiec
d6a003dc59 gc_clock: Store max_ttl as duration rather than time_point
Extra wrapping in time_point has no point.
2015-05-06 17:22:35 +02:00
Tomasz Grabiec
49f9794b1b schema: Introduce schema_builder
It's more flexible than schema's constructor. Primarily meant to be
used from tests, but not necessarily only there.
2015-05-06 16:40:48 +02:00
Tomasz Grabiec
5bd68aa514 data_input: Support read<bytes_view>() 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
2ab006949a atomic_cell: Extend interface of collection_mutation wrappers 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
45f66c5c9f atomic_cell: Add missing header 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
04846ed3d2 mutation: Make mutation equality comparable 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
87e00d97fa mutation: Introduce row_tombstones() getter 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
6762a92299 mutation_partition: Take schema by reference in apply_row_tombstone() 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
0aef6f2ff7 mutation_partition: Allow row lookup by clustering_key_view 2015-05-06 16:40:42 +02:00
Tomasz Grabiec
96bbac8a57 dht: Make partitioner work on partition_key_view 2015-05-06 15:52:56 +02:00
Tomasz Grabiec
0deec287d4 tests: keys: Add test for key <-> view coversions 2015-05-06 15:52:56 +02:00
Tomasz Grabiec
0bc0071373 keys: Introduce view wrappers
We now have partition_key_view, clustering_key_view, etc.
Database APIs will be extended to also accept views.

This will alows us to avoid allocations in certain scenarios.
2015-05-06 15:52:56 +02:00
Calle Wilund
46e0676a7d commitlog: Add reader stream/subscription
Generic read-all-stream from a commit log segmen file.
Provides a byte view for each data entry, doing CRC checks and padding skips.

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:45:13 +03:00
Calle Wilund
2a14037f79 db: add data_input constructors & methods
* Construct from temporary_buffer<>
* Construct from basic_string_view<*> (cast)
* Allow skipping bytes

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:45:06 +03:00
Avi Kivity
1c8b0a4b1c Merge branch 'master' of github.com:cloudius-systems/urchin into db 2015-05-06 16:31:44 +03:00
Calle Wilund
7f685abca0 commitlog: added file header space twice
Checked wrong var == 0 when creating second mem buffer in segment

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:56 +03:00
Calle Wilund
897083e213 commitlog: Sync check time delta was inverted
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:49 +03:00
Calle Wilund
d2d3547115 commitlog: Initial data chunk offset should be 16
As in "after file header".

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:43 +03:00
Tomasz Grabiec
36ad6c9aa8 Merge tag 'avi/memtables/v3' from seastar-dev.git
Multiple memtable support from Avi.
2015-05-06 15:02:42 +02:00
Avi Kivity
4875028fb3 tests: add test for range scan memtable merging 2015-05-06 15:43:43 +03:00
Avi Kivity
eccb6bfe08 tests: add tests for multiple memtables in column family (single partition) 2015-05-06 15:43:09 +03:00
Avi Kivity
ef5c661d11 db: add variant of column_family::for_all_partitions() for unit tests
Since it's for tests, we can pass a slower std::function<>.
2015-05-06 15:43:06 +03:00
Avi Kivity
1d6ac071c0 db: add API to seal current active memtable 2015-05-06 15:39:31 +03:00
Avi Kivity
b717b4b6bb db: add ostream operators for more types (mutation_partition and components) 2015-05-06 15:39:31 +03:00
Avi Kivity
22969aeb18 db: support for multiple memtables
Each column family now contains multiple memtables, with one designated as
"active" receiving all writes, while the others only serve reads.
2015-05-06 15:39:29 +03:00
Raphael S. Carvalho
c3deca9575 sstables: fix problem with scope of deletion_time
write() takes a reference to deletion_time which means that the
object must survive until the continuation is scheduled.
This patch fix the problem the use of do_with().
The problem was making debug version of urchin tests fail.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-06 14:38:53 +02:00
Avi Kivity
5e81b92dc0 db: split column_family::partitions into a new memtable class
In preparation for multiple memtables, move column_family::partitions into
its own class, and forward relevant calls from column_family.

A testonly_all_memtables() function was added to support sstable_test.
2015-05-06 15:35:14 +03:00
Avi Kivity
bc669add40 schema: const correctness
Make schema accessors const, and make schema_ptr refer to a const schema.
2015-05-06 13:52:59 +02:00
Avi Kivity
cc291d7e3b db: improve sharding
Currently we use the first byte of the token for determining the local
shard.  This is suboptimal for two reasons:

 1. the first bytes of the token were already used to select the node,
    so they are not randomly distributed
 2. using a single byte is not sufficient for large core counts, as the
    modulo operation will not return evenly distributed results

Fix by using the final two bytes of the token.
2015-05-06 13:19:44 +02:00
Avi Kivity
f0e30bd1be Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-06 13:32:31 +03:00
Avi Kivity
f3afe9051f Merge branch 'shared_ptr'
const support for lw_shared_ptr<>.
2015-05-06 13:27:00 +03:00
Amnon Heiman
f4f6c0a4a3 smp: add all_cpus() helper
This adds a static method to return a range object to smp.
with this patch it is possible to use:

for (auto i : smp::all_cpus())

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-06 11:49:45 +03:00
Avi Kivity
fe9c5a2750 shared_ptr: const correctness for lw_shared_ptr::use_count() 2015-05-06 11:41:51 +03:00
Avi Kivity
211c77bf52 shared_ptr: add const support for lw_shared_ptr 2015-05-06 11:41:33 +03:00
Pekka Enberg
7a965f1cf2 service: Use mem_fn() in storage_proxy::query_local()
Suggested by Avi.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 11:28:38 +03:00
Avi Kivity
63be3f1dc1 Merge branch 'penberg/commitlog-cleanups' of github.com:cloudius-systems/seastar-dev into db
Minor commitlog code cleanups, from Pekka.
2015-05-06 10:01:27 +03:00
Pekka Enberg
33b37f7954 cql: Fix replication strategy class lookup
Fix replication strategy class lookup to support unqualified names such
as "SimpleStrategy". The AbstractReplicationStrategy.getClass() method
in origin does the same thing.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 10:01:00 +03:00
Pekka Enberg
bf1734c480 db/commitlog: Use 'pragma once' in commitlog.hh
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:47:42 +03:00
Pekka Enberg
a32ae69b2b db/commitlog: Minor formatting fixes
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:47:21 +03:00
Pekka Enberg
9920d58b70 db/commitlog: Use C++ type aliases
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:43:04 +03:00
Avi Kivity
e811690588 db: return smart pointers for column_family read-side lookups
A lookup can cause several data sources to be merged, in which case we will
have to return a temporary (containing data from all the data sources).

For simplicity, we start by always returning a temporary.
2015-05-05 20:21:04 +03:00
Avi Kivity
cdfe03a5b2 db: implement mutation_partition assignment operators 2015-05-05 20:21:04 +03:00
Avi Kivity
aa94cd62dc db: implement mutation_partition copy constructor
As a temporary measure, we will return temporary mutation_partition
objects in response to queries, so we need an easy way to construct
them.
2015-05-05 20:21:04 +03:00
Avi Kivity
8028fb441a db: make column_family a class, not a struct
Don't expose privates in public.
2015-05-05 20:21:03 +03:00