Commit Graph

3171 Commits

Author SHA1 Message Date
Tomasz Grabiec
96bbac8a57 dht: Make partitioner work on partition_key_view 2015-05-06 15:52:56 +02:00
Tomasz Grabiec
0deec287d4 tests: keys: Add test for key <-> view coversions 2015-05-06 15:52:56 +02:00
Tomasz Grabiec
0bc0071373 keys: Introduce view wrappers
We now have partition_key_view, clustering_key_view, etc.
Database APIs will be extended to also accept views.

This will alows us to avoid allocations in certain scenarios.
2015-05-06 15:52:56 +02:00
Calle Wilund
46e0676a7d commitlog: Add reader stream/subscription
Generic read-all-stream from a commit log segmen file.
Provides a byte view for each data entry, doing CRC checks and padding skips.

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:45:13 +03:00
Calle Wilund
2a14037f79 db: add data_input constructors & methods
* Construct from temporary_buffer<>
* Construct from basic_string_view<*> (cast)
* Allow skipping bytes

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:45:06 +03:00
Avi Kivity
1c8b0a4b1c Merge branch 'master' of github.com:cloudius-systems/urchin into db 2015-05-06 16:31:44 +03:00
Calle Wilund
7f685abca0 commitlog: added file header space twice
Checked wrong var == 0 when creating second mem buffer in segment

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:56 +03:00
Calle Wilund
897083e213 commitlog: Sync check time delta was inverted
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:49 +03:00
Calle Wilund
d2d3547115 commitlog: Initial data chunk offset should be 16
As in "after file header".

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:43 +03:00
Tomasz Grabiec
36ad6c9aa8 Merge tag 'avi/memtables/v3' from seastar-dev.git
Multiple memtable support from Avi.
2015-05-06 15:02:42 +02:00
Avi Kivity
4875028fb3 tests: add test for range scan memtable merging 2015-05-06 15:43:43 +03:00
Avi Kivity
eccb6bfe08 tests: add tests for multiple memtables in column family (single partition) 2015-05-06 15:43:09 +03:00
Avi Kivity
ef5c661d11 db: add variant of column_family::for_all_partitions() for unit tests
Since it's for tests, we can pass a slower std::function<>.
2015-05-06 15:43:06 +03:00
Avi Kivity
1d6ac071c0 db: add API to seal current active memtable 2015-05-06 15:39:31 +03:00
Avi Kivity
b717b4b6bb db: add ostream operators for more types (mutation_partition and components) 2015-05-06 15:39:31 +03:00
Avi Kivity
22969aeb18 db: support for multiple memtables
Each column family now contains multiple memtables, with one designated as
"active" receiving all writes, while the others only serve reads.
2015-05-06 15:39:29 +03:00
Raphael S. Carvalho
c3deca9575 sstables: fix problem with scope of deletion_time
write() takes a reference to deletion_time which means that the
object must survive until the continuation is scheduled.
This patch fix the problem the use of do_with().
The problem was making debug version of urchin tests fail.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-06 14:38:53 +02:00
Avi Kivity
5e81b92dc0 db: split column_family::partitions into a new memtable class
In preparation for multiple memtables, move column_family::partitions into
its own class, and forward relevant calls from column_family.

A testonly_all_memtables() function was added to support sstable_test.
2015-05-06 15:35:14 +03:00
Avi Kivity
bc669add40 schema: const correctness
Make schema accessors const, and make schema_ptr refer to a const schema.
2015-05-06 13:52:59 +02:00
Avi Kivity
cc291d7e3b db: improve sharding
Currently we use the first byte of the token for determining the local
shard.  This is suboptimal for two reasons:

 1. the first bytes of the token were already used to select the node,
    so they are not randomly distributed
 2. using a single byte is not sufficient for large core counts, as the
    modulo operation will not return evenly distributed results

Fix by using the final two bytes of the token.
2015-05-06 13:19:44 +02:00
Avi Kivity
f0e30bd1be Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-06 13:32:31 +03:00
Avi Kivity
f3afe9051f Merge branch 'shared_ptr'
const support for lw_shared_ptr<>.
2015-05-06 13:27:00 +03:00
Amnon Heiman
f4f6c0a4a3 smp: add all_cpus() helper
This adds a static method to return a range object to smp.
with this patch it is possible to use:

for (auto i : smp::all_cpus())

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-06 11:49:45 +03:00
Avi Kivity
fe9c5a2750 shared_ptr: const correctness for lw_shared_ptr::use_count() 2015-05-06 11:41:51 +03:00
Avi Kivity
211c77bf52 shared_ptr: add const support for lw_shared_ptr 2015-05-06 11:41:33 +03:00
Pekka Enberg
7a965f1cf2 service: Use mem_fn() in storage_proxy::query_local()
Suggested by Avi.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 11:28:38 +03:00
Avi Kivity
63be3f1dc1 Merge branch 'penberg/commitlog-cleanups' of github.com:cloudius-systems/seastar-dev into db
Minor commitlog code cleanups, from Pekka.
2015-05-06 10:01:27 +03:00
Pekka Enberg
33b37f7954 cql: Fix replication strategy class lookup
Fix replication strategy class lookup to support unqualified names such
as "SimpleStrategy". The AbstractReplicationStrategy.getClass() method
in origin does the same thing.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 10:01:00 +03:00
Pekka Enberg
bf1734c480 db/commitlog: Use 'pragma once' in commitlog.hh
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:47:42 +03:00
Pekka Enberg
a32ae69b2b db/commitlog: Minor formatting fixes
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:47:21 +03:00
Pekka Enberg
9920d58b70 db/commitlog: Use C++ type aliases
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:43:04 +03:00
Avi Kivity
e811690588 db: return smart pointers for column_family read-side lookups
A lookup can cause several data sources to be merged, in which case we will
have to return a temporary (containing data from all the data sources).

For simplicity, we start by always returning a temporary.
2015-05-05 20:21:04 +03:00
Avi Kivity
cdfe03a5b2 db: implement mutation_partition assignment operators 2015-05-05 20:21:04 +03:00
Avi Kivity
aa94cd62dc db: implement mutation_partition copy constructor
As a temporary measure, we will return temporary mutation_partition
objects in response to queries, so we need an easy way to construct
them.
2015-05-05 20:21:04 +03:00
Avi Kivity
8028fb441a db: make column_family a class, not a struct
Don't expose privates in public.
2015-05-05 20:21:03 +03:00
Avi Kivity
f57d7067ff Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-05 20:17:45 +03:00
Avi Kivity
3a0de14aa8 db: more const correctness for column_family and component types
Ensure that read-side accessors are const.  This is important in preparation
for multiple memtables (and later, sstables) since a read-side
mutation_partition may be a temporary object coming from multiple memtables
(and sstables) while a write-side mutation_partition is guaranteed to belong
to a single memtable (and thus, not be temporary).

Since writers will want non-const mutation_partitions to write to, they won't
be able to use the read-side accessors by accident.
2015-05-05 19:37:21 +03:00
Avi Kivity
e1526a697e nway_merger: allow for comparators without default constructors 2015-05-05 19:37:21 +03:00
Avi Kivity
9d9c7471da nway_merger: use pop_front() instead of erase()
pop_front() is a more relaxed requirement than erase() -- it is provided
by more containers, specifically boost::iterator_range<>, which we'd like
to use.
2015-05-05 19:37:21 +03:00
Avi Kivity
b7757617bb nway_merger: strip osv namespace 2015-05-05 19:37:21 +03:00
Avi Kivity
a25c92eda6 Import nway_merger<> from osv 2015-05-05 19:37:21 +03:00
Glauber Costa
1b844325dd sstables: remove leftover printfs
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 17:32:23 +03:00
Avi Kivity
00dfa05833 Merge branch 'mutation-v2' of github.com:glommer/urchin into db
Read mutations from sstables, from Glauber.

Conflicts:
	sstables/key.cc
	sstables/key.hh
	sstables/sstables.cc
	sstables/sstables.hh

[avi: adjust sstables/partition.cc:149 for ambiguity due to new
      basic_sstring range constructor]
2015-05-05 17:17:08 +03:00
Glauber Costa
233a84eb6b sstables: convert row to internal representation
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Glauber Costa
e8b3198580 sstables: bloom filter api
Currently always return true.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Glauber Costa
f6e8141e0d binary search: allow direct token passing
In some situations, we already have the token computed, and there is no need to
compute it again during binary search. As a matter of fact, even in the
simplest of the situations, we'll run a binary search over the summary, and if
the key is believed to be in a particular bucket, we will run a search over the
index. That's two token computations right away.

It would be better to have an interface that allows the token to be passed
directly if known. We provide a wrapper that allows the binary search to
operate on a key only.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Glauber Costa
ea81c44095 sstable key: composite view
Can be used to parse a composite value (like a column name)

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Glauber Costa
d977220b8e sstable key: explode a partition key
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Asias He
7516966db5 message: Fix passing by reference
From Avi:
'''
start() returns immediately, then the listen object is destroyed (it is
on the stack), then the lambda runs. [&] is also very dangerous.
'''

This fixes a bug in initialization of messaging_service.

[asias@hjpc urchin]$ ./message --listen-address 127.0.0.100
listen =127.0.0.100
protocol::server::server: addr=127.0.0.100:7000
opts.reuse_address=1, _reuseport=1
messaging_service on cpu_id=0, ip=127.0.0.100,
listen_address=127.0.0.100
protocol::server::server: addr=0.36.145.24:7000
opts.reuse_address=1, _reuseport=1
messaging_service on cpu_id=1, ip=0.36.145.24,
listen_address=0.36.145.24
Messaging server listening on port 7000 ...
2015-05-05 13:50:43 +03:00
Gleb Natapov
de2e297016 rpc: handle future<> return type of an rpc handler properly 2015-05-05 12:08:06 +03:00