Commit Graph

53948 Commits

Author SHA1 Message Date
Avi Kivity
fe9c5a2750 shared_ptr: const correctness for lw_shared_ptr::use_count() 2015-05-06 11:41:51 +03:00
Avi Kivity
211c77bf52 shared_ptr: add const support for lw_shared_ptr 2015-05-06 11:41:33 +03:00
Pekka Enberg
7a965f1cf2 service: Use mem_fn() in storage_proxy::query_local()
Suggested by Avi.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 11:28:38 +03:00
Avi Kivity
63be3f1dc1 Merge branch 'penberg/commitlog-cleanups' of github.com:cloudius-systems/seastar-dev into db
Minor commitlog code cleanups, from Pekka.
2015-05-06 10:01:27 +03:00
Pekka Enberg
33b37f7954 cql: Fix replication strategy class lookup
Fix replication strategy class lookup to support unqualified names such
as "SimpleStrategy". The AbstractReplicationStrategy.getClass() method
in origin does the same thing.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 10:01:00 +03:00
Pekka Enberg
bf1734c480 db/commitlog: Use 'pragma once' in commitlog.hh
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:47:42 +03:00
Pekka Enberg
a32ae69b2b db/commitlog: Minor formatting fixes
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:47:21 +03:00
Pekka Enberg
9920d58b70 db/commitlog: Use C++ type aliases
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-06 09:43:04 +03:00
Avi Kivity
e811690588 db: return smart pointers for column_family read-side lookups
A lookup can cause several data sources to be merged, in which case we will
have to return a temporary (containing data from all the data sources).

For simplicity, we start by always returning a temporary.
2015-05-05 20:21:04 +03:00
Avi Kivity
cdfe03a5b2 db: implement mutation_partition assignment operators 2015-05-05 20:21:04 +03:00
Avi Kivity
aa94cd62dc db: implement mutation_partition copy constructor
As a temporary measure, we will return temporary mutation_partition
objects in response to queries, so we need an easy way to construct
them.
2015-05-05 20:21:04 +03:00
Avi Kivity
8028fb441a db: make column_family a class, not a struct
Don't expose privates in public.
2015-05-05 20:21:03 +03:00
Avi Kivity
f57d7067ff Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-05 20:17:45 +03:00
Avi Kivity
3a0de14aa8 db: more const correctness for column_family and component types
Ensure that read-side accessors are const.  This is important in preparation
for multiple memtables (and later, sstables) since a read-side
mutation_partition may be a temporary object coming from multiple memtables
(and sstables) while a write-side mutation_partition is guaranteed to belong
to a single memtable (and thus, not be temporary).

Since writers will want non-const mutation_partitions to write to, they won't
be able to use the read-side accessors by accident.
2015-05-05 19:37:21 +03:00
Avi Kivity
e1526a697e nway_merger: allow for comparators without default constructors 2015-05-05 19:37:21 +03:00
Avi Kivity
9d9c7471da nway_merger: use pop_front() instead of erase()
pop_front() is a more relaxed requirement than erase() -- it is provided
by more containers, specifically boost::iterator_range<>, which we'd like
to use.
2015-05-05 19:37:21 +03:00
Avi Kivity
b7757617bb nway_merger: strip osv namespace 2015-05-05 19:37:21 +03:00
Avi Kivity
a25c92eda6 Import nway_merger<> from osv 2015-05-05 19:37:21 +03:00
Glauber Costa
1b844325dd sstables: remove leftover printfs
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 17:32:23 +03:00
Avi Kivity
00dfa05833 Merge branch 'mutation-v2' of github.com:glommer/urchin into db
Read mutations from sstables, from Glauber.

Conflicts:
	sstables/key.cc
	sstables/key.hh
	sstables/sstables.cc
	sstables/sstables.hh

[avi: adjust sstables/partition.cc:149 for ambiguity due to new
      basic_sstring range constructor]
2015-05-05 17:17:08 +03:00
Glauber Costa
233a84eb6b sstables: convert row to internal representation
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Glauber Costa
e8b3198580 sstables: bloom filter api
Currently always return true.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Glauber Costa
f6e8141e0d binary search: allow direct token passing
In some situations, we already have the token computed, and there is no need to
compute it again during binary search. As a matter of fact, even in the
simplest of the situations, we'll run a binary search over the summary, and if
the key is believed to be in a particular bucket, we will run a search over the
index. That's two token computations right away.

It would be better to have an interface that allows the token to be passed
directly if known. We provide a wrapper that allows the binary search to
operate on a key only.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Glauber Costa
ea81c44095 sstable key: composite view
Can be used to parse a composite value (like a column name)

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Glauber Costa
d977220b8e sstable key: explode a partition key
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-05 09:28:48 -04:00
Asias He
7516966db5 message: Fix passing by reference
From Avi:
'''
start() returns immediately, then the listen object is destroyed (it is
on the stack), then the lambda runs. [&] is also very dangerous.
'''

This fixes a bug in initialization of messaging_service.

[asias@hjpc urchin]$ ./message --listen-address 127.0.0.100
listen =127.0.0.100
protocol::server::server: addr=127.0.0.100:7000
opts.reuse_address=1, _reuseport=1
messaging_service on cpu_id=0, ip=127.0.0.100,
listen_address=127.0.0.100
protocol::server::server: addr=0.36.145.24:7000
opts.reuse_address=1, _reuseport=1
messaging_service on cpu_id=1, ip=0.36.145.24,
listen_address=0.36.145.24
Messaging server listening on port 7000 ...
2015-05-05 13:50:43 +03:00
Gleb Natapov
de2e297016 rpc: handle future<> return type of an rpc handler properly 2015-05-05 12:08:06 +03:00
Avi Kivity
a31d8fcc49 Merge branch 'sstable-write' into db
Generation of SSTable Data File, from Raphael.
2015-05-05 11:19:47 +03:00
Raphael S. Carvalho
73a2fd04a7 tests: Add testcase to generation of SSTable data file
Each testcase generates a data file from a given schema + a mutation,
and then check the correctness of the generated data file.
The data used to check the correctness of the generated data file
is from C* data file itself.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-05 11:15:46 +03:00
Raphael S. Carvalho
c646307711 sstables: add initial support to generation of data file
This initial version supports:
Regular columns
Clustering key
Compound Partition key
Compound Clustering key
Static Row

What's not supported:
Counters
Range tombstones
Collections
Compression
anything else that wasn't mentioned in the support list.

The generation of the data file consists of iterating through
a set of mutation_partition from a column_family, then writing
the SSTable rows according to the format.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-05 11:15:46 +03:00
Raphael S. Carvalho
9a263328d5 mutation_partition: rename tombstone_for_static_row to partition_tombstone
tombstone from mutation partition is also used for static row purposes,
but under the hood, it's the partition tombstone.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-05 11:15:46 +03:00
Raphael S. Carvalho
f328609837 sstables: add write support to bytes_view
disk_string provides an easy way of serializing a string into the form
{ size, string[size] }. sstables::key, atomic_cell, among other types
provides a bytes_view for the view of data, so that's why this change
is needed. Otherwise, I would have to convert bytes_view into bytes,
which requires copy.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-05 11:15:45 +03:00
Raphael S. Carvalho
e0949cdb82 sstables: add support to create composite from clustering_key
from_components() is made more generic so as to be re-used for
the creation of a composite from a clustering_key.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-05 11:15:45 +03:00
Raphael S. Carvalho
9d7de45ca4 mutation_partition: add function to return a reference to _rows
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-05 11:15:45 +03:00
Asias He
f689ef705a gossip: Forward gossip message to cpu0
There is one gossiper instance per node and it runs on cpu0 only. We can
not guarantee there will always be a core to core tcp connection within
messaging service, so messaging service needs to listen on all cpus.
When a remote node connects to local node with a connection bound to cpu
other than cpu0, we need to forward this message to cpu0.
2015-05-05 10:56:55 +03:00
Calle Wilund
2835da3d8c config: make sure aliases actually work properly
The nature of how boost::program_options are applied made
alias definitions clash and overwrite each other.
Make aliases not have default, and overrride bpo handling
of defaults so that they are not applied to the config object
(only used in pretty-print)

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-05 10:56:54 +03:00
Avi Kivity
89e4fb5991 Merge branch 'gleb/merge' of github.com:cloudius-systems/seastar-dev into db
Merge Gleb's merge of the rpc changes into seastar.
2015-05-05 10:41:34 +03:00
Gleb Natapov
78d36eabc2 Merge remote-tracking branch 'origin/master' into tmpmerge
message_service: create object using placement new()

Object reference may point to a unit member that was not properly
created, so assign will not work on it. Use placement new() instead.
2015-05-05 10:39:51 +03:00
Raphael S. Carvalho
36701cafcb iostream: make close() flush
This change is intended to make close() flush the stream before
proceeding with the close itself. This improves the situation
where we have to guarantee that the stream is flushed before
closing it.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-05 10:11:18 +03:00
Asias He
e5499ec96e README: Add missing yaml-cpp-devel package
db/config.o db/config.cc
db/config.cc:6:27: fatal error: yaml-cpp/yaml.h: No such file or directory
 #include <yaml-cpp/yaml.h>
                           ^
A README for Urchin is added.
2015-05-05 10:09:59 +03:00
Asias He
061de14d01 README: Rename README-OSv to README-OSv.md 2015-05-05 10:09:18 +03:00
Calle Wilund
e1c0ddea93 commitlog: make sure we don't race creating segments
Use a semaphore as gatekeeper to make sure we finish creating a segment
before starting a new one.

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-05 09:59:27 +03:00
Gleb Natapov
2ec7535969 rpc: allow handler to receive non default constructable types
Currently it is not possible because a type is instantiated before been
deserialized. Fix that by allocating space for an object by using
std::aligned_storage.
2015-05-03 19:02:33 +03:00
Avi Kivity
6d4868a357 Merge branch 'fix-bsearch-v2' of github.com:glommer/urchin into db
Fix sstable binary search key compare, from Glauber.
2015-05-01 11:01:43 +03:00
Glauber Costa
cd001d208c sstable: calculate data size
It would be useful in some situations to know where does the data ends. If the
file is uncompressed, this is equivalent to the file length. If the data file
is compressed, this information needs to come from the compression structure.

Provide a method that encodes that.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-04-30 16:04:19 -04:00
Glauber Costa
fe4b33053e sstable_test: move some interesting functions to a header file
This test is too big already. The mutation tests would be better off in their own files

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-04-30 15:50:16 -04:00
Glauber Costa
cccc3c8597 data_input: allow other sizes to be used for reading a blob.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-04-30 15:49:36 -04:00
Glauber Costa
08d7a7c3d5 binary search: clarify return value
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-04-30 15:31:51 -04:00
Glauber Costa
9993344ab8 tests: test the binary search over a full index list
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-04-30 15:15:36 -04:00
Glauber Costa
1bd12ec2ff binary search: fix token comparator
compare_unsigned cannot be used directly on a token. We need to use the
dht-provided token comparators.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-04-30 14:09:45 -04:00