Commit Graph

3196 Commits

Author SHA1 Message Date
Avi Kivity
55332b379f Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-07 13:39:00 +03:00
Gleb Natapov
c18e76cb10 rpc: fix out stream flushing
Flush it only once per rpc call instead of once per argument.
2015-05-07 13:29:34 +03:00
Gleb Natapov
3a01796eef tests: fix message test
Fix inconsistency between handler's parameter types and what is actually
passed to rpc call. Also get rid of unneeded local variables.

Reviewed-by: Asias He <asias@cloudius-systems.com>
2015-05-07 13:28:47 +03:00
Asias He
f12e955b4e message: Drop register_handler_oneway 2015-05-07 13:16:18 +03:00
Gleb Natapov
1a8c4b75f5 message: do not erase client's rpc call type
Currently we cast rpc call lambda to std::function (to put all lambdas
in one container) which removes its template properties and gives it
strictly typed interface. This is inconvenient since interface may either
receive an object by value (which requires copy during invocation),
or by reference (which require lifetime management), but not both. This
patch changes the implementation to no store lambda in one central
place, but create it with rpc::make_client() call when used. This way
template properties are preserved and send_message() arguments are
forwarded to rpc call with original types.

Reviewed-by: Asias He <asias@cloudius-systems.com>
2015-05-07 13:16:16 +03:00
Avi Kivity
e17f88e948 Merge branch 'asias/gossip_fix' of github.com:cloudius-systems/seastar-dev into db
Gossip fixes, from Asias.
2015-05-07 12:59:21 +03:00
Asias He
2106d3b165 README: Add more missing packages for building urchin
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-07 12:51:50 +03:00
Asias He
c79844e4e5 gossip: Fix passing by reference
Fix the same issue as we do in 7516966db5 (message: Fix passing by
reference).
2015-05-07 15:29:46 +08:00
Asias He
177498375d gossip: Add storage_service_value_factory helper
Switch to use real version when storage_service is ready.
2015-05-07 15:29:46 +08:00
Asias He
7ac5277822 gossip: Fix link error with versioned_value
With the next patch "gossip: Add storage_service_value_factory helper"
in this series.

[asias@hjpc urchin]$ ninja-build

[8/10] LINK build/release/seastar

build/release/gms/gossiper.o: In function
`gms::versioned_value::versioned_value_factory::removing_nonlocal(utils::UUID
const&)':
/home/asias/src/cloudius-systems/urchin/./gms/versioned_value.hh:201:
undefined reference to `gms::versioned_value::DELIMITER_STR'
build/release/gms/gossiper.o: In function
`gms::versioned_value::versioned_value_factory::removal_coordinator(utils::UUID
const&)':
/home/asias/src/cloudius-systems/urchin/./gms/versioned_value.hh:211:
undefined reference to `gms::versioned_value::DELIMITER_STR'
build/release/gms/gossiper.o: In function
`gms::versioned_value::versioned_value_factory::removed_nonlocal(utils::UUID
const&, long)':
/home/asias/src/cloudius-systems/urchin/./gms/versioned_value.hh:206:
undefined reference to `gms::versioned_value::DELIMITER_STR'
/home/asias/src/cloudius-systems/urchin/./gms/versioned_value.hh:206:
undefined reference to `gms::versioned_value::DELIMITER_STR'
collect2: error: ld returned 1 exit status

Fix by defining the symbol in gms/versioned_value.cc.
2015-05-07 15:29:13 +08:00
Avi Kivity
7bc465ae8a Merge branch 'tgrabiec/store-ttl-in-atomic-cell' of github.com:cloudius-systems/seastar-dev into db
Store both ttl and expiry time in atomic_cell, from Tomasz.
2015-05-07 10:23:52 +03:00
Tomasz Grabiec
b1e45e4401 db: Store ttl in atomic_cell
Origin does that, so should we. Both ttl and expiry time are stored in
sstables. The value of ttl seems to be used to calculate the read
digest (expiry is not used for that).

The API for creating atomic_cells changed a bit.

To create a non-expiring cell:

  atomic_cell::make_live(timestamp, value);

To create an expiring cell:

  atomic_cell::make_live(timestamp, value, expiry, ttl);

or:

  // Expiry is calculated based on current clock reading
  atomic_cell::make_live(timestamp, value, ttl_optional);
2015-05-06 19:42:38 +02:00
Tomasz Grabiec
784f841b8b tests: Add test for cell ordering 2015-05-06 18:32:17 +02:00
Tomasz Grabiec
f43836eb68 db: Handle expired cells in compare_atomic_cell_for_merge()
While at it, clarify some comments.
2015-05-06 18:31:21 +02:00
Avi Kivity
195a90a175 Merge branch 'tgrabiec/frozen-mutation-prep-v2' of github.com:cloudius-systems/seastar-dev into db
Preparatory changes before introducing frozen_mutation, from Tomasz.
2015-05-06 18:39:01 +03:00
Tomasz Grabiec
5ba1486ae7 db: Rename "ttl" to "expiry" when it's used as time point
To avoid confusion with "ttl" the duration.
2015-05-06 17:27:22 +02:00
Tomasz Grabiec
d6a003dc59 gc_clock: Store max_ttl as duration rather than time_point
Extra wrapping in time_point has no point.
2015-05-06 17:22:35 +02:00
Tomasz Grabiec
49f9794b1b schema: Introduce schema_builder
It's more flexible than schema's constructor. Primarily meant to be
used from tests, but not necessarily only there.
2015-05-06 16:40:48 +02:00
Tomasz Grabiec
5bd68aa514 data_input: Support read<bytes_view>() 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
2ab006949a atomic_cell: Extend interface of collection_mutation wrappers 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
45f66c5c9f atomic_cell: Add missing header 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
04846ed3d2 mutation: Make mutation equality comparable 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
87e00d97fa mutation: Introduce row_tombstones() getter 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
6762a92299 mutation_partition: Take schema by reference in apply_row_tombstone() 2015-05-06 16:40:48 +02:00
Tomasz Grabiec
0aef6f2ff7 mutation_partition: Allow row lookup by clustering_key_view 2015-05-06 16:40:42 +02:00
Tomasz Grabiec
96bbac8a57 dht: Make partitioner work on partition_key_view 2015-05-06 15:52:56 +02:00
Tomasz Grabiec
0deec287d4 tests: keys: Add test for key <-> view coversions 2015-05-06 15:52:56 +02:00
Tomasz Grabiec
0bc0071373 keys: Introduce view wrappers
We now have partition_key_view, clustering_key_view, etc.
Database APIs will be extended to also accept views.

This will alows us to avoid allocations in certain scenarios.
2015-05-06 15:52:56 +02:00
Calle Wilund
46e0676a7d commitlog: Add reader stream/subscription
Generic read-all-stream from a commit log segmen file.
Provides a byte view for each data entry, doing CRC checks and padding skips.

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:45:13 +03:00
Calle Wilund
2a14037f79 db: add data_input constructors & methods
* Construct from temporary_buffer<>
* Construct from basic_string_view<*> (cast)
* Allow skipping bytes

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:45:06 +03:00
Avi Kivity
1c8b0a4b1c Merge branch 'master' of github.com:cloudius-systems/urchin into db 2015-05-06 16:31:44 +03:00
Calle Wilund
7f685abca0 commitlog: added file header space twice
Checked wrong var == 0 when creating second mem buffer in segment

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:56 +03:00
Calle Wilund
897083e213 commitlog: Sync check time delta was inverted
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:49 +03:00
Calle Wilund
d2d3547115 commitlog: Initial data chunk offset should be 16
As in "after file header".

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:19:43 +03:00
Tomasz Grabiec
36ad6c9aa8 Merge tag 'avi/memtables/v3' from seastar-dev.git
Multiple memtable support from Avi.
2015-05-06 15:02:42 +02:00
Avi Kivity
4875028fb3 tests: add test for range scan memtable merging 2015-05-06 15:43:43 +03:00
Avi Kivity
eccb6bfe08 tests: add tests for multiple memtables in column family (single partition) 2015-05-06 15:43:09 +03:00
Avi Kivity
ef5c661d11 db: add variant of column_family::for_all_partitions() for unit tests
Since it's for tests, we can pass a slower std::function<>.
2015-05-06 15:43:06 +03:00
Avi Kivity
1d6ac071c0 db: add API to seal current active memtable 2015-05-06 15:39:31 +03:00
Avi Kivity
b717b4b6bb db: add ostream operators for more types (mutation_partition and components) 2015-05-06 15:39:31 +03:00
Avi Kivity
22969aeb18 db: support for multiple memtables
Each column family now contains multiple memtables, with one designated as
"active" receiving all writes, while the others only serve reads.
2015-05-06 15:39:29 +03:00
Raphael S. Carvalho
c3deca9575 sstables: fix problem with scope of deletion_time
write() takes a reference to deletion_time which means that the
object must survive until the continuation is scheduled.
This patch fix the problem the use of do_with().
The problem was making debug version of urchin tests fail.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-06 14:38:53 +02:00
Avi Kivity
5e81b92dc0 db: split column_family::partitions into a new memtable class
In preparation for multiple memtables, move column_family::partitions into
its own class, and forward relevant calls from column_family.

A testonly_all_memtables() function was added to support sstable_test.
2015-05-06 15:35:14 +03:00
Avi Kivity
bc669add40 schema: const correctness
Make schema accessors const, and make schema_ptr refer to a const schema.
2015-05-06 13:52:59 +02:00
Avi Kivity
cc291d7e3b db: improve sharding
Currently we use the first byte of the token for determining the local
shard.  This is suboptimal for two reasons:

 1. the first bytes of the token were already used to select the node,
    so they are not randomly distributed
 2. using a single byte is not sufficient for large core counts, as the
    modulo operation will not return evenly distributed results

Fix by using the final two bytes of the token.
2015-05-06 13:19:44 +02:00
Avi Kivity
f0e30bd1be Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-06 13:32:31 +03:00
Avi Kivity
f3afe9051f Merge branch 'shared_ptr'
const support for lw_shared_ptr<>.
2015-05-06 13:27:00 +03:00
Amnon Heiman
f4f6c0a4a3 smp: add all_cpus() helper
This adds a static method to return a range object to smp.
with this patch it is possible to use:

for (auto i : smp::all_cpus())

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-06 11:49:45 +03:00
Avi Kivity
fe9c5a2750 shared_ptr: const correctness for lw_shared_ptr::use_count() 2015-05-06 11:41:51 +03:00
Avi Kivity
211c77bf52 shared_ptr: add const support for lw_shared_ptr 2015-05-06 11:41:33 +03:00