Commit Graph

4801 Commits

Author SHA1 Message Date
Avi Kivity
f8a7c5bd2b main: allow configuring log destination (stdout and/or syslog) 2015-07-15 16:21:12 +03:00
Avi Kivity
91d8d46709 logger: support for syslog()
Note that while syslog() may block, we still call it synchronously via
the reactor thread.  Anything else would make logging a nightmare.
2015-07-15 16:20:37 +03:00
Avi Kivity
5f88707d66 Merge "API: Replacing empty string to void" from Amnon 2015-07-15 14:52:15 +03:00
Amnon Heiman
83a64d75b5 Cleaning the gossiper API
This replaces the void method in the gossiper API to be json_void

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-15 14:20:28 +03:00
Amnon Heiman
b6fa2187af Cleanup the cache_service API
This replaces the void method in the cache service API to be json_void

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-15 14:20:28 +03:00
Amnon Heiman
2b0393525f Cleaning the hinted_handoff API
This replaces the void method in the hinted handoff API to be json_void

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-15 14:20:27 +03:00
Amnon Heiman
161c37d607 Cleaning the storage_proxy API
This replaces the void method in the storage proxy API to be json_void

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-15 14:20:27 +03:00
Amnon Heiman
14aafc83b6 API: Adding the get_host_id implementation to the API
This adds the implementation for the get host id API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-15 14:19:16 +03:00
Amnon Heiman
40d0d58a50 Cleaning the storage_service API
This changes the return type of void API to json_void, for a cleaner
API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-15 14:18:37 +03:00
Asias He
5d68f33fd1 messaging_service: Introduce wrapper for rpc::protocol::{client, server}
In order to make user of incomplete types of them, we need the wrappers.

Later, we can move the definition of the wrappers to .cc files.
2015-07-15 14:17:04 +03:00
Asias He
6191fcb62e messaging_service: Add wrapper for STREAM_INIT_MESSAGE verb
This patch serves as an example of how we can add wrappers for
ms.send_message and ms.register_handler.

When we convert all the users of them, we can make messaging_service.hh
do not include rpc.hh.
2015-07-15 14:17:04 +03:00
Tomasz Grabiec
cce2c648c5 Merge branch 'dev/gleb/for_urchin' from seastar-dev.git
Beginning of read implementation for range queries from Gleb.
2015-07-15 12:53:43 +03:00
Gleb Natapov
ed0faffd12 storage_proxy: beginning of read implementation for ranges queries 2015-07-15 12:41:38 +03:00
Gleb Natapov
6dc5b5ad76 storage_proxy: add resolver classes facilitate query resolving
The responsibility of the classes is to collect all results and do
reconciliation if needed.
2015-07-15 12:41:38 +03:00
Gleb Natapov
92a016fe9f add split() for range<ring_position> 2015-07-15 12:41:38 +03:00
Gleb Natapov
253ba71747 add comparison functions for ring_position
Two ring_positions are equal if tokens and keys are equal or tokens are
equal and one or both of them do not specify key. So ring_positions
without a key is a wildcard that equals any ring_positions with the same
token.
2015-07-15 12:41:31 +03:00
Tomasz Grabiec
c206699c9d Merge tag 'avi/logger-config/v1' from seastar-dev.git
Logger configuration from Avi.
2015-07-15 11:27:09 +02:00
Tomasz Grabiec
e1b4d0c29b Merge branch 'pdziepak/order-by-fixes/v2' from seastar-dev.git
"Fixes for ORDER BY clauses" from Pawel.

"The patches fix several issues in CQL3 frontend related to ORDER BY
clauses. Also, column component indexes are now handled properly and it
is possible to create tables with more than one column in clustering key."
2015-07-15 12:09:55 +03:00
Avi Kivity
ea691ef0cc Merge "Fixes for sstable::read_range_rows()" from Tomasz
First patch:
  Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-07-15 11:25:14 +03:00
Avi Kivity
d33a77e3be Merge "Improve messaging_service build time, part 1" from Asias
"This series moves as much code as possible to source file."
2015-07-15 10:31:02 +03:00
Asias He
3376b15157 messaging_service: Define _server as unique_ptr 2015-07-15 14:56:40 +08:00
Asias He
569a533d89 messaging_service: Switch rpc_protocol _rpc to std::unique_ptr<rpc_protocol> _rpc
Preparation for not including rpc.hh. Make it a pointer so we can use
rpc_protocol as incomplete type.
2015-07-15 14:56:37 +08:00
Asias He
91de9fe94f messaging_service: using rpc_protocol = rpc::protocol<serializer, messaging_verb> 2015-07-15 14:56:35 +08:00
Asias He
ef8cfb3db9 messaging_service: Move code of messaging_service to source file 2015-07-15 14:22:35 +08:00
Asias He
c0335f38d6 messaging_service: Move code for shard_id into source file
Also, no reason to keep shard_id inside messaging_service class, move
it out.
2015-07-15 14:22:02 +08:00
Tomasz Grabiec
161744e8f9 tests: Add more tests for sstable range queries 2015-07-14 19:58:18 +02:00
Tomasz Grabiec
0bd4b77498 tests: Introduce reader_assertions::produces() which takes a vector 2015-07-14 19:58:18 +02:00
Tomasz Grabiec
19efb43354 tests: Add missing include to mutation_reader_assertions.hh 2015-07-14 19:58:18 +02:00
Tomasz Grabiec
6a5943267a tests: Extract tmpdir class from commitlog_test to tmpdir.hh
It will be useful in other tests too.
2015-07-14 19:58:18 +02:00
Tomasz Grabiec
0dc49434ad tests: Make tmpdir more generic 2015-07-14 19:58:17 +02:00
Tomasz Grabiec
5cab8bc759 tests: Deal with non-empty directory in ~tmpdir 2015-07-14 19:58:17 +02:00
Tomasz Grabiec
4e5cb05aa4 sstables: Fix heap-buffer-overflow in read_range_rows()
When passing tokens corresponding to 129th key in the sstable to
read_range_rows(), it failed with heap-buffer-overflow pointing to:

      return make_ready_future<uint64_t>(index_list[min_index_idx].position);

The scenario is as follows. We pass the lower bound token, which
corresponds to the first partition of some (not first) summary
page. That token will compare less than any entry in that page (even
less with the key we took it from, cause we want all partitions with
that token), so min_idx will point to the previous summary page
(correct). Then this code tries to locate the position in the previous
page:

  auto m = adjust_binary_search_index(this->binary_search(index_list, minimum_key(), min_token));
  auto min_index_idx = m >= 0 ? m : 0;

binary_search() will return ((-index.list_size()) -1), because the
token is greater than anything in that page. So "m" and
"min_index_idx" will be (index.list_size()-1) after adjusting.

Then the code tried this:

        auto candidate = key_view(bytes_view(index_list[min_index_idx]));
        auto tcandidate = dht::global_partitioner().get_token(candidate);
        if (tcandidate < min_token) {
            min_index_idx++;
        }

The last key compared less than the token also, so min_index_idx is
bumped up to index_list.size(). It then tried to use this too large
index on index_list, which caused buffer overflow.

We clearly need to return the first position of the next page in this
case, and this change does it indirectly by calling
data_end_position(), which also handles edge cases like if there is no
next summary page.

I reimplemented the logic top-down, and found that the last special
casing for tcandidate was not needed, so I removed it.
2015-07-14 19:58:17 +02:00
Tomasz Grabiec
2a491b2076 sstables: Fix bug in read_range_rows()
The method was using the same summary page for both min and max
tokens, whereas they can be different if they're distant enough from
each other.
2015-07-14 19:58:17 +02:00
Tomasz Grabiec
0a4651cd28 dht: Add field getters to decorated_key 2015-07-14 19:57:37 +02:00
Paweł Dziepak
56bcf40c5e tests/cql: add test cases for ORDER BY
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-14 19:34:28 +02:00
Paweł Dziepak
8c21a62865 db/legacy_schema_tables: read column component index
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-14 19:34:28 +02:00
Paweł Dziepak
54268889a1 schema: allow specifying column component index
The order of columns that belong to partition key or clustering key
needs to be preserved.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-14 19:34:27 +02:00
Paweł Dziepak
01b2f76fee transport: send back only the requested columns
result_set may contain more columns than user requested (for instance if
some of them were needed to properly sort output). The additional columns
are always the last ones and are not included in metadata::column_count().

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-14 19:34:27 +02:00
Paweł Dziepak
f829cf373e cql3: selection::raw_selector::alias may be null
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-14 19:34:27 +02:00
Paweł Dziepak
c5e9434183 cql3: change orderings_type to std::vector
We need to preserve the order of orderings in order to validate them
properly.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-14 19:34:27 +02:00
Paweł Dziepak
922779747d Cql.g: orderings should be passed by reference
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-14 19:34:27 +02:00
Avi Kivity
aa06eb2b79 main: support for configuring logging on an individual logger basis 2015-07-14 18:17:01 +03:00
Avi Kivity
5b7e283451 logger: provide lexical_cast support for log_level
A boost bug prevents automatic generation, so help it.
2015-07-14 18:16:19 +03:00
Avi Kivity
5a5e411814 fix iostream 2015-07-14 18:15:52 +03:00
Avi Kivity
99a15de9e5 logger: de-thread_local-ize logger
The logger class constructor registers itself with the logger registry,
in order to enable dynamically setting log levels.  However, since
thread_local variables may be (and are) initialized at the time of first
use, when the program starts up no loggers are registered.

Fix by making loggers global, not thread_local.  This requires that the
registry use locking to prevent registration happening on different threads
from corrupting the registry.

Note that technically global variables can also be initialized at the
point of first use, and there is no portable way for classes to self-register.
However this is the best we can do.
2015-07-14 17:18:11 +03:00
Avi Kivity
26cd683c32 Merge "streaming updates" from Asias
"With this series, we can now create and start a stream_plan, ask it to stream a
user created table. The negotiation messages can be exchanged and the mutations
can be sent and received.

I'm still trying to find a way to verify the mutation reach to the sstable
correclty on the peer node."
2015-07-14 15:59:58 +03:00
Asias He
0313be256c streaming: Handle non-exist cf_name in get_column_family_stores 2015-07-14 20:56:37 +08:00
Asias He
d720dadf7b streaming: Switch to use logger class 2015-07-14 20:56:28 +08:00
Asias He
e82bdf2995 streaming: Swith to use shared_ptr from std::shared_ptr
Since our shared_ptr works with incomplete types now, switch to it.
2015-07-14 20:41:14 +08:00
Asias He
8fd8f39d63 streaming: Add more debug info for message exchange 2015-07-14 20:41:14 +08:00