Commit Graph

53948 Commits

Author SHA1 Message Date
Glauber Costa
f04f7f2819 sstables: turn composite into a class
Encapsulating it into a composite class is a more natural way to handle this,
and will allow for extending that class later.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-25 20:58:50 -04:00
Glauber Costa
0519dec0d7 sstable tests: remove filter file
It is not being remove, and is thus left as garbage in the sstable directory.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-25 20:47:16 -04:00
Glauber Costa
8b2f5979c9 sstable tests: replace last then statement with finally
Currently, a failed test is leaving files behind.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-25 20:47:16 -04:00
Avi Kivity
d115f9cef6 future: modernize lambda_task, make_task
Use inline, override, final, make_unique.
2015-05-25 22:21:49 +03:00
Avi Kivity
d1bc08e9e7 future: merge two variants of make_task()
Using universal references we can unify the two cases.
2015-05-25 22:18:24 +03:00
Avi Kivity
8812504e4b future: sprinke always_inline attributes in strategic places
gcc makes poor inlining decisions sometimes, which cause all the value-
tracking optimizations to be lost.  Forcing it to inline (particularly
around destructors) allows ready futures to be inlined with fewer always-
true tests and data movements.
2015-05-25 21:27:40 +03:00
Pekka Enberg
89a466ce7f db/legacy_schema_tables: Fix use-after-move in add_table_to_schema_mutation()
Spotted by Tomek.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-25 20:49:16 +03:00
Pekka Enberg
f092f33848 cql_test_env: Create keyspace for all test cases
Make sure keyspace exists for all test cases, not just ones that call
the create_table() helper.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-25 19:31:51 +03:00
Amnon Heiman
f61dcfddc2 Adding the gossiper API implementation
This implement the gossipier API. All actions perform on the local
gossiper.

The following functionality is supported:
/gossiper/downtime/{addr}
/gossiper/generation_number/{addr}
/gossiper/assassinate/{addr}

The following are extened API beyond the MBean definition:
/gossiper/endpoint/down
/gossiper/endpoint/live

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-25 18:28:47 +03:00
Amnon Heiman
313074c980 API: Add the gossiper API
This holds the swagger definition equivelent to the GossiperMBean API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-25 18:28:47 +03:00
Amnon Heiman
588fb4fdcd Gossiper: Add global function
The current gossiper implementation runs the gossiper on CPU 0, this is
irelevent to user of the gossiper, that may want to inquire it.

This adds a globally available API for get_unreachable_members,
get_live_members, get_endpoint_downtime, get_current_generation_number,
unsafe_assassinate_endpoint and assassinate_endpoint that returns a
future and perform on the correct CPU.

The target user is the API that would use this function to expose the
gossiper.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-25 18:28:38 +03:00
Amnon Heiman
6d6872063d http: simplify using the parameters in the request
The parameters in the request holds the path parameters. The current
implementation based on unorder_map has a few downside:

1. Because path parameters are sometime used to mark a file path and
they can extend to multiple url section (may contain /) the value will
always contain the leading slash. Though this is true for files, for
the common case, it would be easier to return the value without it.

2. The []operator of the hash map is non const, this mean that it cannot
be used in the common case where the request object is passed as a const
reference.

3. There is no exists method in an ordered_map - Usually query
parameters are mandatory, still it is usefull to have an easy way of
testing if a parameter is found.

This series wrap the unordered_map implementation in a manner more
suitable for the request.

The [] operator will be const and retrun a copy of the parameter value
without the leading slash.

The at method will keep the old meaning. For clearer code, a path method
was added to indicate that the return value has a leading slash.

A set method is used for assignment.

Older code will continue to work without changes.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-25 15:47:56 +03:00
Asias He
898233ddcf Remove redundant const in static constexpr const
From http://en.cppreference.com/w/cpp/language/constexpr:

  A constexpr specifier used in an object declaration implies const.
2015-05-25 13:09:23 +03:00
Glauber Costa
ab5ccf1a87 sstables: write filter component
Before sealing an sstable, here's what we have to do, filter-wise:
1) determine wether or not we should have a filter file. We won't write
   one, if our fp_chance is 1.0,
2) add each index key to the filter,
3) and write the actual filter.

This patch implements those steps.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-25 12:49:17 +03:00
Asias He
88e7dcfa86 Remove redundant const in static constexpr const
From http://en.cppreference.com/w/cpp/language/constexpr:

   A constexpr specifier used in an object declaration implies const.

However, We can not change from
   static constexpr const char* TIME_FORMAT = "%a %b %d %I:%M:%S %Z %Y";
to
   static constexpr char* TIME_FORMAT = "%a %b %d %I:%M:%S %Z %Y";

The compiler complains:

   In file included from json/formatter.cc:22:0:
   json/formatter.hh:132:42: error: deprecated conversion from string
   constant to ‘char*’ [-Werror=write-strings]
        static constexpr char* TIME_FORMAT = "%a %b %d %I:%M:%S %Z %Y";

Since, unlike const, constexpr does not modify a type. It just applies
to an object (or function), and incidentally implies const to the
top-level type.
2015-05-25 11:57:19 +03:00
Avi Kivity
5b259ff9e3 Merge "sstable range reads"
Sstable range reads, from Glauber.
2015-05-25 10:53:00 +03:00
Avi Kivity
b5beef0529 Merge seastar upstream 2015-05-25 10:53:00 +03:00
Pekka Enberg
4dc488afb2 database: Store metadata in 'struct keyspace'
Store a lw_shared_ptr<keyspace_metadata> in struct keyspace so callers
in migration manager, for example, can look it up.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-25 09:12:29 +02:00
Avi Kivity
0bd48b37ed future: consolidate task_with_state, task_with_ready_state
The internal structs task_with_state and task_with_ready_state are
identical except for their constructors.  Merge them into a new
struct continuation.
2015-05-23 11:40:39 +03:00
Raphael S. Carvalho
28e14a08d8 sstables: add FIXME comment
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-22 09:30:08 +02:00
Raphael S. Carvalho
617484c8f1 sstables: use u-ref to avoid copy of partitions
Suggested-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-22 09:29:20 +02:00
Glauber Costa
fc735de91c tests: test range reader
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:31:07 -04:00
Glauber Costa
62327ff25d sstables: allow to convert a full range
While trying to write code to populate the system table's data, I found out
that currently, there is no publicly exported way for an entity outside the
sstable to find out which are the keys available in the index.

Without the full list of keys, it is very hard to make the kind of
"read-everything" queries we will need for that to work.

Although I haven't checked the internals fully, Origin seems to do that through
token-based range searches, where you can specify start and end tokens and
return everything in the middle (this is how the WHERE clauses are
implemented).

I am proposing in this patchset a subscription interface that will allow us
to extract data from the sstables in this fashion.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
6a8049dce1 dht: add maximum_token
Analogous to minimum_token.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
d4ed0d4c85 sstables: add a maximum and minimum key
They will compare as greater and lesser than any other given key, respectively.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
f1ad76cd33 sstable: make comparison logic part of key
With this, we will be able to override it if needed.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
5134631fed sstables: fix bug when reading from index not in the last summary group
The current code for position determination works well if we are in the
last summary group. For the ones in the middle, it won't.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
b0a990d9c6 sstables: factor out code to compute end positions
The readability improvements alone justify it, but on top of that, I would like
to reuse this one.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
490d3bab6e sstables: factor out binary search adjustment code
So it can be reused.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
72194762ec sstables: allow for empty key parameter in row_consumer
It would be helpful to sometimes avoid passing a key parameter from the
consumer, since we not always know what to expect. In those situations, we can
easily construct it from the data we see in consume_row_start.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
9b17d33fdf mutation: add a move assignment operator
For that to work, we need to unfortunately lose the const specifier in the decorated_key

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
522b5b0302 sstable: allow for empty key_views
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
7ebfc7ef99 sstables: use a key_view instead of a key in the converter
Now we can explode key_views, let's use it instead of a key.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
e69411cf32 sstables: allow a key_view to be exploded as well
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-21 16:27:35 -04:00
Glauber Costa
84cbe7c862 sstables: fix test position
This is my turn to make the same mistake as Nadav recently made: I have
written down the expected position from the test output itself, instead
of checking the file.

A position of 0x400c0000000000 is obviously ridiculous. The file itself is not
that big. My recent patch to fix the summary broke the test, and so we need to
fix it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 18:28:55 +02:00
Glauber Costa
1e44a5ecbc sstables: summary: fix a bug when reading position
Since that whole entries array is memory mapped, we should not call
read_integer - which will byteswap it.

We should instead just fetch the position directly.

This was not spotted before because our test sstables tend to be relatively
small.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 16:47:32 +03:00
Avi Kivity
26e5a485dc Merge branch 'master' of github.com:cloudius-systems/urchin into db 2015-05-21 16:47:01 +03:00
Gleb Natapov
0c549eeca6 reactor: fix unintentional timer copying in reactor destructor 2015-05-21 16:33:20 +03:00
Avi Kivity
bbc8dbbab5 db: abstract memtable empty test 2015-05-21 15:48:51 +03:00
Gleb Natapov
59f4b9af46 messaging: bind rpc client to listen-address 2015-05-21 15:17:42 +03:00
Gleb Natapov
be5676fc95 main: use rpc-address config option for cql and thrift
Currently if you run more than one instance on the same machine, they
all bind to the same local address and port (we enable an option that
makes this non error) and cql/thrift requests are distributed randomly
among them.
2015-05-21 15:17:41 +03:00
Gleb Natapov
a61007d6b2 stub dns resolver
Some urchin parameters has default value of localhost and need to be
resolved into IP address. Add stub resolver that do just that: if it
sees "localhost" string it translates it to a loopback address,
otherwise assume that string is IP address already.
2015-05-21 15:17:34 +03:00
Avi Kivity
c1a97ce9c5 Merge seastar upstream 2015-05-21 15:17:23 +03:00
Avi Kivity
068a3190d1 db: add written sstable to read set 2015-05-21 14:44:04 +03:00
Avi Kivity
21f7975b57 db: don't flush empty memtables
You can't have an empty sstable, so don't flush an empty memtable.
2015-05-21 13:28:14 +03:00
Avi Kivity
3961e0391a db: fix sstable use-after-free in seal_active_memtable()
We write the new sstable, but we don't wait for the write to complete,
so do_with() kills it.
2015-05-21 13:18:35 +03:00
Avi Kivity
b483d506eb Merge "generation of sstable files"
More sstable component generation, from Raphael.
2015-05-21 11:11:55 +03:00
Glauber Costa
3ffad6f127 tests: add sstable mutation test to the full set of regression tests
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 09:01:46 +02:00
Glauber Costa
ba60176d4d bloom_filter: fix index calculation to work on int, not uint
Java uses long, so we should use int64_t. Using uint64_t causes the wrong
indexes to be calculated, and therefore, the filter to respond incorrectly
to a given key.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 09:01:44 +02:00
Glauber Costa
601bb48fe7 bloom_filter: no need for a unique_ptr
I actually wrote this code before I learned that we could just return a local
vector by copy without major hassles.

Never too late for a cleanup.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 09:00:37 +02:00