Commit Graph

53948 Commits

Author SHA1 Message Date
Tomasz Grabiec
96ab69d39d mutation_reader: Introduce make_combined_reader()
Based on column_family::for_all_partitions().
2015-06-18 15:47:40 +02:00
Tomasz Grabiec
b35bc8e75a mutation_reader: State monotonicy requirement in the interafce contract
Currently column_family::for_all_partitions() relies on monotonicity
of keys. Adding strict monotonicity requirement doesn't hurt
implementaitons, but makes some consumers simpler.
2015-06-18 15:47:40 +02:00
Tomasz Grabiec
7f1ff0401e db: Move mutation_reader definition to separate header 2015-06-18 15:47:40 +02:00
Tomasz Grabiec
3779506990 db: query: Make partition_range hold ring_position
Current model was not really correct because Origin doesn't support
querying of partition ranges by their value. We can query slices
according to dht::decorated_key ordering, which orders partitions
first by token then by key value.

ring_position encapsulates range constraint. Key value is optional, in
which case only token is constrained.
2015-06-18 15:47:40 +02:00
Tomasz Grabiec
0740f49599 query: range: Introduce transform() 2015-06-18 15:47:39 +02:00
Tomasz Grabiec
0c45deeadf cql3: statement_restrictions: move definitions to source file 2015-06-18 15:47:39 +02:00
Glauber Costa
4238ecffd3 sstables: make sure sstable directory exists
In theory, when we create a new column family, we should also make sure
that the underlying directory exist. However, this would be quite challenging:
there are a lot of entry points for, add_column_family, none of them are futurized,
and futurizing them could prove challenging up the call chain.

Because we can guarantee that the keyspace directory will exist - now that we
have unified that, it is actually a lot simpler to just make sure that the
directory exist when writing the sstable.

If the keyspace directory wouldn't exist we would have to recurse through the
path.  As previously said, this patch will assume this away.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-18 09:22:58 -04:00
Glauber Costa
2f5b1b642b database: be more forgiving with sstables parsing
Currently, Origin generates sstables in the form CF-UUID, where UUID
is a string of numbers.

We also do CF-UUID, but for us, UUID has dashes separating the UUID components.

Due to the current test, we fails to load our current sstables. That test
really isn't that important, since we are currently not doing anything with the
UUID. And if we were, we should be able to accept both formats anyway.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-18 09:22:20 -04:00
Glauber Costa
efc57ef65e create system directory if it doesn't exist
Because system keyspace is not created using the same way as the others - and
it would be hard to convert, due to the fact that it is created inside the
database constructor, make sure that it is created when the database boots.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-18 09:22:20 -04:00
Glauber Costa
057c38b61c only populate system keyspace
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-18 09:22:20 -04:00
Glauber Costa
2a36eb7b6e cql_test_env: apply the vegas doctrine
"What happens in memory, stays in memory"

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-18 09:22:20 -04:00
Glauber Costa
e6090146e5 merge tables: properly generate column family config
We are currently generating an empty config, which is wrong and won't
propagate important characteristics of the keyspace.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-18 09:22:20 -04:00
Glauber Costa
a95a529865 database: allow for empty data file directories
A lot of our tests run in memory only, but now that our write path is complete,
we may start running into problems soon, as we write down the sstables.

It would be nice to force the database to run in-memory only in some situations.
Even in the real world, some scenarios may benefit from that in the future.

This patch forces durable_writes to be always false in case we force the data
directory to be an empty list.

For system tables, the patch also fixes a bug. Because system tables were
forceably initialized with durable_writes = false, we would never write them to
disk, even when we were supposed to.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-18 09:22:20 -04:00
Avi Kivity
5717869ffc Merge "streaming updates" from Asias 2015-06-18 15:49:07 +03:00
Gleb Natapov
d8dcceea09 stop storage and messaging services during exit 2015-06-18 15:13:02 +03:00
Gleb Natapov
89c74707d0 message: terminate all connections while stopping 2015-06-18 15:13:02 +03:00
Avi Kivity
8aa7e99051 Merge seastar upstream 2015-06-18 15:00:43 +03:00
Gleb Natapov
f7f445d563 distributed: deallocate _instance array during stop.
Otherwise tester may crash if _instances destructor is called when thread
responsible for the allocation (which tester spawned to run seastar in)
no longer running.
2015-06-18 14:43:38 +03:00
Gleb Natapov
46f47b7ea2 rpc: implement stop for server/client connection 2015-06-18 14:43:16 +03:00
Gleb Natapov
3c4dcd871a add default and move constructor/assignment to server_socket 2015-06-18 14:43:16 +03:00
Avi Kivity
8ec6826dca Merge "Enable snitch via config" from Shlomi 2015-06-18 13:46:09 +03:00
Avi Kivity
daa0009eaf Merge "add support to snappy and deflate compressors" from Raphael 2015-06-18 13:44:49 +03:00
Avi Kivity
7c68180d67 Merge "more streaming handlers" from Asias 2015-06-18 13:21:05 +03:00
Avi Kivity
27f332c315 Merge seastar upstream 2015-06-18 13:20:46 +03:00
Avi Kivity
e22e27a9ba build: fix spelling of _FORTIFY_SOURCE 2015-06-18 12:53:06 +03:00
Glauber Costa
92803a2db3 touch_directory: EEXIST-ignoring make directory
There are many situations in which we would like to make sure a directory
exists.  We can do that by creating the directory we want, and just ignoring
the relevant error.

It is a lot of code though, and I believe it is an idiom common enough to exist
on its own.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-06-18 12:51:49 +03:00
Raphael S. Carvalho
534401c91f fstream: use dma_alignment constant instead of a hardcoded value
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-06-18 12:49:33 +03:00
Raphael S. Carvalho
02bdf380c4 fstream: abort instead of silently returning a ready future
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-06-18 12:49:33 +03:00
Gleb Natapov
56193079a4 tests: add one more semaphore test
Test that wait with timeout and without work on the same semaphore
simultaneously.
2015-06-18 12:35:27 +03:00
Gleb Natapov
a2c1ab66b7 semaphore: fix access to freed memory in wait(timeout)
wait() with timeout takes reference to an entry, but when
circular_buffer is resized it may be moved, so freed instance will be
accessed on timeout. Fix it by using std::list instead.
2015-06-18 12:35:25 +03:00
Tomasz Grabiec
4682ca0209 Merge branch 'seastar/master' from seastar.git 2015-06-18 11:25:24 +02:00
Avi Kivity
3d75bab9b4 build: disable _FORTIFY_SOURCE
_FORTIFY_SOURCE complains about false positives in our thread implementation
due to longjmp()s.  Disable it.
2015-06-18 12:11:15 +03:00
Amnon Heiman
c787fa557e API adding a stub implementation for the storage_proxy
This adds a stub implementation for the storage_proxy.

To simplify future implementation, it takes the passed parameter and
return the correct types with fixed values.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 11:32:09 +03:00
Amnon Heiman
c41ccb4ce4 API Adding the storage_proxy definition file
This adds the swagger definition file for the storage_proxy, it is based
on the StorageProxyMBean definition.

The following command where define:
get_total_hints
get_hinted_handoff_enabled
set_hinted_handoff_enabled
get_hinted_handoff_enabled_by_dc
set_hinted_handoff_enabled_by_dc_list
get_max_hint_window
set_max_hint_window
get_max_hints_in_progress
set_max_hints_in_progress
get_rpc_timeout
set_rpc_timeout
get_read_rpc_timeout
set_read_rpc_timeout
get_write_rpc_timeout
set_write_rpc_timeout
get_counter_write_rpc_timeout
set_counter_write_rpc_timeout
get_cas_contention_timeout
set_cas_contention_timeout
get_range_rpc_timeout
set_range_rpc_timeout
get_truncate_rpc_timeout
set_truncate_rpc_timeout
reload_trigger_classes
get_read_repair_attempted
get_read_repair_repaired_blocking
get_read_repair_repaired_background
get_schema_versions

Matricses are not added yet, the basic integration and naming
get_cas_read_timeouts was added under the URL:
/storage_proxy/metrics/
That will holds the related matricses.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-18 11:30:30 +03:00
Asias He
d1c04ec2e3 streaming: Convert StreamReceiveTask.java to C++ 2015-06-18 14:55:07 +08:00
Asias He
0391f55ffd streaming: Import StreamReceiveTask.java 2015-06-18 14:55:07 +08:00
Asias He
334b1f81fc streaming: Convert StreamTransferTask.java to C++ 2015-06-18 14:55:07 +08:00
Asias He
0caad5f8f2 streaming: Import StreamTransferTask.java 2015-06-18 14:55:07 +08:00
Asias He
b315e6505a streaming: Convert StreamTask to C++ 2015-06-18 14:55:07 +08:00
Asias He
e119cb9463 streaming: Add StreamTask.java 2015-06-18 14:55:07 +08:00
Asias He
2f7ab6f293 streaming: Add fields to stream_init_message 2015-06-18 14:55:07 +08:00
Asias He
6c8b002e41 streaming: Enable requests and summaries in prepare_message 2015-06-18 14:55:07 +08:00
Asias He
d1889fe7e5 streaming: Convert StreamSummary.java to C++ 2015-06-18 14:55:07 +08:00
Asias He
501eebce39 streaming: Import StreamSummary.java 2015-06-18 14:55:07 +08:00
Asias He
ab76026938 streaming: Convert StreamRequest.java to C++ 2015-06-18 14:55:06 +08:00
Asias He
a41445ac08 streaming: Import StreamRequest.java 2015-06-18 14:55:06 +08:00
Asias He
cde1676274 streaming: Add SESSION_FAILED_MESSAGE handler 2015-06-18 14:55:06 +08:00
Asias He
8170f6ee0a streaming: Add COMPLETE_MESSAGE handler 2015-06-18 14:55:06 +08:00
Asias He
e2b986910b streaming: Add RETRY_MESSAGE handler 2015-06-18 14:55:06 +08:00
Avi Kivity
b0e58486d4 main: set line buffering on stdout
More friendly when teeing stdout to a file.
2015-06-18 09:42:27 +03:00