Commit Graph

53948 Commits

Author SHA1 Message Date
Glauber Costa
7fa8071dde fix creation of keyspaces
When we call merge_keyspaces, we end up calling db.add_keyspace, which means we
don't do parts of creation of a keyspace - like creating the directory. This works
in some situations where all the work was already done, but not in others.

We should be calling create_keyspace instead, which will take care of all that
for us. That will include creating a directory when one is needed.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-24 12:44:05 -04:00
Glauber Costa
4d07c952cc remove global create_keyspace
There is only one user in tree, and as Tomek pointed out, it is buggy. The
reason is that ksm is a shard-local structure, and it is currently used
indiscriminately by all shards which can easily lead to problems.

We could fix it with some tricks, but it is way better and safer to make sure
the callers are doing it right instead. There is only one caller, so let's fix
that.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-24 12:44:05 -04:00
Glauber Costa
f0f04892d3 database: no longer replicate create_keyspace code
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-24 12:44:05 -04:00
Raphael S. Carvalho
79532b6603 sstables: merge prepare_statistics and add_statistics_metadata
The two separate functions can now be merged. As a result, the code
that generates statistics data is now much easier to understand.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-06-24 18:58:17 +03:00
Raphael S. Carvalho
74febd97f2 sstables: avoid assert when generating cardinality metadata
Assert should be completely avoided. Instead, we should trigger an
exception, allowing the db to proceed with a "smooth" shutdown.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-06-24 18:58:17 +03:00
Avi Kivity
e404b5349a README: update DPDK instructions for bundled DPDK 2015-06-24 18:49:38 +03:00
Tomasz Grabiec
dc6bfb63da system_keyspace: Introduce query()
Simple method to get all data from a system table.

Note: Perhaps we should move storage_proxy::query_local() here as well.
2015-06-24 18:13:02 +03:00
Avi Kivity
a723942a54 Merge seastar upstream 2015-06-24 18:12:43 +03:00
Avi Kivity
716c7c04db build: configure dpdk MAX_MEMSEG to a higher value
Prevent failure on a high memory machine.
2015-06-24 18:09:17 +03:00
Avi Kivity
ab22932899 build: make dpdk config more generic
Allow setting a variable to any value, not just disabling it.
2015-06-24 18:07:11 +03:00
Avi Kivity
e639b56952 Add swagger-ui submodule
Only needed for runtime, not build.
2015-06-24 17:26:14 +03:00
Avi Kivity
3d22623a6b Merge "Flush schema changes to disk" from Glauber
"This is the current patchset to flush and persist schema changes to disk.
It is not perfect, in the sense that older changes still in flight won't be
waited for. But as we discussed - at this moment we'll just note that, and
leave the fix for later"
2015-06-24 17:08:33 +03:00
Paweł Dziepak
e5a2e63785 cql3: fix properties allowed in table creation statements
KW_MINCOMPACTIONTHRESHOLD and KW_MAXCOMPACTIONTHRESHOLD shouldn't be
allowed, they are properties of compaction strategy not the table itself.
KW_COMPACTION and KW_COMPRESSION should be allowed.

Reported-by: Shlomi Livne <shlomi@cloudius-systems.com>
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
2015-06-24 16:45:19 +03:00
Nadav Har'El
27c238d6b7 sstable: fix load of new sstable
Apparently, after writing a new sstable, with write_components(), it
is necessary to load() it. I'm not sure why, but we get a crash on
an aio to a closed file descriptor if we don't.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-06-24 16:44:25 +03:00
Nadav Har'El
0b297b9f6c sstable compaction: simplify compact_sstables() function
Instead of requiring the user to subclass a "sstable_creator" class to
specify how to create a new sstable (or in the future, several of them),
switch to an std::function.

In practice, it is much easier to specify a lambda than a class, especialy
since C++11 made it easy to capture variables into lambdas - but not into
local classes.

The "commit()" function is also unnecessary. Then intention there was to
provide a function to "commit" the new sstables (i.e., rename them).
But the caller doesn't need to supply this function - it can just wait
for the future of the end of compaction, and do his own committing code
right then.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-06-24 16:44:11 +03:00
Avi Kivity
e93ac50666 Merge "Adding the cache_service stub API" from Amnon
"This series adds the cache_service definition file with a stub implementation.
The API is based on the CacheServiceMBean interface.
The stubbed API return stubbed value of the correct type."
2015-06-24 15:52:48 +03:00
Avi Kivity
66afef0bd7 Merge seastar upstream
Conflicts:
	NOTICE
	configure.py
2015-06-24 15:21:55 +03:00
Amnon Heiman
4258b005db API: Adding stub implementation for the cache service
This adds a stub implementation for the cache_service. It also register
the API doc.
The cache_service doc will be available at:
http://localhost:10000/api-doc/cache_service/

The stub implementation returns the correct value type with stubbed
values.

After this patch the following path will be available:
/cache_service/row_cache_save_period
/cache_service/key_cache_save_period
/cache_service/counter_cache_save_period
/cache_service/row_cache_keys_to_save
/cache_service/key_cache_keys_to_save
/cache_service/counter_cache_keys_to_save
/cache_service/invalidate_key_cache
/cache_service/invalidate_counter_cache
/cache_service/row_cache_capacity
/cache_service/key_cache_capacity
/cache_service/counter_cache_capacity
/cache_service/save_caches

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-24 14:30:08 +03:00
Amnon Heiman
d4b5e65b7c API Adding the cache_service definition file
This adds the definition file for the cache service, the API is based on
the CacheServiceMBean definition and contain the following command:
get_row_cache_save_period_in_seconds
set_row_cache_save_period_in_seconds
get_key_cache_save_period_in_seconds
set_key_cache_save_period_in_seconds
get_counter_cache_save_period_in_seconds
set_counter_cache_save_period_in_seconds
get_row_cache_keys_to_save
set_row_cache_keys_to_save
get_key_cache_keys_to_save
set_key_cache_keys_to_save
get_counter_cache_keys_to_save
set_counter_cache_keys_to_save
invalidate_key_cache
invalidate_counter_cache
set_row_cache_capacity_in_mb
set_key_cache_capacity_in_mb
set_counter_cache_capacity_in_mb
save_caches

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-24 14:29:22 +03:00
Avi Kivity
63e7f8d99c docs: exclude dpdk 2015-06-24 13:09:51 +03:00
Avi Kivity
768201a406 build: add support for building with included dpdk
dpdk will be auto-configured in ./configure.py, and auto-built with
ninja whenever something changes.
2015-06-24 13:09:51 +03:00
Avi Kivity
d1766f6031 NOTICE: mention dpdk 2015-06-24 13:09:51 +03:00
Avi Kivity
d55e89f695 Add dpdk submodule 2015-06-24 13:09:37 +03:00
Gleb Natapov
97a4b0ee40 Store frozen_mutation in shared pointer while processing it
If local mutation write takes longer then write timeout mutation will
be deleted while it is processed by database engine. Fix this by storing
mutation in shared pointer and hold to the pointer until mutation is
locally processed.
2015-06-24 12:51:34 +03:00
Gleb Natapov
7d846e842c use write_request_timeout_in_ms for write request timeout
Fixes another fixme. Also change default value to 2000 which seams to
be what origin uses.
2015-06-24 12:51:33 +03:00
Avi Kivity
9faf46f047 Merge "Complete the storage service definition" from Amnon
"This series complete the definition of the storage service API. It is based on
the StorageServiceMBean definition."
2015-06-24 11:55:23 +03:00
Avi Kivity
b91b34afcc Merge "more streaming updates" from Asias 2015-06-24 11:36:10 +03:00
Avi Kivity
0365f3bb53 Merge seastar upstream 2015-06-24 11:35:40 +03:00
Asias He
29cb6fe003 streaming: Add message type for session_failed_message 2015-06-24 16:13:31 +08:00
Asias He
f9a7d20919 streaming: Add message type for retry_message 2015-06-24 16:13:31 +08:00
Asias He
3586de623c streaming: Add message type for received_message 2015-06-24 16:13:31 +08:00
Asias He
8555a28ef6 streaming: Add message type for prepare_message 2015-06-24 16:13:31 +08:00
Asias He
79d2bf1629 streaming: Add message type for outgoing_file_message 2015-06-24 16:13:31 +08:00
Asias He
044dbca5a4 streaming: Add message type for incoming_file_message 2015-06-24 16:13:31 +08:00
Asias He
8425cb126f streaming: Add message type for complete_message 2015-06-24 16:13:30 +08:00
Asias He
052a126d10 streaming: Add stream_message type 2015-06-24 16:13:30 +08:00
Asias He
320c6b57de streaming: Add constructor to incoming_file_message 2015-06-24 16:13:30 +08:00
Asias He
fb3df916b9 streaming: Complete session_info 2015-06-24 16:13:30 +08:00
Asias He
a936ff0ac3 streaming: Add class ss_table_streaming_sections 2015-06-24 16:13:30 +08:00
Asias He
8c3294d9d7 streaming: Add constructor for stream_session 2015-06-24 16:13:30 +08:00
Asias He
442f68af19 streaming: Implement virtual functions in stream_transfer_task 2015-06-24 16:13:30 +08:00
Asias He
ec945320a2 streaming: Implement virtual functions in stream_receive_task 2015-06-24 16:13:30 +08:00
Asias He
09c5f40360 streaming: Reduce dependency to stream_session in stream_task 2015-06-24 16:13:30 +08:00
Asias He
76c6139a57 streaming: Reduce dependency to stream_session in stream_receive_task 2015-06-24 16:13:30 +08:00
Asias He
4c9af76261 streaming: Move add_transfer_file to source file
Reduce dependency to stream_session
2015-06-24 16:13:30 +08:00
Asias He
705e4de58c streaming: Fix uninitialized error in outgoing_file_message
./streaming/messages/file_message_header.hh:63:39: error: ‘format’ is
used uninitialized in this function [-Werror=uninitialized]
2015-06-24 16:13:30 +08:00
Asias He
4a7ef682ab streaming: Use stream_session_state
In session_info, stream_coordinator and stream_session
2015-06-24 16:13:30 +08:00
Asias He
5c7aac3da9 streaming: Add stream_session_state
Added to reduce cyclic dependency.
2015-06-24 16:13:30 +08:00
Asias He
8b3d5b2da0 streaming: Reduce dependency to stream_session in connection_handler 2015-06-24 16:13:30 +08:00
Asias He
827b332a31 streaming: Move has_active_sessions to source file 2015-06-24 16:13:30 +08:00