Currently, we control incremental backups behavior from the storage service.
This creates some very concrete problems, since the storage service is not
always available and initialized.
The solution is to move it to the column family (and to the keyspace so we can
properly propagate the conf file value). When we change this from the api, we will
have to iterate over all of them, changing the value accordingly.
Signed-off-by: Glauber Costa <glommer@scylladb.com>
This adds the implementation that return the estimated total latency of
the read and of the write.
First the method that sum the count was renamed to get_cf_stats_count
and a method was added named get_cf_stats_sum to sum the estimated
latencies.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The histogram that are used typically only sample the data, so to get an
estimation of the actual sum, we use the estimated mean multiply by the
actuall count.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the API definition with stub implementation that would make
the nodetool cfstats to run.
After this patch the nodetool cfstats command would work, but with stub
imlementation.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch add some missing definition for cas read an write, the API
definition is for completness only as we do not support cas yet.
It also change a part of the definition from storage_service to
storage_proxy as it should be.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
For us, everything is "off heap", so this will just be the total amount of
memory used by the filters.
Fixes#339
Signed-off-by: Glauber Costa <glommer@scylladb.com>
This address issue #154
Flush command should wait for the commmand completion before returning.
This change replaces the for loop with a parallel_for_each, it will now
wait for all the flushes to complete before returning.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Until there will be an API for the compaction manager, the API return 0
for the number of total compaction.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The following function where added to column family:
is_auto_compaction_disabled
get_built_indexes
get_compression_metadata_off_heap_memory_used
get_compression_parameters
get_compression_ratio
get_read_latency_estimated_histogram
get_write_latency_estimated_histogram
And the get and set compaction strategy methods and a stub
implementation for the compression parameter, crc chec and sstable
count.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The get_load_map method should return a map between nodes addresses and
their load. In origin the implementation is based on the load
broadcaster that we currently do not have.
This workaround return a map with a single entry of the current node
address and its load
compression
Timers support expect the API to return histogram. This adds the swagger
definition for the following timer in column_family:
get_coordinator_read_latency
get_coordinator_scan_latency
get_waiting_on_free_memtable_space
The following estimated histogram were added to column_family:
get_read_latency_estimated_recent_histogram
get_read_latency_estimated_histogram
get_range_latency_estimated_recent_histogram
get_range_latency_estimated_histogram
get_write_latency_recent_histogram
get_write_latency_estimated_recent_histogram
get_cas_prepare_estimated_recent_histogram
get_cas_prepare_estimated_histogram
get_cas_propose_estimated_recent_histogram
get_cas_propose_estimated_histogram
get_cas_commit_estimated_recent_histogram
get_cas_commit_estimated_histogram
And the following timers in commitlog:
get_waiting_on_segment_allocation
get_waiting_on_commit
To column family API the following API were added:
set_compaction_strategy_class
get_compaction_strategy_class
set_compression_parameters
set_crc_check_chance
get_sstable_count_per_level
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
API: Completing the column_family Swagger definition
This adds the missing definition in the column_family to make it
compatible to ColumnFamilyStoreMbean
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The bloom filter memory calculation is missing, as a workaround until
it will be completed, the memory calculation will return 0.
It is needed by the nodetool info command.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
We do not save the cache, so the get for save priod in second should
return 0, to indicate never like it is done in origin.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch addresses issue #155. It register an exception handler
API of the routes object that handle the no_such_keyspace exception.
The handler just throw a bad_parameter_exception with the error message
it got from the no_such_keyspace exception.
After this patch a call with a keyspace that does not exist, will return
a 400 result.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Until we'll support key and counter cache, it is reasonable to return 0
for their statistic and sizes.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the implementation for hte stream_manager API.
It goes over all stream, on all shards and combine the result to a
vector of streams.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the stream_manager swagger definition file. It is based on the
StreamManagerMBean definition and the return class based on StreamState
class and its sub classes.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch clear the ambiguity in the swagger definition file and adds
the implementation for the memtable memory related methods.
For each column family there is an active memtable and a list of non
active.
when refering the all the memtable in the column family, the nick name
will contain cf_all_memtables.
Each URL has two versions, one, with a column family name, that is
relevant to a specific column family and one without, which is the
result of running the method on all column families.
This patch adds the following implementation to column_family:
get_memtable_on_heap_size
get_all_memtable_on_heap_size
get_memtable_off_heap_size
get_all_memtable_off_heap_size
get_memtable_live_data_size
get_all_memtable_live_data_size
get_all_memtables_on_heap_size
get_all_all_memtables_on_heap_size
get_all_memtables_off_heap_size
get_all_all_memtables_off_heap_size
get_all_memtables_live_data_size
get_all_all_memtables_live_data_size
Memory consumption is map this way: All memory assume to be off heap, so
on heap will return 0, and off heap will return the memory consumption
After this patch the following URL will be available:
/column_family/metrics/memtable_on_heap_size/{name}
/column_family/metrics/memtable_on_heap_size
/column_family/metrics/memtable_off_heap_size/{name}
/column_family/metrics/memtable_off_heap_size
/column_family/metrics/memtable_live_data_size/{name}
/column_family/metrics/memtable_live_data_size
/column_family/metrics/all_memtables_on_heap_size/{name}
/column_family/metrics/all_memtables_on_heap_size
/column_family/metrics/all_memtables_off_heap_size/{name}
/column_family/metrics/all_memtables_off_heap_size
/column_family/metrics/all_memtables_live_data_size/{name}
/column_family/metrics/all_memtables_live_data_size
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"This series modify the stub implementation of unimplemented API method to
return a 500 Http error.
It does so by adding a new API exception unimplemented_exception and a helper
function unimplemented that throw that exception.
A call to unimplemented was added to each of the stub API methods.
After this series a call to an unimplemented to API would return a 500."
"It moves the API configuration from the command line argument to the general
config, it also move the api-doc directory to be configurable instead of hard
coded."
"This series enable the nodetool info, by completing the missing APIs.
The main change is returning fixed value for storage_service
is_rpc_server_running, is_native_transport_running and get_exception_count.
After this series it will be possible to run:
nodetool info (while the jmx is runnning) and to get the results without errors
or crashes."
Because of the different implementation the right way of getting the
load (which is the sum of all the live diskspace used) is using the
column_family api and not through the storage_service API.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Some APIs other then the column_family need to use the get_cf_stats,
this adds the helper method decleration to the column_family.hh and
change the implementation decleration to be non-static
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The API contains stub API methods, this adds a call to unimplemented
method in each of the stubed method that is not implemented.
The return remains the same to help the compiler deduce the return type
of the lambda function.
After this patch a call to an unimplemented API function will return
500.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Route request to CPU 0. _operation_mode is not replicated to other CPUS.
Without this:
$ curl -X GET --header "Accept: application/json"
"http://127.0.0.1:10000/storage_service/operation_mode"
returns "NORMAL" and "STARTING" randomly.
Only cpu 0 instance of gossip has the correct information, route request
to cpu 0.
Fix a bug where
$ curl -X GET --header "Accept: application/json"
"http://172.31.5.77:10000/storage_service/gossiping"
returns true and false randomly.
Not all the API command are implemented it would be better that the user
would receive an error if it tries to call an unimplmeneted API call.
This adds an unimplemented_exception that would be thrown when an API
call is not implemented.
The unimplemented method, simply throws the exception.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
