Commit Graph

239 Commits

Author SHA1 Message Date
Asias He
2d55e5e8ff api: Implement storage_service::move
Start node 1 and node 2:

node 1:
$ scylla --initial-token 0

node 2:
$ scylla --initial-token 2000000

To move node 2 to a new token 8000000

$ curl -X POST --header "Content-Type: application/json" --header "Accept: application/json"
"http://127.0.0.2:10000/storage_service/move?new_token=8000000"
2015-11-09 08:43:04 +08:00
Amnon Heiman
5f1cae67ff API: compaction_manager should catch the field reference by value
The get_cm_stats gets a pointer to a field in the stats object. It
should capture it by value or segmentation falut may occure when the
caller gets out of scope.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-05 14:17:12 +02:00
Amnon Heiman
ef3c6b2647 API: Add describe ring API implementation
This patch chanages the API to support describe ring instead of describe
ring jmx that will be implemented in the jmx server.

The API will return a list of objects instead of string.

An additional api was added as the equivelent to the jmx call with an
empty param.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 16:33:39 +02:00
Amnon Heiman
e562736e82 API: A workaround for nodetool cleanup
The nodetool cleanup command is used in many of the tests, because the
API call is not implemented it causes the tests to fail.

This is a workaround until the cleanup will be implemented, the method
return successfuly.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 17:38:48 +02:00
Amnon Heiman
fc7ea6207c API: messaging service, Add getter for the pending and completed server
This patch adds getter for the messeging service server pending and sent
messages.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:47 +02:00
Amnon Heiman
55d7ddafe5 API: Add repond complete to the messaging service swagger
This patch do the following:
It adds a getter for the completed respond messages (i.e. the total
messages that were sent by the server)

It replaces the return mapping for the statistics to use the key, value
notation that is used in the jmx side.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:36 +02:00
Amnon Heiman
afb921a643 API: add read repair stats to the storage_proxy
This adds the API to the storage_proxy read repair stats that replaces
the stub implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:22 +02:00
Asias He
f2e5e318d7 storage_service: Serialize effective_ownershipp with read api lock 2015-10-29 08:58:41 +08:00
Asias He
94a49121c2 storage_service: Serialize get_ownership with read api lock 2015-10-29 08:58:41 +08:00
Asias He
d41cd48a5f storage_service: Serialize is_joined with read api lock 2015-10-29 08:58:41 +08:00
Asias He
55b76a8963 api: No cpu zero trick for remove_node
storage_service::remove_node is guaranteed to run on cpu zero only.
2015-10-27 21:48:37 +08:00
Asias He
1469cec5bf gossiper: Kill free function helper to get heart version and generation number
They can only be executed on cpu 0. Make the gossiper member
functions for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
f573059698 gossiper: Kill free function helper for {unsafe_,}assassinate_endpoint
They can only be executed on cpu 0. Make the gossiper member functions
for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
c5f377eb8b gossip: Simplify get_endpoint_downtime
_unreachable_endpoints is replicated to call cores. No need to query
on core 0.
2015-10-27 21:48:37 +08:00
Asias He
6f1db4fb72 gossip: Simplify get_unreachable_members
_unreachable_endpoints is replicated to call cores. No need to query on
core 0.

This also fixes a bug in storage_proxy::truncate_blocking
which might access _unreachable_endpoints on non-zero cores.
2015-10-27 21:48:37 +08:00
Asias He
2439a2a982 gossip: Simplify get_live_members
_live_endpoints is replicated to call cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
a28ba9cde8 api: Simplify get_tokens get_node_tokens and get_token_endpoint
token_metadata is replicated to all cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
4e886f0399 storage_service: Implement is_rpc_server_running 2015-10-27 21:48:37 +08:00
Asias He
ca66bea619 storage_service: Implement is_native_transport_running 2015-10-27 21:48:37 +08:00
Asias He
201d439440 storage_service: Enable remove_node support 2015-10-26 10:33:19 +02:00
Asias He
06b19867d8 api: Fix storage_service remove_node parameter
nodetool removenode takes the parameter of host_id instead of token.

Refs: #496
2015-10-26 10:32:47 +02:00
Avi Kivity
e2e869f150 Merge "API: storage proxy metrics" from Amnon
"This series adds two types of functionality to the storage_proxy, it adds the
API that returns the timeout constants from the config and it aligned the
metrics of the read, write and range to origin StorageProxy metrics."
2015-10-25 11:40:46 +02:00
Asias He
bc8d3f0d24 storage_service: Implement get_load_map using load_broadcaster
$nodetool -p 7199 status

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens  Owns    Host ID Rack
UN  127.0.0.1  60711      3       ?  78ce8d76-69f8-44fc-a5ae-b932eb4f641b  rack1
UN  127.0.0.2  66237      3       ?  b1eaaee6-c334-4a65-bc0c-95af49bca0b3  rack1
2015-10-23 11:53:46 +08:00
Asias He
270e713fea Revert "API: Workaround for load_map"
This reverts commit 497e403387.
2015-10-23 10:45:54 +08:00
Amnon Heiman
14482d4496 API: Add read, write, range estimated histogram implementation for storage_proxy
This patch adds the implmentation for the read, write and range
estimated histogram and total latency.

After this patch the following url will be available:
/storage_proxy/metrics/read/estimated_histogram/
/storage_proxy/metrics/read
/storage_proxy/metrics/write/estimated_histogram/
/storage_proxy/metrics/write
/storage_proxy/metrics/range/estimated_histogram/
/storage_proxy/metrics/range

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 18:54:45 +03:00
Amnon Heiman
e1168c0a34 API: Add estimated histogram to storage_proxy
This patch close the gap between the storage_proxy read, write and range
metrics and the API.

For each of the metrics there will be a histogram, estimated histogram
and total.

The patch contains the definitions for the following:
get_read_estimated_histogram
get_read_latency
get_write_estimated_histogram
get_write_latency
get_range_estimated_histogram
get_range_latency

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>

need mrege storage_proxy
2015-10-22 18:54:45 +03:00
Amnon Heiman
a071cdce66 API: Add getters to storage_proxy timers timeout
This patch expose the configuration timeout values of the timers.
The timers will return their values in seconds, the swagger definition
file was modified to reflect the change.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 18:54:11 +03:00
Avi Kivity
a699bc20bc Merge "Adding the stream metrics API" from Amnon
"This series adds the stream metrics API, the swagger definition are based on
the StreamMetrics class in origin."
2015-10-22 17:18:01 +03:00
Amnon Heiman
5323b29699 API: Add compaction history to the API
This patch adds a definition and a stub for the compaction history. The
implementation should read fromt the compaction history table and return
an array of results.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 17:16:58 +03:00
Avi Kivity
5d37ee29f8 Merge "Adding log level support to the API" from Amnon
"This series adds an API to get and set the log level.
After this series it will be possible to use the folloing url:
GET/POST:
/system/logger
/system/logger/{name}"
2015-10-22 17:15:41 +03:00
Avi Kivity
b4e5b9dcf1 Merge "Add support to nodetool describecluster"
"This series adds the functionality that is required for nodetool
describecluster

It uses the gossiper for get cluster name and get partitioner.  The
describe_schema_versions functionality is missing and a workaround is used so
the command would work.

After this series an example for nodetool describecluster:
./bin/nodetool describecluster
Cluster Information:
	Name: Test Cluster
	Snitch: org.apache.cassandra.locator.SimpleSnitch
	Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
	Schema versions:
		127.0.0.1: [48c4e6c8-5d6a-3800-9a3a-517d3f7b2f26]"
2015-10-22 17:11:30 +03:00
Avi Kivity
f7087da054 Merge "GET methods for snapshots" from Glauber
"The snapshots API need to expose GET methods so people can
query information on them. Now that taking snapshots is supported,
this relatively simple series implement get_snapshot_details, a
column family method, and wire that up through the storage_service."
2015-10-22 15:23:45 +03:00
Avi Kivity
5f3a46eabb Merge "load_new_sstables" from Glauber
"This patchset implements load_new_sstables, allowing one to move tables inside the
data directory of a CF, and then call "nodetool refresh" to start using them.

Keep in mind that for Cassandra, this is deemed an unsafe operation:
https://issues.apache.org/jira/browse/CASSANDRA-6245

It is still for us something we should not recommend - unless the CF is totally
empty and not yet used, but we can do a much better job in the safety front.

To guarantee that, the process works in four steps:

1) All writes to this specific column family are disabled. This is a horrible thing to
   do, because dirty memory can grow much more than desired during this. Throughout out
   this implementation, we will try to keep the time during which the writes are disabled
   to its bare minimum.

   While disabling the writes, each shard will tell us about the highest generation number
   it has seen.

2) We will scan all tables that we haven't seen before. Those are any tables found in the
   CF datadir, that are higher than the highest generation number seen so far. We will link
   them to new generation numbers that are sequential to the ones we have so far, and end up
   with a new generation number that is returned to the next step

3) The generation number computed in the previous step is now propagated to all CFs, which
   guarantees that all further writes will pick generation numbers that won't conflict with
   the existing tables. Right after doing that, the writes are resumed.

4) The tables we found in step 2 are passed on to each of the CFs. They can now load those
   tables while operations to the CF proceed normally."
2015-10-22 13:42:24 +03:00
Amnon Heiman
c130381284 Adding live_scanned and tombstone scaned histogram to column family
This series adds a histogrm to the column family for live scanned and
tombstone scaned.

It expose those histogram via the API instead of the stub implmentation,
currently exist.

The implementation update of the histogram will be added in a different
series.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 11:13:28 +03:00
Amnon Heiman
378a97b66b API: Add row cahe hits and miss per column family
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 11:12:14 +03:00
Glauber Costa
fd8e5c7e4c api: load new sstables
Just a wrapper into the storage_service's homonymous call.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 18:30:04 +02:00
Glauber Costa
71c1b2fe69 api: get true snapshot size
Thin wrapper around storage service's facility.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 13:48:44 +02:00
Glauber Costa
c0630bedc2 api: get_snapshot_details
That's basically conversion work between what the storage_service returns
and the json types.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 13:48:44 +02:00
Glauber Costa
3a95a9cbe6 api: implement clear snapshot
Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-20 15:58:31 +02:00
Glauber Costa
19bb50f450 api: implement take_snapshot
Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-20 15:58:31 +02:00
Amnon Heiman
1e8752d55e API: Fix a confusion in the storage service snapshot details
There was a confusion between the snapshot key and the keyspace in the
snapshot details, this fixes it.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-20 13:53:09 +03:00
Amnon Heiman
91d396760e API: add a workaround for the get schema_versions
This adds a workaround for the get schema_version, it will return only a
shcema version of the local node, this is a temporary workaround until
describe_schema_versions will be implemented.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-20 10:56:21 +03:00
Amnon Heiman
383d7ccf4d Add the get cluster and partitioner names to the API
This adds the implementation for the get cluster name and get
partitioner name to the storage_service API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-20 10:54:55 +03:00
Avi Kivity
2ccb5feabd Merge "Support nodetool cfhistogram"
"This series adds the missing estimated histogram to the column family and to
the API so the nodetool cfhistogram would work."
2015-10-19 17:11:46 +03:00
Amnon Heiman
5998ee718a API: Add logger API implementation to the system API
This patch adds the ability to set one or all log levels get a log level
and get all logs name.

After this patch the following url will be available:
GET/POST
/system/logger
/system/logger/{name}

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-19 15:05:46 +03:00
Amnon Heiman
ba1e6adf2a Adding the system swagger definition
The system api will include system related command, currently it holds
the logger related API. It holds definition for the following commands:
get_all_logger_names
set_all_logger_level
get_logger_level
set_logger_level

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-19 14:59:54 +03:00
Amnon Heiman
1acd47ddce API: Adding implementation for the stream manager metrics
This adds an implementation for the stream manager metrics.

And the following url will be available:
/stream_manager/metrics/outbound
/stream_manager/metrics/incoming/{peer}
/stream_manager/metrics/incoming
/stream_manager/metrics/outgoing/{peer}
/stream_manager/metrics/outgoing

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-15 16:04:36 +03:00
Amnon Heiman
164ccd0525 API: Adding the stream manager Swagger definition
This adds the swagger definition file for the stream manager. The API is
based on the StreamManagerMBean and the StreamMetrics.
The following commands where added:
get_current_streams
get_current_streams_state
get_all_active_streams_outbound
get_total_incoming_bytes
get_all_total_incoming_bytes
get_total_outgoing_bytes
get_all_total_outgoing_bytes
2015-10-15 16:04:11 +03:00
Asias He
c58ae5432c storage_service: Fix nodetool info return wrong gossiper status
Before:
$ nodetool info
ID                     : a5adfbbf-cfd8-4c88-ab6b-6a34ccc2857c
Gossip active          : false

After:
$ nodetool info
ID                     : a5adfbbf-cfd8-4c88-ab6b-6a34ccc2857c
Gossip active          : true

Fix #354.
2015-10-14 12:37:51 +03:00
Amnon Heiman
6fd3c81db5 keyspace clean up should be a POST not a GET 2015-10-11 15:51:56 +03:00