Commit Graph

250 Commits

Author SHA1 Message Date
Amnon Heiman
54b4f26cb0 API: Change the compaction summary to use an object
In origin, there are two APIs to get the information about the current
running compactions. Both APIs do the string formatting.

This patch changes the API to have a single API get_compaction that
would return a list of summary object.

The jmx would do the string formatting for the two APIs.

This change gives a better API experience is it's better documented and
would make it easier to support future format changes in origin.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-03 11:57:37 +02:00
Amnon Heiman
1812fe9e70 API: Add the get_version to messaging_service swagger definition file
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-02 14:45:44 +02:00
Amnon Heiman
ae53604ed7 API: Add the get_version implementation to messaging service
This patch adds the implementation to the get_version.
After this patch the following url will be available:
messaging_service/version?addr=127.0.0.1

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-02 13:29:40 +02:00
Avi Kivity
53e3e79349 Merge "API: Stubing the compaction manager" from Amnon
"This series allows the compaction manager to be used by the nodetool as a stub implementation.

It has two changes:
* Add to the compaction manager API a method that returns a compaction info
object

* Stub all the compaction method so that it will create an unimplemented
warning but will not fail, the API implementation will be reverted when the
work on compaction will be completed."
2015-12-02 13:28:34 +02:00
Amnon Heiman
3674ee2fc1 API: get snapshot size
This patch adds the column family API that return the snapshot size.
The changes in the swagger definition file follo origin so the same API will be used for the metric and the
column_family.

The implementation is based on the get_snapshot_details in the
column_family.

This fix:
425

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-01 11:41:52 +02:00
Amnon Heiman
374414ffd0 API: failure_detector modify the get_all_endpoint_states
In origin, get_all_endpoint_states perform all the information
formatting and returns a string.

This is not a good API approach, this patch replaces the implementation
so the API will return an array of values and the JMX will do the
formatting.

This is a better API and would make it simpler in the future to stay in
sync with origin output.

This patch is part of #508

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-18 14:59:09 +02:00
Avi Kivity
e04a6ea80a Merge "Add the natural_endpoints API" from Amnon
"This series adds the natural_endpoints API. It adds the implementation to the storage_service and to the storage_service API.

After this series the noodtool command getendpoints should work.

example:
$ bin/nodetool getendpoints keyspace1 standard1 0x5032394c323239385030127.0.0.2
127.0.0.2"
2015-11-16 13:16:42 +02:00
Amnon Heiman
c2051c200e API: messaging service add timeout and dropped support
This patch adds the API for timeout messages and dropped messages.

For dropped messages, origin has two APIs one for messages and one for
command.

droped messages return the number of messages per ver, so our API was
rename to reflect that.

For dropped messages (command) we currently do not have this logic of
throwing messages before sending, so the API will always return 0.

The total timeout API was removed and will be done on the jmx proxy
level.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-16 13:15:59 +02:00
Amnon Heiman
cda79c9a31 API: add get_natural_endpoints to storage_service
This adds the get_natural_endpoints implementation to the
storage_service API.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-12 14:55:19 +02:00
Amnon Heiman
27737d702b API: Stubing the compaction manager - workaround
Until the compaction manager api would be ready, its failing command
causes problem with nodetool related tests.
Ths patch stub the compaction manager logic so it will not fail.

It will be replaced by an actuall implementation when the equivelent
code in compaction will be ready.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-11 16:58:13 +02:00
Amnon Heiman
66e428799f API: add compaction info object
This patch adds a compaction info object and an API that returns it.
It will be mapped to the JMX getCompactions that returns a map.

The use of an object is more RESTFull and will be better documented in
the swagger definition file.
2015-11-11 16:57:44 +02:00
Asias He
2d55e5e8ff api: Implement storage_service::move
Start node 1 and node 2:

node 1:
$ scylla --initial-token 0

node 2:
$ scylla --initial-token 2000000

To move node 2 to a new token 8000000

$ curl -X POST --header "Content-Type: application/json" --header "Accept: application/json"
"http://127.0.0.2:10000/storage_service/move?new_token=8000000"
2015-11-09 08:43:04 +08:00
Amnon Heiman
5f1cae67ff API: compaction_manager should catch the field reference by value
The get_cm_stats gets a pointer to a field in the stats object. It
should capture it by value or segmentation falut may occure when the
caller gets out of scope.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-05 14:17:12 +02:00
Amnon Heiman
ef3c6b2647 API: Add describe ring API implementation
This patch chanages the API to support describe ring instead of describe
ring jmx that will be implemented in the jmx server.

The API will return a list of objects instead of string.

An additional api was added as the equivelent to the jmx call with an
empty param.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 16:33:39 +02:00
Amnon Heiman
e562736e82 API: A workaround for nodetool cleanup
The nodetool cleanup command is used in many of the tests, because the
API call is not implemented it causes the tests to fail.

This is a workaround until the cleanup will be implemented, the method
return successfuly.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 17:38:48 +02:00
Amnon Heiman
fc7ea6207c API: messaging service, Add getter for the pending and completed server
This patch adds getter for the messeging service server pending and sent
messages.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:47 +02:00
Amnon Heiman
55d7ddafe5 API: Add repond complete to the messaging service swagger
This patch do the following:
It adds a getter for the completed respond messages (i.e. the total
messages that were sent by the server)

It replaces the return mapping for the statistics to use the key, value
notation that is used in the jmx side.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:36 +02:00
Amnon Heiman
afb921a643 API: add read repair stats to the storage_proxy
This adds the API to the storage_proxy read repair stats that replaces
the stub implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:22 +02:00
Asias He
f2e5e318d7 storage_service: Serialize effective_ownershipp with read api lock 2015-10-29 08:58:41 +08:00
Asias He
94a49121c2 storage_service: Serialize get_ownership with read api lock 2015-10-29 08:58:41 +08:00
Asias He
d41cd48a5f storage_service: Serialize is_joined with read api lock 2015-10-29 08:58:41 +08:00
Asias He
55b76a8963 api: No cpu zero trick for remove_node
storage_service::remove_node is guaranteed to run on cpu zero only.
2015-10-27 21:48:37 +08:00
Asias He
1469cec5bf gossiper: Kill free function helper to get heart version and generation number
They can only be executed on cpu 0. Make the gossiper member
functions for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
f573059698 gossiper: Kill free function helper for {unsafe_,}assassinate_endpoint
They can only be executed on cpu 0. Make the gossiper member functions
for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
c5f377eb8b gossip: Simplify get_endpoint_downtime
_unreachable_endpoints is replicated to call cores. No need to query
on core 0.
2015-10-27 21:48:37 +08:00
Asias He
6f1db4fb72 gossip: Simplify get_unreachable_members
_unreachable_endpoints is replicated to call cores. No need to query on
core 0.

This also fixes a bug in storage_proxy::truncate_blocking
which might access _unreachable_endpoints on non-zero cores.
2015-10-27 21:48:37 +08:00
Asias He
2439a2a982 gossip: Simplify get_live_members
_live_endpoints is replicated to call cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
a28ba9cde8 api: Simplify get_tokens get_node_tokens and get_token_endpoint
token_metadata is replicated to all cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
4e886f0399 storage_service: Implement is_rpc_server_running 2015-10-27 21:48:37 +08:00
Asias He
ca66bea619 storage_service: Implement is_native_transport_running 2015-10-27 21:48:37 +08:00
Asias He
201d439440 storage_service: Enable remove_node support 2015-10-26 10:33:19 +02:00
Asias He
06b19867d8 api: Fix storage_service remove_node parameter
nodetool removenode takes the parameter of host_id instead of token.

Refs: #496
2015-10-26 10:32:47 +02:00
Avi Kivity
e2e869f150 Merge "API: storage proxy metrics" from Amnon
"This series adds two types of functionality to the storage_proxy, it adds the
API that returns the timeout constants from the config and it aligned the
metrics of the read, write and range to origin StorageProxy metrics."
2015-10-25 11:40:46 +02:00
Asias He
bc8d3f0d24 storage_service: Implement get_load_map using load_broadcaster
$nodetool -p 7199 status

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens  Owns    Host ID Rack
UN  127.0.0.1  60711      3       ?  78ce8d76-69f8-44fc-a5ae-b932eb4f641b  rack1
UN  127.0.0.2  66237      3       ?  b1eaaee6-c334-4a65-bc0c-95af49bca0b3  rack1
2015-10-23 11:53:46 +08:00
Asias He
270e713fea Revert "API: Workaround for load_map"
This reverts commit 497e403387.
2015-10-23 10:45:54 +08:00
Amnon Heiman
14482d4496 API: Add read, write, range estimated histogram implementation for storage_proxy
This patch adds the implmentation for the read, write and range
estimated histogram and total latency.

After this patch the following url will be available:
/storage_proxy/metrics/read/estimated_histogram/
/storage_proxy/metrics/read
/storage_proxy/metrics/write/estimated_histogram/
/storage_proxy/metrics/write
/storage_proxy/metrics/range/estimated_histogram/
/storage_proxy/metrics/range

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 18:54:45 +03:00
Amnon Heiman
e1168c0a34 API: Add estimated histogram to storage_proxy
This patch close the gap between the storage_proxy read, write and range
metrics and the API.

For each of the metrics there will be a histogram, estimated histogram
and total.

The patch contains the definitions for the following:
get_read_estimated_histogram
get_read_latency
get_write_estimated_histogram
get_write_latency
get_range_estimated_histogram
get_range_latency

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>

need mrege storage_proxy
2015-10-22 18:54:45 +03:00
Amnon Heiman
a071cdce66 API: Add getters to storage_proxy timers timeout
This patch expose the configuration timeout values of the timers.
The timers will return their values in seconds, the swagger definition
file was modified to reflect the change.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 18:54:11 +03:00
Avi Kivity
a699bc20bc Merge "Adding the stream metrics API" from Amnon
"This series adds the stream metrics API, the swagger definition are based on
the StreamMetrics class in origin."
2015-10-22 17:18:01 +03:00
Amnon Heiman
5323b29699 API: Add compaction history to the API
This patch adds a definition and a stub for the compaction history. The
implementation should read fromt the compaction history table and return
an array of results.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 17:16:58 +03:00
Avi Kivity
5d37ee29f8 Merge "Adding log level support to the API" from Amnon
"This series adds an API to get and set the log level.
After this series it will be possible to use the folloing url:
GET/POST:
/system/logger
/system/logger/{name}"
2015-10-22 17:15:41 +03:00
Avi Kivity
b4e5b9dcf1 Merge "Add support to nodetool describecluster"
"This series adds the functionality that is required for nodetool
describecluster

It uses the gossiper for get cluster name and get partitioner.  The
describe_schema_versions functionality is missing and a workaround is used so
the command would work.

After this series an example for nodetool describecluster:
./bin/nodetool describecluster
Cluster Information:
	Name: Test Cluster
	Snitch: org.apache.cassandra.locator.SimpleSnitch
	Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
	Schema versions:
		127.0.0.1: [48c4e6c8-5d6a-3800-9a3a-517d3f7b2f26]"
2015-10-22 17:11:30 +03:00
Avi Kivity
f7087da054 Merge "GET methods for snapshots" from Glauber
"The snapshots API need to expose GET methods so people can
query information on them. Now that taking snapshots is supported,
this relatively simple series implement get_snapshot_details, a
column family method, and wire that up through the storage_service."
2015-10-22 15:23:45 +03:00
Avi Kivity
5f3a46eabb Merge "load_new_sstables" from Glauber
"This patchset implements load_new_sstables, allowing one to move tables inside the
data directory of a CF, and then call "nodetool refresh" to start using them.

Keep in mind that for Cassandra, this is deemed an unsafe operation:
https://issues.apache.org/jira/browse/CASSANDRA-6245

It is still for us something we should not recommend - unless the CF is totally
empty and not yet used, but we can do a much better job in the safety front.

To guarantee that, the process works in four steps:

1) All writes to this specific column family are disabled. This is a horrible thing to
   do, because dirty memory can grow much more than desired during this. Throughout out
   this implementation, we will try to keep the time during which the writes are disabled
   to its bare minimum.

   While disabling the writes, each shard will tell us about the highest generation number
   it has seen.

2) We will scan all tables that we haven't seen before. Those are any tables found in the
   CF datadir, that are higher than the highest generation number seen so far. We will link
   them to new generation numbers that are sequential to the ones we have so far, and end up
   with a new generation number that is returned to the next step

3) The generation number computed in the previous step is now propagated to all CFs, which
   guarantees that all further writes will pick generation numbers that won't conflict with
   the existing tables. Right after doing that, the writes are resumed.

4) The tables we found in step 2 are passed on to each of the CFs. They can now load those
   tables while operations to the CF proceed normally."
2015-10-22 13:42:24 +03:00
Amnon Heiman
c130381284 Adding live_scanned and tombstone scaned histogram to column family
This series adds a histogrm to the column family for live scanned and
tombstone scaned.

It expose those histogram via the API instead of the stub implmentation,
currently exist.

The implementation update of the histogram will be added in a different
series.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 11:13:28 +03:00
Amnon Heiman
378a97b66b API: Add row cahe hits and miss per column family
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-22 11:12:14 +03:00
Glauber Costa
fd8e5c7e4c api: load new sstables
Just a wrapper into the storage_service's homonymous call.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 18:30:04 +02:00
Glauber Costa
71c1b2fe69 api: get true snapshot size
Thin wrapper around storage service's facility.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 13:48:44 +02:00
Glauber Costa
c0630bedc2 api: get_snapshot_details
That's basically conversion work between what the storage_service returns
and the json types.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 13:48:44 +02:00
Glauber Costa
3a95a9cbe6 api: implement clear snapshot
Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-20 15:58:31 +02:00