Commit Graph

104 Commits

Author SHA1 Message Date
Amnon Heiman
6c1858b275 API:storage_service should support metrics load
Following C* API there are two APIs for getting the load from
storage_service:
/storage_service/metrics/load
/storage_service/load

This patch adds the implementation for
/storage_service/metrics/load

The alternative, is to drop on of the API and modify the JMX
implementation to use the same API.

Fixes #2245

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20170401181520.19506-1-amnon@scylladb.com>
2017-04-05 18:14:19 +03:00
Nadav Har'El
d49aa7abd2 storage_service: make is_joined() an immediate function
Commit d41cd48a made the is_joined() method a future<bool> because
only cpu 0 knows its real value. This makes this function inconvenient
to use. So this patch reverts commit d41cd48a, and instead sets this
flag's value on all shards, so each shard can read its value locally
(and immediately).

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20161228160450.5831-1-nyh@scylladb.com>
2016-12-28 18:37:22 +02:00
Vlad Zolotarov
9606db2f08 api::set_tracing_probability: prevent a server from returning 500 for a bad probability value
- Change an exception type thrown by a tracing::tracing::set_trace_probability()
     to make it different from the one thrown by an std::stod() when it fails to
     parse a given string.
   - Catch the std::out_of_range exception thrown by a tracing::tracing::set_trace_probability() and
     wrap the exception string into the httpd::bad_param_exception() object.
   - Throw a httpd::bad_param_exception() with a
     "Bad format in a probability value: <a user given probability string value>"
     message if std::invalid_argument is caught.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1465300738-1557-1-git-send-email-vladz@cloudius-systems.com>
2016-12-27 12:07:09 +02:00
Calle Wilund
5eb54f9bc4 api::storage_service: c3 compat - make query keyspaces a trinary choice
all, user or non-local strategy ones.
2016-11-08 12:22:04 +00:00
Tomasz Grabiec
c1a7e2090e Revert "database: change find_column_families signature so it returns a lw_shared_ptr"
This reverts commit f3528ede65.
2016-11-04 10:48:21 +01:00
Glauber Costa
f3528ede65 database: change find_column_families signature so it returns a lw_shared_ptr
There are places in which we need to use the column family object many
times, with deferring points in between. Because the column family may
have been destroyed in the deferring point, we need to go and find it
again.

If we use lw_shared_ptr, however, we'll be able to at least guarantee
that the object will be alive. Some users will still need to check, if
they want to guarantee that the column family wasn't removed. But others
that only need to make sure we don't access an invalid object will be
able to avoid the cost of re-finding it just fine.

Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <722bf49e158da77ff509372c2034e5707706e5bf.1478111467.git.glauber@scylladb.com>
2016-11-03 13:27:31 +01:00
Vlad Zolotarov
006999f46c api::storage_service::slow_query: don't use duration_cast in GET
The slow_query_record_ttl() and slow_query_threshold() return the duration
of the appropriate type already - no need for an additional cast.
In addition there was a mistake in a cast of ttl.

Fixes #1734

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1475669400-5925-1-git-send-email-vladz@cloudius-systems.com>
2016-10-09 18:09:13 +03:00
Amnon Heiman
11c687dd93 API: Add slow query logging implementation
This adds the implementation for the slow query logging API.

After this patch the following will be available:

curl -X GET  "http://localhost:10000/storage_service/slow_query"
curl -X POST
"http://localhost:10000/storage_service/slow_query?enable=true&ttl=10&threshold=6000"

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-09-03 01:15:22 +03:00
Amnon Heiman
e66a1cd705 API: Add implementation for the scylla release version
This adds the implementation to the scylla release version API.

After this patch a call to:

curl -X GET "http://localhost:10000/storage_service/scylla_release_version"

Will return the current scylla release version.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-07-03 16:29:09 +03:00
Vlad Zolotarov
0611417c76 api::storage_service: add set_trace_probability/get_trace_probability
Trace probability defines a probability for the next CQL command
to be traced.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-06-06 15:44:28 +03:00
Asias He
1e84699a64 api: Wire up storage_service removal_status and force_remove_completion
They are used by nodetool removenode:

$ nodetool removenode force
$ nodetool removenode status

For example:

$ nodetool removenode status
RemovalStatus: Removing token (-8969872965815280276). Waiting for
replication confirmation from [127.0.0.3,127.0.0.1].

$ nodetool removenode force
RemovalStatus: No token removals in process.

Tested with:

1)
- start 3 nodes
- inject data with
  cassandra-stress write no-warmup cl=TWO n=2000000 -schema 'replication(factor=2)'
- kill -9 node2
- wait for node2 to be in DOWN state
- run nodetool removenode host2_host_id on node1

2)
- start 3 nodes
- inject data with
  cassandra-stress write no-warmup cl=TWO n=2000000 -schema 'replication(factor=2)'
- kill -9 node2
- wait for node2 to be in DOWN state
- run nodetool removenode host2_host_id on node1
- kill -9 node3
- nodetool removenode will wait forever since node3 is gonne, node3
  will never send the replication confirmation to node1
- run nodetool removenode force on node1
  nodetool removenode completes with the following error:
    $ nodetool removenode 31690b82-ebb0-4594-8bcf-1ce82b6e0f6e
    nodetool: Scylla API server HTTP POST to URL
    '/storage_service/remove_node' failed: nodetool removenode force is called by user
  nodetool removenode force completes sucessfully
    $ nodetool removenode force
    RemovalStatus: Removing token (-9171569494049085776). Waiting for
    replication confirmation from [127.0.0.3,127.0.0.1].

Fixes 1135.
2016-04-13 14:53:28 +08:00
Asias He
891e947314 storage_service: Rename remove_node to removenode
nodetool uses removenode command to remove a node. Rename the
implementation in storage_service to match the command.
2016-04-13 14:53:28 +08:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Vlad Zolotarov
2cd836a02e api::set_storage_service(): fix the 'nodetool enablebackup' API
'nodetool enable/disablebackup' callback was modifying only the
existing keyspaces and column families configurations.
However new keyspaces/column families were using
the original 'incremental_backups' configuration value which could
be different from the value configured by 'nodetool enable/disablebackup'
user command.

This patch updates the database::_enable_incremental_backups per-shard
value in addition to updating the existing keyspaces and column families
configurations.

Fixes #845

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-03-06 17:26:31 +02:00
Nadav Har'El
f9ee74f56f repair: options for repairing only a subrange
To implement nodetool's "--start-token"/"--end-token" feature, we need
to be able to repair only *part* of the ranges held by this node.
Our REST API already had a "ranges" option where the tool can list the
specific ranges to repair, but using this interface in the JMX
implementation is inconvenient, because it requires the *Java* code
to be able to intersect the given start/end token range with the actual
ranges held by the repaired node.

A more reasonable approach, which this patch uses, is to add new
"startToken"/"endToken" options to the repair's REST API. What these
options do is is to find the node's token ranges as usual, and only
then *intersect* them with the user-specified token range. The JMX
implementation becomes much simpler (in a separate patch for scylla-jmx)
and the real work is done in the C++ code, where it belongs, not in
Java code.

With the additional scylla-jmx patch to use the new REST API options
provided here, this fixes #917.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <1455807739-25581-1-git-send-email-nyh@scylladb.com>
2016-02-18 17:13:56 +02:00
Raphael S. Carvalho
a53cfc8127 compaction manager: add support to wait for termination of cleanup
'nodetool cleanup' must wait for termination of cleanup, however,
cleanup is handled asynchronously. To solve that, a mechanism is
added here to wait for termination of a cleanup. This mechanism is
about using promise to notificate waiter of cleanup completion.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <6dc0a39170f3f51487fb8858eb443573548d8bce.1455655016.git.raphaelsc@scylladb.com>
2016-02-18 17:01:18 +02:00
Amnon Heiman
e33710d2ca API: storage_service get_logging_level
This patch adds the get_loggin_level command that returns a map between
the log name and its level.
To test the API do:
curl -X GET "http://localhost:10000/storage_service/logging_level"

this would enable the `nodetool getlogginglevels` command.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1453365106-27294-3-git-send-email-amnon@scylladb.com>
2016-01-21 11:58:54 +02:00
Raphael S. Carvalho
fc6a1934b0 api: implement force_keyspace_cleanup
This will add support for an user to clean up an entire keyspace
or some of its column families.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2016-01-12 03:53:22 -02:00
Pekka Enberg
67ccd05bbe api/storage_service: Wire up 'compaction_throughput_mb_per_sec'
The API is needed by nodetool compactionstats command.
2016-01-05 13:01:05 +02:00
Amnon Heiman
6942b41693 API: rename the map of string, double to map_string_double
This replaces the confusing name to a more meaningful name.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1451466952-405-1-git-send-email-amnon@scylladb.com>
2016-01-03 19:10:49 +02:00
Pekka Enberg
0aa105c9cf Merge "load report a negative value" from Amnon
"This series solve an issue with the load broadcaster that reports negative
 values due to an integer wrap around.  While fixing this issue an additional
 change was made so that the load_map would return doubles and not formatted
 string.  This is a better API, safer and better documented."
2015-12-30 10:21:55 +02:00
Amnon Heiman
ec379649ea API: repair to use documented params
The repair API use to have an undocumented parameter list similiar to
origin.

This patch changes the way repair is getting its parameters.
Instead of a one undocumented string it now lists all the different
optional parameters in the swagger file and accept them explicitely.

Reviewed-by: Nadav Har'El <nyh@scylladb.com>
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-29 15:38:44 +02:00
Amnon Heiman
71905081b1 API: report the load map as an unformatted double
In origin the storage_serivce report the load map as a formatted string.
As an API a better option is to report the load map as double and let
the JMX proxy do the formatting.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-29 11:55:34 +02:00
Paweł Dziepak
39a65e6294 api: enable storage_service::drain()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-17 14:06:41 +01:00
Paweł Dziepak
8ee1a44720 storage_service: implement get_drain_progress()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-17 14:06:40 +01:00
Asias He
e9a4d93d1b storage_service: Fix added node not showing up in nodetool in status joining
The get_token_endpoint API should return a map of tokens to endpoints,
including the bootstrapping ones.

Use get_local_storage_service().get_token_to_endpoint_map() for it.

$ nodetool -p 7100 status

Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens  Owns    Host ID Rack
UN  127.0.0.1  12645      256     ?  eac5b6cf-5fda-4447-8104-a7bf3b773aba  rack1
UN  127.0.0.2  12635      256     ?  2ad1b7df-c8ad-4cbc-b1f1-059121d2f0c7  rack1
UN  127.0.0.3  12624      256     ?  61f82ea7-637d-4083-acc9-567e0c01b490  rack1
UJ  127.0.0.4  ?          256     ?  ced2725e-a5a4-4ac3-86de-e1c66cecfb8d  rack1

Fixes #617
2015-12-09 10:43:51 +08:00
Amnon Heiman
cda79c9a31 API: add get_natural_endpoints to storage_service
This adds the get_natural_endpoints implementation to the
storage_service API.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-12 14:55:19 +02:00
Asias He
2d55e5e8ff api: Implement storage_service::move
Start node 1 and node 2:

node 1:
$ scylla --initial-token 0

node 2:
$ scylla --initial-token 2000000

To move node 2 to a new token 8000000

$ curl -X POST --header "Content-Type: application/json" --header "Accept: application/json"
"http://127.0.0.2:10000/storage_service/move?new_token=8000000"
2015-11-09 08:43:04 +08:00
Amnon Heiman
ef3c6b2647 API: Add describe ring API implementation
This patch chanages the API to support describe ring instead of describe
ring jmx that will be implemented in the jmx server.

The API will return a list of objects instead of string.

An additional api was added as the equivelent to the jmx call with an
empty param.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 16:33:39 +02:00
Amnon Heiman
e562736e82 API: A workaround for nodetool cleanup
The nodetool cleanup command is used in many of the tests, because the
API call is not implemented it causes the tests to fail.

This is a workaround until the cleanup will be implemented, the method
return successfuly.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 17:38:48 +02:00
Asias He
f2e5e318d7 storage_service: Serialize effective_ownershipp with read api lock 2015-10-29 08:58:41 +08:00
Asias He
94a49121c2 storage_service: Serialize get_ownership with read api lock 2015-10-29 08:58:41 +08:00
Asias He
d41cd48a5f storage_service: Serialize is_joined with read api lock 2015-10-29 08:58:41 +08:00
Asias He
55b76a8963 api: No cpu zero trick for remove_node
storage_service::remove_node is guaranteed to run on cpu zero only.
2015-10-27 21:48:37 +08:00
Asias He
1469cec5bf gossiper: Kill free function helper to get heart version and generation number
They can only be executed on cpu 0. Make the gossiper member
functions for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
a28ba9cde8 api: Simplify get_tokens get_node_tokens and get_token_endpoint
token_metadata is replicated to all cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
4e886f0399 storage_service: Implement is_rpc_server_running 2015-10-27 21:48:37 +08:00
Asias He
ca66bea619 storage_service: Implement is_native_transport_running 2015-10-27 21:48:37 +08:00
Asias He
201d439440 storage_service: Enable remove_node support 2015-10-26 10:33:19 +02:00
Asias He
bc8d3f0d24 storage_service: Implement get_load_map using load_broadcaster
$nodetool -p 7199 status

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens  Owns    Host ID Rack
UN  127.0.0.1  60711      3       ?  78ce8d76-69f8-44fc-a5ae-b932eb4f641b  rack1
UN  127.0.0.2  66237      3       ?  b1eaaee6-c334-4a65-bc0c-95af49bca0b3  rack1
2015-10-23 11:53:46 +08:00
Asias He
270e713fea Revert "API: Workaround for load_map"
This reverts commit 497e403387.
2015-10-23 10:45:54 +08:00
Avi Kivity
b4e5b9dcf1 Merge "Add support to nodetool describecluster"
"This series adds the functionality that is required for nodetool
describecluster

It uses the gossiper for get cluster name and get partitioner.  The
describe_schema_versions functionality is missing and a workaround is used so
the command would work.

After this series an example for nodetool describecluster:
./bin/nodetool describecluster
Cluster Information:
	Name: Test Cluster
	Snitch: org.apache.cassandra.locator.SimpleSnitch
	Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
	Schema versions:
		127.0.0.1: [48c4e6c8-5d6a-3800-9a3a-517d3f7b2f26]"
2015-10-22 17:11:30 +03:00
Avi Kivity
f7087da054 Merge "GET methods for snapshots" from Glauber
"The snapshots API need to expose GET methods so people can
query information on them. Now that taking snapshots is supported,
this relatively simple series implement get_snapshot_details, a
column family method, and wire that up through the storage_service."
2015-10-22 15:23:45 +03:00
Glauber Costa
fd8e5c7e4c api: load new sstables
Just a wrapper into the storage_service's homonymous call.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 18:30:04 +02:00
Glauber Costa
71c1b2fe69 api: get true snapshot size
Thin wrapper around storage service's facility.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 13:48:44 +02:00
Glauber Costa
c0630bedc2 api: get_snapshot_details
That's basically conversion work between what the storage_service returns
and the json types.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-21 13:48:44 +02:00
Glauber Costa
3a95a9cbe6 api: implement clear snapshot
Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-20 15:58:31 +02:00
Glauber Costa
19bb50f450 api: implement take_snapshot
Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-20 15:58:31 +02:00
Amnon Heiman
383d7ccf4d Add the get cluster and partitioner names to the API
This adds the implementation for the get cluster name and get
partitioner name to the storage_service API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-10-20 10:54:55 +03:00
Asias He
c58ae5432c storage_service: Fix nodetool info return wrong gossiper status
Before:
$ nodetool info
ID                     : a5adfbbf-cfd8-4c88-ab6b-6a34ccc2857c
Gossip active          : false

After:
$ nodetool info
ID                     : a5adfbbf-cfd8-4c88-ab6b-6a34ccc2857c
Gossip active          : true

Fix #354.
2015-10-14 12:37:51 +03:00