Commit Graph

267 Commits

Author SHA1 Message Date
Asias He
2345cda42f messaging_service: Rename shard_id to msg_addr
Use shard_id as the destination of the messaging_service is confusing,
since shard_id is used in the context of cpu id.
Message-Id: <8c9ef193dc000ef06f8879e6a01df65cf24635d8.1452155241.git.asias@scylladb.com>
2016-01-07 10:36:35 +02:00
Pekka Enberg
67ccd05bbe api/storage_service: Wire up 'compaction_throughput_mb_per_sec'
The API is needed by nodetool compactionstats command.
2016-01-05 13:01:05 +02:00
Amnon Heiman
6942b41693 API: rename the map of string, double to map_string_double
This replaces the confusing name to a more meaningful name.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1451466952-405-1-git-send-email-amnon@scylladb.com>
2016-01-03 19:10:49 +02:00
Asias He
89b79d44de streaming: Get rid of the _connecting_ parameter
messaging_service will use private ip address automatically to connect a
peer node if possible. There is no need for the upper level like
streaming to worry about it. Drop it simplifies things a bit.
2015-12-31 11:25:08 +01:00
Amnon Heiman
8d31c27f7b api: stream_state should include the stream_info
The stream info was creted but was left out of the stream state. This
patch adds the created stream_info to the stream state vector.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-31 12:01:03 +02:00
Gleb Natapov
2bcfe02ee6 messaging: remove unused verbs 2015-12-30 15:06:35 +01:00
Pekka Enberg
0aa105c9cf Merge "load report a negative value" from Amnon
"This series solve an issue with the load broadcaster that reports negative
 values due to an integer wrap around.  While fixing this issue an additional
 change was made so that the load_map would return doubles and not formatted
 string.  This is a better API, safer and better documented."
2015-12-30 10:21:55 +02:00
Amnon Heiman
ec379649ea API: repair to use documented params
The repair API use to have an undocumented parameter list similiar to
origin.

This patch changes the way repair is getting its parameters.
Instead of a one undocumented string it now lists all the different
optional parameters in the swagger file and accept them explicitely.

Reviewed-by: Nadav Har'El <nyh@scylladb.com>
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-29 15:38:44 +02:00
Amnon Heiman
71905081b1 API: report the load map as an unformatted double
In origin the storage_serivce report the load map as a formatted string.
As an API a better option is to report the load map as double and let
the JMX proxy do the formatting.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-29 11:55:34 +02:00
Amnon Heiman
b0856f7acf API: Init value for cf_map reduce should be of type int64_t
The helper function for summing statistic over the column family are
template function that infer the return type acording to the type of the
Init param.

In the API the return value should be int64_t, passing an integer would
cause a number wrap around.

A partial output from the nodetool cfstats after the fix

nodetool cfstats keyspace1
Keyspace: keyspace1
	Read Count: 0
	Read Latency: NaN ms.
	Write Count: 4050000
	Write Latency: 0.009178098765432099 ms.
	Pending Flushes: 0
		Table: standard1
		SSTable count: 12
		Space used (live): 1118617445
		Space used (total): 23336562465

Fixes #682

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-22 17:33:13 +02:00
Paweł Dziepak
39a65e6294 api: enable storage_service::drain()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-17 14:06:41 +01:00
Paweł Dziepak
8ee1a44720 storage_service: implement get_drain_progress()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-17 14:06:40 +01:00
Raphael S. Carvalho
833a78e9f7 api: implement get_compactions
get_compactions returns progress information about each
ongoing compaction.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:50:36 -02:00
Raphael S. Carvalho
87fbe29cf9 api: add support to compaction history
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:00:21 -02:00
Asias He
e9a4d93d1b storage_service: Fix added node not showing up in nodetool in status joining
The get_token_endpoint API should return a map of tokens to endpoints,
including the bootstrapping ones.

Use get_local_storage_service().get_token_to_endpoint_map() for it.

$ nodetool -p 7100 status

Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens  Owns    Host ID Rack
UN  127.0.0.1  12645      256     ?  eac5b6cf-5fda-4447-8104-a7bf3b773aba  rack1
UN  127.0.0.2  12635      256     ?  2ad1b7df-c8ad-4cbc-b1f1-059121d2f0c7  rack1
UN  127.0.0.3  12624      256     ?  61f82ea7-637d-4083-acc9-567e0c01b490  rack1
UJ  127.0.0.4  ?          256     ?  ced2725e-a5a4-4ac3-86de-e1c66cecfb8d  rack1

Fixes #617
2015-12-09 10:43:51 +08:00
Amnon Heiman
3ce7fa181c API: Add the implementation for index_summary_off_heap_memory
This adds the implementation for the index_summary_off_heap_memory for a
single column family and for all of them.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-07 15:15:39 +02:00
Tomasz Grabiec
934d3f06d1 api: Make histogram reduction work on domain value instead of json objects
Objects extending json_base are not movable, so we won't be able to
pass them via future<>, which will assert that types are nothrow move
constructible.

This problem only affects httpd::utils_json::histogram, which is used
in map-reduce. This patch changes the aggregation to work on domain
value (utils::ihistrogram) instead of json objects.
2015-12-07 09:50:28 +01:00
Amnon Heiman
54b4f26cb0 API: Change the compaction summary to use an object
In origin, there are two APIs to get the information about the current
running compactions. Both APIs do the string formatting.

This patch changes the API to have a single API get_compaction that
would return a list of summary object.

The jmx would do the string formatting for the two APIs.

This change gives a better API experience is it's better documented and
would make it easier to support future format changes in origin.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-03 11:57:37 +02:00
Amnon Heiman
1812fe9e70 API: Add the get_version to messaging_service swagger definition file
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-02 14:45:44 +02:00
Amnon Heiman
ae53604ed7 API: Add the get_version implementation to messaging service
This patch adds the implementation to the get_version.
After this patch the following url will be available:
messaging_service/version?addr=127.0.0.1

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-02 13:29:40 +02:00
Avi Kivity
53e3e79349 Merge "API: Stubing the compaction manager" from Amnon
"This series allows the compaction manager to be used by the nodetool as a stub implementation.

It has two changes:
* Add to the compaction manager API a method that returns a compaction info
object

* Stub all the compaction method so that it will create an unimplemented
warning but will not fail, the API implementation will be reverted when the
work on compaction will be completed."
2015-12-02 13:28:34 +02:00
Amnon Heiman
3674ee2fc1 API: get snapshot size
This patch adds the column family API that return the snapshot size.
The changes in the swagger definition file follo origin so the same API will be used for the metric and the
column_family.

The implementation is based on the get_snapshot_details in the
column_family.

This fix:
425

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-01 11:41:52 +02:00
Amnon Heiman
374414ffd0 API: failure_detector modify the get_all_endpoint_states
In origin, get_all_endpoint_states perform all the information
formatting and returns a string.

This is not a good API approach, this patch replaces the implementation
so the API will return an array of values and the JMX will do the
formatting.

This is a better API and would make it simpler in the future to stay in
sync with origin output.

This patch is part of #508

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-18 14:59:09 +02:00
Avi Kivity
e04a6ea80a Merge "Add the natural_endpoints API" from Amnon
"This series adds the natural_endpoints API. It adds the implementation to the storage_service and to the storage_service API.

After this series the noodtool command getendpoints should work.

example:
$ bin/nodetool getendpoints keyspace1 standard1 0x5032394c323239385030127.0.0.2
127.0.0.2"
2015-11-16 13:16:42 +02:00
Amnon Heiman
c2051c200e API: messaging service add timeout and dropped support
This patch adds the API for timeout messages and dropped messages.

For dropped messages, origin has two APIs one for messages and one for
command.

droped messages return the number of messages per ver, so our API was
rename to reflect that.

For dropped messages (command) we currently do not have this logic of
throwing messages before sending, so the API will always return 0.

The total timeout API was removed and will be done on the jmx proxy
level.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-16 13:15:59 +02:00
Amnon Heiman
cda79c9a31 API: add get_natural_endpoints to storage_service
This adds the get_natural_endpoints implementation to the
storage_service API.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-12 14:55:19 +02:00
Amnon Heiman
27737d702b API: Stubing the compaction manager - workaround
Until the compaction manager api would be ready, its failing command
causes problem with nodetool related tests.
Ths patch stub the compaction manager logic so it will not fail.

It will be replaced by an actuall implementation when the equivelent
code in compaction will be ready.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-11 16:58:13 +02:00
Amnon Heiman
66e428799f API: add compaction info object
This patch adds a compaction info object and an API that returns it.
It will be mapped to the JMX getCompactions that returns a map.

The use of an object is more RESTFull and will be better documented in
the swagger definition file.
2015-11-11 16:57:44 +02:00
Asias He
2d55e5e8ff api: Implement storage_service::move
Start node 1 and node 2:

node 1:
$ scylla --initial-token 0

node 2:
$ scylla --initial-token 2000000

To move node 2 to a new token 8000000

$ curl -X POST --header "Content-Type: application/json" --header "Accept: application/json"
"http://127.0.0.2:10000/storage_service/move?new_token=8000000"
2015-11-09 08:43:04 +08:00
Amnon Heiman
5f1cae67ff API: compaction_manager should catch the field reference by value
The get_cm_stats gets a pointer to a field in the stats object. It
should capture it by value or segmentation falut may occure when the
caller gets out of scope.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-05 14:17:12 +02:00
Amnon Heiman
ef3c6b2647 API: Add describe ring API implementation
This patch chanages the API to support describe ring instead of describe
ring jmx that will be implemented in the jmx server.

The API will return a list of objects instead of string.

An additional api was added as the equivelent to the jmx call with an
empty param.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 16:33:39 +02:00
Amnon Heiman
e562736e82 API: A workaround for nodetool cleanup
The nodetool cleanup command is used in many of the tests, because the
API call is not implemented it causes the tests to fail.

This is a workaround until the cleanup will be implemented, the method
return successfuly.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 17:38:48 +02:00
Amnon Heiman
fc7ea6207c API: messaging service, Add getter for the pending and completed server
This patch adds getter for the messeging service server pending and sent
messages.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:47 +02:00
Amnon Heiman
55d7ddafe5 API: Add repond complete to the messaging service swagger
This patch do the following:
It adds a getter for the completed respond messages (i.e. the total
messages that were sent by the server)

It replaces the return mapping for the statistics to use the key, value
notation that is used in the jmx side.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:36 +02:00
Amnon Heiman
afb921a643 API: add read repair stats to the storage_proxy
This adds the API to the storage_proxy read repair stats that replaces
the stub implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:22 +02:00
Asias He
f2e5e318d7 storage_service: Serialize effective_ownershipp with read api lock 2015-10-29 08:58:41 +08:00
Asias He
94a49121c2 storage_service: Serialize get_ownership with read api lock 2015-10-29 08:58:41 +08:00
Asias He
d41cd48a5f storage_service: Serialize is_joined with read api lock 2015-10-29 08:58:41 +08:00
Asias He
55b76a8963 api: No cpu zero trick for remove_node
storage_service::remove_node is guaranteed to run on cpu zero only.
2015-10-27 21:48:37 +08:00
Asias He
1469cec5bf gossiper: Kill free function helper to get heart version and generation number
They can only be executed on cpu 0. Make the gossiper member
functions for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
f573059698 gossiper: Kill free function helper for {unsafe_,}assassinate_endpoint
They can only be executed on cpu 0. Make the gossiper member functions
for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
c5f377eb8b gossip: Simplify get_endpoint_downtime
_unreachable_endpoints is replicated to call cores. No need to query
on core 0.
2015-10-27 21:48:37 +08:00
Asias He
6f1db4fb72 gossip: Simplify get_unreachable_members
_unreachable_endpoints is replicated to call cores. No need to query on
core 0.

This also fixes a bug in storage_proxy::truncate_blocking
which might access _unreachable_endpoints on non-zero cores.
2015-10-27 21:48:37 +08:00
Asias He
2439a2a982 gossip: Simplify get_live_members
_live_endpoints is replicated to call cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
a28ba9cde8 api: Simplify get_tokens get_node_tokens and get_token_endpoint
token_metadata is replicated to all cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
4e886f0399 storage_service: Implement is_rpc_server_running 2015-10-27 21:48:37 +08:00
Asias He
ca66bea619 storage_service: Implement is_native_transport_running 2015-10-27 21:48:37 +08:00
Asias He
201d439440 storage_service: Enable remove_node support 2015-10-26 10:33:19 +02:00
Asias He
06b19867d8 api: Fix storage_service remove_node parameter
nodetool removenode takes the parameter of host_id instead of token.

Refs: #496
2015-10-26 10:32:47 +02:00
Avi Kivity
e2e869f150 Merge "API: storage proxy metrics" from Amnon
"This series adds two types of functionality to the storage_proxy, it adds the
API that returns the timeout constants from the config and it aligned the
metrics of the read, write and range to origin StorageProxy metrics."
2015-10-25 11:40:46 +02:00