Commit Graph

7233 Commits

Author SHA1 Message Date
Avi Kivity
389e035d19 Update scylla-ami submodule
* dist/ami/files/scylla-ami c6ddbea...3f37184 (1):
  > Update reflector URL

(cherry picked from commit 440b403089)
2015-11-03 08:12:51 +08:00
Takuya ASADA
a0feaf801c dist: add scylla.repo to fetch scylla rpms on ami
Mistakenly didn't included on yum repository for AMI patchset, but it's needed

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
(cherry picked from commit 8587c4d6b3)
2015-11-03 08:01:09 +08:00
Takuya ASADA
a5a69279ef dist: update packages on ec2 instance first bootup
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
(cherry picked from commit aaeccdee60)
2015-11-03 07:50:43 +08:00
Takuya ASADA
db634dfe01 dist: use scylla repo on ami, instead of locally built rpms
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
(cherry picked from commit 8b98fe5a1c)
2015-11-03 07:50:33 +08:00
Takuya ASADA
63d16f5816 dist: move inline script to setup-ami.sh
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
(cherry picked from commit 2863b8098f)
2015-11-03 07:50:12 +08:00
Amnon Heiman
e562736e82 API: A workaround for nodetool cleanup
The nodetool cleanup command is used in many of the tests, because the
API call is not implemented it causes the tests to fail.

This is a workaround until the cleanup will be implemented, the method
return successfuly.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 17:38:48 +02:00
Amnon Heiman
80e5289359 Adding API as an unimplemented cause
Normally an API call that is not implemented should fail, there are
cases that as a workaround an API call is stub, in those cases a warning
is added to indicate that the API is not implemented.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 17:38:48 +02:00
Avi Kivity
c9b8c368bd Merge "Adding missing statistics to enable nodetool netstats" from Amnon
"The series adds read repair and storage_service server statistics."
2015-11-02 17:10:43 +02:00
Amnon Heiman
fc7ea6207c API: messaging service, Add getter for the pending and completed server
This patch adds getter for the messeging service server pending and sent
messages.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:47 +02:00
Amnon Heiman
55d7ddafe5 API: Add repond complete to the messaging service swagger
This patch do the following:
It adds a getter for the completed respond messages (i.e. the total
messages that were sent by the server)

It replaces the return mapping for the statistics to use the key, value
notation that is used in the jmx side.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:36 +02:00
Amnon Heiman
afb921a643 API: add read repair stats to the storage_proxy
This adds the API to the storage_proxy read repair stats that replaces
the stub implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:22 +02:00
Amnon Heiman
b6034572dc storage_proxy: Add read repair statistics
This adds the read repair statistics to he storage_proxy stats and adds
to its implementation incrementing the counters value.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:16:40 +02:00
Amnon Heiman
d5d0653210 messaging_service: Add a function that goes over all the server stats
The API needs to get the stats from the rpc server, that is hidden from the
messaging service API.

This patch adds a foreach function that goes over all the server stats
without exposing the server implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:15:52 +02:00
Avi Kivity
9c09f43311 Merge "add [background] read/write statistics" from Gleb
"The main objective of the series is to introduce statistics about ongoing
read/writes and especially those that are done in the background (acknowledged,
but uncompleted), but it contains some cleanups as well."
2015-11-02 15:46:07 +02:00
Avi Kivity
e2b661bb78 sstables: fix compile error 2015-11-02 15:32:44 +02:00
Gleb Natapov
9af55b3a7b storage_proxy: add ongoing read statistics
Add statistics for ongoing reads and ongoing background reads. Read is
a background one if it was acknowledged, but there still work to do to
complete it.
2015-11-02 15:02:13 +02:00
Gleb Natapov
463d330a24 storage_proxy: do not capture 'this' needlessly. 2015-11-02 15:01:16 +02:00
Gleb Natapov
5067d027ba storage_proxy: export statistics via collectd 2015-11-02 15:01:16 +02:00
Gleb Natapov
287cf894a0 storage_proxy: count background mutation writes
Count how many writes are running in the background. Write is a
background write if it was acknowledged, but there still work to do to
complete it.
2015-11-02 15:01:12 +02:00
Gleb Natapov
afa61cb87c storage_proxy: separate methods from members with access specifier 2015-11-02 15:00:47 +02:00
Gleb Natapov
62e5ad27d8 storage_proxy: devirtualize total_block_for()
total_block_for() is defined only in base class, so make it non virtual.
2015-11-02 15:00:47 +02:00
Gleb Natapov
59d8f9f392 storage_proxy: move proxy pointer to write response handler
We will need it later there.
2015-11-02 15:00:47 +02:00
Gleb Natapov
ca0177f5cc storage_proxy: simplify timeout handling
Use _cl_achieved instead of calculating if there is enough acks.
2015-11-02 15:00:47 +02:00
Gleb Natapov
4838de008f storage_proxy: get rid of semaphore in mutation write response handler
Acks are counted in _cl_acks, so there is no need to count them once
more in semaphore, just make promise available when there is enough of
them.
2015-11-02 15:00:47 +02:00
Gleb Natapov
9381ad0741 storage_proxy: initialize _stats 2015-11-02 15:00:47 +02:00
Avi Kivity
b1d13636bb Merge "locator: use the correct snitch instance in gossiping_property_file_snitch::reload_configuration()" from Vlad
"Commit 4cd9c4c0c5441cf55e280c6f2f2e5529426b9c98 introduced a minor
issue: a wrong snitch instance may be used when updating a Gossiper state
(if I/O CPU is different from CPU0).

In order to fix this issue a local snitch instance on CPU0 should be used,
just like a Gossiper local instance.

We have to move some interfaces to i_endpoint_snitch
from being private in a gossiping_property_file_snitch in order to be
able to access it using snitch_ptr handle."
2015-11-02 14:09:27 +02:00
Vlad Zolotarov
b654d942b4 locator::gossiping_property_file_snitch: don't ignore a returned future
Don't ignore yet another returned future in reload_configuration().

Since commit 5e8037b50a
storage_service::gossip_snitch_info() returns a future.

This patch takes this into an account.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-11-02 13:44:53 +02:00
Vlad Zolotarov
689d5fb000 locator::gossiping_property_file_snitch: fix in reload_configuration()
When we access a gossiper instance we use a _gossip_started
state of a snitch, which is set in a gossiper_starting() method.

gossiper_starting() method however is invoked by a gossiper on CPU0
only therefore the _gossip_started snitch state will be set for an
instance on CPU0 only.

Therefore instead of synchronizing the _gossip_started state between
all shards we just have to make sure we check it on the right CPU,
which is CPU0.

This patch fixes this issue.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-11-02 13:44:53 +02:00
Vlad Zolotarov
5da4e62a59 locator::i_endpoint_snitch: align the _prefer_local parameter with _my_dc and _my_rack
Adjust the interface and distribution of prefer_local parameter read
from a snitch property file with the rest of similar parameters (e.g. dc and rack):
they are read and their values are distributed (copied) across all shards'
instances.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-11-02 13:44:53 +02:00
Vlad Zolotarov
5042f3c952 locator::i_endpoint_snitch_base: make reload_gossiper_state() a virtual function
Make reload_gossiper_state() be a virtual method
of a base class in order to allow calling it using a snitch_ptr
handle.

A base class already has a ton of virtual methods so no harm is
done performance-wise. Using virtual methods instead of doing
dynamic_cast results in a much cleaner code however.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-11-02 13:44:53 +02:00
Vlad Zolotarov
926ce145db locator::i_endpoint_snitch_base: move _gossip_started to the base class
Move the member and add an access method.
This is needed in order to be able to access this state using
snitch_ptr handle.

This also allows to get rid of ec2_multi_region_snitch::_helper_added
member since it duplicates _gossip_started semantics.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-11-02 13:44:31 +02:00
Takuya ASADA
919d94a52b dist: update Ubuntu's g++ to 5.x
Now ubuntu-toolchain-r repo has g++-5 package for 14.04, so switch to it

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-02 13:31:19 +02:00
Avi Kivity
4e3d281402 Merge 2015-11-02 13:31:02 +02:00
Avi Kivity
d436617adc Merge seastar upstream
* seastar 9ae6407...258daf9 (6):
  > rpc server: Add pending and sent messages to server
  > scripts: posix_net_conf.sh: Use a generic logic for RPS configuring
  > scripts: posix_net_conf.sh: allow passing a NIC name as a parameter
  > doc: link to the tutorial
  > tutorial: begin documenting the network API
  > slab: remove bogus uintptr_t definition
2015-11-02 13:30:26 +02:00
Avi Kivity
c6968b2940 Make abstract_type::serialize(data_value) a member of data_value
Reduces duplication.  Fixes #517.
2015-11-02 12:25:35 +01:00
Asias He
66a576c0c0 tests: Add documents to tests/gossip.cc
- How to run the test
- What to expect
2015-11-02 11:29:43 +02:00
Avi Kivity
4b1fffe54c Merge "storage_service: serialize more management operations" from Asias
- Add run_with_no_api_lock
- Do not hold the api lock while do the streaming for rebuild
- Include "config: Enable storage_port option" in this series
2015-11-02 10:55:57 +02:00
Avi Kivity
107877f33a Merge "Futurize all callers of add_local_application_state" from Asias
"In 5e8037b50a (gossip: Futurize
add_local_application_state()) , we futurized add_local_application_state.
However, not all of the callers are futurized. Fix it up."
2015-11-02 10:22:01 +02:00
Asias He
f9ff98f4c4 tests: Fix ignored future for add_local_application_state 2015-11-02 09:34:17 +08:00
Asias He
6748bc686f load_broadcaster: Fix ignored future in start_broadcasting 2015-11-02 09:22:05 +08:00
Asias He
a8724a046e migration_manager: Fix ignored future in passive_announce
add_local_application_state returns future. Do not ignore it.
2015-11-02 09:21:55 +08:00
Asias He
e3c5a31e85 gossip: Futurize gossiper_starting
gossiper_starting calls gossiper::add_local_application_state which
returns a future, so futurize gossiper_starting as well.
2015-11-02 09:10:48 +08:00
Avi Kivity
6470cfc63e Merge "Updates for EC2Snitchs"
"- Fix snitch names from EC2XXX to Ec2XXX to align with configuration.
- Copy cassandra-rackdc.properties file to /var/lib/scylla/conf
- Set SCYLLA_HOME before booting process"
2015-11-01 18:52:23 +02:00
Shlomi Livne
67899b7b51 dist: set SCYLLA_HOME used to find the configuration and property files
Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 17:31:47 +02:00
Shlomi Livne
f0cd241478 dist: fedora/rhel/centos copy cassandra-rackdc.properties
Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 15:21:54 +02:00
Shlomi Livne
b35f56cf8d dist: ubuntu copy cassandra-rackdc.properties
Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 15:21:34 +02:00
Shlomi Livne
50cdcbd255 Update snitch registration EC2MultiRegionSnitch --> Ec2MultiRegionSnitch
Update snitch EC2MultiRegionSnitch to Ec2MultiRegionSnitch,
org.apache.cassandra.locator.EC2MultiRegionSnitch to
org.apache.cassandra.locator.Ec2MultiRegionSnitch

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 15:21:26 +02:00
Shlomi Livne
72b27a91fe Update snitch registration EC2Snitch --> Ec2Snitch
Update EC2Snitch to Ec2Snitch, org.apache.cassandra.locator.EC2Snitch to
org.apache.cassandra.locator.Ec2Snitch

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 15:20:28 +02:00
Lucas Meneghel Rodrigues
6a2f5409c9 test.py: Use shlex to split the patch
Let's use shlex to do the path splitting instead of the
simple .split() call.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-11-01 14:33:32 +02:00
Avi Kivity
212fcdc159 Remove kvm/ subdirectory
It's a seastar thing, not a Scylla thing.
2015-11-01 12:48:32 +02:00