Commit Graph

53948 Commits

Author SHA1 Message Date
Asias He
66a576c0c0 tests: Add documents to tests/gossip.cc
- How to run the test
- What to expect
2015-11-02 11:29:43 +02:00
Avi Kivity
4b1fffe54c Merge "storage_service: serialize more management operations" from Asias
- Add run_with_no_api_lock
- Do not hold the api lock while do the streaming for rebuild
- Include "config: Enable storage_port option" in this series
2015-11-02 10:55:57 +02:00
Avi Kivity
107877f33a Merge "Futurize all callers of add_local_application_state" from Asias
"In 5e8037b50a (gossip: Futurize
add_local_application_state()) , we futurized add_local_application_state.
However, not all of the callers are futurized. Fix it up."
2015-11-02 10:22:01 +02:00
Asias He
f9ff98f4c4 tests: Fix ignored future for add_local_application_state 2015-11-02 09:34:17 +08:00
Asias He
6748bc686f load_broadcaster: Fix ignored future in start_broadcasting 2015-11-02 09:22:05 +08:00
Asias He
a8724a046e migration_manager: Fix ignored future in passive_announce
add_local_application_state returns future. Do not ignore it.
2015-11-02 09:21:55 +08:00
Asias He
e3c5a31e85 gossip: Futurize gossiper_starting
gossiper_starting calls gossiper::add_local_application_state which
returns a future, so futurize gossiper_starting as well.
2015-11-02 09:10:48 +08:00
Avi Kivity
6470cfc63e Merge "Updates for EC2Snitchs"
"- Fix snitch names from EC2XXX to Ec2XXX to align with configuration.
- Copy cassandra-rackdc.properties file to /var/lib/scylla/conf
- Set SCYLLA_HOME before booting process"
2015-11-01 18:52:23 +02:00
Shlomi Livne
67899b7b51 dist: set SCYLLA_HOME used to find the configuration and property files
Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 17:31:47 +02:00
Shlomi Livne
f0cd241478 dist: fedora/rhel/centos copy cassandra-rackdc.properties
Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 15:21:54 +02:00
Shlomi Livne
b35f56cf8d dist: ubuntu copy cassandra-rackdc.properties
Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 15:21:34 +02:00
Shlomi Livne
50cdcbd255 Update snitch registration EC2MultiRegionSnitch --> Ec2MultiRegionSnitch
Update snitch EC2MultiRegionSnitch to Ec2MultiRegionSnitch,
org.apache.cassandra.locator.EC2MultiRegionSnitch to
org.apache.cassandra.locator.Ec2MultiRegionSnitch

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 15:21:26 +02:00
Shlomi Livne
72b27a91fe Update snitch registration EC2Snitch --> Ec2Snitch
Update EC2Snitch to Ec2Snitch, org.apache.cassandra.locator.EC2Snitch to
org.apache.cassandra.locator.Ec2Snitch

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2015-11-01 15:20:28 +02:00
Lucas Meneghel Rodrigues
6a2f5409c9 test.py: Use shlex to split the patch
Let's use shlex to do the path splitting instead of the
simple .split() call.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-11-01 14:33:32 +02:00
Avi Kivity
212fcdc159 Remove kvm/ subdirectory
It's a seastar thing, not a Scylla thing.
2015-11-01 12:48:32 +02:00
Lucas Meneghel Rodrigues
c1dde5dc4d service/storage_service.cc: Fix control reaches end of non-void function error
During testing build, the debugging statement at the end
of the function body (after return statements) causes compilation to
fail due to the flag -Werror=return-type:

service/storage_service.cc: In member function ‘future<> service::storage_service::clear_snapshot(sstring, std::vector<basic_sstring<char, unsigned int, 15u> >)’:
service/storage_service.cc:1358:1: error: control reaches end of non-void function [-Werror=return-type]

Which traces back to 21f84d77. Let's attach a then_wrapped()
clause to parallel_for_each() adding the debug message as
suggested by Avi.

CC: Glauber Costa <glommer@scylladb.com>
Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-11-01 11:59:44 +02:00
Asias He
5e8037b50a gossip: Futurize add_local_application_state()
We are ignoring the future returned by seastar::async. Futurize it so
caller can wait for the application state to be actually applied.

In addition, dropping the unused add_local_application_states function.
2015-11-01 11:20:52 +02:00
Avi Kivity
2c3591cbd9 data_value de-any-fication
We use boost::any to convert to and from database values (stored in
serlialized form) and native C++ values.  boost::any captures information
about the data type (how to copy/move/delete etc.) and stores it inside
the boost::any instance.  We later retrieve the real value using
boost::any_cast.

However, data_value (which has a boost::any member) already has type
information as a data_type instance.  By teaching data_type intances about
the corresponding native type, we can elimiante the use of boost::any.

While boost::any is evil and eliminating it improves efficiency somewhat,
the real goal is growing native type support in data_type.  We will use that
later to store native types in the cache, enabling O(log n) access to
collections, O(1) access to tuples, and more efficient large blob support.
2015-10-30 17:38:51 +01:00
Pekka Enberg
b478d9d8a6 Merge "Wire reconnectable_snitch_helper to gossiping_property_file_snitch" from Vlad
"gossiping_property_file_snitch checks its property
file (cassandra-rackdc.properties) for changes every minute and
if there were changes it re-registers the helper and initiates
re-read of the new DC and Rack values in the corresponding places.

Therefore we need the ability to unregister/register the corresponding subscriber
at the same time when a subscriber list is possibly iterated by
some other asynchronous context on the current CPU.

The current gossiper implementation assumes that subscribers list may not be
changed from the context different from the one that iterates on their list.

So, this had to be fixed.

There was also missing an update_endpoint(ep) interface in the locator::topology
class and the corresponding token_metadata::update_topology(ep) wrapper.

Also there were some bugs in the gossiping_property_file::reload_configuration()
method."
2015-10-30 15:48:42 +02:00
Pekka Enberg
a9a8d1e8a2 Merge "Fixes for dependency packages, fix upstart script" from Takuya
"Fixes #510, (part of) #493."
2015-10-30 15:41:40 +02:00
Lucas Meneghel Rodrigues
a3d26f699e test.py: Only print test stdout if non None, non empty
On hindsight, it doesn't make much sense to print an
empty string, so let's only print stdout if it's non
None, non empty.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-10-30 09:11:07 +02:00
Avi Kivity
91a8cfc029 cql3: don't declare counter-related functions
This works now, but fails when an in-progress patch I'm working on calls
one of the type's methods.
2015-10-30 09:06:48 +02:00
Vlad Zolotarov
6b4b983f9d locator::gossiping_property_file_snitch: implement gossiper_starting() and reload_gossiper_state()
This functions were empty and now they have the intended code:
   - Register the reconnectable_snitch_helper if "prefer_local"
     parameter was given the TRUE value.
   - Set the application INTERNAL_IP state to listen_address().

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
17294a3bc7 locator::gossiping_property_file_snitch: fix some issues in reload_configuration()
- Invoke reload_gossiper_state() and gossip_snitch_info() on CPU0 since
     gossiper is effectively running on CPU0 therefore all methods
     modifying its state should be invoked on CPU0 as well.
   - Don't invoke any method on external "distributed" objects unless their
     corresponding per-shard service object have already been initialized.
   - Update a local Node info in a storage_service::token_metadata::topology
     when reloading snitch configuration when DC and/or Rack info has changed.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
b3504f9b1f locator::token_metadata: added topology::update_endpoint(ep) and update_topology(ep)
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
a3d55ba882 locator::reconnectable_snitch_helper: remove unused constructor parameter
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
33b195760b gms::gossiper: allow the modification of _subscribers while it's being iterated
Introduce a subscribers_list class that exposes 3 methods:
  - push_back(s) - adds a new element s to the back of the list
  - remove(s) - removes an element s from the list
  - for_each(f) - invoke f on each element of the list
  - make a subscriber_list store shared_ptr to a subscriber
    to allow removing (currently it stores a naked pointer to the object).

subscribers_list allows push_back() and remove() to be called while
another thread (e.g. seastar::async()) is in the middle of for_each().

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Simplify subscribers_list::remove() method.
   - load_broadcaster: inherit from enable_shared_from_this instead
     of async_sharded_service.
2015-10-30 00:16:16 +02:00
Pekka Enberg
763b718418 Update 'docker run' instructions in README
Explicitly mention how to enable port forwarding so that external
applications are able to connect.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 20:07:04 +02:00
Takuya ASADA
3bf69db90c dist: handle SIGKILL correctly on upstart, also do not respawn process
Fixes #510
2015-10-30 00:55:47 +09:00
Takuya ASADA
6aac944588 dist: fix warning when building scylla-server ubuntu package 2015-10-30 00:53:20 +09:00
Takuya ASADA
37e00d3527 dist: fix warning when building thrift ubuntu package 2015-10-30 00:52:11 +09:00
Takuya ASADA
a165d6aa01 dist: fix warning when building antlr3-c++-dev ubuntu package 2015-10-30 00:52:00 +09:00
Takuya ASADA
9eb8aa7ea3 dist: fix warning when building antlr3-tool ubuntu package 2015-10-30 00:51:42 +09:00
Avi Kivity
f7f996d473 Merge seastar upstream
* seastar 9d8913a...9ae6407 (2):
  > core/memory.cc: Declare member min_free_pages from cpu_pages struct
  > http: All http replies should have version set
2015-10-29 16:49:10 +02:00
Asias He
6259dd73b6 token: Print token using the partitioner defined method
Make nodetool ring output align with c* output.
2015-10-29 15:53:47 +02:00
Pekka Enberg
f4849000c8 cql3: Fix exception message in lists::setter_by_index::execute()
Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 15:36:56 +02:00
Pekka Enberg
6634f37464 cql3: Fix exception message in lists::discarder_by_index::execute()
Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 15:36:51 +02:00
Pekka Enberg
2344bd806d cql3: Fix grammar 'WITH WITH' bug that causes a SIGSEGV
Fix an issue where 'WITH WITH' in CREATE and ALTER KEYSPACE would bring
down the whole server.

  https://issues.apache.org/jira/browse/CASSANDRA-9565

  c939422637

Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 10:59:57 +01:00
Raphael S. Carvalho
6bea503f9a db: fallback to sizetiered if compaction strategy isn't supported
It may happen that the user will migrate a table to Scylla which
compaction strategy isn't supported yet, such as Data tiered.
Let's handle that by falling back to size-tiered compaction
strategy and printing a warning message.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-10-29 09:33:28 +02:00
Asias He
c030e6713b storage_service: Serialize rebuild
Do not hold the api lock while streaming the data since it might take a
long time, so we need to reconcile other operations while we are in the
middle of rebuild.
2015-10-29 09:51:52 +08:00
Asias He
cc6d4e0fc9 storage_service: Introduce run_with_no_api_lock
Run on cpu zero without take the api lock.
2015-10-29 09:50:03 +08:00
Asias He
2c8867c348 config: Enable storage_port option 2015-10-29 08:58:41 +08:00
Asias He
97d2bdf663 storage_service: Serialize gossiping operations 2015-10-29 08:58:41 +08:00
Asias He
f2e5e318d7 storage_service: Serialize effective_ownershipp with read api lock 2015-10-29 08:58:41 +08:00
Asias He
94a49121c2 storage_service: Serialize get_ownership with read api lock 2015-10-29 08:58:41 +08:00
Asias He
d41cd48a5f storage_service: Serialize is_joined with read api lock 2015-10-29 08:58:41 +08:00
Asias He
24a452133a storage_service: Serialize join_ring with write api lock 2015-10-29 08:58:41 +08:00
Asias He
08337b0df1 storage_service: Serialize more node management operations with read api lock
- get_operation_mode
- is_starting
- is_rpc_server_running
- is_native_transport_running
- get_load_map
- is_initialized
2015-10-29 08:58:41 +08:00
Asias He
c6bf157610 storage_service: Introduce run_with_read_api_lock
Similar to run_with_write_api_lock, but the operations only query the
cluster.
2015-10-29 08:58:41 +08:00
Asias He
93154e0184 storage_service: Serialize more node management operations with write api lock
- start_rpc_server
- stop_rpc_server
- start_native_transport
- stop_native_transport
- remove_node
2015-10-29 08:57:45 +08:00