Commit Graph

7157 Commits

Author SHA1 Message Date
Vlad Zolotarov
b3504f9b1f locator::token_metadata: added topology::update_endpoint(ep) and update_topology(ep)
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
a3d55ba882 locator::reconnectable_snitch_helper: remove unused constructor parameter
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
33b195760b gms::gossiper: allow the modification of _subscribers while it's being iterated
Introduce a subscribers_list class that exposes 3 methods:
  - push_back(s) - adds a new element s to the back of the list
  - remove(s) - removes an element s from the list
  - for_each(f) - invoke f on each element of the list
  - make a subscriber_list store shared_ptr to a subscriber
    to allow removing (currently it stores a naked pointer to the object).

subscribers_list allows push_back() and remove() to be called while
another thread (e.g. seastar::async()) is in the middle of for_each().

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Simplify subscribers_list::remove() method.
   - load_broadcaster: inherit from enable_shared_from_this instead
     of async_sharded_service.
2015-10-30 00:16:16 +02:00
Avi Kivity
f7f996d473 Merge seastar upstream
* seastar 9d8913a...9ae6407 (2):
  > core/memory.cc: Declare member min_free_pages from cpu_pages struct
  > http: All http replies should have version set
2015-10-29 16:49:10 +02:00
Asias He
6259dd73b6 token: Print token using the partitioner defined method
Make nodetool ring output align with c* output.
2015-10-29 15:53:47 +02:00
Pekka Enberg
f4849000c8 cql3: Fix exception message in lists::setter_by_index::execute()
Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 15:36:56 +02:00
Pekka Enberg
6634f37464 cql3: Fix exception message in lists::discarder_by_index::execute()
Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 15:36:51 +02:00
Pekka Enberg
2344bd806d cql3: Fix grammar 'WITH WITH' bug that causes a SIGSEGV
Fix an issue where 'WITH WITH' in CREATE and ALTER KEYSPACE would bring
down the whole server.

  https://issues.apache.org/jira/browse/CASSANDRA-9565

  c939422637

Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 10:59:57 +01:00
Raphael S. Carvalho
6bea503f9a db: fallback to sizetiered if compaction strategy isn't supported
It may happen that the user will migrate a table to Scylla which
compaction strategy isn't supported yet, such as Data tiered.
Let's handle that by falling back to size-tiered compaction
strategy and printing a warning message.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-10-29 09:33:28 +02:00
Raphael S. Carvalho
c2a98807c7 compaction_manager: fix remove
remove() is the function used to remove every reference to a cf from
the compaction manager. This function works by removing cf from the
queue, and waiting for possible ongoing compaction on cf.
However, a cf may be re-queued by compaction manager task if there
is pending compaction by the end of compaction.

If cf is still referenced by the time remove() returns, we could end
up with an use-after-free. To fix that, a task shouldn't re-queue a
cf if it was asked to stop. The stat pending_tasks was also not
being updated when a cf was removed from the task queue.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-10-28 17:35:26 +02:00
Gleb Natapov
ac5f92db70 storage_proxy: clean up local_dc checking
The only place local_dc is checked during mutation sending is in
send_to_live_endpoints(), but current code pass it there throw several
function call layers. Simplify the code by getting local_dc when it is
used directly.
2015-10-28 16:10:18 +02:00
Paweł Dziepak
6e5916161c test/cql_test_env: merge client state after executing query
Since 4641dfff24 "service: Copy client
state to query state" after executing a query client state needs to be
merged back. If that's not done client_state::_last_timestamp_micros
won't be advanced properly and mutations originating from the same
source may have exactly the same timestamp.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-10-28 14:31:11 +02:00
Avi Kivity
4d23712c9c Merge "Fix gossip test" from Asias 2015-10-28 12:59:19 +02:00
Asias He
25c898fe9c gossip: Enable too more log prints for debug 2015-10-28 16:13:57 +08:00
Asias He
e0e8e9a1ed tests: Remove redundant debug info for gossip
The debug info is printed in logger already. Avoid to print it twice.
2015-10-28 16:13:57 +08:00
Asias He
01ee5d002a failure_detector: Remove debug print in operator<< 2015-10-28 16:13:57 +08:00
Asias He
f205ae30c3 tests: Fix gossip
- scylla/seastar/core/sharded.hh:439: Service& seastar::sharded<Service>::local()
  [with Service = locator::snitch_ptr]: Assertion `local_is_initialized()' failed.
- ./utils/fb_utilities.hh:74: static const inet_address utils::fb_utilities::get_broadcast_address():
  Assertion `broadcast_address()' failed.
2015-10-28 16:13:57 +08:00
Takuya ASADA
5aab69e72f dist: do not remove build/ dir when scylla-server ubuntu package building
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-10-27 18:23:09 +02:00
Avi Kivity
7f88db8625 Merge "storage_service and gossip update" from Asias 2015-10-27 18:14:43 +02:00
Lucas Meneghel Rodrigues
a9a33d5a99 test.py: PEP8 Fixes
Fix some PEP8 problems found in the tester code:

 * Wrong spacing around operators
 * Lines between class and function definitions
 * Fixed some of the larger than 80 column statements
 * Removed an unused import

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-10-27 17:54:54 +02:00
Tomasz Grabiec
491eae58a4 Merge branch 'gleb/read-repair' from seastar-dev.git
From Gleb:

Now that we can calculate mutation diffs do so on digest mismatch and send
them out.
2015-10-27 16:36:05 +01:00
Asias He
7e6e90dc52 storage_service: Serialize decommission
Prevent two operations happen simultaneously.
2015-10-27 21:48:38 +08:00
Asias He
4b75815306 storage_service: Introduce run_with_write_api_lock helper
It is useful to run code on cpu zero with the api lock.
2015-10-27 21:48:38 +08:00
Asias He
62228726a3 storage_service: Introduce a rwlock to serialize management operations 2015-10-27 21:48:38 +08:00
Asias He
1bbc1920d2 range_streamer: Start to use get_preferred_ip
It is available now.
2015-10-27 21:48:37 +08:00
Asias He
306bab9ead storage_service: Use get_preferred_ip
Now that it is available, use it.
2015-10-27 21:48:37 +08:00
Asias He
55b76a8963 api: No cpu zero trick for remove_node
storage_service::remove_node is guaranteed to run on cpu zero only.
2015-10-27 21:48:37 +08:00
Asias He
b3c7305d25 storage_service: Make remove_node runs on cpu 0 only
We need to serialize nodetool operations to avoid two operations
happening simultaneously. Running on cpu 0 is one step toward this
goal.
2015-10-27 21:48:37 +08:00
Asias He
6c6b1c4ba7 storage_service: Make decommission runs on cpu 0 only
We need to serialize nodetool operations to avoid two operations
happening simultaneously. Running on cpu 0 is one step toward this goal.
2015-10-27 21:48:37 +08:00
Asias He
49daba2599 storage_service: Do not ignore future in decommission
gossiper::stop returns a future which we can not ignore.
2015-10-27 21:48:37 +08:00
Asias He
00311817bd storage_service: Implement shutdown_client_servers 2015-10-27 21:48:37 +08:00
Asias He
83eb36796f storage_service: Kill FIXME for LoadBroadcaster.BROADCAST_INTERVAL
It is available now.
2015-10-27 21:48:37 +08:00
Asias He
1469cec5bf gossiper: Kill free function helper to get heart version and generation number
They can only be executed on cpu 0. Make the gossiper member
functions for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
f573059698 gossiper: Kill free function helper for {unsafe_,}assassinate_endpoint
They can only be executed on cpu 0. Make the gossiper member functions
for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
c5f377eb8b gossip: Simplify get_endpoint_downtime
_unreachable_endpoints is replicated to call cores. No need to query
on core 0.
2015-10-27 21:48:37 +08:00
Asias He
6f1db4fb72 gossip: Simplify get_unreachable_members
_unreachable_endpoints is replicated to call cores. No need to query on
core 0.

This also fixes a bug in storage_proxy::truncate_blocking
which might access _unreachable_endpoints on non-zero cores.
2015-10-27 21:48:37 +08:00
Asias He
a9f96d1f5a gossip: Replicate _unreachable_endpoints to all cores 2015-10-27 21:48:37 +08:00
Asias He
2439a2a982 gossip: Simplify get_live_members
_live_endpoints is replicated to call cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
a28ba9cde8 api: Simplify get_tokens get_node_tokens and get_token_endpoint
token_metadata is replicated to all cores. No need to query on core 0.
2015-10-27 21:48:37 +08:00
Asias He
4e886f0399 storage_service: Implement is_rpc_server_running 2015-10-27 21:48:37 +08:00
Asias He
7f0634b429 storage_service: Implement stop_rpc_server 2015-10-27 21:48:37 +08:00
Asias He
ca66bea619 storage_service: Implement is_native_transport_running 2015-10-27 21:48:37 +08:00
Asias He
7bc49a1efe storage_service: Implement stop_native_transport 2015-10-27 21:48:37 +08:00
Asias He
8218ab7922 storage_service: Implement start_native_transport and start_rpc_server
They are used for APIs. Share the code in main.cc as well.
2015-10-27 21:48:37 +08:00
Lucas Meneghel Rodrigues
42c0acfc44 test.py: Return test output only if subprocess succeeded
The current code will try to print the output of a
subprocess.Popen().communicate() call even if that
call raised an exception and that output is None.

Let's fix this problem by only printing the output
if it's not None.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-10-27 15:17:05 +02:00
Asias He
3c47844e8c storage_service: Complete check_for_endpoint_collision
Part of it was stubbed.
2015-10-27 21:17:02 +08:00
Asias He
77a87cb2b6 storage_service: Implement prepare_replacement_info
Needed by replace node operation.
2015-10-27 21:17:02 +08:00
Vlad Zolotarov
5cdbc3701a tests: set broadcast address
Since commit 5613979a85
broadcast address has to be set before it's used for the first
time.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-27 15:16:13 +02:00
Avi Kivity
345f739f28 Merge seastar upstream
* seastar 501e4cb...9d8913a (3):
  > Add mutable to with_lock and do_with
  > app-template: disable collectd by default
  > reactor: use fdatasync() instead of fsync()
2015-10-27 15:13:36 +02:00
Avi Kivity
42a5c8b92e Merge "CQL request load balancing" from Pekka
"Currently, CQL requests are processed on the same CPU core where the
connection lives in. This series adds infrastructure for migrating CQL
processing to other cores and implements a round-robin load balancing
algorithm that can be enabled with the "--load-balance=round-robin"
command line option. Load balancing is not enabled by default because we
need to first run performance tests to determine if the simple
round-robin algorithm is sufficient, or wheter we need to implement more
sophisticated dynamic load balancing."
2015-10-27 15:04:44 +02:00