Commit Graph

7170 Commits

Author SHA1 Message Date
Avi Kivity
2c3591cbd9 data_value de-any-fication
We use boost::any to convert to and from database values (stored in
serlialized form) and native C++ values.  boost::any captures information
about the data type (how to copy/move/delete etc.) and stores it inside
the boost::any instance.  We later retrieve the real value using
boost::any_cast.

However, data_value (which has a boost::any member) already has type
information as a data_type instance.  By teaching data_type intances about
the corresponding native type, we can elimiante the use of boost::any.

While boost::any is evil and eliminating it improves efficiency somewhat,
the real goal is growing native type support in data_type.  We will use that
later to store native types in the cache, enabling O(log n) access to
collections, O(1) access to tuples, and more efficient large blob support.
2015-10-30 17:38:51 +01:00
Pekka Enberg
b478d9d8a6 Merge "Wire reconnectable_snitch_helper to gossiping_property_file_snitch" from Vlad
"gossiping_property_file_snitch checks its property
file (cassandra-rackdc.properties) for changes every minute and
if there were changes it re-registers the helper and initiates
re-read of the new DC and Rack values in the corresponding places.

Therefore we need the ability to unregister/register the corresponding subscriber
at the same time when a subscriber list is possibly iterated by
some other asynchronous context on the current CPU.

The current gossiper implementation assumes that subscribers list may not be
changed from the context different from the one that iterates on their list.

So, this had to be fixed.

There was also missing an update_endpoint(ep) interface in the locator::topology
class and the corresponding token_metadata::update_topology(ep) wrapper.

Also there were some bugs in the gossiping_property_file::reload_configuration()
method."
2015-10-30 15:48:42 +02:00
Pekka Enberg
a9a8d1e8a2 Merge "Fixes for dependency packages, fix upstart script" from Takuya
"Fixes #510, (part of) #493."
2015-10-30 15:41:40 +02:00
Lucas Meneghel Rodrigues
a3d26f699e test.py: Only print test stdout if non None, non empty
On hindsight, it doesn't make much sense to print an
empty string, so let's only print stdout if it's non
None, non empty.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-10-30 09:11:07 +02:00
Avi Kivity
91a8cfc029 cql3: don't declare counter-related functions
This works now, but fails when an in-progress patch I'm working on calls
one of the type's methods.
2015-10-30 09:06:48 +02:00
Vlad Zolotarov
6b4b983f9d locator::gossiping_property_file_snitch: implement gossiper_starting() and reload_gossiper_state()
This functions were empty and now they have the intended code:
   - Register the reconnectable_snitch_helper if "prefer_local"
     parameter was given the TRUE value.
   - Set the application INTERNAL_IP state to listen_address().

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
17294a3bc7 locator::gossiping_property_file_snitch: fix some issues in reload_configuration()
- Invoke reload_gossiper_state() and gossip_snitch_info() on CPU0 since
     gossiper is effectively running on CPU0 therefore all methods
     modifying its state should be invoked on CPU0 as well.
   - Don't invoke any method on external "distributed" objects unless their
     corresponding per-shard service object have already been initialized.
   - Update a local Node info in a storage_service::token_metadata::topology
     when reloading snitch configuration when DC and/or Rack info has changed.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
b3504f9b1f locator::token_metadata: added topology::update_endpoint(ep) and update_topology(ep)
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
a3d55ba882 locator::reconnectable_snitch_helper: remove unused constructor parameter
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-30 00:16:53 +02:00
Vlad Zolotarov
33b195760b gms::gossiper: allow the modification of _subscribers while it's being iterated
Introduce a subscribers_list class that exposes 3 methods:
  - push_back(s) - adds a new element s to the back of the list
  - remove(s) - removes an element s from the list
  - for_each(f) - invoke f on each element of the list
  - make a subscriber_list store shared_ptr to a subscriber
    to allow removing (currently it stores a naked pointer to the object).

subscribers_list allows push_back() and remove() to be called while
another thread (e.g. seastar::async()) is in the middle of for_each().

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Simplify subscribers_list::remove() method.
   - load_broadcaster: inherit from enable_shared_from_this instead
     of async_sharded_service.
2015-10-30 00:16:16 +02:00
Pekka Enberg
763b718418 Update 'docker run' instructions in README
Explicitly mention how to enable port forwarding so that external
applications are able to connect.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 20:07:04 +02:00
Takuya ASADA
3bf69db90c dist: handle SIGKILL correctly on upstart, also do not respawn process
Fixes #510
2015-10-30 00:55:47 +09:00
Takuya ASADA
6aac944588 dist: fix warning when building scylla-server ubuntu package 2015-10-30 00:53:20 +09:00
Takuya ASADA
37e00d3527 dist: fix warning when building thrift ubuntu package 2015-10-30 00:52:11 +09:00
Takuya ASADA
a165d6aa01 dist: fix warning when building antlr3-c++-dev ubuntu package 2015-10-30 00:52:00 +09:00
Takuya ASADA
9eb8aa7ea3 dist: fix warning when building antlr3-tool ubuntu package 2015-10-30 00:51:42 +09:00
Avi Kivity
f7f996d473 Merge seastar upstream
* seastar 9d8913a...9ae6407 (2):
  > core/memory.cc: Declare member min_free_pages from cpu_pages struct
  > http: All http replies should have version set
2015-10-29 16:49:10 +02:00
Asias He
6259dd73b6 token: Print token using the partitioner defined method
Make nodetool ring output align with c* output.
2015-10-29 15:53:47 +02:00
Pekka Enberg
f4849000c8 cql3: Fix exception message in lists::setter_by_index::execute()
Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 15:36:56 +02:00
Pekka Enberg
6634f37464 cql3: Fix exception message in lists::discarder_by_index::execute()
Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 15:36:51 +02:00
Pekka Enberg
2344bd806d cql3: Fix grammar 'WITH WITH' bug that causes a SIGSEGV
Fix an issue where 'WITH WITH' in CREATE and ALTER KEYSPACE would bring
down the whole server.

  https://issues.apache.org/jira/browse/CASSANDRA-9565

  c939422637

Spotted by C* unit tests.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-29 10:59:57 +01:00
Raphael S. Carvalho
6bea503f9a db: fallback to sizetiered if compaction strategy isn't supported
It may happen that the user will migrate a table to Scylla which
compaction strategy isn't supported yet, such as Data tiered.
Let's handle that by falling back to size-tiered compaction
strategy and printing a warning message.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-10-29 09:33:28 +02:00
Raphael S. Carvalho
c2a98807c7 compaction_manager: fix remove
remove() is the function used to remove every reference to a cf from
the compaction manager. This function works by removing cf from the
queue, and waiting for possible ongoing compaction on cf.
However, a cf may be re-queued by compaction manager task if there
is pending compaction by the end of compaction.

If cf is still referenced by the time remove() returns, we could end
up with an use-after-free. To fix that, a task shouldn't re-queue a
cf if it was asked to stop. The stat pending_tasks was also not
being updated when a cf was removed from the task queue.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-10-28 17:35:26 +02:00
Gleb Natapov
ac5f92db70 storage_proxy: clean up local_dc checking
The only place local_dc is checked during mutation sending is in
send_to_live_endpoints(), but current code pass it there throw several
function call layers. Simplify the code by getting local_dc when it is
used directly.
2015-10-28 16:10:18 +02:00
Paweł Dziepak
6e5916161c test/cql_test_env: merge client state after executing query
Since 4641dfff24 "service: Copy client
state to query state" after executing a query client state needs to be
merged back. If that's not done client_state::_last_timestamp_micros
won't be advanced properly and mutations originating from the same
source may have exactly the same timestamp.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-10-28 14:31:11 +02:00
Avi Kivity
4d23712c9c Merge "Fix gossip test" from Asias 2015-10-28 12:59:19 +02:00
Asias He
25c898fe9c gossip: Enable too more log prints for debug 2015-10-28 16:13:57 +08:00
Asias He
e0e8e9a1ed tests: Remove redundant debug info for gossip
The debug info is printed in logger already. Avoid to print it twice.
2015-10-28 16:13:57 +08:00
Asias He
01ee5d002a failure_detector: Remove debug print in operator<< 2015-10-28 16:13:57 +08:00
Asias He
f205ae30c3 tests: Fix gossip
- scylla/seastar/core/sharded.hh:439: Service& seastar::sharded<Service>::local()
  [with Service = locator::snitch_ptr]: Assertion `local_is_initialized()' failed.
- ./utils/fb_utilities.hh:74: static const inet_address utils::fb_utilities::get_broadcast_address():
  Assertion `broadcast_address()' failed.
2015-10-28 16:13:57 +08:00
Takuya ASADA
5aab69e72f dist: do not remove build/ dir when scylla-server ubuntu package building
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-10-27 18:23:09 +02:00
Avi Kivity
7f88db8625 Merge "storage_service and gossip update" from Asias 2015-10-27 18:14:43 +02:00
Lucas Meneghel Rodrigues
a9a33d5a99 test.py: PEP8 Fixes
Fix some PEP8 problems found in the tester code:

 * Wrong spacing around operators
 * Lines between class and function definitions
 * Fixed some of the larger than 80 column statements
 * Removed an unused import

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-10-27 17:54:54 +02:00
Tomasz Grabiec
491eae58a4 Merge branch 'gleb/read-repair' from seastar-dev.git
From Gleb:

Now that we can calculate mutation diffs do so on digest mismatch and send
them out.
2015-10-27 16:36:05 +01:00
Asias He
7e6e90dc52 storage_service: Serialize decommission
Prevent two operations happen simultaneously.
2015-10-27 21:48:38 +08:00
Asias He
4b75815306 storage_service: Introduce run_with_write_api_lock helper
It is useful to run code on cpu zero with the api lock.
2015-10-27 21:48:38 +08:00
Asias He
62228726a3 storage_service: Introduce a rwlock to serialize management operations 2015-10-27 21:48:38 +08:00
Asias He
1bbc1920d2 range_streamer: Start to use get_preferred_ip
It is available now.
2015-10-27 21:48:37 +08:00
Asias He
306bab9ead storage_service: Use get_preferred_ip
Now that it is available, use it.
2015-10-27 21:48:37 +08:00
Asias He
55b76a8963 api: No cpu zero trick for remove_node
storage_service::remove_node is guaranteed to run on cpu zero only.
2015-10-27 21:48:37 +08:00
Asias He
b3c7305d25 storage_service: Make remove_node runs on cpu 0 only
We need to serialize nodetool operations to avoid two operations
happening simultaneously. Running on cpu 0 is one step toward this
goal.
2015-10-27 21:48:37 +08:00
Asias He
6c6b1c4ba7 storage_service: Make decommission runs on cpu 0 only
We need to serialize nodetool operations to avoid two operations
happening simultaneously. Running on cpu 0 is one step toward this goal.
2015-10-27 21:48:37 +08:00
Asias He
49daba2599 storage_service: Do not ignore future in decommission
gossiper::stop returns a future which we can not ignore.
2015-10-27 21:48:37 +08:00
Asias He
00311817bd storage_service: Implement shutdown_client_servers 2015-10-27 21:48:37 +08:00
Asias He
83eb36796f storage_service: Kill FIXME for LoadBroadcaster.BROADCAST_INTERVAL
It is available now.
2015-10-27 21:48:37 +08:00
Asias He
1469cec5bf gossiper: Kill free function helper to get heart version and generation number
They can only be executed on cpu 0. Make the gossiper member
functions for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
f573059698 gossiper: Kill free function helper for {unsafe_,}assassinate_endpoint
They can only be executed on cpu 0. Make the gossiper member functions
for them to do so.
2015-10-27 21:48:37 +08:00
Asias He
c5f377eb8b gossip: Simplify get_endpoint_downtime
_unreachable_endpoints is replicated to call cores. No need to query
on core 0.
2015-10-27 21:48:37 +08:00
Asias He
6f1db4fb72 gossip: Simplify get_unreachable_members
_unreachable_endpoints is replicated to call cores. No need to query on
core 0.

This also fixes a bug in storage_proxy::truncate_blocking
which might access _unreachable_endpoints on non-zero cores.
2015-10-27 21:48:37 +08:00
Asias He
a9f96d1f5a gossip: Replicate _unreachable_endpoints to all cores 2015-10-27 21:48:37 +08:00