Commit Graph

33 Commits

Author SHA1 Message Date
Vlad Zolotarov
a9f6e5f8da db::batchlog_manager: move collectd registration to the metrics registration layer
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
2017-01-10 16:24:54 -05:00
Avi Kivity
c94fb1bf12 build: reduce inclusions of messaging_service.hh
Remove inclusions from header files (primary offender is fb_utilities.hh)
and introduce new messaging_service_fwd.hh to reduce rebuilds when the
messaging service changes.

Message-Id: <1475584615-22836-1-git-send-email-avi@scylladb.com>
2016-10-05 11:46:49 +03:00
Vlad Zolotarov
b36b69c1d6 service::storage_proxy: remove a default value for a tracing::trace_state_ptr parameter
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:59 +03:00
Calle Wilund
88ffe60138 batchlog_manager: Change replay mutation CL to ALL
Try to emulate the origin behaviour for batch reply. They use an
explicit write handler, combinging

1.) Hinting to all known dead endpoints
2.) Sending to all persumed live, requiring ack from all
3.) Hinting to endpoint to which send failed.

We don't have hints, so try to work around by doing send with
cl=ALL, and if send fails (wholly or partially), retain the
batch in the log.

This is still slight behavioural difference, and we also risk
filling up the batch log in extreme cases. (Though probably not
in any real environment).

Refs #1222

Message-Id: <1466444170-23797-1-git-send-email-calle@scylladb.com>
2016-06-21 09:41:09 +03:00
Vlad Zolotarov
4ef5b11e9b batchlog_manager: add a counter for a total number of write attempts
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-04-21 11:29:21 +03:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Asias He
5550aeba1d batchlog_manager: Avoid stopping batchlog_manager more than once
We can stop batchlog_manager in decommission and drain.
Avoid stopping it more than once.

Fix the following error:

$ nodetool decommission
$ nodetool drain

storage_service - DECOMMISSIONING: stop_gossiping done
storage_service - messaging_service stopped
storage_service - DECOMMISSIONING: stop messaging_service done
storage_service - DECOMMISSIONING: set_bootstrap_state done
storage_service - DECOMMISSIONED:
storage_service - DECOMMISSIONING: done
storage_service - DRAINING: starting drain process
gossip - gossip is already stopped
scylla: ./seastar/core/gate.hh:93: future<> seastar::gate::close():
Assertion `!_stopped && "seastar::gate::close() cannot be called
more than once"' failed.
2016-03-30 20:54:30 +08:00
Asias He
cdb43c5586 batchlog_manager: Allow user initiated bachlog replay operation
During decommission, the storage_service::unbootstrap() needs to
initiate a batchlog replay operation. To sync the replay operation
initiated by the timer in batchlog_manager and storage_service, a
semaphore is introduced.  To simplify the semaphore locking, the
management code now always runs on shard zero, but the real work is
distruted to all shards.
2016-03-30 20:54:30 +08:00
Tomasz Grabiec
697d9bfa56 serializer: Introduce as_input_stream(bytes_view) 2016-02-26 12:26:13 +01:00
Paweł Dziepak
1b52264dfd batchlog_manager: use new canonical_mutation serializers
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 23:12:00 +00:00
Gleb Natapov
c509e48674 Parallelize batchlog replay
Current code is serialized by get_truncated_at(). Use map_reduce to make
it run in parallel.
Message-Id: <1454421603-13080-4-git-send-email-gleb@scylladb.com>
2016-02-02 17:08:54 +01:00
Gleb Natapov
42e3999a00 Check batchlog version before replaying
In case batchlog serialization format changes check it before trying
to interpret raw data.
Message-Id: <1454421603-13080-3-git-send-email-gleb@scylladb.com>
2016-02-02 17:08:54 +01:00
Tomasz Grabiec
ec12b75426 batchlog_manager: Store canonical_mutations
We need to be able to replay mutations created using older versions of
the table's schema. frozen_mutation can be only read using the version
it was serialized with, and there is no guarantee that the node will
know this version at the time of replay. Currently versions kept
in-memory so a node forgets all past versions when it restarts.

To solve this, let's store canonical_mutations which, like data in
sstables, can be read using any later schema version of given table.
2016-01-19 13:46:28 +01:00
Tomasz Grabiec
e21049328f batchlog_manager: Add more debug logging 2016-01-19 13:46:28 +01:00
Tomasz Grabiec
a9c00cbc11 batchlog_manager: Use requested schema version 2016-01-11 10:34:52 +01:00
Asias He
f57ba6902b storage_service: Introduce ring_delay_ms option
It is hard-coded as 30 seconds at the moment.

Usage:
$ scylla --ring-delay-ms 5000

Time a node waits to hear from other nodes before joining the ring in
milliseconds.

Same as -Dcassandra.ring_delay_ms in cassandra.
2015-12-25 15:08:22 +08:00
Tomasz Grabiec
179b587d62 Abstract timestamp creation behind new_timestamp()
Replace db_clock::now_in_usec() and db_clock::now() * 1000 accesses
where the intent is to create a new auto-generate cell timestamp with
a call to new_timestamp(). Now the knowledge of how to create timestamps
is in a single place.
2015-12-15 15:16:04 +02:00
Avi Kivity
47499dcf18 data_value: make conversion from bytes explicit
Since bytes is a very generic value that is returned from many calls,
it is easy to pass it by mistake to a function expecting a data_value,
and to get a wrong result.  It is impossible for the data_value constructor
to know if the argument is a genuine bytes variable, a data_value of another
type, but serialized, or some other serialized data type.

To prevent misuse, make the data_value(bytes) constructor
(and complementary data_value(optional<bytes>) explicit.
2015-11-13 17:12:29 +02:00
Calle Wilund
42c086a5cd batchlog_manager: Fixup includes + exception handling
* Fix exception handling in batch loop (report + still re-arm)
* Cleanup seastar include reference style
2015-10-07 17:06:34 +03:00
Calle Wilund
a4c14d3d1d batchlog_manager: Add hint of which cpu timer callback is running on 2015-10-07 14:57:55 +02:00
Calle Wilund
b46496da34 batchlog_manager: Rename logger
* More useful/referrable on command line (--log*)
* Matches class name (though not origin)
2015-10-07 14:30:09 +02:00
Calle Wilund
6f94a3bdad batchlog_manager: Use gate instead of semaphore
Since that exists now.
2015-10-07 14:30:09 +02:00
Calle Wilund
874da0eb67 batchlog_manager: Run timer loop on only one shard
Since replay is a "node global" operation, we should not attempt to
do it in parallel on each shard. It will just overlap/interfere.
Could just run this on cpu 0 or but since this _could_ be a
lengty operation, each timer callback is round-robined shards just in case...
2015-10-07 14:30:09 +02:00
Calle Wilund
b3c95ce42d system_keyspace: Change truncation record method to use context qp
Align with rest of file (for better or worse). This allows calls from
entity without query_processor handy (i.e. storage_proxy).

Added "minimal" setup method for the "global" state, to facilitate
tests. Doing a full setup either in cql_test_env or after it is created
breaks badly. (Not sure why). So quick workaround.

Updated the current two users (batchlog_manager and commitlog_replayer)
callsites to conform.
2015-09-30 09:09:41 +02:00
Pekka Enberg
f43f0d6f04 keys: Add compound_wrapper::from_singular()
Clean up code by adding a from_singular() helper function to compound
wrapper and use it in.
2015-09-28 16:29:44 +02:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Paweł Dziepak
830a86258b db: avoid copying mutations
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:32 +02:00
Paweł Dziepak
ddec2b4d09 batchlog_manager: pass mutations by const ref
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:29 +02:00
Calle Wilund
9a52ad84b1 BatchlogManager: make blm globally reachable distributed like other objects 2015-08-11 17:10:17 +02:00
Calle Wilund
0ded44eeee BatchlogManager: make endpoint_filter method + implement 2015-08-11 17:10:16 +02:00
Avi Kivity
1d4805236b messaging_service: don't include config.hh in .hh
config.hh changes rapidly, so don't force lots of recompiles by including it.

Need to place seed_provider_type in namespace scope, so we can forward
declare it for that.
2015-07-16 12:26:02 +03:00
Avi Kivity
99a15de9e5 logger: de-thread_local-ize logger
The logger class constructor registers itself with the logger registry,
in order to enable dynamically setting log levels.  However, since
thread_local variables may be (and are) initialized at the time of first
use, when the program starts up no loggers are registered.

Fix by making loggers global, not thread_local.  This requires that the
registry use locking to prevent registration happening on different threads
from corrupting the registry.

Note that technically global variables can also be initialized at the
point of first use, and there is no portable way for classes to self-register.
However this is the best we can do.
2015-07-14 17:18:11 +03:00
Calle Wilund
ef2cc9b05d BatchLogManager.java -> C++
Somewhat simplifies version of the Origin code, since from what I 
can see, there is less need for us to do explicit query sends in 
the BLM itself, instead we can just go through storage_proxy. 
I could be wrong though.
2015-07-08 10:59:57 +02:00