Commit Graph

9990 Commits

Author SHA1 Message Date
Duarte Nunes
2d7c322805 thrift: Actually concatenate strings
This patch fixes concatenating a char[] with an int by using sprint
instead of just increasing the pointer.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1468971542-9600-1-git-send-email-duarte@scylladb.com>
(cherry picked from commit 64dff69077)
2016-07-20 11:09:15 +03:00
Tomasz Grabiec
13f18c6445 database: Add table name to log message about sealing
Message-Id: <1468917744-2539-1-git-send-email-tgrabiec@scylladb.com>
(cherry picked from commit 0d26294fac)
2016-07-20 10:13:32 +03:00
Tomasz Grabiec
9c430c2cff schema_tables: Add more logging
Message-Id: <1468917771-2592-1-git-send-email-tgrabiec@scylladb.com>
(cherry picked from commit a0832f08d2)
2016-07-20 10:13:28 +03:00
Pekka Enberg
c84e030fe9 release: prepare for 1.3.rc1 scylla-1.3-rc1 2016-07-19 20:15:26 +03:00
Avi Kivity
dc50b845b4 Merge seastar upstream
* seastar 823bc05...103543a (1):
  > core: add a seastar::format()
2016-07-19 19:09:42 +03:00
Avi Kivity
1a1b7fe3f2 Merge "CQL Tracing patch bomb" from Vlad
"This series includes the following:
   - Introduction of a formatted message support in trace().
   - Major rename: s/flush_/write_/, s/flush()/kick()/, s/store_/write_/.
   - Some cosmetic fixes found on the way.
   - Fix a bug in a shutdown flow.
   - Instrumentation to MUTATE, PREPARE, EXECUTE and BATCH flow and some
     related changes.
   - A patch that aligns the QUERY tracing format with the Origin.
   - Methods and functions description in tracing/trace_state.hh."
2016-07-19 18:46:59 +03:00
Vlad Zolotarov
7c590295ef SELECT instrumentation: add a nice trace point
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:59 +03:00
Vlad Zolotarov
a197323b47 tracing::trace_state.hh: Add descriptions for main methods and functions
Add a proper description to a tracing::trace() that clarifies
that the tracing message string and the positional parameters
are going to be copied if tracing state is initialized.

Add a description for trace_state::begin() methods and for a
tracing::begin() helper function.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:59 +03:00
Vlad Zolotarov
b36b69c1d6 service::storage_proxy: remove a default value for a tracing::trace_state_ptr parameter
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:59 +03:00
Vlad Zolotarov
baa6496816 service::storage_proxy: READ instrumentation: store trace state object in abstract_read_executor
Having a trace_state_ptr in the storage_proxy level is needed to trace code bits in this level.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:59 +03:00
Vlad Zolotarov
b0a39f210d transport: CQL tracing: QUERY instrumentation: align the session creation parameters with origin
- Don't put the query name as a 'request' but rather save it as one of entries in a
     'params' map.
   - Save some additional query parameters in 'params'.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
962bddf8fe transport: CQL tracing: instrument a BATCH command
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
d21eaabcfe transport: CQL tracing: instrument EXECUTE command
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
89a49c346c tracing::trace_state: add begin() overload for seastar::value_of given as a "request" parameter.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
1f9b858d83 cql3: prepared_statement: add raw_cql_statement field
This field will contain an original statement given to a PREPARE
command.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
147dd72517 transport: CQL tracing: instrument a PREPARE command
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
be88074f47 service::query_state: get rid of begin_tracing()
Use tracing::begin() directly.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
982d301178 service::client_state: add a const version of get_trace_state()
tracing::begin() requires a non-const version, tracing::trace()
requires a const version.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
da56aa4256 service::client_state: rename: trace_state_ptr() -> get_trace_state()
Rename the method for consistency with other classes methods returning
the same value.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
4c16df9e4c service: instrument MUTATE flow with tracing
Store the trace state in the abstract_write_response_handler.
Instrument send_mutation RPC to receive an additional
rpc::optional parameter that will contain optional<trace_info>
value.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
54a758dfff cql3::select_statement: simplify the tracing code by using a tracing::make_trace_info() helper
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
952dc8a3d4 query_state: add get_trace_state() method
Adding this method allows to use tracing helper functions
and remove the no longer needed accessors in the query_state.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
0552ffcd17 service/storage_proxy: tracing: adjust the existing SELECT instrumentation with the new trace() interface
From now on trace_state::trace() is able to receive the sprint-ready
format string with the arguments that will be applied only during
the flush event.

This patch also optimizes the way the source address is evaluated -
do it only once instead of twice if tracing is requested.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
c1bb4d147d query::read_command: std::move() std::experimental::optional when initializing trace_info
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
0689843e79 tracing::trace_state: add method to set the session's "params" map entries
Sometimes we want to be able to set "params" map after we
started a tracing session, e.g. when the parameters values,
like a consistency level parsed from the "options" part of a binary frame,
are available only after some heavy part of a flow we would like
to trace.

This patch includes the following changes:

   - No longer pass a map to the begin().
   - Limit the parameters to the known set.
   - Define a method to set each such parameter and save its
     value till the final sstring->sstring map is created.
   - Construct the final sstring->sstring map in the destructor of the trace_state
     object in order to defer all the formatting to be after the traced flow.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
9c0a725c56 tracing: add a _local_tracing to a i_tracing_backend_helper
A backend helper has to constantly communicate with the corresponding
tracing::tracing instance. By saving a reference to the tracing::tracing instance
will save us a lot of tracing::get_local_tracing_instance() calls and thus
a lot of dereferencing.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
2bb054748e tracing: record events' time stamps
- Extend the i_tracing_backend_helper interface to accept the event
     record timestamp.
   - Grab the current timestamp when the event record is taken.
   - Add the instrumentation to the trace_keyspace_helper to create a unique time-UUID
     from a given std::chrono::duration object.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
f64f27beb9 utils: add get_time_UUID(system_clock::time_point)
Creates a type 1 UUID (time-based UUID) with the given system_clock::time_point

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
06d4221382 tracing: add tracing::make_trace_info() helper
This helper returns an std::experimental::optional<trace_info>
which is initialized or not initialized depending on whether
a given trace_state_ptr is initialized or not.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:57 +03:00
Vlad Zolotarov
7a5fc9fcdc tracing::trace_state: add const qualifiers to a trace_state_ptr parameter
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:57 +03:00
Vlad Zolotarov
b0673aabd5 tracing: fix a logger name
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:57 +03:00
Vlad Zolotarov
da4836becc tracing::trace_state: add support for a formatted message in trace()
Add an support for passing a format string plus positional parameters
for creation of a trace point message.

Format string should be given in a fmt library native format described
here: http://fmtlib.net/latest/syntax.html#syntax .

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:57 +03:00
Vlad Zolotarov
ee0e986e96 tracing: make a service shutdown stages more strict
kick() backend during shutdown and restrict accessing a backend
after that.

Flush pending records when service is being shut down.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:57 +03:00
Vlad Zolotarov
6e38133f82 tracing: prevent a destruction of a tracing::tracing while it's used
Prevent the destruction of tracing::tracing instances while there
are still tracing::trace_state objects that are using it:

   - Make tracing::tracing inherit from seastar::async_sharded_service<tracing::tracing>.
   - Grab a tracing::tracing.shared_from_this() in each
     tracing::trace_state object using it.
   - Use a saved pointer to the local tracing::tracing instance in a destructor
     instead of accessing it via tracing::get_local_tracing_instance()
     to avoid "local is not initialized" assert when sessions are
     being destroyed after the service was stopped.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:57 +03:00
Vlad Zolotarov
a5022a09a4 tracing: use 'write' instead of 'flush' and 'store' for consistency with seastar's API
In names of functions and variables:
s/flush_/write_/
s/store_/write_/

In a i_tracing_backend_helper:
s/flush()/kick()/

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:57 +03:00
Pekka Enberg
cbf5283a93 Merge "Populate size_estimates table" from Duarte
"This patchset implements the size_estimates_recorder, which periodically
writes estimations for all the non-system column families in the
size_estimates system table. This table is updated per schema with a set
of token ranges and the associated estimations of how many partitions
there are and their mean size.

Fixes #352"
2016-07-19 14:31:12 +03:00
Duarte Nunes
9ffdf4a5cd db: Implement size_estimates_recorder
This patch implements the size_estimates_recorder, which periodically
writes estimations for all the non-system column families in the
size_estimates system table. The size_estimates_recorder class
corresponds to the one in Cassandra's SizeEstimatesRecorder.java.

Estimation is carried out by shard 0. Since we're estimating based on
data in shared sstables, having multiple shards doing this would skew
the results.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-19 09:44:58 +00:00
Avi Kivity
86661db178 Merge seastar upstream
* seastar ceb0c94...823bc05 (1):
  > Revert "util::lazy_eval: add an implicit cast operator overload"
2016-07-19 12:02:44 +03:00
Duarte Nunes
f8f61cf246 system_keyspace: Record and clear size estimates
This patch implements functions that allow the size_estimates system
table to be updated and cleared. The size_estimates table is updated
per schema with a set of token ranges and the associated estimations
of how many partitions there are and their mean size.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-18 23:58:31 +00:00
Duarte Nunes
3518db531e database: Get non-system column_families
This patch adds an utility function that allows fetching the set of
column_families that do not belong to the system keyspace.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-18 23:58:31 +00:00
Duarte Nunes
4bc00c2055 database: Expose selection of sstables by a range
This patch allows a set of a column_family's sstables to be
selected according to a range of ring_positions.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-18 23:58:31 +00:00
Duarte Nunes
d7ae25c572 range: Make transform template arguments deductable
This patch makes it so that the template arguments of
range<T>::transform are more easily deducible by the compiler.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-18 23:58:31 +00:00
Duarte Nunes
3c05ea2f80 types: Add to_bytes_view for sstrings
This patch adds an overload of to_bytes_view for sstrings

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-18 23:58:31 +00:00
Tomasz Grabiec
ce768858f5 types: Fix update_types()
We should replace the old type, not insert the new type before the old type.

Fixes #1465

Message-Id: <1468861076-20397-1-git-send-email-tgrabiec@scylladb.com>
2016-07-18 20:14:22 +03:00
Avi Kivity
f886f7a2f5 Merge seastar upstream
* seastar a45823a...ceb0c94 (2):
  > print: switch to fmtlib
  > logging: simplify stringer array building
2016-07-18 19:37:34 +03:00
Avi Kivity
d261927fa3 logalloc: change sprint() of a pointer to use void* explicitly
Otherwise, fmtlib dislikes it.
2016-07-18 19:37:16 +03:00
Avi Kivity
1d1b03a7cb cql3: change sprint() of a pointer to use void* explicitly
Otherwise, fmtlib dislikes it.
2016-07-18 19:36:35 +03:00
Raphael S. Carvalho
7b9cf528ad tests: fix occassional failure in date tiered test
That was a bug in the test itself. It could happen that a sstable would
incorrectly belong to the next time window if the current minute is
approaching its end. Fix is about having all sstables that we want in
the same time window with the same min/max timestamp.

Fixes #1448.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <ee25d49e7ed12b4cf7d018a08163404c3d122e56.1468782787.git.raphaelsc@scylladb.com>
2016-07-18 15:18:29 +02:00
Paweł Dziepak
4497204b7d streamed_mutation: do not leave mutation in an invalid state
This patch avoids moving entries from range tombstones and clustering
rows sets in streamed_mutation_from_mutation(). Such action breaks these
sets as the entries will be left in some unknown state.

Instead, the sets are being broken in a supported and predictable way
using unlink_leftmost_without_rebalance().

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Message-Id: <1468843205-18852-1-git-send-email-pdziepak@scylladb.com>
2016-07-18 15:14:21 +02:00
Avi Kivity
9d1b813f45 Merge seastar upstream
* seastar d699205...a45823a (5):
  > rpc: do not call shutdown function on already closed fd
  > log: Do not crash if logger is invoked from non-reactor thread
  > rpc: remove unaligned_cast and reinterpret_cast uses
  > unaligned: note unaligned_cast<> is deprecated
  > byteorder: add unaligned read/write helpers

Fixes #1463.
2016-07-18 15:24:43 +03:00