Commit Graph

10306 Commits

Author SHA1 Message Date
Vlad Zolotarov
c8cf2ef82c tracing::trace_state: introduce is_in_state() and set_state() accessors
Use these new methods to manipulate trace_state::_state value.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
39b23cd084 tracing::trace_state: rename: get_write_on_close() -> write_on_close()
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
09624f704f tracing::trace_state: rename: get_type() -> type()
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
b40a819d1e tracing::trace_state: rename: get_session_id() -> session_id()
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
ed21398ce9 trace_keyspace_helper: create a system_traces.node_slow_log table
This table is going to be used to store information about queries
which are slower than a specified threshold.

Also added a column caching and mutation creation functions

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
efeb62e72f tracing: trace_keyspace_helper: introduce a check_column_definition() helper function
Checks if a given column definition exists and has a requested type.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
6c3e1935b0 tracing::session_record: change a type of a "ttl" field to be std::chrono::seconds
TTL is always defined in seconds - make its type explicitly reflect that.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
e017533229 tracing: set a username session parameter
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
93c2502be4 tracing: set a table_name in a BATCH query
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
25be28bb3c tracing: set a table_name parameter in a SELECT statement
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
abae3b05e7 tracing: set table_name in a modification statement
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
372da7e71b tracing: add support for setting a username and a table name parameters
- "username" is a name used in the authentication process.
  - "table name" is a <keyspace>.<cf name> string representing a name
    of a table used for a query in question.

Note that there may be more than one table name in a batch query. Therefore
we store an unordered set of tables names.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
eaf5db66a8 tracing::session_record: store "parameters" data in an std::map instead of in an unordered_map
Avoid sorting (and creating a new one) container at a backend code when a sorted
container is needed.

The overhead for the backends where it's not needed is minimal since the size of the
map is very small.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:10 +03:00
Takuya ASADA
80f7449095 dist/ubuntu: support scylla-housekeeping service on all Ubuntu versions
Current scylla-housekeeping support on Ubuntu has bug, it does not installs .service/.timer for Ubuntu 16.04.
So fix it to make it work.

Fixes #1502

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Tested-by: Amos Kong <amos@scylladb.com>
Message-Id: <1471607903-14889-1-git-send-email-syuu@scylladb.com>
2016-08-23 13:49:44 +03:00
Takuya ASADA
aac60082ae dist/common/systemd: don't use .in for scylla-housekeeping.*, since these are not template file
.in is the name for template files witch requires to rewrite on building time, but these systemd unit files does not require rewrite, so don't name .in, reference directly from .spec.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1471607533-3821-1-git-send-email-syuu@scylladb.com>
2016-08-23 13:49:09 +03:00
Duarte Nunes
440c1b2189 thrift: Avoid always recording size estimates
Size estimates for a particular column family are recorded every 5
minutes. However, when a user calls the describe_splits(_ex) verbs,
they may want to see estimates for a recently created and updated
column family; this is legitimate and common in testing. However, a
client may also call describe_splits(_ex) very frequently and
recording the estimates on every call is wasteful and, worse, can
cause clients to give up. This patch fixes this by only recording
estimates if the first attempt to query them produces no results.

Refs #1139

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1471900595-4715-1-git-send-email-duarte@scylladb.com>
2016-08-23 13:08:25 +03:00
Takuya ASADA
383148de13 dist/common/scripts/scylla_bootparam_setup: fix failing setup hugepages= variable on boot parameter
This is caused because mistakenly dropped sourcing sysconfig file, so source it again.
Fixes #1599

Message-Id: <1471943742-19684-1-git-send-email-syuu@scylladb.com>
2016-08-23 12:41:39 +03:00
Takuya ASADA
1ad578ecf1 dist/common/scripts/scylla_bootparam_setup: use distribution standard grub.cfg update command on Ubuntu
Result is almost same, but let's do it in ubuntu/debian flavor.

Message-Id: <1471943898-24490-1-git-send-email-syuu@scylladb.com>
2016-08-23 12:41:34 +03:00
Paweł Dziepak
5feed84e32 sstables: do not call consume_end_partition() after proceed::no
After state_processor().process_state() returns proceed::no the upper
layer should have a chance to act before more data is pushed to the
consumer. This means that in case of proceed::no verify_end_state()
should not be called immediately since it may invoke
consume_end_partition().

Fixes #1605.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Message-Id: <1471943032-7290-1-git-send-email-pdziepak@scylladb.com>
2016-08-23 12:24:39 +03:00
Pekka Enberg
9d1d8baf37 dist/docker: Separate supervisord config files
Move scylla-server and scylla-jmx supervisord config files to separate
files and make the main supervisord.conf scan /etc/supervisord.conf.d/
directory. This makes it easier for people to extend the Docker image
and add their own services.

Message-Id: <1471588406-25444-1-git-send-email-penberg@scylladb.com>
2016-08-22 17:20:23 +03:00
Avi Kivity
55f2cf1626 thrift: do not generate wrapping ring_position ranges
As part of the move to unwrap ranges, don't generate wrapping ranges from
thrift.  A little extra motivation is to avoid the need for the solution
to #1573 to be able to handle wrapping ranges.

This patch may also be fixing a bug in that the range (token, token] was
previously translated as (-inf, +inf), while now it is translated as
{(token, +inf), (-inf, token]}; the new translation respects ordering
better.

Reviewed-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1471869587-12972-1-git-send-email-avi@scylladb.com>
2016-08-22 14:43:45 +01:00
Avi Kivity
46caff5b06 Merge "Store frozen_mutation in fragmented buffer" from Paweł
"This series switches frozen_mutations and to use bytes_ostream
internally so that the size of a single allocation is bounded.
Deserializers are also enhanced so that they can cope with reading
from fragmented buffers.

The goal of the change is to reduce memory pressure in case of
large partitions.

Performance as measured by perf_simple_query (median of 30).

            before       after      diff
read     705270.74   702906.35     -0.3%
write    814504.81   836462.33     +2.7%

Refs #1440.
Refs #1545.
Fixes #1546."
2016-08-22 13:01:34 +03:00
Paweł Dziepak
1315090bf0 query-result: no need to linearize buffer any more
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
3fe5ed3cd9 query: use result_view::consume() where appropriate
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
cb2a557cf7 query::result: reduce chunk count
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
ea3ac0a270 frozen_mutation: reduce chunk count in constructor
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
1daf4c73a3 frozen_mutation: avoid buffer linearization and copy
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
3707d7fec3 frozen_mutation: use bytes_ostream internally
Unlike bytes, bytes_ostream supports fragmented buffers, thus reducing
the pressure on the memory allocator caused by large frozen partitions.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
c0425b63ff frozen_mutation: add mutation_view()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
89f7b46f61 idl: switch to utils::input_stream
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
dcf794b04d idl: make bytes compatible with bytes_ostream
This patch makes idl type "bytes" compatible with both bytes and
bytes_ostream.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
2c5ec44281 atomic_cell: add overloads taking const bytes&
Deserialization code is going to use a proxy object that will be casted
to either bytes or bytes_ostream depending on the demand. It cannot be
casted directly to bytes_view though as it won't extend the lifetime of
the buffer appropriately. The simples solution is just to add overloads
that accept const bytes&.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
387434a76c bytes_ostream: add reduce_chunk_count()
Deserialization code has now two variants. The faster one can be used
only when the source buffer is not fragmented. reduce_chunk_count() aims
to increase number of cases when the fast path can be used.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
7d4b7fd5fc bytes_ostream: add equality operator
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
222bde7e6f bytes_ostream: introduce upper bound on chunk size
This patch makes append() and write() limit the maximum size of a single
allocation to bytes_ostream::max_chunk_size.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
0ee98ea4c4 tests: add fragmented input stream test
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
e76203c927 utils: add input_stream
input_stream performs a type erasure on seastar::simple_input_stream and
fragmented_input_stream. The main goal is to keep the overhead for the
cases when simple_input_stream is used minimum.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
29827a9726 utils: add fragmented_input_stream
fragmented_input_stream is an input stream usable by IDL-generated
deserializers which can read from fragmented buffers.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Asias He
4ffd867ad0 gossip: Add log when cluster or partioner mismatch
It is easier for user to figure out the configuration error.

The log looks like:

   WARN  2016-08-22 15:04:56,214 [shard 0] gossip - ClusterName mismatch
   from 127.0.0.2 test2!=test

   WARN  2016-08-22 15:06:16,106 [shard 0] gossip - Partitioner mismatch from 127.0.0.2
   org.apache.cassandra.dht.RandomPartitioner!=org.apache.cassandra.dht.Murmur3Partitioner

Fixes: #1587

Message-Id: <745ed8857da6f70745735b94eef7b226d2f22e10.1471849834.git.asias@scylladb.com>
2016-08-22 11:06:31 +03:00
Raphael S. Carvalho
77d4cd21d7 sstables: Fix estimation of pending tasks for leveled strategy
There were two underflow bugs.

1) in variable i, causing get_level() to see an invalid level and
throw an exception as a result.
2) when estimating number of pending tasks for a level.

Fixes #1603.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <cce993863d9de4d1f49b3aabe981c475700595fc.1471636164.git.raphaelsc@scylladb.com>
2016-08-22 10:37:15 +03:00
Vlad Zolotarov
92921fe110 tracing::trace_state: push the UUID to the end of an error message in a destructor
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1471780783-25406-1-git-send-email-vladz@cloudius-systems.com>
2016-08-21 16:50:52 +03:00
Vlad Zolotarov
0683d4bd29 tracing::trace_state: don't throw in a destructor
The condition in question is sanity check for a SW bug.

This SW bug (if occurs) is not critical - there is an additional protection
against it in the stop_foreground_and_write().

Having said all that, since we shell not throw from a destructor,
replace throwing of a std::logic_error with an logger error message.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1471773320-7398-1-git-send-email-vladz@cloudius-systems.com>
2016-08-21 13:50:52 +03:00
Avi Kivity
2dcbbf9006 Merge seastar upstream
* seastar 4254111...6fadd98 (6):
  > simple_input_stream: remove explicit copy constructor
  > simple_stream: add [[gnu::always_inline]]
  > perf_fstream: don't initialize variable-size array inline
  > rpc: annotate template function calls with "template"
  > iotune: avoid "auto" parameters
  > file: ext4 support
2016-08-21 13:38:43 +03:00
Paweł Dziepak
e60bb83688 sstables: optimise clustering rows filtering
Clustering rows in the sstables are sorted in the ascending order so we
can use that to minimise number of comparisons when checking if a row is
in the requested range.

Refs #1544.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Reviewed-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <1471608921-30818-1-git-send-email-pdziepak@scylladb.com>
2016-08-19 18:11:11 +03:00
Pekka Enberg
2bf5e8de6e dist/docker: Use Scylla mascot as the logo
Glauber "eagle eyes" Costa pointed out that the Scylla logo used in our
Docker image documentation looks broken because it's missing the Scylla
text.

Fix the problem by using the Scylla mascot instead.
Message-Id: <1471525154-2800-1-git-send-email-penberg@scylladb.com>
2016-08-19 12:50:02 +03:00
Pekka Enberg
4d90e1b4d4 dist/docker: Fix bug tracker URL in the documentation
The bug tracker URL in our Docker image documentation is not clickable
because the URL Markdown extracts automatically is broken.

Fix that and add some more links on how to get help and report issues.
Message-Id: <1471524880-2501-1-git-send-email-penberg@scylladb.com>
2016-08-19 12:49:52 +03:00
Yoav Kleinberger
25fb5e831e docker: extend supervisor capabilities
allow user to use the `supervisorctl' program to start and stop
services. `exec` needed to be added to the scylla and scylla-jmx starter
scripts - otherwise supervisord loses track of the actual process we
want to manage.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1471442960-110914-1-git-send-email-yoav@scylladb.com>
2016-08-18 15:08:11 +03:00
Avi Kivity
d33d958239 Merge "tracing: cleanups" from Vlad
"This series includes a time stamp representation changes Avi asked.
In addition is fixes a session "duration" semantics to be the time
it took to satisfy the user's request and not a time it took to
achieve the complete replication factor."
2016-08-18 14:36:19 +03:00
Pekka Enberg
1553bec57a dist/docker: Documentation cleanups
- Fix invisible characters to be space so that Markdown to PDF
  conversion works.

- Fix formatting of examples to be consistent.

- Spellcheck.

Message-Id: <1471514924-29361-1-git-send-email-penberg@scylladb.com>
2016-08-18 13:09:37 +03:00
Pekka Enberg
4ca260a526 dist/docker: Document image command line options
This patch documents all the command line options Scylla's Docker image supports.

Message-Id: <1471513755-27518-1-git-send-email-penberg@scylladb.com>
2016-08-18 13:01:58 +03:00