Commit Graph

10088 Commits

Author SHA1 Message Date
Avi Kivity
64d0cf58ea size_estimates_recorder: unwrap ranges before searching for sstables
column_family::select_sstables() requires unwrapped ranges, so unwrap
them.  Fixes crash with Leveled Compaction Strategy.

Fixes #1507.

Reviewed-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469563488-14869-1-git-send-email-avi@scylladb.com>
2016-07-27 10:06:21 +03:00
Takuya ASADA
b542581d97 dist: add protobuf to dependencies, for newly added prometheus API support
Since we added prometheus API support, we need protobuf compiler and header to build seastar/scylla, so add dependencies on packaging.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1469573936-8551-1-git-send-email-syuu@scylladb.com>
2016-07-27 08:48:58 +03:00
Paweł Dziepak
efa690ce8c stables: fix skipping partitions with no rows
If partition contains no static and clustering rows or range tombstones
mp_row_consumer will return disengaged mutation_fragment_opt with
is_mutation_end flag set to mark end of this partition.

Current, mutation_reader::impl code incorrectly recognized disengaged
mutation fragment as end of the stream of all mutations. This patch
fixes that by using is_mutation_end flag to determine whether end of
partition or end of stream was reached.

Fixes #1503.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Message-Id: <1469525449-15525-1-git-send-email-pdziepak@scylladb.com>
2016-07-26 13:02:03 +03:00
Amos Kong
64530e9686 scylla-housekeeping: fix typo of script path
I tried to start scylla-housekeeping service by:
 # sudo systemctl restart scylla-housekeeping.service

But it's failed for wrong script path, error detail:
 systemd[5605]: Failed at step EXEC spawning
 /usr/lib/scylla/scylla-Housekeeping: No such file or directory

The right script name is 'scylla-housekeeping'

Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <c11319a3c7d3f22f613f5f6708699be0aa6bd740.1469506477.git.amos@scylladb.com>
2016-07-26 09:18:43 +03:00
Raphael S. Carvalho
b7cdfafbdd tests: fix compilation of partitioner test
For some unknown reason, there were some duplicated definitions
of bytes3. They were not needed at all.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <7f2b87211c592e573f45f277dc0ab4a8c037f258.1469490327.git.raphaelsc@scylladb.com>
2016-07-26 09:04:08 +03:00
Avi Kivity
1d1f61ac65 bytes_ostream: fix assignment operators
Protect against self-assignment, and provide the strong exception guarantees
for copying assignment.

Fixes #1499.
Message-Id: <1469466156-11114-1-git-send-email-avi@scylladb.com>
2016-07-25 19:06:42 +02:00
Avi Kivity
9e515c17a3 Merge "Optimize deserialization" from Tomasz
Fixes #1330.
2016-07-25 19:52:28 +03:00
Tomasz Grabiec
cea4957a11 serializer: Avoid copying when deserializing bytes_ostream 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
22664b9b7f serializer: Optimize serializer<>::skip() for remaining types 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
2e121d4b18 serializer: Implement serializer<std::vector<T>>::skip() using skip_array<T>() 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
af94ed538e serializer: Optimize serializer<std::array<T, N>>::skip() 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
fd5ccce919 serializer: Move skip() to serializer.hh
It is part of the core API, like serialize() and deserialize().
2016-07-25 17:35:42 +02:00
Tomasz Grabiec
d3658b33da tests: Add test for skip() not doing full deserialization 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
c3707ff754 idl: Avoid full deserialization in skip()
Fixes #1330.
2016-07-25 17:35:34 +02:00
Tomasz Grabiec
033312f686 serializer: Add serializer<enum_set<T>> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
1ddba66861 serializer: Add serializer<std::unique_ptr<T>> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
4ec29d88d3 serializer: Add serializer<sstring> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
517a501ace serializer: Add serializer<std::experimental::optional<T>> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
c67e047b92 serializer: Add serializer<bytes_ostream> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
1bc63a133b serializer: Add serializer<bytes> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
51e25cb50e serializer: Add serializer<std::chrono::time_point<Clock, Duration>> 2016-07-25 17:22:25 +02:00
Tomasz Grabiec
f965e64a05 serializer: Add serializer<std::map<K, V>> 2016-07-25 17:22:21 +02:00
Tomasz Grabiec
5ffaccfa7d serializer: Add serializer<std::array<T, N>> 2016-07-25 17:20:52 +02:00
Tomasz Grabiec
43a69e64f6 serializer: Add serializer<std::chrono::duration<T, Ratio>> 2016-07-25 17:20:08 +02:00
Tomasz Grabiec
445f763fa3 serializer: Add serializer<std::vector<T>> 2016-07-25 17:20:08 +02:00
Tomasz Grabiec
953fce3f7b serializer: Define serializer<> specializations for integral types 2016-07-25 17:19:50 +02:00
Avi Kivity
c908e630f0 Merge seastar upstream
* seastar 2425aab...64ae228 (1):
  > Merge "Adding prometheus API support" from Amnon
2016-07-25 15:32:40 +03:00
Avi Kivity
45e1274064 Merge "Add GDB commands for working with thread lists" from Tomasz 2016-07-25 15:32:02 +03:00
Tomasz Grabiec
0b8aafd72c scylla-gdb.py: Introduce "scylla thread apply all ..."
Similar to gdb's "thread apply all". Executes given command in the
context of each seastar thread.

For example to print backtrace of all threads:

  scylla thread apply-all bt
2016-07-25 12:40:47 +02:00
Tomasz Grabiec
2d6341d4cb scylla-gdb.py: Introduce "scylla threads" command
Lists all seastar threads.

Example:

(gdb) scylla threads
[shard  1] (seastar::thread_context*) 0x602008c9aa00
[shard  1] (seastar::thread_context*) 0x602008c9ca00
[shard  1] (seastar::thread_context*) 0x602008cf5800
[shard  1] (seastar::thread_context*) 0x602008cbe000
[shard  1] (seastar::thread_context*) 0x602008c4bc00
[shard  2] (seastar::thread_context*) 0x601008d6b800
[shard  2] (seastar::thread_context*) 0x601008d89400
[shard  2] (seastar::thread_context*) 0x601008c95a00
[shard  0] (seastar::thread_context*) 0x600008d84400
[shard  0] (seastar::thread_context*) 0x6000000a1600
[shard  0] (seastar::thread_context*) 0x600008ce9a00
[shard  0] (seastar::thread_context*) 0x600008df9a00
[shard  0] (seastar::thread_context*) 0x600008dfee00
[shard  0] (seastar::thread_context*) 0x600008d85800
[shard  0] (seastar::thread_context*) 0x600008df9000
[shard  0] (seastar::thread_context*) 0x600008d82c00
[shard  0] (seastar::thread_context*) 0x600008cece00
2016-07-25 12:32:48 +02:00
Tomasz Grabiec
0e1b21eab6 scylla-gdb.py: Make intrusive_list() work with newer version of boost 2016-07-25 12:32:48 +02:00
Duarte Nunes
d8a4bd6b1a sstables: Remove duplication in extract_clustering_key
This patch removes some duplicated code in extract_clustering_key(),
which is already handled in composite_view.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469397806-8067-1-git-send-email-duarte@scylladb.com>
2016-07-25 10:07:25 +02:00
Duarte Nunes
b2278697ce sstables: Remove superfluous call to check_static()
When building a column we're calling check_static() two times;
refector things a bit so that this doesn't happen and we reuse the
previous calculation.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469397748-7987-1-git-send-email-duarte@scylladb.com>
2016-07-25 10:06:56 +02:00
Duarte Nunes
7a81553d17 compound_compat: Only compound values can be static
If a composite is not a compound, then it doesn't carry a length
prefix where static information is encoded. In its absence, a
non-compound composite can never be static.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469397561-7748-1-git-send-email-duarte@scylladb.com>
2016-07-25 10:05:19 +02:00
Avi Kivity
f2939b125e dist: reduce kernel's inclincation to OOM-kill Scylla
Scylla is likely the most important process on the machine, make the kernel
less inclined to kill it, on systemd-enabled hosts.

See #1393.
Message-Id: <1468847986-9429-1-git-send-email-avi@scylladb.com>
2016-07-25 10:00:19 +02:00
Duarte Nunes
5c4a2044d5 thrift: Fail when creating mixed CF
This patch ensures we fail when creating a mixed column family, either
when adding columns to a dynamic CF through updated_column_family() or
when adding a dynamic column upon insertion.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469378658-19853-1-git-send-email-duarte@scylladb.com>
2016-07-25 10:35:37 +03:00
Duarte Nunes
560cc12fd7 thrift: Correctly translate no_such_column_family
The no_such_column_family exception is translated to
InvalidRequestException instead of to NotFoundException.

8991d35231 exposed this problem.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469376674-14603-1-git-send-email-duarte@scylladb.com>
2016-07-25 10:35:07 +03:00
Avi Kivity
df346d8b49 Merge "thrift: Implements describe_splits verbs" from Duarte
"This patchset implements the describe_splits and describe_splits_ex verbs
in Thrift. It also contains a couple of related fixes."
2016-07-25 10:08:00 +03:00
Raphael S. Carvalho
56ef3d4fde sstables: get rid of magic number
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <c0b3acf58aa6f7632bf15a2dd0dcac4d1b45d444.1469389290.git.raphaelsc@scylladb.com>
2016-07-25 10:06:51 +03:00
Takuya ASADA
b5bb702b35 dist: add dependency for lspci
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1469429320-13645-1-git-send-email-syuu@scylladb.com>
2016-07-25 10:02:30 +03:00
Duarte Nunes
ab08561b89 thrift: Implement describe_splits verb
This patch implements the describe_splits verb on top of
describe_splits_ex.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-24 22:44:07 +00:00
Duarte Nunes
472c23d7d2 thrift: Implement describe_splits_ex verb
This patch implements the describe_splits_ex verbs by querying the
size_estimates system table for all the estimates in the specified
token range.

If the keys_per_split argument is bigger then the
estimated partitions count, then we merge ranges until keys_per_split
is met. Note that the tokens can't be split any further,
keys_per_split might be less than the reported number of keys in one
or more ranges.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-24 22:44:01 +00:00
Duarte Nunes
ecfa04da77 system_keyspace: Add query_size_estimates() function
The query_size_estimates() function queries the size_estimates system
table for a given keyspace and table, filtering out the token ranges
according to the specified tokens.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-24 22:43:58 +00:00
Duarte Nunes
d984cc30bf size_estimates_recorder: Fix stop()
This patch fixes stop() by checking if the current CPU instead of
whether the service is active (which it won't be at the time stop() is
called).

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-24 22:43:58 +00:00
Duarte Nunes
e16f3f2969 system_keyspace: Avoid pointers in range_estimates
This patch makes range_estimates a proper struct, where tokens are
represented as dht::tokens rather than dht::ring_position*.

We also pass other arguments to update_ and clear_size_estimates by
copy, since one will already be required.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-24 22:43:35 +00:00
Asias He
2f4cd86809 random_partitioner: Implement random_partitioner
Cassandra 1.x clusters often use RandomPartitioner. Supporting
RandomPartitioner will allow easier migration to Scylla

Tests are added to make sure scylla generates the same token as
Cassandra does for the same partition key.

Fixes #1438

Message-Id: <3bc8b7f06fad16d59aaaa96e2827198ce74214c6.1469166766.git.asias@scylladb.com>
2016-07-24 16:25:25 +03:00
Avi Kivity
a0369a6d5b Merge seastar upstream
* seastar 9d1db3f...2425aab (5):
  > Merge "Track all threads globally" from Tomasz
  > net: provide remote address on native_server_socket_impl<Protocol>::accept()
  > Introduce install-dependencies.sh
  > reactor: make sure a poll cycle always happens when later is called
  > Merge "rpc: various fixes and cleanups" from Gleb
2016-07-24 16:22:05 +03:00
Avi Kivity
b339337287 Merge "Add more tools to the GDB script" from Tomasz 2016-07-24 16:21:31 +03:00
Duarte Nunes
2be45c4806 thrift: Handle and convert invalid_request_exception
This patch converts an exceptions::invalid_request_exception
into a Thrift InvalidRequestException instead of into a generic one.

This makes TitanDB work correctly, which expects an
InvalidRequestException when setting a non-existent keyspace.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469362086-1013-1-git-send-email-duarte@scylladb.com>
2016-07-24 14:08:58 +02:00
Duarte Nunes
b5968ac244 sstables: Fix format string
This patch fixes a format string by using {} instead of %s.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469352081-2167-1-git-send-email-duarte@scylladb.com>
2016-07-24 14:01:04 +02:00