Commit Graph

11716 Commits

Author SHA1 Message Date
Avi Kivity
35ef82e78d cql3: copy create_index_statement when preparing it
To prepare for the separation of prepared and raw schema_altering_statements,
avoid the reliance this class implementing both the raw and prepared
variants and the use of shared_from_this().
2016-07-30 23:12:55 +03:00
Avi Kivity
23eef0a610 cql3: copy alter_type_statement when preparing it
To prepare for the separation of prepared and raw schema_altering_statements,
avoid the reliance this class implementing both the raw and prepared
variants and the use of shared_from_this().
2016-07-30 23:12:24 +03:00
Avi Kivity
8b50f75958 cql3: copy alter_table_statement when preparing it
To prepare for the separation of prepared and raw schema_altering_statements,
avoid the reliance this class implementing both the raw and prepared
variants and the use of shared_from_this().
2016-07-30 23:10:43 +03:00
Avi Kivity
81b75dfa47 cql3: copy alter_keyspace_statement when preparing it
To prepare for the separation of prepared and raw schema_altering_statements,
avoid the reliance this class implementing both the raw and prepared
variants and the use of shared_from_this().
2016-07-30 22:45:08 +03:00
Avi Kivity
b881945d45 estimated_histograms: fix indentation, bracing 2016-07-30 20:13:16 +03:00
Avi Kivity
75ee8fc2a7 size_estimates_recorder: adjust indentation 2016-07-30 20:10:12 +03:00
Piotr Jastrzebski
ca9c29e296 Cache information about partition being wide
Once we encounter a wide partition store information
about this in cache entry and don't try to read it all
and cache next time it's requested.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
[Paweł: rebased, moved large partition reading logic to
cache_entry::read_wide()]
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-07-29 18:39:22 +01:00
Paweł Dziepak
42f566433e tests/row_cache_test: use BOOST_REQUIRE_EQUAL() istead of raw assert()
In case of failure BOOST_REQUIRE_EQUAL() is nicer and prints the actual
values that were supposed to be equal, but aren't.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-07-29 17:19:18 +01:00
Paweł Dziepak
ee1f1ee1c4 row_cache: fix creating readers for large partitions
There were cases of use-after-free introduced by the code responsible
for creating mutation_readers for large partitions – the lifetimes
of partition ranges and the readers themselves weren't sufficiently
extended.

Another problem, was that if the partition was no longer present in the
sstable the reader would return EOS which was then returned by
range_populating_reader itself causing its users to incorrectly
interpret that as an end of stream.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-07-29 17:02:17 +01:00
Paweł Dziepak
7b479d8b41 clarify relations between mutation_reader and streamed_mutation
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-07-29 15:58:42 +01:00
Paweł Dziepak
3a27582cd4 sstables: allow streamed_mutation to outlive mutation_reader
This patch makes sstable_streamed_mutation keep a reference to
sstable_data_source object which contains full state necessary to read
the sstable. That state is also shared with parent mutation_reader (only
for range queries), but now its lifetime is appropriately extended if
the mutation_reader is destoryed before streamed_mutation.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-07-29 15:53:09 +01:00
Paweł Dziepak
5ba4cd1a0b sstables: enable_lw_shared_from_this for sstable
sstable has member functions that create objects which need to extend
lifetime of the sstable (for example mutation_readers), the easiest way
to achieve that is to enable_lw_shared_from_this for sstable.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-07-29 15:51:12 +01:00
Duarte Nunes
7d1b7e8da3 storage_service: Fix get_range_to_address_map_in_local_dc
This patch fixes a couple of bugs in
get_range_to_address_map_in_local_dc.

Fixes #1517

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469782666-21320-1-git-send-email-duarte@scylladb.com>
2016-07-29 11:11:07 +02:00
Gleb Natapov
3531dd8d71 api: fix use after free in sum_sstable
get_sstables_including_compacted_undeleted() may return temporary shared
ptr which will be destroyed before the loop if not stored locally.

Fixes #1514

Message-Id: <20160728100504.GD2502@scylladb.com>
2016-07-28 14:25:40 +03:00
Piotr Jastrzebski
fdfd1af694 Use continuity flag correctly with concurrent invalidations
Between reading cache entry and actually using it
invalidations can happen so we have to check if no flag was
cleared if it was we need to read the entry again.

Fixes #1464.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <7856b0ded45e42774ccd6f402b5ee42175bd73cf.1469701026.git.piotr@scylladb.com>
2016-07-28 11:55:18 +01:00
Duarte Nunes
25a44ee6cf sstables: Validate static cell is on static column
This patch enforces compatibility between a cell and the
corresponding column definition with regards to them being
static.

[tgrabiec: Fixed typo in "definition"]

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469699532-26984-1-git-send-email-duarte@scylladb.com>
2016-07-28 12:01:31 +02:00
Duarte Nunes
6fc6adbdeb sstable_mutation_test: Test non-compound cell name
This patch adds a test case for reading non-compound cell names,
validating that such a cell is not incorrectly marked as static.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469616205-4550-5-git-send-email-duarte@scylladb.com>
2016-07-28 11:12:20 +02:00
Duarte Nunes
40f388f2b5 sstables: Don't assume cell name is compound
The current code assumes cell names are always compound and may
wrongly report a non-static row as such, since it looks at the first
bytes of the name assuming they are the component's length.

Tables with compact storage (which cannot contain static rows) may not
have a compound comparator, so we check for the table's compoundness
before checking for the static marker. We do this by delegating to
composite_view::is_static.

Fixes #1495

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469616205-4550-4-git-send-email-duarte@scylladb.com>
2016-07-28 11:12:06 +02:00
Duarte Nunes
05c3d4f22b composite: Use operator[] instead of at()
Since we already do bounds checking on is_static(), we can use
bytes_view::operator[] instead of bytes_view::at() to avoid repeating
the bounds checking.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469616205-4550-3-git-send-email-duarte@scylladb.com>
2016-07-28 11:11:54 +02:00
Duarte Nunes
6c9076fdd7 composite_view: Fix is_static
composite_view's is_static function is wrong because:

1) It doesn't guard against the composite being a compound;
2) Doesn't deal with widening due to integral promotions and
   consequent sign extension.

This patch fixes this by ensuring there's only one correct
implementation of is_static, to avoid code duplication and
enforce test coverage.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469616205-4550-2-git-send-email-duarte@scylladb.com>
2016-07-28 11:11:38 +02:00
Amnon Heiman
406fa11cc5 scylla-housekeeping: check version should use the current version
This patch handle two issues with check version:
* When checking for a version, the script send the current version
* Instead of string compare it uses parse_version to compare the
versions.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-07-28 11:31:05 +03:00
Amnon Heiman
641e5dc57c scylla-housekeeping: Switchng to pythno2
There is a problem with python module installation in pythno3,
especially on centos. Though pytho34 has a normal package, alot of the
modules are missing yum installation and can only be installed by pip.

This patch switch the  scylla-housekeeping implementation to use
python2, we should switch back to python3 when CeontOS python3 will be
more mature.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-07-28 11:27:52 +03:00
Tomasz Grabiec
27f6c0d62b tests: lsa_async_eviction_test: Use chunked_fifo<>
To protect against large reallocations during push() which are done
under reclaim lock and may fail.
2016-07-28 09:33:12 +02:00
Tomasz Grabiec
17b45fae9e tests/memory_footprint: Fix runtime errors
The test bit rot. One thing is that cache is no longer empty right
after creation, since we added ghost entries to it. Second problem is
that mutation serializer needs storage service to be initialized, so
we need to setup a full cql test env.
Message-Id: <1469626546-4279-1-git-send-email-tgrabiec@scylladb.com>
2016-07-27 14:38:36 +01:00
Duarte Nunes
f468453cbe README.md: Replace yum with dnf
yum is démodé.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469616395-5537-1-git-send-email-duarte@scylladb.com>
2016-07-27 13:50:31 +03:00
Duarte Nunes
e4464f7500 README.md: Add protobuf dependencies
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469616338-5199-1-git-send-email-duarte@scylladb.com>
2016-07-27 13:50:19 +03:00
Duarte Nunes
5aaf43d1bc thrift: Preserve partition order when accumulating
This patch changes the column_visitor so that it preservers the order
of the partitions it visits when building the accumulation result.

This is required by verbs such as get_range_slice, on top of which
users can implement paging. In such cases, the last key returned by
the query will be that start of the range for the next query. If
that key is not actually the last in the partitioner's order, then
the new request will likely result in duplicate values being sent.

Ref #693

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469568135-19644-1-git-send-email-duarte@scylladb.com>
2016-07-27 10:34:29 +02:00
Avi Kivity
64d0cf58ea size_estimates_recorder: unwrap ranges before searching for sstables
column_family::select_sstables() requires unwrapped ranges, so unwrap
them.  Fixes crash with Leveled Compaction Strategy.

Fixes #1507.

Reviewed-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469563488-14869-1-git-send-email-avi@scylladb.com>
2016-07-27 10:06:21 +03:00
Takuya ASADA
b542581d97 dist: add protobuf to dependencies, for newly added prometheus API support
Since we added prometheus API support, we need protobuf compiler and header to build seastar/scylla, so add dependencies on packaging.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1469573936-8551-1-git-send-email-syuu@scylladb.com>
2016-07-27 08:48:58 +03:00
Paweł Dziepak
efa690ce8c stables: fix skipping partitions with no rows
If partition contains no static and clustering rows or range tombstones
mp_row_consumer will return disengaged mutation_fragment_opt with
is_mutation_end flag set to mark end of this partition.

Current, mutation_reader::impl code incorrectly recognized disengaged
mutation fragment as end of the stream of all mutations. This patch
fixes that by using is_mutation_end flag to determine whether end of
partition or end of stream was reached.

Fixes #1503.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Message-Id: <1469525449-15525-1-git-send-email-pdziepak@scylladb.com>
2016-07-26 13:02:03 +03:00
Amos Kong
64530e9686 scylla-housekeeping: fix typo of script path
I tried to start scylla-housekeeping service by:
 # sudo systemctl restart scylla-housekeeping.service

But it's failed for wrong script path, error detail:
 systemd[5605]: Failed at step EXEC spawning
 /usr/lib/scylla/scylla-Housekeeping: No such file or directory

The right script name is 'scylla-housekeeping'

Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <c11319a3c7d3f22f613f5f6708699be0aa6bd740.1469506477.git.amos@scylladb.com>
2016-07-26 09:18:43 +03:00
Raphael S. Carvalho
b7cdfafbdd tests: fix compilation of partitioner test
For some unknown reason, there were some duplicated definitions
of bytes3. They were not needed at all.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <7f2b87211c592e573f45f277dc0ab4a8c037f258.1469490327.git.raphaelsc@scylladb.com>
2016-07-26 09:04:08 +03:00
Avi Kivity
1d1f61ac65 bytes_ostream: fix assignment operators
Protect against self-assignment, and provide the strong exception guarantees
for copying assignment.

Fixes #1499.
Message-Id: <1469466156-11114-1-git-send-email-avi@scylladb.com>
2016-07-25 19:06:42 +02:00
Avi Kivity
9e515c17a3 Merge "Optimize deserialization" from Tomasz
Fixes #1330.
2016-07-25 19:52:28 +03:00
Tomasz Grabiec
cea4957a11 serializer: Avoid copying when deserializing bytes_ostream 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
22664b9b7f serializer: Optimize serializer<>::skip() for remaining types 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
2e121d4b18 serializer: Implement serializer<std::vector<T>>::skip() using skip_array<T>() 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
af94ed538e serializer: Optimize serializer<std::array<T, N>>::skip() 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
fd5ccce919 serializer: Move skip() to serializer.hh
It is part of the core API, like serialize() and deserialize().
2016-07-25 17:35:42 +02:00
Tomasz Grabiec
d3658b33da tests: Add test for skip() not doing full deserialization 2016-07-25 17:35:42 +02:00
Tomasz Grabiec
c3707ff754 idl: Avoid full deserialization in skip()
Fixes #1330.
2016-07-25 17:35:34 +02:00
Tomasz Grabiec
033312f686 serializer: Add serializer<enum_set<T>> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
1ddba66861 serializer: Add serializer<std::unique_ptr<T>> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
4ec29d88d3 serializer: Add serializer<sstring> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
517a501ace serializer: Add serializer<std::experimental::optional<T>> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
c67e047b92 serializer: Add serializer<bytes_ostream> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
1bc63a133b serializer: Add serializer<bytes> 2016-07-25 17:22:28 +02:00
Tomasz Grabiec
51e25cb50e serializer: Add serializer<std::chrono::time_point<Clock, Duration>> 2016-07-25 17:22:25 +02:00
Tomasz Grabiec
f965e64a05 serializer: Add serializer<std::map<K, V>> 2016-07-25 17:22:21 +02:00
Tomasz Grabiec
5ffaccfa7d serializer: Add serializer<std::array<T, N>> 2016-07-25 17:20:52 +02:00