This change fixes the dependencies between the clock implementation headers. All
the clocks share the common clock offset, but are otherwise independent (though
the `db_clock` does depend on `gc_clock` for time point conversions).
`gc_clock` reports system time, and these conversion functions allow for
manipulating time points produced by the clock without making assumptions about
its epoch.
"Fix minor issues found while building with clang trunk."
* 'clang' of https://github.com/avikivity/scylla:
seastarx: don't make seastar namespace inline
seastarx: add missing make_shared forward declaration
tests: fix call to seastar::sleep()
dht: fix bad to_sstring() call
* 'lcs_improvements_v1' of github.com:raphaelsc/scylla:
lcs: remove useless code for choosing L0 candidates
lcs: remove some dead code
lcs: make logger static
lcs: actually prefer oldest sstables of L0 when it falls behind
lcs: remove useless expensive check for overlapping L1 sstables
so now user can look at nodetool compactionstats and determine
whether or not resharding is running, for example:
$ ./bin/nodetool compactionstats
pending tasks: 3
id compaction type keyspace table completed total unit progress
<none> RESHARD system compaction_history 11 256 keys 4.30%
<none> RESHARD system compaction_history 2 256 keys 0.78%
<none> RESHARD system compaction_history 10 256 keys 3.91%
<none> RESHARD system compaction_history 8 256 keys 3.12%
<none> RESHARD system compaction_history 7 256 keys 2.73%
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20170620175733.25882-1-raphaelsc@scylladb.com>
The code filters CFs by name to not include system keyspace, but v3
schema added yet another system namespace. Better filter according to
replication strategy to accommodate for schema v4 adding even more
system keyspaces.
Fixes: #2516
Message-Id: <20170621073816.GB3944@scylladb.com>
This version optionally reads the include paths for Seastar from pkg-config and
uses file globbing to register all source and header files.
In comparison to the previous version, I see all files in the project explorer
view are "active" (rather than just .cc files). I believe there are also fewer
errors reported by the editor.
Add support Debian new stable release.
Also including following changes:
- update libthrift due to unable to compile on Debian 9
- drop dist/debian/supported_release since distribution check code moved to pbuilderrc
- add libssl-dev for build-depends
- add sudo for pbuilder extra packages (Debian doesn't have it by default install)
Signed-off-by: syuu <syuu@dokukino.com>
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1498047515-19972-1-git-send-email-syuu@scylladb.com>
The memtable destructor can take a long time if the memtable is full; use
clear_gently() to clear it without impacting latency.
Fixes#2477.
Message-Id: <20170620093550.16121-1-avi@scylladb.com>
Although usually one can fast_forward_to() on the result of a
sstable::as_mutation_source(), earlier we had an optimization
where if a single partition was specified, it was read exactly,
and fast_forward_to() was *NOT* allowed.
With the mutation_reader::forwarding flag patch, when this flag
was on - requesting fast_forward_to() - we disabled this optimization.
This makes sense, but is not backward compatible with the code which
previously assumes this optimization exists... So this patch returns
this optimization, despite this meaning that we blatently ignore
the fwd_mr flag in that case.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20170620081107.14335-1-nyh@scylladb.com>
* seastar 621b7ed...9e2b7ec (8):
> Merge "Low-resolution clocks" from Jesse
> build: disable -Wattributes when gcc -fvisibility=hidden bug strikes
> build: work around ragel 7 generated code bug
> rpc: make unmarshall_exception() inline
> future-utils: make functions global
> fix reactor stall detector rate limiting on an mostly idle system
> prometheus: Add ability to add /metrics to any http_server
> prometheus: fix memory leak in http_server_control
multi_range_mutation_reader uses fast_forward_to() to skip between
ranges, so we always need to create the underlying reader with with
mutation_reader::forwarding::yes if there is more than one range,
irrespective of whether multi_range_mutation_reader itself will be
forwarded or not.
Fixes#2510.
Introduced in commit 3018df1.
Message-Id: <1497943032-18696-1-git-send-email-tgrabiec@scylladb.com>
The code being removed could be used if parallel compaction were
allowed for LCS, but the current code isn't even allowing that.
At the moment, it's only wasting cycles.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Strategy prefers promoting oldest sstables in L0. Because sort
procedure is incorrectly sorting elements in descending order,
newest sstables will be promoted first *if and only if* L0 falls
behind (more than 32 sstables). If L0 doesn't fall behind, we'll
have all L0 sstables compacted with overlapping ones in L1.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
there's no way a L1 sstable will be in candidates set which was
previously built from list of L0 sstables.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Enable pbuilder for Ubuntu/Debian to prevent build enviroment dependent issues.
Also support cross building by pbuilder.
(cross-building from Fedora 25 and Ubuntu 16.04 are tested)
closes#629
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1497895661-26376-1-git-send-email-syuu@scylladb.com>
As Avi noticed, the "forwarding_tag" which was meant to be local in
streamed_mutation, became global. If another class copied the same trick,
it would share the same type instead of being distinct types as intended.
The problem is that in:
using forwarding = bool_class<class forwarding_tag>;
Apparently, the "class forwarding_tag" forward-declares a global type - it
does not create a local-scope type as intended, which the following apparently
does (even though no actual definition is given for that class):
class forwarding_tag;
using forwarding = bool_class<forwarding_tag>;
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20170619153933.13116-1-nyh@scylladb.com>
In commit c63e88d556, support was added for
fast_forward_to() in data_consume_rows(). Because an input stream's end
cannot be changed after creation, that patch ignores the specified end
byte, and uses the end of file as the end position of the stream.
As result of this, even when we want to read a specific byte range (e.g.,
in the repair code to checksum the partitions in a given range), the code
reads an entire 128K buffer around the end byte, or significantly more, with
read-ahead enabled. This causes repair to do more than 10 times the amount
of I/O it really has to do in the checksumming phase (which in the current
implementation, reads small ranges of partitions at a time).
This patch has two levels:
1. In the lower level, sstable::data_consume_rows(), which reads all
partitions in a given disk byte range, now gets another byte position,
"last_end". That can be the range's end, the end of the file, or anything
in between the two. It opens the disk stream until last_end, which means
1. we will never read-ahead beyond last_end, and 2. fast_fordward_to() is
not allowed beyond last_end.
2. In the upper level, we add to the various layers of sstable readers,
mutation readers, etc., a boolean flag mutation_reader::forwarding, which
says whether fast_forward_to() is allowed on the stream of mutations to
move the stream to a different partition range.
Note that this flag is separate from the existing boolean flag
streamed_mutation::fowarding - that one talks about skipping inside a
single partition, while the flag we are adding is about switching the
partition range being read. Most of the functions that previously
accepted streamed_mutation::forwarding now accept *also* the option
mutation_reader::forwarding. The exception are functions which are known
to read only a single partition, and not support fast_forward_to() a
different partition range.
We note that if mutation_reader::forwarding::no is requested, and
fast_forward_to() is forbidden, there is no point in reading anything
beyond the range's end, so data_consume_rows() is called with last_end as
the range's end. But if forwarding::yes is requested, we use the end of the
file as last_end, exactly like the code before this patch did.
Importantly, we note that the repair's partition reading code,
column_family::make_streaming_reader, uses mutation_reader::forwarding::no,
while the other existing reading code will use the default forwarding::yes.
In the future, we can further optimize the amount of bytes read from disk
by replacing forwarding::yes by an actual last partition that may ever be
read, and use its byte position as the last_end passed to data_consume_rows.
But we don't do this yet, and it's not a regression from the existing code,
which also opened the file input stream until the end of the file, and not
until the end of the range query. Moreover, such an improvement will not
improve of anything if the overall range is always very large, in which
case not over-reading at its end will not improve performance.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20170619152629.11703-1-nyh@scylladb.com>
We mistakenly placed ./configure.py to dh_auto_build, but it's should place at
dh_auto_configure.
This bug causes issue #2505, since we haven't defined dh_auto_configure task yet(It seems running cmake on top of the dir is one of default behavior of dh_auto_configure).
Fixes#2505
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1497866068-32097-1-git-send-email-syuu@scylladb.com>
This patch add the CMakeLists.txt file for IDEs based on cmake, like
CLion.
This file assumes the existence of a build/release/gen directory,
containing generated files.
Refs #867
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170618151333.94714-1-duarte@scylladb.com>
"This patch set ensures we quote the name of a UDT when it
contains characters that may cause parsing by the CQL parser
to fail.
Fixes#2491"
* 'cql3-quote-type/v1' of https://github.com/duarten/scylla:
cql3/util: Make maybe_quote() take argument by const reference
cql3/cql3_type: Quote UDT name if needed
schema: Lift maybe_quote() into cql3/util
Boost 1.55 (ubuntu 14) fails to compile because an iterator produce by
boost::adaptors::transformed() when std::ref to lambda is passed to
it do not match iterator concept. It cannot be default constructed
because std::reference_wrapper is not default constructable.
boost::range::min_element() never actually default construct it, but
concept is checked anyway. The patch fixes it by providing an explicit
functor that is default constructable.
Message-Id: <20170618131836.GD3944@scylladb.com>
When making the schema mutations for a view update, we should only
include the base table schema mutations (in case the target node
doesn't contain them) when the view is being directly updated. When it
is being updated as a side effect of updating the base table, then
including the base schema mutations will hide the actual changes being
performed on the base.
Fixes#2500
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1497782822-2711-1-git-send-email-duarte@scylladb.com>
This reverts commit 317d7fc253 (and also the
related 2c57ab84b2). It causes crashes
during range scans, reported by Gleb:
"To reproduce I run SELECT * FROM keyspace1.standard1; on typical c-s
dataset and 3 node cluster.
Backtrace:
at /home/gleb/work/seastar/seastar/core/apply.hh:36
rvalue=<unknown type in /home/gleb/work/seastar/build/release/scylla, CU 0x54cf307, DIE 0x55ebf2a>) at /home/gleb/work/seastar/seastar/core/do_with.hh:57
range=std::vector of length 6, capacity 8 = {...}) at /home/gleb/work/seastar/seastar/core/future-util.hh:142
at ./seastar/core/future.hh:890
at /home/gleb/work/seastar/seastar/core/future-util.hh:119
at /home/gleb/work/seastar/seastar/core/future-util.hh:142
Add a performance test for deletion in addition to the existing update
and query tests. The deletion performance test is executed using the
'--delete' argument to perf_simple_query.
Fixes#2417.
Signed-off-by: Etienne Kruger <el@loadavg.io>
Message-Id: <20170615232500.26987-1-el@loadavg.io>
This patch ensures we properly quote a UDT name, which may contain
characters like ".", which can lead the name to be interpreted as a
keyspace qualified name when parsed by the CQL parser.
Fixes#2491
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
