This will be used to persist CDC streams generation timestamp
proposed by a joining node in case the node crashes or restarts,
similarly to the way tokens are persisted.
The get_saved_cdc_streams_timestamp method retrieves the generation
timestamp from the system table. It will be used by a restarting
node.
The update_cdc_streams_timestamp method saves CDC stream
generation timestamp of the calling node in the system table.
A joining node will persist the timestamp before it proposes it to other
nodes.
The cdc_topology_description table will be used internally
by nodes to send new CDC stream generations to other nodes.
The cdc_description table is a user-facing table,
used to inform users about new sets of CDC streams.
Regenerate sstables and digests for schema_change_test.
We don't need to protect this change by a schema feature:
when a node creates these tables, it announces them
to all other nodes. If schema agreement happens before
this migration, all nodes will use a digest calculated
without these tables. If it happens after, then all nodes
will eventually know about these tables and use a digest
calculated with these tables.
cdc::topology_description describes a mapping of tokens to CDC streams.
The cdc::generate_topology_description function is given:
1. a set of tokens which split the token ring into token ranges (vnodes),
2. information on how each token range is distributed among its owning
node's shards
and tries to generate a set of CDC stream identifiers such that for each
shard and vnode pair there exists a stream whose token falls into this
vnode and is owned by this shard.
It then builds a cdc::topology_description which maps tokens to these
found stream identifiers, such that if token T is owned by shard S in
vnode V, it gets mapped to the stream identifier generated for (S, V).
This is a class that will be used for storing information
required to perform CDC operations, i.e. assignment of token ranges to
CDC streams.
It is serializable to bytes and will be stored
in such a form in a distributed table accessible
by all nodes.
Allows calculating the shard of the given token using custom values of
shard_count and sharding_ignore_msb (instead of the ones used by the
particular partitioner instance).
Merged patch series from Piotr Sarna:
This series adds the following to alternator:
- TagResource request
- UntagResource request
- ListTagsOfResource request
- Honoring "Tags" parameter in CreateTable
It also provides more tests for above features and extended docs.
Tagging is backed by a schema extension, which is in turn backed
by entries in system_schema.tables.extensions map.
Tags are considered part of the schema, and in particular
they are updated via an equivalent of:
ALTER TABLE table WITH scylla_tags = {'key1':'v1', 'key2':'v2'}
Each tag change is therefore a schema change, which also means
that editing tags for the same table on different nodes may be
subject to races, until the schema agreement issues are resolved
in Scylla.
Fixes#5066
Tests: alternator-test(local, remote)
Piotr Sarna (6):
alternator,main: add tags schema extension
alternator: add creating values from string views
alternator: implement tagging
alternator: allow tagging on table creation
docs: add entries for alternator tags and arn
alternator-test: make test tables case sensitive
alternator-test/test_tag.py | 63 ++++++++++-
alternator-test/util.py | 2 +-
alternator/executor.cc | 191 ++++++++++++++++++++++++++++++++--
alternator/executor.hh | 3 +
alternator/rjson.cc | 4 +
alternator/rjson.hh | 1 +
alternator/server.cc | 3 +
alternator/tags_extension.hh | 52 +++++++++
docs/alternator/alternator.md | 14 ++-
main.cc | 5 +
10 files changed, 325 insertions(+), 13 deletions(-)
create mode 100644 alternator/tags_extension.hh
Attempting to apply timed-out writes is a wasted effort. The coordinator
have already given up on the write and reported it as failed to the
client. Any cycles spent on this write is a waste at this point.
We currently only check the timeout if the write is blocked on memory,
otherwise, if the system is not under pressure, we will happily apply
timed out writes. If the system is under pressure we will make it worse
by wasting cycles on processing a timed out write.
Prevent this by checking the timeout as early as possible in
`database::apply()` and `database::apply_counter_update()`.
This patch doesn't solve all our problems related to timed out writes.
They can still sit and accumulate in various queues without expiring, a
prominent example being the smp queues. It is however a good first step
towards reducing wasted effort spent on them.
Refs: #5055
Ref #5251
Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20200129093007.550250-1-bdenes@scylladb.com>
After 546556b71b we can have mixed writes into commitlog,
some do flush immediately some do not. If non flushing write races with
flushing one and becomes responsible for writing back its buffer into a
file flush will be skipped which will cause assert in batch_cycle() to
trigger since flush position will not be advanced. Fix that by checking
that flush was skipped and in this case flush explicitly our file
position.
Fixes#5670
Message-Id: <20200128145103.GI26048@scylladb.com>
During table creation, it's now possible to provide a 'Tags' parameter,
which will add tags to a newly created table. Note that creating a table
and tagging it is not atomic, so in case of failure it's possible to end
up with a created table, but without appropriate tags.
This commit comes with a test.
Message-Id: <00c2e202e9075d2c61e4ee5ba322ff4d5dbe718c.1579618972.git.sarna@scylladb.com>
A schema extension is introduced for alternator - tags.
This schema extension can be used to store arbitrary tags for a table,
in the form of a map<text, text>.
Updating tags for a table is equivalent to the following CQL query:
ALTER TABLE table WITH scylla_tags = {'key1':'v1', 'key2':'v2'}
The extension, as all other extensions, is backed by the entry
in the system_schema.tables table.
Add "const" attributes to `assignment_testable::test_assignment`
and `term::raw::prepare` methods. These should have been marked as
"const" even before the change but for some reason were missing
these qualifiers.
Mark other supplementary methods with "const" attributes as
necessary.
Tests: unit(dev, debug)
Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
Message-Id: <20200127213215.494000-1-pa.solodovnikov@scylladb.com>
"
Throw an error in case we hit an invalid time UUID
rather than hitting an assert.
Fixes#5552
(Ref #5588 that was dequeued and fixed here)
Test: UUID_test, cql_query_test(debug)
"
* 'validate-time-uuid' of https://github.com/bhalevy/scylla:
cql3: abstract_function_selector: provide assignment_testable_source_context
test: cql_query_test: add time uuid validation tests
cql3: time_uuid_fcts: validate timestamp arg
cql3: make_max_timeuuid_fct: delete outdated FIXME comment
cql3: time_uuid_fcts: validate time UUID
test: UUID_test: add tests for time uuid
utils: UUID: create_time assert nanos_since validity
utils/UUID_gen: make_nanos_since
utils: UUID: assert UUID.is_timestamp
"
This PR makes named_value respect allowed_values and then use it to transition away from old deprecated RandomPartitioner and ByteOrderedPartitioner. Then it removes the code that's no longer used.
We want to remove deprecated partitioners because, on one hand, they lead to performance problems and hot nodes. Moreover, we're planning to unify the token representation which would allow per table partitioner support. That, in turn, is a feature helpful in multiple efforts like CDC, materialized views, secondary indexes and multi-tenancy.
tests: unit(dev)
"
* 'remove_deprecated_partitioners' of https://github.com/haaawk/scylla:
partitioners: remove random_partitioner
partitioners: Make it impossible to use RandomPartitioner
partitioners: remove byte_ordered_partitioner
partitioners: Make it impossible to use ByteOrderedPartitioner
partitioners: Remove leftovers of OrderPreservingPartitioner
i_partitioner.cc: stop including byte_ordered_partitioner.hh
i_partitioner.cc: stop including random_partitioner.hh
config: use allowed_values to verify named_value input
config: add operator<< for seed_provider_type
Since we now default to lld if present, and since lld is a faster
linker than either ld or gold, it makes sense to install it
as a dependency and to make it available as part of the frozen
toolchain.
"
Grab the lowest hanging fruits.
This patch-set makes three important changes:
* Consume the memory for I/O operations on tracked files, *before* they
are forwarded to the underlying file.
* Track memory consumed by buffers created for parsing in
`continuous_data_consumer`. As this is the basis for the data, index
and promoted index parsers, all three are covered now in this regard.
* Track the index file.
The remaining, not-so-low handing fruits in order of
gain/cost(performance) ratio:
* Track in-memory index lists.
* Track in-memory promoted index blocks.
* Track reader buffer memory.
Note that this ordering might change based on the workload and other
environmental factors.
Also included in this series is an infrastructure refactoring to make
tracking memory easier and involve including lighter headers, as well as
a manual test designed to allow testing and experimenting with the
effects of changes to the accuracy of the tracking of reader memory
consumption.
Refs: #4176
Refs: #2778
Tests: unit(dev), manual(sstable_scan_footprint_test)
The latter was run as:
build/dev/test/manual/sstable_scan_footprint_test -c1 -m2G --reads=4000
--read-concurrency=1 --logger-log-level test=trace --collect-stats
--stats-period-ms=20
This will trickle reads until the semaphore blocks, then wait until the
wait queue drains before sending new reads. This way we are not testing
the effectiveness of the pre-admission estimation (which is terribly
optimistic) and instead check that with slowly ramping up read load the
semaphore will block on memory preventing OOM.
This now runs to completion without a single `std::bad_alloc`. The read
concurrency semaphore allows between 15-30 reads, and is always blocked
on memory.
"
* 'more-accurate-reader-resource-tracking/v1' of ssh://github.com/denesb/scylla:
test/manual/sstable_scan_footprint_test: improve memory consumption diagnostics
tests/manual/sstable_scan_footprint_test: use the semaphore to determine read rate
tests/manual: Add test measuring memory demand of concurrent sstable reads
index_reader: make the index file tracked
sstables/continuous_data_consumer: track buffers used for parsing
reader_concurrency_semaphore: tracking_file_impl: consume memory speculatively
reader_concurrency_semaphore: bye reader_resource_tracker
treewide: replace reader_resource_tracer with reader_permit
reader_permit: expose make_tracked_temporary_buffer()
reader_permit: introduce make_tracked_file()
reader_permit: introduce memory_units
reader_concurrency_semaphore: mv reader_resources and reader_permit to reader_permit.hh
reader_concurrency_semaphore: reader_permit: make it a value type
reader_concurrency_semaphore: s/resources/reader_resources/
reader_concurrency_semaphore::reader_permit: move methods out-of-line
In Scylla all query processing activity should run under the
"statement" scheduling group. The scheduling group is
important for maintaining the balance between background and
foreground tasks in Scylla.
Testing: In order to test the correctness of the patch.
First, the following assert was inserted before any call
to one of the executor functions in the http route:
assert(current_scheduling_group().name() == "statement"
Then all alternator tests ran and passed.
The second stage was to change the name so the assert
will fail:
assert(current_scheduling_group().name() == "no-statement"
And ran the tests again - validating that Scylla coredumps.
The asserts were then removed.
Fixes#5008
Signed-off-by: Eliran Sinvani <eliransin@scylladb.com>
Message-Id: <20200127154341.10020-1-eliransin@scylladb.com>
"
There seem to be two problems with handling snapshot API -- one
on start and the other one on stop. Here's the set that addresses
both.
The fix moved snapshot API registration later in time that required
Amnon's ACK. Now we have it :) so -- the rebase and resend.
Tests: unit(dev), start-stop
"
* 'br-snapshot-bugs-2' of https://github.com/xemul/scylla:
snapshot: Pass requests through gate
api: Register snapshot API later
api: Unwrap wrap_ks_cf
"
With these changes and a binutils compiled with
--enable-deterministic-archives, the only difference I get in the
build directory if I build scylla twice from scratch are:
* The various CMakeError.log because they have temporary file names.
* The various CMakeOutput.log for the same reason.
* .ninja_log and .ninja_deps. I am not sure what the contents are.
"
* 'espindola/fix-determinism' of https://github.com/espindola/scylla:
build: remove timestamps from then antlr output
build: Make the output of idl-compiler deterministic
This depends on the patch
mk: avoid combining -r and -export-dynamic linker options
being added to dpdk.
I benchmarked this on top of my patches to get a reproducible build. I
first compiled with ccache, deleted the build directory and recompiled
so that all the "gcc -c" invocations were served by ccache. The times
of the second "ninja release" invocations were:
lld:
ninja release 155.68s user 71.89s system 2077% cpu 10.953 total
gold:
ninja release 953.79s user 254.71s system 2533% cpu 47.699 total
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20200127171516.26268-1-espindola@scylladb.com>
> memory: add scoped_heap_profiling
> build: add switch to enable heap profiling support
> io_tester: do not abort on end of test
> resource: clean up cgroups version determination.
> prometheus: Silence a bogus gcc warning in http server
> Update dpdk submodule
> resource: Support cgroups v2
> net: Don't use variable length arrays
> core/memory.hh: document set_heap_profiling_enabled()
> Revert "net: Don't use variable length arrays"
> cmake: fix pkgconfig boost deps
> thread: Avoid confusing comment by switching value
> net: posix-stack: fix allocator in ap listening sockets
> net: posix-stack: fix passing allocator to new sockets
> stall_detector: Add a counter for stall detector report
> Merge "Don't use variable length arrays" from Rafael
> treewide: fix minor issues reported by clang
> thread: Call mprotect in make_stack
> thread: Always allocate stack with aligned_alloc
> build: Make SEASTAR_THREAD_STACK_GUARDS private
> thread: Move code out of a header
Merged patch series from Konstantin Osipov:
This series sets cql_repl core count to 1 and adds LWT
unit tests.
test.py: invoke cql_repl with smp=1
lwt: add lightweight transactions unit tests
Merged patch series from Piotr Sarna:
In order to minimize the usage of throws and catches in code paths
that are potentially hot, these paths instead return appropriate errors
directly.
The server layer is still able to catch and translate errors,
but the preferred way is to return api_error directly in places
that may be performance-sensitive.
Tests: alternator-test(local)
Fixes#5472
Piotr Sarna (3):
alternator: change request return type to variant<value, error>
alternator: elide throwing in condition checks
alternator: replace top-level throws with returns in executor
alternator/executor.hh | 28 ++++----
alternator/server.hh | 4 +-
alternator/executor.cc | 141 +++++++++++++++++++++--------------------
alternator/server.cc | 44 ++++++++-----
4 files changed, 117 insertions(+), 100 deletions(-)
Exclude cql_repl from the list of tests, since it's not a test.
Build it as a separate app. Do not strip, so that any CQL test
failure is easy to debug without a rebuild.
All test-related targets are converted from lists to sets to avoid
quadratic lookup cost in the check inside the loop which creates the
ninja file.
Currently --ami does not check instance types, creates invalid
io_properties.yaml on unsupported instance types.
It actually won't occur on AMI startup, since scylla_ami_setup only
invoke scylla_io_setup --ami when the instance is supported, so we don't
get the issue on startup, but we still get when we run scylla_io_setup
manually.
It's better to check instance type on scylla_io_setup, too.
Refs #5438
Conditional updates inform the user that the condition is not met
by returning an error. An initial implementation was based on rethrowing
these errors, but returning them directly is considered better
for performance.
This will prevent contention in case of parallel updates of the same row
by the same coordinator. The patch does it by introducing a new per key
lock map and taking it before running PAXOS protocol (either for write
of for read).
Message-Id: <20200117101228.GA14816@scylladb.com>
In order to minimize the use of exceptions during normal operations,
each request handler is now able to return either a proper JSON value,
or an instance of api_error, which indicates that something went wrong,
but without having to throw, catch and rethrow C++ exceptions.
This is especially important for conditional updates, since it's
expected to be common to return ConditionalCheckFailedException.
Message-Id: <d8996a0a270eb0d9db8fdcfb7046930b96781e69.1579515640.git.sarna@scylladb.com>
Docker restricts the number of processes in a container to some
limit it calculates. This limit turns out to be too low on large
machines, since we run multiple links in parallel, and each link
runs many threads.
Remove the limit by specifying --pids-limit -1. Since dbuild is
meant to provide a build environment, not a security barrier,
this is okay (the container is still restricted by host limits).
I checked that --pids-limit is supported by old versions of
docker and by podman.
Fixes#5651.
Message-Id: <20200127090807.3528561-1-avi@scylladb.com>
"
Final set of changes for full scan metrics.
- allow filtering
- full scan (Note: non-system tables only)
- full scan without BYPASS CACHE option
- tests for all metrics (bypass cache, allow filtering, full scan)
- works with prepared statements (tested, too)
"
* 'as_full_scan_metrics' of https://github.com/alecco/scylla:
Range scan query counter
Counter of queries doing full scan.
ALLOW FILTERING query counter
This test is all about tracking measured memory consumption vs. real
memory consumption. To make this easier add additional diagnostics:
* enable seastar heap profiler for the duration of the reads (seastar
has to be compiled with `-DSEASTAR_HEAPPROF`).
* Add a stats collector, which periodically collects stats such as
non-LSA free/used memory, LSA free/used memory and memory tracked by
the reader concurrency semaphore. These stats are written to a `.csv`
file, allowing importing them into a spreadsheet and processing them.
Currently the test fires the configured amount of reads at once. This is
somewhat restricting in the number of testable scenarios. For example,
it doesn't allow one to see if the semaphore correctly tracks the memory
consumption of existing reads, by firing new reads after a while.
Replace this algorithm by one which fires reads with a configured
concurrency, then waits for the semaphore's queue (if any) to drain,
before firing new reads. The test can now be configured with the total
amount of reads to fire, and with the read-concurrency, i.e. the number
of reads to fire at once in each iteration.
This allows for much greater flexibility in the different test
scenarios. The previous behaviour can still be achieved by configuring
a concurrency of 100.
This patch also adds better error handling. Reads are aborted on the
first error and errors are caught and not allowed to bubble up past the
test's main function and are logged instead.
Extensive logging is also added to be able to monitor the system while
the test is running.
Allow manual experimentation with the effectiveness of the accuracy of
the tracking of the resource consumption of readers, and hence the
system's ability to prevent overload and the dreaded `std::bad_alloc`.
This patch was originally developed by
Tomasz Grabiec <tgrabiec@scylladb.com>, I only adapted it to compile and
link on current master.
Based on heap profiling, buffers used for storing half-parsed fields are
a major contributor to the overall memory consumption of reads. This
memory was completely "under the radar" before. Track it by using
tracked `temporary_buffer` instances everywhere in
`continuous_data_consumer`. As `continuous_data_consumer` is the basis
for parsing all index and data files, adding the tracing here
automatically covers all data, index and promoted index parsing.
I'm almost convinced that there is a better place to store the `permit`
then the three places now, but so far I was unable to completely
decipher the our data/index file parsing class hierarchy.
Consume the memory before even submitting the I/O to the underlying
`file` object. This is in line with the underlying `file` object
allocating the buffer before it forwards the I/O request to the kernel.
This extends the "visibility" over the memory consumed by I/O greatly,
as it turns out buffers spend most time alive waiting for the I/O to
complete and are parsed shortly afterwards.
The former was never really more than a reader_permit with one
additional method. Currently using it doesn't even save one from any
includes. Now that readers will be using reader_permit we would have to
pass down both to mutation_source. Instead get rid of
reader_resource_tracker and just use reader_permit. Instead of making it
a last and optional parameter that is easy to ignore, make it a
first class parameter, right after schema, to signify that permits are
now a prominent part of the reader API.
This -- mostly mechanical -- patch essentially refactors mutation_source
to ask for the reader_permit instead of reader_resource_tracking and
updates all usage sites.
Previously `tracking_file_impl::make_tracked_buf()`. In the next patches
we plan on using this outside `tracking_file_impl`, so make it public
and templatize on the char type.
Similar to `seastar::semaphore_units`, this allows consuming and
releasing memory via an RAII object. In addition to that, it also allows
tracking changing values. This feature was designed to be used for
tracking the ever changing memory consumption of the buffers of
`flat_mutation_reader`:s.
This is now the only supported way of consuming memory from a permit.
