Add a function that checks if there is an index
which supports one of the columns present in
the given expression.
This functionality will soon be needed for
clustering and nonprimary columns so it's
good to separate into a reusable function.
Signed-off-by: Jan Ciolek <jan.ciolek@scylladb.com>
The code is copied from:
single_column_primary_key_restrictions<clustering_key>
::num_prefix_columns_that_need_not_be_filtered
Signed-off-by: Jan Ciolek <jan.ciolek@scylladb.com>
Instead of writing
_clustering_columns_restrictions->expression
It's better to use the new field:
_new_clustering_columns_restrictions
These expressions should be the same.
It removes another use of the unwanted restrictions field.
Signed-off-by: Jan Ciolek <jan.ciolek@scylladb.com>
Add a function which checks that an expression
contains only binary operators with '='.
Right now this check is done only in a single place,
but soon the same check will have to be done
for clustering columns as well, so the code
is moved to a separate function to prevent duplication.
Signed-off-by: Jan Ciolek <jan.ciolek@scylladb.com>
All occurences of _clustering_columns_restrictions->empty()
have been replaced with code that operates on the new
expression representation: _new_clustering_columns_restrictions.
Signed-off-by: Jan Ciolek <jan.ciolek@scylladb.com>
storage_service/keyspaces?type=user along with user keyspaces returned
the keyspaces that were internal but non-system.
The list of the keyspaces for the user option
(storage_service/keyspaces?type=user) contains neither system nor
internal but only user keyspaces.
Fixes: #11042Closes#11049
Remove some stale entries, add new entries for docs/.
Closes#11046
* github.com:scylladb/scylla:
CODEOWNERS: add owners for docs/
CODEOWNERS: remove @haaawk
User documentation was recently migrated to scylla.git, and this is
maintained by non scylla-core people. Add entries for docs/ so they are
notified when somebody submits changes to docs/.
These are cleanups needed for upcoming series that will make manager switch to table abstraction.
Closes#11037
* github.com:scylladb/scylla:
compaction_manager: remove unused variable in rewrite_sstable()
table: remove ref from on_compaction_completion() signature
table: use compaction_completion_desc to describe changes for off-strategy
compaction_manager: rename table_state's get_sstable_set to main_sstable_set
* seastar 7d8d846b26...6d4a0cb7a3 (18):
> io: Adjust IO latency goal on fair-queue level
Fixes#10927
> coroutine: exception: deprecate return_exception(exception_ptr)
> Merge "Make fair-queue class manipulations noexcept" from Pavel E
> lowres_timers: Put timeout to infinity if no timers armed
> util/conversion: support IEC prefix like "Ki"
> util/conversion: use string_view instead of string
> thread: fix backtrace termination for s390x on clang
> *: add fmt::ostream_formatter<> so {fmt} can use operator<<
> net: Remove operator<< for ipv4_addr
> rpc-impl: Log "caught exception" when catching exception
> rpc: Don't format non-trivial types with format specifier
> sharded: use std::invoke() to call mapper function
> Merge 'Avoid false-positive warnings in Gcc 12.1.1' from Nadav Har'El
> tls_test: Remove dns bottle neck + improve read loop in google connect test
> Revert "sstring: restore compatibility with std::string"
> sstring: restore compatibility with std::string
> tls_test: Make google https connect routine loop buffer reads
> coroutine: add buffer support to async generator
Closes#11033
Now update_sstable_lists_on_off_strategy_completion() and
on_compaction_completion() can be called from the same unified
interface.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
To make it possible to add a single interface in table_state for
updating sstable list on behalf of both off-strategy and in-strategy
compactions, update_sstable_lists_on_off_strategy_completion() will
work with compaction_completion_desc too for describing sstable set
changes.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
With compaction_manager switching to table_state, we'll need to
introduce a method in table_state to return maintenance set.
So better to have a descriptive name for main set.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
- bytes_ostream has a default initial chunk size of 512.
- let's say we call bytes_ostream::write() to write 500 bytes.
- as next_alloc_size() takes into account space to hold chunk metadata
(24 bytes) + chunk data, then 512 bytes is not enough, so it returns
500 + 24 instead to be allocated.
- when allocating next chunk, next_alloc_size() will use the size of
existing chunk, which is 500 bytes (without metadata) and multiply it
to 2 (growth factor), so 1000 bytes is allocated for it.
So allocations can be non power-of-two, resulting in memory waste.
When seastar is allocating from small pools, the waste is not terrible
(although accumulated small wastes can be problematic), but once
allocations pass the large threshold (16k), then alignment is 4k
(page size) and the waste is not negligible.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Closes#11027
cql-pytest contains subdirectories with tests
ported from Cassandra. It's desirable to preserve
the same layout and file names for these tests as in
the original source tree. To do that, add support for recursive
search of tests to PythonTestSuite. The log files for
the tests which are found recursively are created in subdirs
of the test tmpdir.
While implementing the feature, switch to using pathlib,
since a) it supports rglob (recursive glob) and b) it
was requested in one of the earlier reviews.
Closes#11018
This patch adds several more tests for Alternator's UpdateItem operation.
These tests verify a few simple cases that, surprisingly, never had test
coverage. The new tests pass (on both DynamoDB and Alternator) so did not
expose any bug.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Closes#11025
If the compaction_descriptor returned by `time_window_compaction_strategy::get_sstables_for_compaction`
is marked with `has_only_fully_expired::yes` it should always be compacted
since `time_window_compaction_strategy::get_sstables_for_compaction` is not idempotent.
It sets `_last_expired_check` and if compaction is postponed and retried before
`expired_sstable_check_frequency` has passed, it will not look for those fully-expired
sstables again. Plus, compacting them is the cheapest possible as it does not require
reading anything, just deleting the input sstables, so there's no reason not postpone it.
Also, extend `max_ongoing_compaction_test` to test serialization of compaction jobs with the same weight.
Fixes#10989Closes#10990
* github.com:scylladb/scylla:
compaction_manager: always register descriptor with fully expired sstables for compaction
test: max_ongoing_compaction_test: test serialization of regular compaction with same weight
test: max_ongoing_compaction_test: reindent refactored code
test: max_ongoing_compaction_test: define compact_all_tables lambda
test: max_ongoing_compaction_test: refactor make_table_with_single_fully_expired_sstable
test: max_ongoing_compaction_test: reduce number of tables
This series adds the infrastructure needed for testing user permissions, like the ability to create temporary roles and CQL sessions which log in as different users, and a few initial test cases for granting and revoking permissions.
Closes#10998
* github.com:scylladb/scylla:
cql-pytest: add a case for granting/revoking data permissions
cql-pytest: add new_user and new_session utils
cql-pytest: speed up permissions refresh period for tests
Said parameter is a convenience so downstream consumers of the
mutation compactors don't have to check the `bool is_live` already
passed to them. This convenience however causes a template parameter and
additional logic for the compactor. As the most prominent of these
consumers (the query result builder) will soon have to switch to
`emit_only_live_rows::no` for other reasons anyway (it will want to count
tombstones), we take the opportunity to switch everybody to ::no. This
can be done with very little additional complexity to these consumers --
basically an additional if or two. With everybody using the `::no` variant
of the compactor, we can remove this template parameter and the logic
associated with it altogether.
Closes#10931
* github.com:scylladb/scylla:
multishard_mutation_query: remove now pointless compact_for_result_state typedef
mutation_compactor: remove only-live related logic
mutation_compactor: remove emit_only_live_rows template parameter
mutation_compactor: remove unused compact_mutation_state::parameters
querier: remove {data,mutation}_querier aliases
querier: remove now pointless emit_only_live_rows template parameter
tree: use emit_only_live_rows::no
querier: querier_cache: de-override insert() methods
This is part of support installing executables from PIP package,
now we support installing executable from PIP package but it will
install under /opt/scylladb/python3/bin.
To call these commands without speciying full path, we also need to install
symlink to /usr/bin.
To do this, we need new list which specifies command name for symlink.
Closes#10748
This series includes an assortment of loosely related improvements developed for a recent investigation. The changes include:
* Fix broken `std_deque` wrapper.
* Make `scylla smp-queues` fast.
* Teach `scylla smp-queues` to filter for both sender CPU (`--from`) and receiver CPU (`--to`) or both.
* Teach `scylla smp-queues` to make histogram over content of the queues -- i.e. the type of tasks in the smp queues.
* Teach `scylla smp-queues` to filter for tasks belonging to a certain scheduling group.
* Teach `scylla task_histogram` to include only tasks in the histogram.
* Teach `scylla task_histogram` to filter for tasks belonging to a certain scheduling group.
* Teach `scylla-fiber` to walk in both directions.
And some refactoring.
Fixes: https://github.com/scylladb/scylla/issues/7059Closes#11019
* github.com:scylladb/scylla:
docs/dev/debugging.md: update continuation chain traversal guide
scylla-gdb.py: scylla fiber: walk continuation chain in both directions
scylla-gdb.py: scylla fiber: allow passing analyzed pointers to _probe_pointer()
scylla-gdb.py: scylla fiber: hoist preparatory code out of _walk()
scylla-gdb.py: scylla task_histogram: add --scheduling-groups option
scylla-gdb.py: scylla task_histogram: add --filter-tasks option
scylla-gdb.py: scylla task_histogram: use histogram class
scylla-gdb.py: scylla-fiber: extract symbol matching logic
scylla-gdb.py: histogram: add limit feature
scylla-gdb.py: histogram: handle formatting errors
scylla-gdb.py: intrusive_slist: avoid infinite recursion in __len__()
scylla-gdb.py: scylla smp-queues: add --scheduling-group option
scylla-gdb.py: scylla smp-queues: add --content switch
scylla-gdb.py: smp-queue: add filtering capability
scylla-gdb.py: make scylla smp-queues fast
scylla-gdb.py: fix disagreement between std_deque len() and iter()
If the compaction_descriptor returned by time_window_compaction_strategy::get_sstables_for_compaction
is marked with has_only_fully_expired::yes it should always be compacted
since time_window_compaction_strategy::get_sstables_for_compaction is not idempotent.
It sets _last_expired_check and if compaction is postponed and retried before
expired_sstable_check_frequency has passed, it will not look for those fully-expired
sstables again. Plus, compacting them is the cheapest possible as it does not require
reading anything, just deleting the input sstables, so there's no reason not postpone it.
Fixes#10989
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
To test both expired and non-expired sstables scenarios
we need to pass this helper function the expected number
of sstables before compaction and after compaction.
When compaction a set of fully-expired sstables,
we expect none to remain, while when the set of sstables
is not fully expired, we'll expect 1 output sstable
after compaction.
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
So we can use the lower-level build blocks to
test compaction serialization of both fully-expired
and non-fully-expired sstables scenarios in the following patches.
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
The command wasn't tested fully,
and when tested, started failing scylla-gdb test.
Before Raft, the list of connections to print in a single-node setup
was always empty, so a mistake in the gdb script command 'scylla netw'
didn't lead to a test failure.
With raft, there is always an RPC connection to self after initial
bootstrap, and the test begins to print connections (and fail, because
there is a bug in the printing code).
Fix that bug.
Closes#11012
Now that we use emit_only_live_rows::no everywhere we can remove this
template parameters. Only the template parameter is removed, the
internal logic around it is left in place (will be removed in a next
patch), by hard-wiring `only_live()`.
