This commit removes the upgrade guides
from ScyllaDB Open Source to Enterprise
for versions we no longer support.
In addition, it removes a link to
one of the removed pages from
the Troubleshooting section (the link is
redundant).
before this change, we rely on the default-generated fmt::formatter
created from operator<<, but fmt v10 dropped the default-generated
formatter.
in this change, we define a formatter for db::operation_type, and
remove their operator<<().
Refs #13245
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16832
this change should silence the warning like
```
/home/kefu/dev/scylladb/repair/repair.cc:222:23: error: comparison of integers of different signs: 'int' and 'size_type' (aka 'unsigned long') [-Werror,-Wsign-compare]
222 | for (int i = 0; i < all.size(); i++) {
| ~ ^ ~~~~~~~~~~
```
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16867
before this change, we rely on the default-generated fmt::formatter
created from operator<<, but fmt v10 dropped the default-generated
formatter.
in this change, we replace operator<< with format_as() for
unimplemented::cause, so that we don't rely on the deprecated behavior,
and neither do we create a fully blown fmt::formatter. as in
fmt v10, format_as() can be used in place of fmt::formatter,
while in fmt v9, format_as() is only allowed to return a integer.
so, to be future-proof, and to be simpler, format_as() is used.
we can even replace `format_as(c)` with `c`, once fmt v10 is
available in future.
Refs #13245
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16866
these unused includes were identified by clangd. see https://clangd.llvm.org/guides/include-cleaner#unused-include-warning for more details on the "Unused include" warning.
Closesscylladb/scylladb#16868
* github.com:scylladb/scylladb:
auth: do not include unused headers
locator: Handle replication factor of 0 for initial_tablets calculations
table: add_sstable_and_update_cache: trigger compaction only in compaction group
compaction_manager: perform_task_on_all_files: return early when there are no sstables to compact
compaction_manager: perform_cleanup: use compaction_manager::eligible_for_compaction
This short series prevents the creation of compaction tasks when we know in advance that they have nothing to do.
This is possible in the clean path by:
- improve the detection of candidates for cleanup by skipping sstables that require cleanup but are already being compacted
- checking that list of sstables selected for cleanup isn't empty before creating the cleanup task
For upgrade sstables, and generally when rewriting all sstable: launch the task only if the list off candidate sstables isn't empty.
For regular compaction, when triggered via `table::add_sstable_and_update_cache`, we currently trigger compaction (by calling `submit`) on all compaction groups while the sstable is added only to one of them.
Also, it is typically called for maintenance sstables that are awaiting offstrategy compaction, in which case we can skip calling `submit` entirely since the caller triggers offstrategy compaction at a later stage.
Refs scylladb/scylladb#15673
Refs scylladb/scylladb#16694Fixesscylladb/scylladb#16803Closesscylladb/scylladb#16808
* github.com:scylladb/scylladb:
table: add_sstable_and_update_cache: trigger compaction only in compaction group
compaction_manager: perform_task_on_all_files: return early when there are no sstables to compact
compaction_manager: perform_cleanup: use compaction_manager::eligible_for_compaction
When calculating per-DC tablets the formula is shards_in_dc / rf_in_dc,
but the denominator in it can be configured to be literally zero and the
division doesn't work.
Fix by assuming zero tablets for dcs with zero rf
fixes: #16844
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Closesscylladb/scylladb#16861
before this change, we always reference the return value of
`make_reader()`, and the return value's type `flat_mutation_reader_v2`
is movable, so we can just pass it by moving away from it.
in this change, instead of using a lambda, let's just have the
return value of it. simpler this way.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16835
Raft rebuild is broken because the session id is not set.
The following was seen when run rebuild
stream_session - [Stream #8cfca940-afc9-11ee-b6f1-30b8f78c1451]
stream_transfer_task: Fail to send to 127.0.70.1:0:
seastar::rpc::remote_verb_error (Session not found:
00000000-0000-0000-0000-000000000000)
with raft topology, e.g.,
scylla --enable-repair-based-node-ops 0 --consistent-cluster-management true --experimental-features consistent-topology-changes
Fix by setting the session id.
Fixes#16741Closesscylladb/scylladb#16814
The test verifies repair brings the missing rows to the owner.
- Shutdown part of the nodes in the cluster
- Insert data
- Start all nodees
- Run repair
- Shutdown part of the nodes
- Check all data is present
Since a given tablet belongs to a single shard on both repair master and repair
followers, row level repair code needs to be changed to work on a single
shard for a given tablet. In order to tell the repair followers which
shard to work on, a dst_cpu_id value is passed over rpc from the repair
master.
A helper to get the dst shard id on the repair follower.
If the repair master specifies the shard id for the follower, use it.
Otherwise, the follower chooses one itself.
If the joining node fails while handling the response from the
topology coordinator, it hangs even though it knows the join
operation has failed. Therefore, we ensure it shuts down in
this patch.
Additionally, we ensure that if the first join request response
was a rejection or the node failed while handling it, the
following acceptances by the (possibly different) coordinator
don't succeed. The node considers the join operation as failed.
We shouldn't add it to the cluster.
Fixesscylladb/scylladb#16333Closesscylladb/scylladb#16650
* github.com:scylladb/scylladb:
topology_coordinator: clarify warnings
raft topology: join: allow only the first response to be a succesful acceptance
storage_service: join_node_response_handler: fix indentation
raft topology: join: shut down a node on error in response handler
Service level controller updates itself in interval. However the interval time is hardcoded in main to 10 seconds and it leads to long sleeps in some of the tests.
This patch moves this value to `service_levels_interval_ms` command line option and sets this value to 0.5s in cql-pytest.
Closesscylladb/scylladb#16394
* github.com:scylladb/scylladb:
test:cql-pytest: change service levels intervals in tests
configure service levels interval
There is no need to trigger compaction in all compaction
groups when an sstable is added to only one of them.
And with that level of control, if the caller passes
sstables::offstrategy::yes, we know it will
trigger offstrategy compaction later on so there
is no need to trigger compaction at all
for this sstable at this time.
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
3b424e391b introduced a loop
in `perform_cleanup` that waits until all sstables that require
cleanup are cleaned up.
However, with f1bbf705f9,
an sstable that is_eligible_for_compaction (i.e. it
is not in staging, awaiting view update generation),
may already be compacted by e.g. regular compaction.
And so perform_cleanup should interrupt that
by calling try_perform_cleanup, since the latter
reevaluates `update_sstable_cleanup_state` with
compaction disabled - that stops ongoing compactions.
Refs scylladb/scylladb#15673
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Refs #16757
Allows waiting for all previous and pending segment deletes to finish.
Useful if a caller of `discard_completed_segments` (i.e. a memtable
flush target) not only wants to ensure segments are clean and released,
but thoroughly deleted/recycled, and hence no treat to resurrecting
data on crash+restart.
Test included.
Closesscylladb/scylladb#16801
before this change, we rely on the default-generated fmt::formatter
created from operator<<, but fmt v10 dropped the default-generated
formatter.
in this change, we define a formatter for auth::role_or_anonymous,
and remove their operator<<().
Refs #13245
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16812
Currently to figure out if a topology request is complete a submitter
checks the topology state and tries to figure out from that the status
of the request. This is not exact. Lets look at rebuild handling for
instance. To figure out if request is completed the code waits for
request object to disappear from the topology, but if another rebuild
starts between the end of the previous one and the code noticing that
it completed the code will continue waiting for the next rebuild.
Another problem is that in case of operation failure there is no way to
pass an error back to the initiator.
This series solves those problems by assigning an id for each request and
tracking the status of each request in a separate table. The initiator
can query the request status from the table and see if the request was
completed successfully or if it failed with an error, which is also
evadable from the table.
The schema for the table is:
CREATE TABLE system.topology_requests (
id timeuuid PRIMARY KEY,
initiating_host uuid,
start_time timestamp,
done boolean,
error text,
end_time timestamp,
);
and all entries have TTL of one month.
The sstables replay_position in stats_metadata is
valid only on the originating node and shard.
Therefore, validate the originating host and shard
before using it in compaction or table truncate.
Fixes#10080
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Closesscylladb/scylladb#16550
And call on_internal_error if process_upload_dir
is called for tablets-enabled keyspace as it isn't
supported at the moment (maybe it could be in the future
if we make sure that the sstables are confined to tablets
boundaries).
Refs #12775Fixes#16743
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Closesscylladb/scylladb#16788
Now that we have explicit status for each request we may use it to
replace shutdown notification rpc. During a decommission, in
left_token_ring state, we set done to true after metadata barrier
that waits for all request to the decommissioning node to complete
and notify the decommissioning node with a regular barrier. At this
point the node will see that the request is complete and exit.
Instead of trying to guess if a request completed by looking into the
topology state (which is sometimes can be error prone) look at the
request status in the new topology_requests. If request failed report
a reason for the failure from the table.
before this change, we rely on the default-generated fmt::formatter
created from operator<<, but fmt v10 dropped the default-generated
formatter.
in this change, we define a formatter for replica::memtable and
replica::memtable_entry, and remove their operator<<().
Refs #13245
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16793
