Commit Graph

36 Commits

Author SHA1 Message Date
Michał Chojnowski
b511d57fc8 Revert "Merge 'Compaction resharding tasks' from Aleksandra Martyniuk"
This reverts commit 2a58b4a39a, reversing
changes made to dd63169077.

After patch 87c8d63b7a,
table_resharding_compaction_task_impl::run() performs the forbidden
action of copying a lw_shared_ptr (_owned_ranges_ptr) on a remote shard,
which is a data race that can cause a use-after-free, typically manifesting
as allocator corruption.

Note: before the bad patch, this was avoided by copying the _contents_ of the
lw_shared_ptr into a new, local lw_shared_ptr.

Fixes #14475
Fixes #14618

Closes #14641
2023-07-11 19:11:37 +03:00
Aleksandra Martyniuk
87c8d63b7a compaction: add shard_reshard_sstables_compaction_task_impl
Add task manager's task covering resharding compaction on one shard.
2023-06-28 11:43:12 +02:00
Aleksandra Martyniuk
db6e4a356b compaction: invoke resharding on sharded database
In reshard_sstables_compaction_task_impl::run() we call
sharded<sstables::sstable_directory>::invoke_on_all. In lambda passed
to that method, we use both sharded sstable_directory service
and its local instance.

To make it straightforward that sharded and local instances are
dependend, we call sharded<replica::database>::invoke_on_all
instead and access local directory through the sharded one.
2023-06-28 11:43:12 +02:00
Aleksandra Martyniuk
1acaed026a compaction: move run_resharding_jobs into reshard_sstables_compaction_task_impl::run() 2023-06-28 11:43:11 +02:00
Aleksandra Martyniuk
837d77ba8c compaction: add reshard_sstables_compaction_task_impl
Add task manager's task covering resharding compaction.
2023-06-28 11:41:43 +02:00
Aleksandra Martyniuk
0d6dd3eeda compaction: replica: copy struct and functions from distributed_loader.cc
As a preparation for integrating resharding compaction with task manager
a struct and some functions are copied from replica/distributed_loader.cc
to compaction/task_manager_module.cc.
2023-06-28 11:41:42 +02:00
Raphael S. Carvalho
83c70ac04f utils: Extract pretty printers into a header
Can be easily reused elsewhere.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2023-06-26 21:58:20 -03:00
Aleksandra Martyniuk
f9a527b06d compaction: move reshape function to shard_reshaping_table_compaction_task_impl::run() 2023-06-23 16:22:53 +02:00
Aleksandra Martyniuk
1960904a72 compaction: add shard_reshaping_compaction_task_impl
shard_reshaping_compaction_task_impl covers reshaping compaction
on one shard.
2023-06-23 16:22:38 +02:00
Aleksandra Martyniuk
e3e2d6b886 compaction: add table_reshaping_compaction_task_impl 2023-06-23 15:57:37 +02:00
Aleksandra Martyniuk
dace5fb004 compaction: copy reshape to task_manager_module.cc
distributed_loader::reshape is copied to compaction/task_manager_module.cc
as it will be used in reshape compaction tasks.
2023-06-23 12:53:16 +02:00
Aleksandra Martyniuk
e317ffe23a compaction: extend signature of some methods
Extend a signature of table::compact_all_sstables and
compaction_manager::perform_major_compaction so that they get
the info of a covering task.

This allows to easily create child tasks that cover compaction group
compaction.
2023-06-20 10:45:34 +02:00
Aleksandra Martyniuk
fecdd75cd6 compaction: fix indentation 2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
53c24c0f7d compaction: create table_upgrade_sstables_compaction_task_impl
Implementation of task_manager's task that covers upgrade sstables
compaction of one table.
2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
143919cfa7 compaction: create table_offstrategy_keyspace_compaction_task_impl
Implementation of task_manager's task that covers offstrategy keyspace
compaction of one table.
2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
55ef1c24e1 compaction: create table_cleanup_keyspace_compaction_task_impl
Implementation of task_manager's task that covers cleanup keyspace
compaction of one table.
2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
5c7832ab59 compaction: create table_major_keyspace_compaction_task_impl
Implementation of task_manager's task that covers major keyspace
compaction of one table.
2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
d0c4028d64 compaction: add helpers for table tasks scheduling
In shard compaction tasks per table tasks will be created all at once
and then they will wait for their turn to run.

A function that allows waking up tasks one after another and a function
that makes the task wait for its turn are added.
2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
6dacc45c70 compaction: add run_on_table
Extract code which runs a function on a particular table from
run_on_existing_tables to run_on_table.
2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
5c65ac00ef compaction: pass std::string to run_on_existing_tables
Keyspace argument passed to run_on_existing_tables has its type
changed from std::string_view to std::string.
2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
f48b57e7b9 compaction: use table_info in compaction tasks
Task manager compaction tasks need table names for logs.
Thus, compaction tasks store table infos instead of table ids.

get_table_ids function is deleted as it isn't used anywhere.
2023-05-30 09:58:55 +02:00
Aleksandra Martyniuk
83d3463d10 compaction: add table_scrub_sstables_compaction_task_impl
Implementation of task_manager's task covering scrub sstables
compaction of one table.
2023-05-09 11:15:25 +02:00
Aleksandra Martyniuk
d8e4a2fee3 compaction: add shard_scrub_sstables_compaction_task_impl
Implementation of task_manager's task covering scrub sstables
compaction on one shard.
2023-05-09 11:14:36 +02:00
Aleksandra Martyniuk
8d32579fe6 compaction: add scrub_sstables_compaction_task_impl
Implementation of task_manager's task covering scrub sstables
compaction.
2023-05-09 11:13:57 +02:00
Pavel Emelyanov
5e201b9120 database: Remove compaction_manager.hh inclusion into database.hh
The only reason why it's there (right next to compaction_fwd.hh) is
because the database::table_truncate_state subclass needs the definition
of compaction_manager::compaction_reenabler subclass.

However, the former sub is not used outside of database.cc and can be
defined in .cc. Keeping it outside of the header allows dropping the
compaction_manager.hh from database.hh thus greatly reducing its fanout
over the code (from ~180 indirect inclusions down to ~20).

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>

Closes #13622
2023-04-23 16:27:11 +03:00
Botond Dénes
525b21042f Merge 'Rewrite sstables keyspace compaction task' from Aleksandra Martyniuk
Task manager task implementations of classes that cover
rewrite sstables keyspace compaction which can be start
through /storage_service/keyspace_compaction/ api.

Top level task covers the whole compaction and creates child
tasks on each shard.

Closes #12714

* github.com:scylladb/scylladb:
  test: extend test_compaction_task.py to test rewrite sstables compaction
  compaction: create task manager's task for rewrite sstables keyspace compaction on one shard
  compaction: create task manager's task for rewrite sstables keyspace compaction
  compaction: create rewrite_sstables_compaction_task_impl
2023-04-12 08:38:59 +03:00
Aleksandra Martyniuk
25cfffc3ae compaction: rename local_offstrategy_keyspace_compaction_task_impl to shard_offstrategy_keyspace_compaction_task_impl
Closes #13475
2023-04-12 08:38:25 +03:00
Aleksandra Martyniuk
a93f044efa compaction: create task manager's task for rewrite sstables keyspace compaction on one shard
Implementation of task_manager's task that covers rewrite sstables keyspace
compaction on one shard.
2023-04-11 13:07:17 +02:00
Aleksandra Martyniuk
c4098df4ec compaction: create task manager's task for rewrite sstables keyspace compaction
Implementation of task_manager's task covering rewrite sstables keyspace
compaction that can be started through storage_service api.
2023-04-11 11:04:21 +02:00
Aleksandra Martyniuk
8afa54d4f6 compaction: create task manager's task for offstrategy keyspace compaction on one shard
Implementation of task_manager's task that covers local offstrategy keyspace compaction.
2023-03-30 10:49:09 +02:00
Aleksandra Martyniuk
73860b7c9d compaction: create task manager's task for offstrategy keyspace compaction
Implementation of task_manager's task covering offstrategy keyspace compaction
that can be started through storage_service api.
2023-03-30 10:44:56 +02:00
Aleksandra Martyniuk
27b999808f compaction: create task manager's task for cleanup keyspace compaction on one shard
Implementation of task_manager's task that covers cleanup keyspace compaction
on one shard.
2023-03-13 16:35:39 +01:00
Aleksandra Martyniuk
7dd27205f6 compaction: create task manager's task for cleanup keyspace compaction
Implementation of task_manager's task covering cleanup keyspace compaction
that can be started through storage_service api.
2023-03-13 16:35:39 +01:00
Aleksandra Martyniuk
b188060535 compaction: create task manager's task for major keyspace compaction on one shard
Implementation of task_manager's task that covers major keyspace compaction
on one shard.
2023-03-01 18:56:26 +01:00
Aleksandra Martyniuk
159e603ac4 compaction: create task manager's task for major keyspace compaction
Implementation of task_manager's task covering major keyspace compaction
that can be started through storage_service api.
2023-02-23 15:48:05 +01:00
Aleksandra Martyniuk
6b1d7f5979 compaction: copy run_on_existing_tables to task_manager_module.cc
Copy run_on_existing_tables from api/storage_service.cc to
compaction/task_manager_module.cc
2023-02-23 15:31:59 +01:00