Commit Graph

25 Commits

Author SHA1 Message Date
Aleksandra Martyniuk
881ac7a9af test: fix regular compaction tasks check
Since 6b87778 regular compaction tasks are removed from task manager
immediately after they are finished.

test_regular_compaction_task lists compaction tasks and then requests
their statuses. Only one regular compaction task is guaranteed to still
be running at that time, the rest of them may finish before their status
is requested and so it will no longer be in task manager, causing the test
to fail.

Fix statuses check to consider the possibility of a regular compaction
task being removed from task manager.

Fixes: #17776.
(cherry picked from commit 80c5eb4ecb)

Closes scylladb/scylladb#17810
2024-03-15 08:54:00 +02:00
Botond Dénes
6c625e8cd3 Merge '[Backport 5.4] tasks: compaction: drop regular compaction tasks after they are finished' from Aleksandra Martyniuk
Make compaction tasks internal. Drop all internal tasks without parents
immediately after they are done.

Fixes: https://github.com/scylladb/scylladb/issues/16735
Refs: https://github.com/scylladb/scylladb/issues/16694.

Closes scylladb/scylladb#16798

* github.com:scylladb/scylladb:
  compaction: make regular compaction tasks internal
  tasks: don't keep internal root tasks after they complete
2024-01-17 09:34:08 +02:00
Aleksandra Martyniuk
081a36e34f compaction: make regular compaction tasks internal
Regular compaction tasks are internal.

Adjust test_compaction_task accordingly: modify test_regular_compaction_task,
delete test_running_compaction_task_abort (relying on regular compaction)
which checks are already achived by test_not_created_compaction_task_abort.
Rename the latter.

(cherry picked from commit 6b87778ef2)
2024-01-16 11:15:41 +01:00
Benny Halevy
3ff8051532 api: add /storage_service/compact
For major compacting all tables in the database.
The advantage of this api is that `commitlog->force_new_active_segment`
happens only once in `database::flush_all_tables` rather than
once per keyspace (when `nodetool compact` translates to
a sequence of `/storage_service/keyspace_compaction` calls).

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
(cherry picked from commit b12b142232)
2024-01-12 15:57:39 +08:00
Benny Halevy
993e6997c0 api: compaction: add flush_memtables option
When flushing is done externally, e.g. by running
`nodetool flush` prior to `nodetool compact`,
flush_memtables=false can be passed to skip flushing
of tables right before they are major-compacted.

This is useful to prevent creation of small sstables
due to excessive memtable flushing.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
(cherry picked from commit 1fd85bd37b)
2024-01-12 15:57:39 +08:00
Aleksandra Martyniuk
bfa3a7325a test: extend test_compaction_task.py to cover compaction group tasks 2023-07-28 10:51:55 +02:00
Aleksandra Martyniuk
1decf86d71 compaction: change sstables compaction tasks type 2023-07-28 10:51:55 +02:00
Aleksandra Martyniuk
4fc4c2527c test: extend test_compaction_task.py to test resharding compaction 2023-07-19 17:19:10 +02:00
Aleksandra Martyniuk
7dbe624dee test: add regular compaction task test 2023-07-17 15:54:33 +02:00
Aleksandra Martyniuk
241b56b7b5 test: drain old compaction tasks from task manager
When running compaction task test on the same scylla instantion
other tests are run, some compaction tasks from other test cases may
be left in task manager. If they stay in memory long enough, they may
get unregistered during the compaction task test and cause bad_request
status.

Drain old compaction tasks before and after each test.

Fixes: #14584.

Closes #14585
2023-07-17 10:57:36 +03:00
Michał Chojnowski
b511d57fc8 Revert "Merge 'Compaction resharding tasks' from Aleksandra Martyniuk"
This reverts commit 2a58b4a39a, reversing
changes made to dd63169077.

After patch 87c8d63b7a,
table_resharding_compaction_task_impl::run() performs the forbidden
action of copying a lw_shared_ptr (_owned_ranges_ptr) on a remote shard,
which is a data race that can cause a use-after-free, typically manifesting
as allocator corruption.

Note: before the bad patch, this was avoided by copying the _contents_ of the
lw_shared_ptr into a new, local lw_shared_ptr.

Fixes #14475
Fixes #14618

Closes #14641
2023-07-11 19:11:37 +03:00
Aleksandra Martyniuk
bf3e0744c1 test: extend test_compaction_task.py to test resharding compaction 2023-06-28 11:43:12 +02:00
Aleksandra Martyniuk
b02a5fd184 test: extend test_compaction_task.py to test reshaping compaction 2023-06-23 16:22:53 +02:00
Aleksandra Martyniuk
8ad6f1f481 test: extend test_compaction_task.py
Extend test_compaction_task.py to test major compaction tasks
covering compaction group compaction.
2023-06-20 12:12:49 +02:00
Aleksandra Martyniuk
648cf4e748 test: use named variable for task tree depth 2023-06-20 12:12:49 +02:00
Aleksandra Martyniuk
3c5094dce8 test: put compaction task checks of the same type together
In test_compaction_task.py tests concerning the same type of compaction
are squashed together so that they are run synchronously and there is
no data race when the tests are run in parallel.
2023-06-07 09:49:42 +02:00
Aleksandra Martyniuk
94a2895874 test: filter tasks of given compaction type
In test_compaction_task.py the tasks are filtered by compaction type
so that each test case checks only the intended tasks.
2023-06-07 09:30:44 +02:00
Aleksandra Martyniuk
b325bf11bc test: extend compaction tasks test
Compaction task test checks whether child-parent relationship
in tasks tree is valid.
2023-05-31 14:59:24 +02:00
Aleksandra Martyniuk
f199ec5ec3 test: extend test_compaction_task.py to test scrub compaction 2023-05-09 11:15:26 +02:00
Aleksandra Martyniuk
e170fa1c99 test: extend test_compaction_task.py to test rewrite sstables compaction 2023-04-11 13:07:22 +02:00
Aleksandra Martyniuk
306d44568f test: extend test_compaction_task.py to test offstrategy compaction 2023-03-30 10:52:27 +02:00
Aleksandra Martyniuk
cb0e6d617a test: extend test_compaction_task.py to test cleanup compaction 2023-03-13 16:36:20 +01:00
Aleksandra Martyniuk
4808220729 test: extend test_compaction_task.py
test/rest_api/test_compaction_task.py is extended so that it checks
validity of major compaction run from column family api.
2023-03-10 15:01:22 +01:00
Aleksandra Martyniuk
24edcd27d4 test: extend major keyspace compaction tasks test 2023-03-01 18:56:31 +01:00
Aleksandra Martyniuk
e9f01c7cce test: add test for major keyspace compaction tasks 2023-02-23 15:48:25 +01:00