mirror of
https://github.com/scylladb/scylladb.git
synced 2026-05-29 19:21:01 +00:00
c8e2bf1596
Currently, applying a schema change on a replica works like this:
Collect all affected keyspaces from incoming mutations
Read current state of schema
Apply the mutations
Read new state of schema
The "Read ... state of schema" step reads all kinds of schema
objects. In particular, to read the "table" objects, it does the
following:
for every affected keyspace k:
read all mutations from system_schema.tables for k
extract all existing table names from those mutations
for every existing table:
read mutations from {tables, columns, indexes, view_virtual_columns, ...} for that table
As you can see, the number of reads performed is O(nr tables in a
keyspace), not O(nr tables in a change). This means that making a
sequence of schema changes, like adding a table, is quadratic.
Another aspect which magnifies this is that we don't read those tables
using a single scan, but issue individual queries for each table
separately.
This patch optimizes this by considering only affected tables when
reading schema for the purpose of diff calculation.
When mutations contain multi-table deletions, we still read the
set of tables, like before. This could be optimized by looking
at the database to get the list, but it's not part of the patch.
I tested this using a test case provided by Kamil (kbr-scylla@53fe154)
./test.py --mode debug test_many_schema_changes -s
The test bootstraps a cluster and then creates about 40 schema
changes. Then a new node is bootstrapped and replays those changes via
group0.
In debug mode, each change takes roughly 2s to process before the
patch, and 0.5s after the patch.
The whole replay is reduced to 56% of what was before:
Before (1m19s) :
INFO 2023-01-20 19:44:35,848 [shard 0] raft_group0 - setup_group0: ensuring that the cluster has fully upgraded to use Raft...
INFO 2023-01-20 19:45:54,844 [shard 0] raft_group0 - setup_group0: waiting for peers to synchronize state...
After (45s):
INFO 2023-01-20 22:02:51,869 [shard 0] raft_group0 - setup_group0: ensuring that the cluster has fully upgraded to use Raft...
INFO 2023-01-20 22:03:36,834 [shard 0] raft_group0 - setup_group0: waiting for peers to synchronize state...
Closes #12592
Closes #12592