mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
afcf17f168c5600a5a4e7ece8126da64370e6021
scylladb/test/boost/schema_change_test.cc
Kamil Braun 30cc07b40d Merge 'Introduce tablets' from Tomasz Grabiec 
This PR introduces an experimental feature called "tablets". Tablets are
a way to distribute data in the cluster, which is an alternative to the
current vnode-based replication. Vnode-based replication strategy tries
to evenly distribute the global token space shared by all tables among
nodes and shards. With tablets, the aim is to start from a different
side. Divide resources of replica-shard into tablets, with a goal of
having a fixed target tablet size, and then assign those tablets to
serve fragments of tables (also called tablets). This will allow us to
balance the load in a more flexible manner, by moving individual tablets
around. Also, unlike with vnode ranges, tablet replicas live on a
particular shard on a given node, which will allow us to bind raft
groups to tablets. Those goals are not yet achieved with this PR, but it
lays the ground for this.

Things achieved in this PR:

  - You can start a cluster and create a keyspace whose tables will use
    tablet-based replication. This is done by setting `initial_tablets`
    option:

    ```
        CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
                        'replication_factor': 3,
                        'initial_tablets': 8};
    ```

    All tables created in such a keyspace will be tablet-based.

    Tablet-based replication is a trait, not a separate replication
    strategy. Tablets don't change the spirit of replication strategy, it
    just alters the way in which data ownership is managed. In theory, we
    could use it for other strategies as well like
    EverywhereReplicationStrategy. Currently, only NetworkTopologyStrategy
    is augmented to support tablets.

  - You can create and drop tablet-based tables (no DDL language changes)

  - DML / DQL work with tablet-based tables

    Replicas for tablet-based tables are chosen from tablet metadata
    instead of token metadata

Things which are not yet implemented:

  - handling of views, indexes, CDC created on tablet-based tables
  - sharding is done using the old method, it ignores the shard allocated in tablet metadata
  - node operations (topology changes, repair, rebuild) are not handling tablet-based tables
  - not integrated with compaction groups
  - tablet allocator piggy-backs on tokens to choose replicas.
    Eventually we want to allocate based on current load, not statically

Closes #13387

* github.com:scylladb/scylladb:
  test: topology: Introduce test_tablets.py
  raft: Introduce 'raft_server_force_snapshot' error injection
  locator: network_topology_strategy: Support tablet replication
  service: Introduce tablet_allocator
  locator: Introduce tablet_aware_replication_strategy
  locator: Extract maybe_remove_node_being_replaced()
  dht: token_metadata: Introduce get_my_id()
  migration_manager: Send tablet metadata as part of schema pull
  storage_service: Load tablet metadata when reloading topology state
  storage_service: Load tablet metadata on boot and from group0 changes
  db, migration_manager: Notify about tablet metadata changes via migration_listener::on_update_tablet_metadata()
  migration_notifier: Introduce before_drop_keyspace()
  migration_manager: Make prepare_keyspace_drop_announcement() return a future<>
  test: perf: Introduce perf-tablets
  test: Introduce tablets_test
  test: lib: Do not override table id in create_table()
  utils, tablets: Introduce external_memory_usage()
  db: tablets: Add printers
  db: tablets: Add persistence layer
  dht: Use last_token_of_compaction_group() in split_token_range_msb()
  locator: Introduce tablet_metadata
  dht: Introduce first_token()
  dht: Introduce next_token()
  storage_proxy: Improve trace-level logging
  locator: token_metadata: Fix confusing comment on ring_range()
  dht, storage_proxy: Abstract token space splitting
  Revert "query_ranges_to_vnodes_generator: fix for exclusive boundaries"
  db: Exclude keyspace with per-table replication in get_non_local_strategy_keyspaces_erms()
  db: Introduce get_non_local_vnode_based_strategy_keyspaces()
  service: storage_proxy: Avoid copying keyspace name in write handler
  locator: Introduce per-table replication strategy
  treewide: Use replication_strategy_ptr as a shorter name for abstract_replication_strategy::ptr_type
  locator: Introduce effective_replication_map
  locator: Rename effective_replication_map to vnode_effective_replication_map
  locator: effective_replication_map: Abstract get_pending_endpoints()
  db: Propagate feature_service to abstract_replication_strategy::validate_options()
  db: config: Introduce experimental "TABLETS" feature
  db: Log replication strategy for debugging purposes
  db: Log full exception on error in do_parse_schema_tables()
  db: keyspace: Remove non-const replication strategy getter
  config: Reformat
2023-04-27 09:40:18 +02:00

1024 lines
50 KiB
C++
Raw Blame History

/*
* Copyright (C) 2015-present ScyllaDB
*/
/*
* SPDX-License-Identifier: AGPL-3.0-or-later
*/
#include <iostream>
#include <seastar/core/thread.hh>
#include "test/lib/scylla_test_case.hh"
#include <seastar/util/defer.hh>
#include "test/lib/cql_test_env.hh"
#include "test/lib/cql_assertions.hh"
#include "test/lib/mutation_source_test.hh"
#include "test/lib/result_set_assertions.hh"
#include "service/migration_manager.hh"
#include "service/storage_proxy.hh"
#include "schema/schema_builder.hh"
#include "schema/schema_registry.hh"
#include "db/schema_tables.hh"
#include "types/list.hh"
#include "types/user.hh"
#include "db/config.hh"
#include "test/lib/tmpdir.hh"
#include "test/lib/exception_utils.hh"
#include "test/lib/log.hh"
#include "serializer_impl.hh"
#include "cdc/cdc_extension.hh"
#include "utils/UUID_gen.hh"
SEASTAR_TEST_CASE(test_new_schema_with_no_structural_change_is_propagated) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
auto partial = schema_builder("tests", "table")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type);
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
auto old_schema = partial.build();
auto& mm = e.migration_manager().local();
{
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
mm.announce(mm.prepare_new_column_family_announcement(old_schema, ts).get(), std::move(group0_guard)).get();
}
auto old_table_version = e.db().local().find_schema(old_schema->id())->version();
auto old_node_version = e.db().local().get_version();
auto new_schema = partial.build();
BOOST_REQUIRE_NE(new_schema->version(), old_schema->version());
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
mm.announce(mm.prepare_column_family_update_announcement(new_schema, false, std::vector<view_ptr>(), ts).get(), std::move(group0_guard)).get();
BOOST_REQUIRE_NE(e.db().local().find_schema(old_schema->id())->version(), old_table_version);
BOOST_REQUIRE_NE(e.db().local().get_version(), old_node_version);
});
});
}
SEASTAR_TEST_CASE(test_schema_is_updated_in_keyspace) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
auto builder = schema_builder("tests", "table")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type);
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
auto old_schema = builder.build();
auto& mm = e.migration_manager().local();
{
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
mm.announce(mm.prepare_new_column_family_announcement(old_schema, ts).get(), std::move(group0_guard)).get();
}
auto s = e.local_db().find_schema(old_schema->id());
BOOST_REQUIRE_EQUAL(*old_schema, *s);
BOOST_REQUIRE_EQUAL(864000, s->gc_grace_seconds().count());
BOOST_REQUIRE_EQUAL(*s, *e.local_db().find_keyspace(s->ks_name()).metadata()->cf_meta_data().at(s->cf_name()));
builder.set_gc_grace_seconds(1);
auto new_schema = builder.build();
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
mm.announce(mm.prepare_column_family_update_announcement(new_schema, false, std::vector<view_ptr>(), ts).get(), std::move(group0_guard)).get();
s = e.local_db().find_schema(old_schema->id());
BOOST_REQUIRE_NE(*old_schema, *s);
BOOST_REQUIRE_EQUAL(*new_schema, *s);
BOOST_REQUIRE_EQUAL(1, s->gc_grace_seconds().count());
BOOST_REQUIRE_EQUAL(*s, *e.local_db().find_keyspace(s->ks_name()).metadata()->cf_meta_data().at(s->cf_name()));
});
});
}
SEASTAR_TEST_CASE(test_tombstones_are_ignored_in_version_calculation) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
auto table_schema = schema_builder("ks", "table")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.build();
auto& mm = e.migration_manager().local();
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
mm.announce(mm.prepare_new_column_family_announcement(table_schema, ts).get(), std::move(group0_guard)).get();
auto old_table_version = e.db().local().find_schema(table_schema->id())->version();
auto old_node_version = e.db().local().get_version();
{
testlog.info("Applying a no-op tombstone to v1 column definition");
auto s = db::schema_tables::columns();
auto pkey = partition_key::from_singular(*s, table_schema->ks_name());
mutation m(s, pkey);
auto ckey = clustering_key::from_exploded(*s, {utf8_type->decompose(table_schema->cf_name()), "v1"});
m.partition().apply_delete(*s, ckey, tombstone(api::min_timestamp, gc_clock::now()));
mm.announce(std::vector<mutation>({m}), mm.start_group0_operation().get()).get();
}
auto new_table_version = e.db().local().find_schema(table_schema->id())->version();
auto new_node_version = e.db().local().get_version();
BOOST_REQUIRE_EQUAL(new_table_version, old_table_version);
BOOST_REQUIRE_EQUAL(new_node_version, old_node_version);
});
});
}
SEASTAR_TEST_CASE(test_concurrent_column_addition) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
service::migration_manager& mm = e.migration_manager().local();
auto s0 = schema_builder("ks", "table")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.build();
auto s1 = schema_builder("ks", "table")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.with_column("v3", bytes_type)
.build();
auto s2 = schema_builder("ks", "table", std::make_optional(s1->id()))
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.with_column("v2", bytes_type)
.build();
{
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
mm.announce(mm.prepare_new_column_family_announcement(s1, ts).get(), std::move(group0_guard)).get();
}
auto old_version = e.db().local().find_schema(s1->id())->version();
// Apply s0 -> s2 change.
{
auto group0_guard = mm.start_group0_operation().get();
auto&& keyspace = e.db().local().find_keyspace(s0->ks_name()).metadata();
auto muts = db::schema_tables::make_update_table_mutations(e.db().local(), keyspace, s0, s2,
group0_guard.write_timestamp(), false);
mm.announce(std::move(muts), std::move(group0_guard)).get();
}
auto new_schema = e.db().local().find_schema(s1->id());
BOOST_REQUIRE(new_schema->get_column_definition(to_bytes("v1")) != nullptr);
BOOST_REQUIRE(new_schema->get_column_definition(to_bytes("v2")) != nullptr);
BOOST_REQUIRE(new_schema->get_column_definition(to_bytes("v3")) != nullptr);
BOOST_REQUIRE(new_schema->version() != old_version);
BOOST_REQUIRE(new_schema->version() != s2->version());
});
});
}
SEASTAR_TEST_CASE(test_sort_type_in_update) {
return do_with_cql_env_thread([](cql_test_env& e) {
service::migration_manager& mm = e.migration_manager().local();
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
auto&& keyspace = e.db().local().find_keyspace("ks").metadata();
auto type1 = user_type_impl::get_instance("ks", to_bytes("type1"), {}, {}, true);
auto muts1 = db::schema_tables::make_create_type_mutations(keyspace, type1, ts);
auto type3 = user_type_impl::get_instance("ks", to_bytes("type3"), {}, {}, true);
auto muts3 = db::schema_tables::make_create_type_mutations(keyspace, type3, ts);
// type2 must be created after type1 and type3. This tests that announce sorts them.
auto type2 = user_type_impl::get_instance("ks", to_bytes("type2"), {"field1", "field3"}, {type1, type3}, true);
auto muts2 = db::schema_tables::make_create_type_mutations(keyspace, type2, ts);
auto muts = muts2;
muts.insert(muts.end(), muts1.begin(), muts1.end());
muts.insert(muts.end(), muts3.begin(), muts3.end());
mm.announce(std::move(muts), std::move(group0_guard)).get();
});
}
SEASTAR_TEST_CASE(test_column_is_dropped) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
e.execute_cql("create table tests.table1 (pk int primary key, c1 int, c2 int);").get();
e.execute_cql("alter table tests.table1 drop c2;").get();
e.execute_cql("alter table tests.table1 add s1 int;").get();
schema_ptr s = e.db().local().find_schema("tests", "table1");
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("c1")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c2")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s1")));
});
});
}
SEASTAR_TEST_CASE(test_static_column_is_dropped) {
return do_with_cql_env_thread([](cql_test_env& e) {
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
e.execute_cql("create table tests.table1 (pk int, c1 int, c2 int static, primary key (pk, c1));").get();
e.execute_cql("alter table tests.table1 drop c2;").get();
e.execute_cql("alter table tests.table1 add s1 int static;").get();
schema_ptr s = e.db().local().find_schema("tests", "table1");
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("c1")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c2")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s1")));
e.execute_cql("alter table tests.table1 drop s1;").get();
s = e.db().local().find_schema("tests", "table1");
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("c1")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c2")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("s1")));
});
}
SEASTAR_TEST_CASE(test_multiple_columns_add_and_drop) {
return do_with_cql_env_thread([](cql_test_env& e) {
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
e.execute_cql("create table tests.table1 (pk int primary key, c1 int, c2 int, c3 int);").get();
e.execute_cql("alter table tests.table1 drop (c2);").get();
e.execute_cql("alter table tests.table1 add (s1 int);").get();
schema_ptr s = e.db().local().find_schema("tests", "table1");
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("c1")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c2")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("c3")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s1")));
e.execute_cql("alter table tests.table1 drop (c1, c3);").get();
e.execute_cql("alter table tests.table1 add (s2 int, s3 int);").get();
s = e.db().local().find_schema("tests", "table1");
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c1")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c2")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c3")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s1")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s2")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s3")));
});
}
SEASTAR_TEST_CASE(test_multiple_static_columns_add_and_drop) {
return do_with_cql_env_thread([](cql_test_env& e) {
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
e.execute_cql("create table tests.table1 (pk int, c1 int, c2 int static, c3 int, primary key(pk, c1));").get();
e.execute_cql("alter table tests.table1 drop (c2);").get();
e.execute_cql("alter table tests.table1 add (s1 int static);").get();
schema_ptr s = e.db().local().find_schema("tests", "table1");
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("c1")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c2")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("c3")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s1")));
e.execute_cql("alter table tests.table1 drop (c3, s1);").get();
e.execute_cql("alter table tests.table1 add (s2 int, s3 int static);").get();
s = e.db().local().find_schema("tests", "table1");
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("c1")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c2")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("c3")));
BOOST_REQUIRE(!s->columns_by_name().contains(to_bytes("s1")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s2")));
BOOST_REQUIRE(s->columns_by_name().contains(to_bytes("s3")));
});
}
SEASTAR_TEST_CASE(test_combined_column_add_and_drop) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
service::migration_manager& mm = e.migration_manager().local();
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
auto s1 = schema_builder("ks", "table1")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.build();
{
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
mm.announce(mm.prepare_new_column_family_announcement(s1, ts).get(), std::move(group0_guard)).get();
}
auto&& keyspace = e.db().local().find_keyspace(s1->ks_name()).metadata();
auto s2 = schema_builder("ks", "table1", std::make_optional(s1->id()))
.with_column("pk", bytes_type, column_kind::partition_key)
.without_column("v1", bytes_type, api::new_timestamp())
.build();
// Drop v1
{
auto group0_guard = mm.start_group0_operation().get();
auto muts = db::schema_tables::make_update_table_mutations(e.db().local(), keyspace, s1, s2,
group0_guard.write_timestamp(), false);
mm.announce(std::move(muts), std::move(group0_guard)).get();
}
// Add a new v1 and drop it
{
auto s3 = schema_builder("ks", "table1", std::make_optional(s1->id()))
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", list_type_impl::get_instance(int32_type, true))
.build();
auto s4 = schema_builder("ks", "table1", std::make_optional(s1->id()))
.with_column("pk", bytes_type, column_kind::partition_key)
.without_column("v1", list_type_impl::get_instance(int32_type, true), api::new_timestamp())
.build();
auto group0_guard = mm.start_group0_operation().get();
auto muts = db::schema_tables::make_update_table_mutations(e.db().local(), keyspace, s3, s4,
group0_guard.write_timestamp(), false);
mm.announce(std::move(muts), std::move(group0_guard)).get();
}
auto new_schema = e.db().local().find_schema(s1->id());
BOOST_REQUIRE(new_schema->get_column_definition(to_bytes("v1")) == nullptr);
assert_that_failed(e.execute_cql("alter table ks.table1 add v1 list<text>;"));
});
});
}
// Tests behavior when incompatible CREATE TABLE statements are
// issued concurrently in the cluster.
// Relevant only for the pre-raft schema management, with raft the scenario should not happen.
SEASTAR_TEST_CASE(test_concurrent_table_creation_with_different_schema) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
service::migration_manager& mm = e.migration_manager().local();
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
auto s1 = schema_builder("ks", "table1")
.with_column("pk1", bytes_type, column_kind::partition_key)
.with_column("pk2", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.build();
auto s2 = schema_builder("ks", "table1")
.with_column("pk1", bytes_type, column_kind::partition_key)
.with_column("pk3", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.build();
auto ann1 = mm.prepare_new_column_family_announcement(s1, api::new_timestamp()).get();
auto ann2 = mm.prepare_new_column_family_announcement(s2, api::new_timestamp()).get();
{
auto group0_guard = mm.start_group0_operation().get();
mm.announce(std::move(ann1), std::move(group0_guard)).get();
}
{
auto group0_guard = mm.start_group0_operation().get();
mm.announce(std::move(ann2), std::move(group0_guard)).get();
}
auto&& keyspace = e.db().local().find_keyspace(s1->ks_name()).metadata();
// s2 should win because it has higher timestamp
auto new_schema = e.db().local().find_schema(s2->id());
BOOST_REQUIRE(*new_schema == *s2);
});
});
}
SEASTAR_TEST_CASE(test_merging_does_not_alter_tables_which_didnt_change) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
service::migration_manager& mm = e.migration_manager().local();
auto&& keyspace = e.db().local().find_keyspace("ks").metadata();
auto s0 = schema_builder("ks", "table1")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.build();
auto find_table = [&] () -> replica::column_family& {
return e.db().local().find_column_family("ks", "table1");
};
std::vector<mutation> muts1;
{
auto group0_guard = mm.start_group0_operation().get();
muts1 = db::schema_tables::make_create_table_mutations(s0, group0_guard.write_timestamp());
mm.announce(muts1, std::move(group0_guard)).get();
}
auto s1 = find_table().schema();
auto legacy_version = s1->version();
{
auto group0_guard = mm.start_group0_operation().get();
mm.announce(muts1, std::move(group0_guard)).get();
}
BOOST_REQUIRE(s1 == find_table().schema());
BOOST_REQUIRE_EQUAL(legacy_version, find_table().schema()->version());
{
auto group0_guard = mm.start_group0_operation().get();
auto muts2 = muts1;
muts2.push_back(db::schema_tables::make_scylla_tables_mutation(s0, group0_guard.write_timestamp()));
mm.announce(muts2, std::move(group0_guard)).get();
}
BOOST_REQUIRE(s1 == find_table().schema());
BOOST_REQUIRE_EQUAL(legacy_version, find_table().schema()->version());
});
});
}
SEASTAR_TEST_CASE(test_merging_creates_a_table_even_if_keyspace_was_recreated) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
service::migration_manager& mm = e.migration_manager().local();
auto&& keyspace = e.db().local().find_keyspace("ks").metadata();
auto s0 = schema_builder("ks", "table1")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("v1", bytes_type)
.build();
auto find_table = [&] () -> replica::column_family& {
return e.db().local().find_column_family("ks", "table1");
};
std::vector<mutation> all_muts;
{
auto group0_guard = mm.start_group0_operation().get();
const auto ts = group0_guard.write_timestamp();
auto muts = e.migration_manager().local().prepare_keyspace_drop_announcement("ks", ts).get0();
boost::copy(muts, std::back_inserter(all_muts));
mm.announce(muts, std::move(group0_guard)).get();
}
{
auto group0_guard = mm.start_group0_operation().get();
const auto ts = group0_guard.write_timestamp();
// all_muts contains keyspace drop.
auto muts = e.migration_manager().local().prepare_new_keyspace_announcement(keyspace, ts);
boost::copy(muts, std::back_inserter(all_muts));
mm.announce(muts, std::move(group0_guard)).get();
}
{
auto group0_guard = mm.start_group0_operation().get();
const auto ts = group0_guard.write_timestamp();
auto muts = e.migration_manager().local().prepare_new_column_family_announcement(s0, ts).get0();
boost::copy(muts, std::back_inserter(all_muts));
mm.announce(all_muts, std::move(group0_guard)).get();
}
auto s1 = find_table().schema();
BOOST_REQUIRE(s1 == find_table().schema());
});
});
}
class counting_migration_listener : public service::migration_listener {
public:
int create_keyspace_count = 0;
int create_column_family_count = 0;
int create_user_type_count = 0;
int create_function_count = 0;
int create_aggregate_count = 0;
int create_view_count = 0;
int update_keyspace_count = 0;
int update_column_family_count = 0;
int columns_changed_count = 0;
int update_user_type_count = 0;
int update_function_count = 0;
int update_aggregate_count = 0;
int update_view_count = 0;
int update_tablets = 0;
int drop_keyspace_count = 0;
int drop_column_family_count = 0;
int drop_user_type_count = 0;
int drop_function_count = 0;
int drop_aggregate_count = 0;
int drop_view_count = 0;
public:
virtual void on_create_keyspace(const sstring&) override { ++create_keyspace_count; }
virtual void on_create_column_family(const sstring&, const sstring&) override { ++create_column_family_count; }
virtual void on_create_user_type(const sstring&, const sstring&) override { ++create_user_type_count; }
virtual void on_create_function(const sstring&, const sstring&) override { ++create_function_count; }
virtual void on_create_aggregate(const sstring&, const sstring&) override { ++create_aggregate_count; }
virtual void on_create_view(const sstring&, const sstring&) override { ++create_view_count; }
virtual void on_update_keyspace(const sstring&) override { ++update_keyspace_count; }
virtual void on_update_column_family(const sstring&, const sstring&, bool columns_changed) override {
++update_column_family_count;
columns_changed_count += int(columns_changed);
}
virtual void on_update_user_type(const sstring&, const sstring&) override { ++update_user_type_count; }
virtual void on_update_function(const sstring&, const sstring&) override { ++update_function_count; }
virtual void on_update_aggregate(const sstring&, const sstring&) override { ++update_aggregate_count; }
virtual void on_update_view(const sstring&, const sstring&, bool) override { ++update_view_count; }
virtual void on_update_tablet_metadata() override { ++update_tablets; }
virtual void on_drop_keyspace(const sstring&) override { ++drop_keyspace_count; }
virtual void on_drop_column_family(const sstring&, const sstring&) override { ++drop_column_family_count; }
virtual void on_drop_user_type(const sstring&, const sstring&) override { ++drop_user_type_count; }
virtual void on_drop_function(const sstring&, const sstring&) override { ++drop_function_count; }
virtual void on_drop_aggregate(const sstring&, const sstring&) override { ++drop_aggregate_count; }
virtual void on_drop_view(const sstring&, const sstring&) override { ++drop_view_count; }
};
SEASTAR_TEST_CASE(test_alter_nested_type) {
return do_with_cql_env_thread([](cql_test_env& e) {
e.execute_cql("CREATE TYPE foo (foo_k int);").get();
e.execute_cql("CREATE TYPE bar (bar_k frozen<foo>);").get();
e.execute_cql("alter type foo add zed_v int;").get();
e.execute_cql("CREATE TABLE tbl (key int PRIMARY KEY, val frozen<bar>);").get();
e.execute_cql("insert into tbl (key, val) values (1, {bar_k: {foo_k: 2, zed_v: 3} });").get();
});
}
SEASTAR_TEST_CASE(test_nested_type_mutation_in_update) {
// ALTER TYPE always creates a mutation with a single type. This
// creates a mutation with 2 types, one nested in the other, to
// show that we can handle that.
return do_with_cql_env_thread([](cql_test_env& e) {
counting_migration_listener listener;
e.local_mnotifier().register_listener(&listener);
e.execute_cql("CREATE TYPE foo (foo_k int);").get();
e.execute_cql("CREATE TYPE bar (bar_k frozen<foo>);").get();
BOOST_REQUIRE_EQUAL(listener.create_user_type_count, 2);
service::migration_manager& mm = e.migration_manager().local();
auto group0_guard = mm.start_group0_operation().get();
auto ts = group0_guard.write_timestamp();
auto&& keyspace = e.db().local().find_keyspace("ks").metadata();
auto type1 = user_type_impl::get_instance("ks", to_bytes("foo"), {"foo_k", "extra"}, {int32_type, int32_type}, true);
auto muts1 = db::schema_tables::make_create_type_mutations(keyspace, type1, ts);
auto type2 = user_type_impl::get_instance("ks", to_bytes("bar"), {"bar_k", "extra"}, {type1, int32_type}, true);
auto muts2 = db::schema_tables::make_create_type_mutations(keyspace, type2, ts);
auto muts = muts1;
muts.insert(muts.end(), muts2.begin(), muts2.end());
mm.announce(std::move(muts), std::move(group0_guard)).get();
BOOST_REQUIRE_EQUAL(listener.create_user_type_count, 2);
BOOST_REQUIRE_EQUAL(listener.update_user_type_count, 2);
});
}
SEASTAR_TEST_CASE(test_notifications) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
counting_migration_listener listener;
e.local_mnotifier().register_listener(&listener);
auto listener_lease = defer([&e, &listener] { e.local_mnotifier().register_listener(&listener); });
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
BOOST_REQUIRE_EQUAL(listener.create_keyspace_count, 1);
e.execute_cql("create table tests.table1 (pk int primary key, c1 int, c2 int);").get();
BOOST_REQUIRE_EQUAL(listener.create_column_family_count, 1);
BOOST_REQUIRE_EQUAL(listener.columns_changed_count, 0);
e.execute_cql("alter table tests.table1 drop c2;").get();
BOOST_REQUIRE_EQUAL(listener.update_column_family_count, 1);
BOOST_REQUIRE_EQUAL(listener.columns_changed_count, 1);
e.execute_cql("alter table tests.table1 add s1 int;").get();
BOOST_REQUIRE_EQUAL(listener.update_column_family_count, 2);
BOOST_REQUIRE_EQUAL(listener.columns_changed_count, 2);
e.execute_cql("alter table tests.table1 alter s1 type blob;").get();
BOOST_REQUIRE_EQUAL(listener.update_column_family_count, 3);
BOOST_REQUIRE_EQUAL(listener.columns_changed_count, 3);
e.execute_cql("drop table tests.table1;").get();
BOOST_REQUIRE_EQUAL(listener.drop_column_family_count, 1);
e.execute_cql("create type tests.type1 (field1 text, field2 text);").get();
BOOST_REQUIRE_EQUAL(listener.create_user_type_count, 1);
e.execute_cql("drop type tests.type1;").get();
BOOST_REQUIRE_EQUAL(listener.drop_user_type_count, 1);
e.execute_cql("create type tests.type1 (field1 text, field2 text);").get();
e.execute_cql("create type tests.type2 (field1 text, field2 text);").get();
BOOST_REQUIRE_EQUAL(listener.create_user_type_count, 3);
e.execute_cql("drop type tests.type1;").get();
BOOST_REQUIRE_EQUAL(listener.drop_user_type_count, 2);
e.execute_cql("alter type tests.type2 add field3 text;").get();
BOOST_REQUIRE_EQUAL(listener.update_user_type_count, 1);
e.execute_cql("alter type tests.type2 alter field3 type blob;").get();
BOOST_REQUIRE_EQUAL(listener.update_user_type_count, 2);
e.execute_cql("alter type tests.type2 rename field2 to field4 and field3 to field5;").get();
BOOST_REQUIRE_EQUAL(listener.update_user_type_count, 3);
});
});
}
SEASTAR_TEST_CASE(test_drop_user_type_in_use) {
return do_with_cql_env_thread([](cql_test_env& e) {
e.execute_cql("create type simple_type (user_number int);").get();
e.execute_cql("create table simple_table (key int primary key, val frozen<simple_type>);").get();
e.execute_cql("insert into simple_table (key, val) values (42, {user_number: 1});").get();
BOOST_REQUIRE_EXCEPTION(e.execute_cql("drop type simple_type;").get(), exceptions::invalid_request_exception,
exception_predicate::message_equals("Cannot drop user type ks.simple_type as it is still used by table ks.simple_table"));
});
}
SEASTAR_TEST_CASE(test_drop_nested_user_type_in_use) {
return do_with_cql_env_thread([](cql_test_env& e) {
e.execute_cql("create type simple_type (user_number int);").get();
e.execute_cql("create table nested_table (key int primary key, val tuple<int, frozen<simple_type>>);").get();
e.execute_cql("insert into nested_table (key, val) values (42, (41, {user_number: 1}));").get();
BOOST_REQUIRE_EXCEPTION(e.execute_cql("drop type simple_type;").get(), exceptions::invalid_request_exception,
exception_predicate::message_equals(
"Cannot drop user type ks.simple_type as it is still used by table ks.nested_table"));
});
}
SEASTAR_TEST_CASE(test_prepared_statement_is_invalidated_by_schema_change) {
return do_with_cql_env([](cql_test_env& e) {
return seastar::async([&] {
logging::logger_registry().set_logger_level("query_processor", logging::log_level::debug);
e.execute_cql("create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
e.execute_cql("create table tests.table1 (pk int primary key, c1 int, c2 int);").get();
auto id = e.prepare("select * from tests.table1;").get0();
e.execute_cql("alter table tests.table1 add s1 int;").get();
try {
e.execute_prepared(id, {}).get();
BOOST_FAIL("Should have failed");
} catch (const not_prepared_exception&) {
// expected
}
});
});
}
// We don't want schema digest to change between Scylla versions because that results in a schema disagreement
// during rolling upgrade.
// This test is *not* supposed to check that the schema does not change.
// It only checks that the digest itself does not change *given* that the schema does not change.
// If the schema changes, the digest will change too (as expected), which will cause the test
// to fail unless you regenerate the test data. That's by design of the test.
// Note that changing the schema may introduce rolling upgrade problems, e.g. if changing the schema
// on an upgraded node doesn't force other nodes to follow-up. This test is not intended to catch such bugs.
// To test that rolling upgrade works in case of schema changes, we need to actually run two different
// versions of Scylla, and that cannot be done in a unit test.
// See also #6582.
future<> test_schema_digest_does_not_change_with_disabled_features(sstring data_dir,
std::set<sstring> disabled_features, std::vector<utils::UUID> expected_digests,
std::function<void(cql_test_env& e)> extra_schema_changes,
std::shared_ptr<db::extensions> extensions = std::make_shared<db::extensions>()) {
using namespace db;
using namespace db::schema_tables;
auto tmp = tmpdir();
// NOTICE: Regenerating data for this test may be necessary when a system table is added.
// This test uses pre-generated sstables and relies on the fact that they are up to date
// with the current system schema. If it is not, the schema will be updated, which will cause
// new timestamps to appear and schema digests will not match anymore.
// Warning: if you regenerate the data (and digests), please make sure that you don't accidentally
// hide a digest calculation bug. Separate commits that touch the schema from commits which
// could potentially modify the digest calculation algorithm (for example). And DO test whether
// rolling upgrade works.
const bool regenerate = false;
auto db_cfg_ptr = ::make_shared<db::config>(std::move(extensions));
auto& db_cfg = *db_cfg_ptr;
db_cfg.enable_user_defined_functions({true}, db::config::config_source::CommandLine);
db_cfg.experimental_features({experimental_features_t::feature::UDF, experimental_features_t::feature::KEYSPACE_STORAGE_OPTIONS}, db::config::config_source::CommandLine);
if (regenerate) {
db_cfg.data_file_directories({data_dir}, db::config::config_source::CommandLine);
} else {
fs::copy(std::string(data_dir), std::string(tmp.path().string()), fs::copy_options::recursive);
db_cfg.data_file_directories({tmp.path().string()}, db::config::config_source::CommandLine);
}
cql_test_config cfg_in(db_cfg_ptr);
cfg_in.disabled_features = std::move(disabled_features);
return do_with_cql_env_thread([expected_digests = std::move(expected_digests), extra_schema_changes = std::move(extra_schema_changes)] (cql_test_env& e) {
if (regenerate) {
// Exercise many different kinds of schema changes.
e.execute_cql(
"create keyspace tests with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
e.execute_cql("create table tests.table1 (pk int primary key, c1 int, c2 int);").get();
e.execute_cql("create type tests.basic_info (c1 timestamp, v2 text);").get();
e.execute_cql("create index on tests.table1 (c1);").get();
e.execute_cql("create table ks.tbl (a int, b int, c float, PRIMARY KEY (a))").get();
e.execute_cql(
"create materialized view ks.tbl_view AS SELECT c FROM ks.tbl WHERE c IS NOT NULL PRIMARY KEY (c, a)").get();
e.execute_cql(
"create materialized view ks.tbl_view_2 AS SELECT a FROM ks.tbl WHERE a IS NOT NULL PRIMARY KEY (a)").get();
e.execute_cql(
"create keyspace tests2 with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };").get();
e.execute_cql("drop keyspace tests2;").get();
extra_schema_changes(e);
}
auto expect_digest = [&] (schema_features sf, utils::UUID expected) {
// Changes in the system_distributed keyspace was a common source of regenerating this test case.
// It was not the original idea of this test to regenerate after each change in the distributed
// system keyspace, since these changes are propagated naturally on their own anyway - the schema
// with highest timestamp will win and be sent to other nodes.
// Thus, system_distributed.* tables are officially not taken into account,
// which makes it less likely that this test case would need to be needlessly regenerated.
auto actual = calculate_schema_digest(e.get_storage_proxy(), sf, std::not_fn(&is_internal_keyspace)).get0();
if (regenerate) {
std::cout << format(" utils::UUID(\"{}\"),", actual) << "\n";
} else {
BOOST_REQUIRE_EQUAL(actual.uuid(), expected);
}
};
auto expect_version = [&] (sstring ks_name, sstring cf_name, utils::UUID expected) {
auto actual = e.local_db().find_column_family(ks_name, cf_name).schema()->version();
if (regenerate) {
std::cout << format(" utils::UUID(\"{}\"),", actual) << "\n";
} else {
BOOST_REQUIRE_EQUAL(actual.uuid(), expected);
}
};
schema_features sf = schema_features::of<schema_feature::DIGEST_INSENSITIVE_TO_EXPIRY>();
expect_digest(sf, expected_digests[0]);
sf.set<schema_feature::VIEW_VIRTUAL_COLUMNS>();
expect_digest(sf, expected_digests[1]);
sf.set<schema_feature::VIEW_VIRTUAL_COLUMNS>();
expect_digest(sf, expected_digests[2]);
sf = schema_features::full();
sf.remove<schema_feature::SCYLLA_KEYSPACES>();
expect_digest(sf, expected_digests[3]);
// Causes tombstones to become expired
// This is in order to test that schema disagreement doesn't form due to expired tombstones being collected
// Refs https://github.com/scylladb/scylla/issues/4485
forward_jump_clocks(std::chrono::seconds(60*60*24*31));
expect_digest(sf, expected_digests[4]);
// FIXME: schema_mutations::digest() is still sensitive to expiry, so we can check versions only after forward_jump_clocks()
// otherwise the results would not be stable.
expect_version("tests", "table1", expected_digests[5]);
expect_version("ks", "tbl", expected_digests[6]);
expect_version("ks", "tbl_view", expected_digests[7]);
expect_version("ks", "tbl_view_2", expected_digests[8]);
// Check that system_schema.scylla_keyspaces info is taken into account
sf = schema_features::full();
expect_digest(sf, expected_digests[9]);
}, cfg_in).then([tmp = std::move(tmp)] {});
}
SEASTAR_TEST_CASE(test_schema_digest_does_not_change) {
std::vector<utils::UUID> expected_digests{
utils::UUID("264f79fc-61bd-3670-8d6e-2794f9787b0a"),
utils::UUID("d2035515-b299-3265-b920-7dbe5306e72a"),
utils::UUID("d2035515-b299-3265-b920-7dbe5306e72a"),
utils::UUID("de49e92f-a00d-3f24-8779-d07de26708cb"),
utils::UUID("de49e92f-a00d-3f24-8779-d07de26708cb"),
utils::UUID("bbf064ed-0a98-3c19-8c4a-5c929cabd936"),
utils::UUID("0db2a3f8-6779-388f-951d-de6a537789a7"),
utils::UUID("21a89984-ffc6-325d-b818-66fc29da51e7"),
utils::UUID("e69a05e8-80a6-3e8f-bd34-1b5837374c79"),
utils::UUID("de49e92f-a00d-3f24-8779-d07de26708cb"),
};
return test_schema_digest_does_not_change_with_disabled_features("./test/resource/sstables/schema_digest_test",
std::set<sstring>{"COMPUTED_COLUMNS", "CDC", "KEYSPACE_STORAGE_OPTIONS"}, std::move(expected_digests), [] (cql_test_env& e) {});
}
SEASTAR_TEST_CASE(test_schema_digest_does_not_change_after_computed_columns) {
std::vector<utils::UUID> expected_digests{
utils::UUID("036153ec-4565-34fb-a878-ce347b94f247"),
utils::UUID("fa2e7735-7604-3202-8ce9-399996305aca"),
utils::UUID("fa2e7735-7604-3202-8ce9-399996305aca"),
utils::UUID("94606636-ae43-3e0a-b238-e7f0e33ef600"),
utils::UUID("94606636-ae43-3e0a-b238-e7f0e33ef600"),
utils::UUID("13d418e8-968e-3bd8-801a-6df5e5327d44"),
utils::UUID("75808956-93e2-331a-96cc-8dc9cdea79ca"),
utils::UUID("7eccb793-c6f4-3e84-ae35-788d03188baf"),
utils::UUID("467deb84-d7de-36cb-a1fa-f3672cb66340"),
utils::UUID("94606636-ae43-3e0a-b238-e7f0e33ef600"),
};
return test_schema_digest_does_not_change_with_disabled_features("./test/resource/sstables/schema_digest_test_computed_columns",
std::set<sstring>{"CDC", "KEYSPACE_STORAGE_OPTIONS"}, std::move(expected_digests), [] (cql_test_env& e) {});
}
SEASTAR_TEST_CASE(test_schema_digest_does_not_change_with_functions) {
std::vector<utils::UUID> expected_digests{
utils::UUID("6fa38d16-bbc4-3da5-bda5-680329789d8f"),
utils::UUID("649bf7ec-fd64-3ccb-adde-3887fc1432be"),
utils::UUID("649bf7ec-fd64-3ccb-adde-3887fc1432be"),
utils::UUID("48fd0c1b-9777-34be-8c16-187c6ab55cfc"),
utils::UUID("48fd0c1b-9777-34be-8c16-187c6ab55cfc"),
utils::UUID("6c7bef38-05b7-3bf3-a52c-352333da7214"),
utils::UUID("cb23910d-ced8-35e5-99f3-57f362860f3c"),
utils::UUID("b0ecf791-0637-34a5-a940-bc217517f1aa"),
utils::UUID("47b87dfc-3c18-324b-b280-58300ac5d3ca"),
utils::UUID("48fd0c1b-9777-34be-8c16-187c6ab55cfc"),
};
return test_schema_digest_does_not_change_with_disabled_features(
"./test/resource/sstables/schema_digest_with_functions_test",
std::set<sstring>{"CDC", "KEYSPACE_STORAGE_OPTIONS"},
std::move(expected_digests),
[] (cql_test_env& e) {
e.execute_cql("create function twice(val int) called on null input returns int language lua as 'return 2 * val';").get();
e.execute_cql("create function my_add(a int, b int) called on null input returns int language lua as 'return a + b';").get();
});
}
SEASTAR_TEST_CASE(test_schema_digest_does_not_change_with_cdc_options) {
auto ext = std::make_shared<db::extensions>();
ext->add_schema_extension<cdc::cdc_extension>(cdc::cdc_extension::NAME);
std::vector<utils::UUID> expected_digests{
utils::UUID("ff69e387-64ca-3335-b488-b7a615908148"),
utils::UUID("7f1ac621-fc68-3420-bc9b-54520da40418"),
utils::UUID("7f1ac621-fc68-3420-bc9b-54520da40418"),
utils::UUID("09899769-4e7f-3119-9769-e3db3d99455b"),
utils::UUID("09899769-4e7f-3119-9769-e3db3d99455b"),
utils::UUID("265be25f-b268-3f43-a54d-9c6e379a901d"),
utils::UUID("c604f5c9-988e-393f-b9d8-2ed55b9a540c"),
utils::UUID("45d9f25d-58a1-3f1e-85a1-f09d82c52588"),
utils::UUID("7ef45dd2-aab9-38f1-bcc6-ba9c94666e36"),
utils::UUID("09899769-4e7f-3119-9769-e3db3d99455b"),
};
return test_schema_digest_does_not_change_with_disabled_features(
"./test/resource/sstables/schema_digest_test_cdc_options",
std::set<sstring>{"KEYSPACE_STORAGE_OPTIONS"},
std::move(expected_digests),
[] (cql_test_env& e) {
e.execute_cql("create table tests.table_cdc (pk int primary key, c1 int, c2 int) with cdc = {'enabled':'true'};").get();
},
std::move(ext));
}
SEASTAR_TEST_CASE(test_schema_digest_does_not_change_with_keyspace_storage_options) {
std::vector<utils::UUID> expected_digests{
utils::UUID("d9f78213-ff9f-3208-9083-47e18cebf06f"),
utils::UUID("30e2cf99-389d-381f-82b9-3fcdcf66a1fb"),
utils::UUID("30e2cf99-389d-381f-82b9-3fcdcf66a1fb"),
utils::UUID("98d63879-6633-3708-880e-8716fcbadda0"),
utils::UUID("98d63879-6633-3708-880e-8716fcbadda0"),
utils::UUID("f4ca70f9-170c-3a69-a274-76e711d2841e"),
utils::UUID("cbf9aa1e-2488-3485-8c28-e42bf42a2dcf"),
utils::UUID("67c0db2d-8fd6-30f4-beee-f15a80a889fd"),
utils::UUID("9c5c996a-6b27-346e-96d4-26d545d4601a"),
utils::UUID("3fc03c97-8010-3746-8cea-e8b9ac27fe4e"),
};
return test_schema_digest_does_not_change_with_disabled_features(
"./test/resource/sstables/schema_digest_test_keyspace_storage_options",
std::set<sstring>{},
std::move(expected_digests),
[] (cql_test_env& e) {
e.execute_cql("create keyspace tests_s3 with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 }"
" and storage = { 'type': 'S3', 'bucket': 'b1', 'endpoint': 'localhost' };").get();
e.execute_cql("create table tests_s3.table1 (pk int primary key, c1 int, c2 int)").get();
});
}
// Regression test, ensuring people don't forget to set the null sharder
// for newly added schema tables.
SEASTAR_TEST_CASE(test_schema_tables_use_null_sharder) {
return do_with_cql_env_thread([] (cql_test_env& e) {
e.db().invoke_on_all([] (replica::database& db) {
{
auto ks_metadata = db.find_keyspace("system_schema").metadata();
auto& cf_metadata = ks_metadata->cf_meta_data();
for (auto [_, s]: cf_metadata) {
std::cout << "checking " << s->cf_name() << std::endl;
BOOST_REQUIRE_EQUAL(s->get_sharder().shard_count(), 1);
}
}
// There are some other tables which reside in the "system" keyspace
// but need to use shard 0, too.
auto ks_metadata = db.find_keyspace("system").metadata();
auto& cf_metadata = ks_metadata->cf_meta_data();
auto it = cf_metadata.find("scylla_table_schema_history");
BOOST_REQUIRE(it != cf_metadata.end());
BOOST_REQUIRE_EQUAL(it->second->get_sharder().shard_count(), 1);
it = cf_metadata.find("raft");
BOOST_REQUIRE(it != cf_metadata.end());
BOOST_REQUIRE_EQUAL(it->second->get_sharder().shard_count(), 1);
it = cf_metadata.find("raft_snapshots");
BOOST_REQUIRE(it != cf_metadata.end());
BOOST_REQUIRE_EQUAL(it->second->get_sharder().shard_count(), 1);
it = cf_metadata.find("raft_snapshot_config");
BOOST_REQUIRE(it != cf_metadata.end());
BOOST_REQUIRE_EQUAL(it->second->get_sharder().shard_count(), 1);
// The schemas returned by all_tables() may be different than those stored in the `db` object:
// the schemas stored inside `db` come from deserializing mutations. The schemas in all_tables()
// are hardcoded. If there is some information in the schema object that is not serialized into
// mutations (say... the sharder - for now, at least), the two schema objects may differ, if
// one is not careful.
for (auto s: db::schema_tables::all_tables(db::schema_features::full())) {
BOOST_REQUIRE_EQUAL(s->get_sharder().shard_count(), 1);
}
}).get();
}, raft_cql_test_config());
}
SEASTAR_TEST_CASE(test_schema_make_reversed) {
auto schema = schema_builder("ks", get_name())
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("ck", bytes_type, column_kind::clustering_key)
.with_column("v1", bytes_type)
.build();
testlog.info(" schema->version(): {}", schema->version());
auto reversed_schema = schema->make_reversed();
testlog.info(" reversed_schema->version(): {}", reversed_schema->version());
BOOST_REQUIRE(schema->version() != reversed_schema->version());
BOOST_REQUIRE(reversed(schema->version()) == reversed_schema->version());
auto re_reversed_schema = reversed_schema->make_reversed();
testlog.info("re_reversed_schema->version(): {}", re_reversed_schema->version());
BOOST_REQUIRE(schema->version() == re_reversed_schema->version());
BOOST_REQUIRE(reversed_schema->version() != re_reversed_schema->version());
return make_ready_future<>();
}
SEASTAR_TEST_CASE(test_schema_get_reversed) {
return do_with_cql_env([] (cql_test_env& e) {
auto schema = schema_builder("ks", get_name())
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("ck", bytes_type, column_kind::clustering_key)
.with_column("v1", bytes_type)
.build();
schema = local_schema_registry().learn(schema);
auto reversed_schema = schema->get_reversed();
testlog.info(" &schema: {}", fmt::ptr(schema.get()));
testlog.info("&reverse_schema: {}", fmt::ptr(reversed_schema.get()));
BOOST_REQUIRE_EQUAL(reversed_schema->get_reversed().get(), schema.get());
BOOST_REQUIRE_EQUAL(schema->get_reversed().get(), reversed_schema.get());
return make_ready_future<>();
});
}
Reference in New Issue View Git Blame Copy Permalink