Files
scylladb/test/lib/cql_test_env.hh
Avi Kivity 0b418fa7cf cql3, transport, tests: remove "unset" from value type system
The CQL binary protocol introduced "unset" values in version 4
of the protocol. Unset values can be bound to variables, which
cause certain CQL fragments to be skipped. For example, the
fragment `SET a = :var` will not change the value of `a` if `:var`
is bound to an unset value.

Unsets, however, are very limited in where they can appear. They
can only appear at the top-level of an expression, and any computation
done with them is invalid. For example, `SET list_column = [3, :var]`
is invalid if `:var` is bound to unset.

This causes the code to be littered with checks for unset, and there
are plenty of tests dedicated to catching unsets. However, a simpler
way is possible - prevent the infiltration of unsets at the point of
entry (when evaluating a bind variable expression), and introduce
guards to check for the few cases where unsets are allowed.

This is what this long patch does. It performs the following:

(general)

1. unset is removed from the possible values of cql3::raw_value and
   cql3::raw_value_view.

(external->cql3)

2. query_options is fortified with a vector of booleans,
   unset_bind_variable_vector, where each boolean corresponds to a bind
   variable index and is true when it is unset.
3. To avoid churn, two compatiblity structs are introduced:
   cql3::raw_value{,_view}_vector_with_unset, which can be constructed
   from a std::vector<raw_value{,_view/}>, which is what most callers
   have. They can also be constructed with explicit unset vectors, for
   the few cases they are needed.

(cql3->variables)

4. query_options::get_value_at() now throws if the requested bind variable
   is unset. This replaces all the throwing checks in expression evaluation
   and statement execution, which are removed.
5. A new query_options::is_unset() is added for the users that can tolerate
   unset; though it is not used directly.
6. A new cql3::unset_operation_guard class guards against unsets. It accepts
   an expression, and can be queried whether an unset is present. Two
   conditions are checked: the expression must be a singleton bind
   variable, and at runtime it must be bound to an unset value.
7. The modification_statement operations are split into two, via two
   new subclasses of cql3::operation. cql3::operation_no_unset_support
   ignores unsets completely. cql3::operation_skip_if_unset checks if
   an operand is unset (luckily all operations have at most one operand that
   tolerates unset) and applies unset_operation_guard to it.
8. The various sites that accept expressions or operations are modified
   to check for should_skip_operation(). This are the loops around
   operations in update_statement and delete_statement, and the checks
   for unset in attributes (LIMIT and PER PARTITION LIMIT)

(tests)

9. Many unset tests are removed. It's now impossible to enter an
   unset value into the expression evaluation machinery (there's
   just no unset value), so it's impossible to test for it.
10. Other unset tests now have to be invoked via bind variables,
   since there's no way to create an unset cql3::expr::constant.
11. Many tests have their exception message match strings relaxed.
   Since unsets are now checked very early, we don't know the context
   where they happen. It would be possible to reintroduce it (by adding
   a format string parameter to cql3::unset_operation_guard), but it
   seems not to be worth the effort. Usage of unsets is rare, and it is
   explicit (at least with the Python driver, an unset cannot be
   introduced by ommission).

I tried as an alternative to wrap cql3::raw_value{,_view} (that doesn't
recognize unsets) with cql3::maybe_unset_value (that does), but that
caused huge amounts of churn, so I abandoned that in favor of the
current approach.

Closes #12517
2023-01-16 21:10:56 +02:00

184 lines
5.7 KiB
C++

/*
* Copyright (C) 2015-present ScyllaDB
*/
/*
* SPDX-License-Identifier: AGPL-3.0-or-later
*/
#pragma once
#include <functional>
#include <vector>
#include <seastar/core/distributed.hh>
#include <seastar/core/sstring.hh>
#include <seastar/core/future.hh>
#include <seastar/core/shared_ptr.hh>
#include "db/view/view_update_generator.hh"
#include "transport/messages/result_message_base.hh"
#include "cql3/query_options_fwd.hh"
#include "cql3/values.hh"
#include "cql3/prepared_statements_cache.hh"
#include "cql3/query_processor.hh"
#include "bytes.hh"
#include "schema.hh"
#include "test/lib/eventually.hh"
namespace replica {
class database;
}
namespace db {
class batchlog_manager;
}
namespace db::view {
class view_builder;
}
namespace auth {
class service;
}
namespace cql3 {
class query_processor;
}
namespace service {
class client_state;
class migration_manager;
class raft_group0_client;
class raft_group_registry;
}
class not_prepared_exception : public std::runtime_error {
public:
not_prepared_exception(const cql3::prepared_cache_key_type& id) : std::runtime_error(format("Not prepared: {}", id)) {}
};
namespace db {
class config;
}
struct scheduling_groups {
scheduling_group compaction_scheduling_group;
scheduling_group memory_compaction_scheduling_group;
scheduling_group streaming_scheduling_group;
scheduling_group statement_scheduling_group;
scheduling_group memtable_scheduling_group;
scheduling_group memtable_to_cache_scheduling_group;
scheduling_group gossip_scheduling_group;
};
// Creating and destroying scheduling groups on each env setup and teardown
// doesn't work because it messes up execution stages due to scheduling groups
// having the same name but not having the same id on each run. So they are
// created once and used across all envs. This method allows retrieving them to
// be used in tests.
// Not thread safe!
future<scheduling_groups> get_scheduling_groups();
class cql_test_config {
public:
seastar::shared_ptr<db::config> db_config;
// Scheduling groups are overwritten unconditionally, see get_scheduling_groups().
std::optional<replica::database_config> dbcfg;
std::set<sstring> disabled_features;
std::optional<cql3::query_processor::memory_config> qp_mcfg;
cql_test_config();
cql_test_config(const cql_test_config&);
cql_test_config(shared_ptr<db::config>);
~cql_test_config();
};
class cql_test_env {
public:
virtual ~cql_test_env() {};
virtual future<::shared_ptr<cql_transport::messages::result_message>> execute_cql(sstring_view text) = 0;
virtual future<::shared_ptr<cql_transport::messages::result_message>> execute_cql(
sstring_view text, std::unique_ptr<cql3::query_options> qo) = 0;
/// Processes queries (which must be modifying queries) as a batch.
virtual future<::shared_ptr<cql_transport::messages::result_message>> execute_batch(
const std::vector<sstring_view>& queries, std::unique_ptr<cql3::query_options> qo) = 0;
virtual future<cql3::prepared_cache_key_type> prepare(sstring query) = 0;
virtual future<::shared_ptr<cql_transport::messages::result_message>> execute_prepared(
cql3::prepared_cache_key_type id,
cql3::raw_value_vector_with_unset values,
db::consistency_level cl = db::consistency_level::ONE) = 0;
virtual future<::shared_ptr<cql_transport::messages::result_message>> execute_prepared_with_qo(
cql3::prepared_cache_key_type id,
std::unique_ptr<cql3::query_options> qo) = 0;
virtual future<std::vector<mutation>> get_modification_mutations(const sstring& text) = 0;
virtual future<> create_table(std::function<schema(std::string_view)> schema_maker) = 0;
virtual future<> require_keyspace_exists(const sstring& ks_name) = 0;
virtual future<> require_table_exists(const sstring& ks_name, const sstring& cf_name) = 0;
virtual future<> require_table_exists(std::string_view qualified_name) = 0;
virtual future<> require_table_does_not_exist(const sstring& ks_name, const sstring& cf_name) = 0;
virtual future<> require_column_has_value(
const sstring& table_name,
std::vector<data_value> pk,
std::vector<data_value> ck,
const sstring& column_name,
data_value expected) = 0;
virtual future<> stop() = 0;
virtual service::client_state& local_client_state() = 0;
virtual replica::database& local_db() = 0;
virtual cql3::query_processor& local_qp() = 0;
virtual distributed<replica::database>& db() = 0;
virtual distributed<cql3::query_processor> & qp() = 0;
virtual auth::service& local_auth_service() = 0;
virtual db::view::view_builder& local_view_builder() = 0;
virtual db::view::view_update_generator& local_view_update_generator() = 0;
virtual service::migration_notifier& local_mnotifier() = 0;
virtual sharded<service::migration_manager>& migration_manager() = 0;
virtual sharded<db::batchlog_manager>& batchlog_manager() = 0;
virtual sharded<gms::gossiper>& gossiper() = 0;
virtual future<> refresh_client_state() = 0;
virtual service::raft_group0_client& get_raft_group0_client() = 0;
virtual sharded<service::raft_group_registry>& get_raft_group_registry() = 0;
virtual db::system_keyspace& get_system_keyspace() = 0;
data_dictionary::database data_dictionary();
};
future<> do_with_cql_env(std::function<future<>(cql_test_env&)> func, cql_test_config = {});
future<> do_with_cql_env_thread(std::function<void(cql_test_env&)> func, cql_test_config = {}, thread_attributes thread_attr = {});
reader_permit make_reader_permit(cql_test_env&);
// CQL test config with raft experimental feature enabled
cql_test_config raft_cql_test_config();