Files
scylladb/cql3/statements/create_keyspace_statement.cc
Nadav Har'El 5984db047d Merge 'mv: forbid IS NOT NULL on columns outside the primary key' from Jan Ciołek
statement_restrictions: forbid IS NOT NULL on columns outside the primary key

IS NOT NULL is currently allowed only when creating materialized views.
It's used to convey that the view will not include any rows that would make the view's primary key columns NULL.

Generally materialized views allow to place restrictions on the primary key columns, but restrictions on the regular columns are forbidden. The exception was IS NOT NULL - it was allowed to write regular_col IS NOT NULL. The problem is that this restriction isn't respected, it's just silently ignored (see #10365).

Supporting IS NOT NULL on regular columns seems to be as hard as supporting any other restrictions on regular columns.
It would be a big effort, and there are some reasons why we don't support them.

For now let's forbid such restrictions, it's better to fail than be wrong silently.

Throwing a hard error would be a breaking change.
To avoid breaking existing code the reaction to an invalid IS NOT NULL restrictions is controlled by the `strict_is_not_null_in_views` flag.

This flag can have the following values:
* `true` - strict checking. Having an `IS NOT NULL` restriction on a column that doesn't belong to the view's primary key causes an error to be thrown.
* `warn` - allow invalid `IS NOT NULL` restrictions, but throw a warning. The invalid restrictions are silently ignored.
* `false` - allow invalid `IS NOT NULL` restricitons, without any warnings or errors. The invalid restrictions are silently ignored.

The default values for this flag are `warn` in `db::config` and `true` in scylla.yaml.

This way the existing clusters will have `warn` by default, so they'll get a warning if they try to create such an invalid view.

New clusters with fresh scylla.yaml will have the flag set to `true`, as scylla.yaml overwrites the default value in `db::config`.
New clusters will throw a hard error for invalid views, but in older existing clusters it will just be a warning.
This way we can maintain backwards compatibility, but still move forward by rejecting invalid queries on new clusters.

Fixes: #10365

Closes #13013

* github.com:scylladb/scylladb:
  boost/restriction_test: test the strict_is_not_null_in_views flag
  docs/cql/mv: columns outside of view's primary key can't be restricted
  cql-pytest: enable test_is_not_null_forbidden_in_filter
  statement_restrictions: forbid IS NOT NULL on columns outside the primary key
  schema_altering_statement: return warnings from prepare_schema_mutations()
  db/config: add strict_is_not_null_in_views config option
  statement_restrictions: add get_not_null_columns()
  test: remove invalid IS NOT NULL restrictions from tests
2023-06-07 12:12:19 +03:00

226 lines
9.8 KiB
C++

/*
* Copyright 2015-present ScyllaDB
*
* Modified by ScyllaDB
*/
/*
* SPDX-License-Identifier: (AGPL-3.0-or-later and Apache-2.0)
*/
#include <seastar/core/coroutine.hh>
#include "cql3/statements/create_keyspace_statement.hh"
#include "cql3/statements/ks_prop_defs.hh"
#include "prepared_statement.hh"
#include "data_dictionary/data_dictionary.hh"
#include "data_dictionary/keyspace_metadata.hh"
#include "mutation/mutation.hh"
#include "service/migration_manager.hh"
#include "service/storage_proxy.hh"
#include "transport/messages/result_message.hh"
#include "cql3/query_processor.hh"
#include "db/config.hh"
#include "gms/feature_service.hh"
#include <boost/regex.hpp>
#include <stdexcept>
bool is_system_keyspace(std::string_view keyspace);
namespace cql3 {
namespace statements {
static logging::logger mylogger("create_keyspace");
create_keyspace_statement::create_keyspace_statement(const sstring& name, shared_ptr<ks_prop_defs> attrs, bool if_not_exists)
: _name{name}
, _attrs{attrs}
, _if_not_exists{if_not_exists}
{
}
const sstring& create_keyspace_statement::keyspace() const
{
return _name;
}
future<> create_keyspace_statement::check_access(query_processor& qp, const service::client_state& state) const
{
return state.has_all_keyspaces_access(auth::permission::CREATE);
}
void create_keyspace_statement::validate(query_processor& qp, const service::client_state& state) const
{
std::string name;
name.resize(_name.length());
std::transform(_name.begin(), _name.end(), name.begin(), ::tolower);
if (is_system_keyspace(name)) {
throw exceptions::invalid_request_exception("system keyspace is not user-modifiable");
}
// keyspace name
boost::regex name_regex("\\w+");
if (!boost::regex_match(name, name_regex)) {
throw exceptions::invalid_request_exception(format("\"{}\" is not a valid keyspace name", _name.c_str()));
}
if (name.length() > schema::NAME_LENGTH) {
throw exceptions::invalid_request_exception(format("Keyspace names shouldn't be more than {:d} characters long (got \"{}\")", schema::NAME_LENGTH, _name.c_str()));
}
_attrs->validate();
if (!bool(_attrs->get_replication_strategy_class())) {
throw exceptions::configuration_exception("Missing mandatory replication strategy class");
}
try {
_attrs->get_storage_options();
} catch (const std::runtime_error& e) {
throw exceptions::invalid_request_exception(e.what());
}
if (!qp.proxy().features().keyspace_storage_options
&& _attrs->get_storage_options().type_string() != "LOCAL") {
throw exceptions::invalid_request_exception("Keyspace storage options not supported in the cluster");
}
#if 0
// The strategy is validated through KSMetaData.validate() in announceNewKeyspace below.
// However, for backward compatibility with thrift, this doesn't validate unexpected options yet,
// so doing proper validation here.
AbstractReplicationStrategy.validateReplicationStrategy(name,
AbstractReplicationStrategy.getClass(attrs.getReplicationStrategyClass()),
StorageService.instance.getTokenMetadata(),
DatabaseDescriptor.getEndpointSnitch(),
attrs.getReplicationOptions());
#endif
}
future<std::tuple<::shared_ptr<cql_transport::event::schema_change>, std::vector<mutation>, cql3::cql_warnings_vec>> create_keyspace_statement::prepare_schema_mutations(query_processor& qp, api::timestamp_type ts) const {
using namespace cql_transport;
const auto& tm = *qp.proxy().get_token_metadata_ptr();
::shared_ptr<event::schema_change> ret;
std::vector<mutation> m;
try {
m = qp.get_migration_manager().prepare_new_keyspace_announcement(_attrs->as_ks_metadata(_name, tm), ts);
ret = ::make_shared<event::schema_change>(
event::schema_change::change_type::CREATED,
event::schema_change::target_type::KEYSPACE,
keyspace());
} catch (const exceptions::already_exists_exception& e) {
if (!_if_not_exists) {
co_return coroutine::exception(std::current_exception());
}
}
co_return std::make_tuple(std::move(ret), std::move(m), std::vector<sstring>());
}
std::unique_ptr<cql3::statements::prepared_statement>
cql3::statements::create_keyspace_statement::prepare(data_dictionary::database db, cql_stats& stats) {
return std::make_unique<prepared_statement>(make_shared<create_keyspace_statement>(*this));
}
future<> cql3::statements::create_keyspace_statement::grant_permissions_to_creator(const service::client_state& cs) const {
return do_with(auth::make_data_resource(keyspace()), [&cs](const auth::resource& r) {
return auth::grant_applicable_permissions(
*cs.get_auth_service(),
*cs.user(),
r).handle_exception_type([](const auth::unsupported_authorization_operation&) {
// Nothing.
});
});
}
// Check for replication strategy choices which are restricted by the
// configuration. This check can throw a configuration_exception immediately
// if the strategy is forbidden by the configuration, or return a warning
// string if the restriction was set to "warn".
// This function is only supposed to check for replication strategies
// restricted by the configuration. Checks for other types of strategy
// errors (such as unknown replication strategy name or unknown options
// to a known replication strategy) are done elsewhere.
std::optional<sstring> check_restricted_replication_strategy(
query_processor& qp,
const sstring& keyspace,
const ks_prop_defs& attrs)
{
if (!attrs.get_replication_strategy_class()) {
return std::nullopt;
}
sstring replication_strategy = locator::abstract_replication_strategy::to_qualified_class_name(
*attrs.get_replication_strategy_class());
// SimpleStrategy is not recommended in any setup which already has - or
// may have in the future - multiple racks or DCs. So depending on how
// protective we are configured, let's prevent it or allow with a warning:
if (replication_strategy == "org.apache.cassandra.locator.SimpleStrategy") {
switch(qp.db().get_config().restrict_replication_simplestrategy()) {
case db::tri_mode_restriction_t::mode::TRUE:
throw exceptions::configuration_exception(
"SimpleStrategy replication class is not recommended, and "
"forbidden by the current configuration. Please use "
"NetworkToplogyStrategy instead. You may also override this "
"restriction with the restrict_replication_simplestrategy=false "
"configuration option.");
case db::tri_mode_restriction_t::mode::WARN:
return format("SimpleStrategy replication class is not "
"recommended, but was used for keyspace {}. The "
"restrict_replication_simplestrategy configuration option "
"can be changed to silence this warning or make it into an error.",
keyspace);
case db::tri_mode_restriction_t::mode::FALSE:
// Scylla was configured to allow SimpleStrategy, but let's warn
// if it's used on a cluster which *already* has multiple DCs:
if (qp.proxy().get_token_metadata_ptr()->get_topology().get_datacenter_endpoints().size() > 1) {
return "Using SimpleStrategy in a multi-datacenter environment is not recommended.";
}
break;
}
}
// The minimum_keyspace_rf configuration option can be used to forbid
// a lower replication factor. We assume that all numeric replication
// options are replication factors - this is true for SimpleStrategy and
// NetworkTopologyStrategy but in the future if we add more strategies,
// we may need to limit this test only to specific options.
// A zero replication factor is not forbidden - it is the traditional
// way to avoid replication on some DC.
// We ignore errors (non-number, negative number, etc.) here,
// these are checked and reported elsewhere.
for (auto opt : attrs.get_replication_options()) {
try {
auto rf = std::stol(opt.second);
if (rf > 0 && rf < qp.proxy().data_dictionary().get_config().minimum_keyspace_rf()) {
throw exceptions::configuration_exception(format(
"Replication factor {}={} is forbidden by the current "
"configuration setting of minimum_keyspace_rf={}. Please "
"increase replication factor, or lower minimum_keyspace_rf "
"set in the configuration.", opt.first, opt.second,
qp.proxy().data_dictionary().get_config().minimum_keyspace_rf()));
}
} catch (std::invalid_argument&) {
} catch (std::out_of_range& ) {
}
}
return std::nullopt;
}
future<::shared_ptr<messages::result_message>>
create_keyspace_statement::execute(query_processor& qp, service::query_state& state, const query_options& options) const {
std::optional<sstring> warning = check_restricted_replication_strategy(qp, keyspace(), *_attrs);
return schema_altering_statement::execute(qp, state, options).then([warning = std::move(warning)] (::shared_ptr<messages::result_message> msg) {
if (warning) {
msg->add_warning(*warning);
mylogger.warn("{}", *warning);
}
return msg;
});
}
lw_shared_ptr<data_dictionary::keyspace_metadata> create_keyspace_statement::get_keyspace_metadata(const locator::token_metadata& tm) {
_attrs->validate();
return _attrs->as_ks_metadata(_name, tm);
}
}
}