Files
scylladb/cql3/term.hh
Avi Kivity 0876248c2b Merge "cql3: cache function calls evaluation for non-deterministic functions" from Pavel S
"
`function_call` AST nodes are created for each function
with side effects in a CQL query, i.e. non-deterministic
functions (`uuid()`, `now()` and some others timeuuid-related).

These nodes are evaluated either when a query itself is executed
or query restrictions are computed (e.g. partition/clustering
key ranges for LWT requests).

We need to cache the calls since otherwise when handling a
`bounce_to_shard` request for an LWT query, we can possibly
enter an infinite bouncing loop (in case a function is used
to calculate partition key ranges for a query), since the
results can be different each time.

Furthermore, we don't support bouncing more than one time.
Returning `bounce_to_shard` message more than one time
will result in a crash.

Caching works only for LWT statements and only for the function
calls that affect partition key range computation for the query.

`variable_specifications` class is renamed to `prepare_context`
and generalized to record information about each `function_call`
AST node and modify them, as needed:
* Check whether a given function call is a part of partition key
  statement restriction.
* Assign ids for caching if above is true and the call is a part
  of an LWT statement.

There is no need to include any kind of statement identifier
in the cache key since `query_options` (which holds the cache)
is limited to a single statement, anyway.

Function calls are indexed by the order in which they appear
within a statement while parsing. There is no need to
include any kind of statement identifier to the cache key
since `query_options` (which holds the cache) is limited
to a single statement, anyway.

Note that `function_call::raw` AST nodes are not created
for selection clauses of a SELECT statement hence they
can only accept only one of the following things as parameters:
* Other function calls.
* Literal values.
* Parameter markers.

In other words, only parameters that can be immediately reduced
to a byte buffer are allowed and we don't need to handle
database inputs to non-pure functions separately since they
are not possible in this context. Anyhow, we don't even have
a single non-pure function that accepts arguments, so precautions
are not needed at the moment.

Add a test written in `cql-pytest` framework to verify
that both prepared and unprepared lwt statements handle
`bounce_to_shard` messages correctly in such scenario.

Fixes: #8604

Tests: unit(dev, debug)

NOTE: the patchset uses `query_options` as a container for
cached values. This doesn't look clean and `service::query_state`
seems to be a better place to store them. But it's not
forwarded to most of the CQL code and would mean that a huge number
of places would have to be amended.
The series presents a trade-off to avoid forwarding `query_state`
everywhere (but maybe it's the thing that needs to be done, nonetheless).
"

* 'lwt_bounce_to_shard_cached_fn_v6' of https://github.com/ManManson/scylla:
  cql-pytest: add a test for non-pure CQL functions
  cql3: cache function calls evaluation for non-deterministic functions
  cql3: rename `variable_specifications` to `prepare_context`
2021-07-30 14:21:11 +03:00

228 lines
7.4 KiB
C++

/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*
* Copyright (C) 2015-present ScyllaDB
*
* Modified by ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include "cql3/assignment_testable.hh"
#include "cql3/query_options.hh"
#include "cql3/values.hh"
namespace cql3 {
class term;
class terminal;
class prepare_context;
/**
* A parsed, non prepared (thus untyped) term.
*
* This can be one of:
* - a constant
* - a collection literal
* - a function call
* - a marker
*/
class term_raw : public virtual assignment_testable {
public:
/**
* This method validates this RawTerm is valid for provided column
* specification and "prepare" this RawTerm, returning the resulting
* prepared Term.
*
* @param receiver the "column" this RawTerm is supposed to be a value of. Note
* that the ColumnSpecification may not correspond to a real column in the
* case this RawTerm describe a list index or a map key, etc...
* @return the prepared term.
*/
virtual ::shared_ptr<term> prepare(database& db, const sstring& keyspace, lw_shared_ptr<column_specification> receiver) const = 0;
virtual sstring to_string() const = 0;
virtual sstring assignment_testable_source_context() const override {
return to_string();
}
friend std::ostream& operator<<(std::ostream& os, const term_raw& r) {
return os << r.to_string();
}
};
class multi_column_term_raw : public virtual term_raw {
public:
virtual ::shared_ptr<term> prepare(database& db, const sstring& keyspace, const std::vector<lw_shared_ptr<column_specification>>& receiver) const = 0;
};
/**
* A CQL3 term, i.e. a column value with or without bind variables.
*
* A Term can be either terminal or non terminal. A term object is one that is typed and is obtained
* from a raw term (Term.Raw) by poviding the actual receiver to which the term is supposed to be a
* value of.
*/
class term : public ::enable_shared_from_this<term> {
public:
virtual ~term() {}
/**
* Collects the column specification for the bind variables in this Term.
* This is obviously a no-op if the term is Terminal.
*
* @param boundNames the variables specification where to collect the
* bind variables of this term in.
*/
virtual void fill_prepare_context(prepare_context& ctx) const = 0;
/**
* Bind the values in this term to the values contained in {@code values}.
* This is obviously a no-op if the term is Terminal.
*
* @param options the values to bind markers to.
* @return the result of binding all the variables of this NonTerminal (or
* 'this' if the term is terminal).
*/
virtual ::shared_ptr<terminal> bind(const query_options& options) = 0;
/**
* A shorter for bind(values).get().
* We expose it mainly because for constants it can avoids allocating a temporary
* object between the bind and the get (note that we still want to be able
* to separate bind and get for collections).
*/
virtual cql3::raw_value_view bind_and_get(const query_options& options) = 0;
/**
* Whether or not that term contains at least one bind marker.
*
* Note that this is slightly different from being or not a NonTerminal,
* because calls to non pure functions will be NonTerminal (see #5616)
* even if they don't have bind markers.
*/
virtual bool contains_bind_marker() const = 0;
virtual sstring to_string() const {
return format("term@{:p}", static_cast<const void*>(this));
}
friend std::ostream& operator<<(std::ostream& out, const term& t) {
return out << t.to_string();
}
using raw = term_raw;
using multi_column_raw = multi_column_term_raw;
};
/**
* A terminal term, one that can be reduced to a byte buffer directly.
*
* This includes most terms that don't have a bind marker (an exception
* being delayed call for non pure function that are NonTerminal even
* if they don't have bind markers).
*
* This can be only one of:
* - a constant value
* - a collection value
*
* Note that a terminal term will always have been type checked, and thus
* consumer can (and should) assume so.
*/
class terminal : public term {
public:
virtual void fill_prepare_context(prepare_context& ctx) const {
}
virtual ::shared_ptr<terminal> bind(const query_options& options) override {
return static_pointer_cast<terminal>(this->shared_from_this());
}
// While some NonTerminal may not have bind markers, no Term can be Terminal
// with a bind marker
virtual bool contains_bind_marker() const override {
return false;
}
/**
* @return the serialized value of this terminal.
*/
virtual cql3::raw_value get(const query_options& options) = 0;
virtual cql3::raw_value_view bind_and_get(const query_options& options) override {
return raw_value_view::make_temporary(get(options));
}
virtual sstring to_string() const = 0;
};
class multi_item_terminal : public terminal {
public:
virtual std::vector<managed_bytes_opt> copy_elements() const = 0;
};
class collection_terminal {
public:
virtual ~collection_terminal() {}
/** Gets the value of the collection when serialized with the given protocol version format */
virtual managed_bytes get_with_protocol_version(cql_serialization_format sf) = 0;
};
/**
* A non terminal term, i.e. a term that can only be reduce to a byte buffer
* at execution time.
*
* We have the following type of NonTerminal:
* - marker for a constant value
* - marker for a collection value (list, set, map)
* - a function having bind marker
* - a non pure function (even if it doesn't have bind marker - see #5616)
*/
class non_terminal : public term {
public:
virtual cql3::raw_value_view bind_and_get(const query_options& options) override {
auto t = bind(options);
if (t) {
return cql3::raw_value_view::make_temporary(t->get(options));
}
return cql3::raw_value_view::make_null();
};
};
}