mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-21 00:50:35 +00:00
In this series we implement the UpdateTable operation to add a GSI to an existing table, or remove a GSI from a table. As the individual commit messages will explained, this required changing how Alternator stores materialized view keys - instead of insisting that these key must be real columns (that is **not** the case when adding a GSI to an existing table), the materialized view can now take as its key any Alternator attribute serialized inside the ":attrs" map holding all non-key attributes. Fixes #11567.
We also fix the IndexStatus and Backfilling attributes returned by DescribeTable - as DynamoDB API users use this API to discover when a newly added GSI completed its "backfilling" (what we call "view building") stage. Fixes #11471.
This series should not be backported lightly - it's a new feature and required fairly large and intrusive changes that can introduce bugs to use cases that don't even use Alternator or its UpdateTable operations - every user of CQL materialized views or secondary indexes, as well as Alternator GSI or LSI, will use modified code. **It should be backported to 2025.1**, though - this version was actually branched long after this PR was sent, and it provides a feature that was promised for 2025.1.
Closes scylladb/scylladb#21989
* github.com:scylladb/scylladb:
alternator: fix view build on oversized GSI key attribute
mv: clean up do_delete_old_entry
test/alternator: unflake test for IndexStatus
test/alternator: work around unrelated bug causing test flakiness
docs/alternator: adding a GSI is no longer an unimplemented feature
test/alternator: remove xfail from all tests for issue 11567
alternator: overhaul implementation of GSIs and support UpdateTable
mv: support regular_column_transformation key columns in view
alternator: add new materialized-view computed column for item in map
build: in cmake build, schema needs alternator
build: build tests with Alternator
alternator: add function serialized_value_if_type()
mv: introduce regular_column_transformation, a new type of computed column
alternator: add IndexStatus/Backfilling in DescribeTable
alternator: add "LimitExceededException" error type
docs/alternator: document two more unimplemented Alternator features
(cherry picked from commit 529ff3efa5)
Closes scylladb/scylladb#22826
107 lines
4.0 KiB
C++
107 lines
4.0 KiB
C++
/*
|
|
* Copyright 2019-present ScyllaDB
|
|
*/
|
|
|
|
/*
|
|
* SPDX-License-Identifier: LicenseRef-ScyllaDB-Source-Available-1.0
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
#include <string>
|
|
#include <string_view>
|
|
#include <optional>
|
|
#include "types/types.hh"
|
|
#include "schema/schema_fwd.hh"
|
|
#include "keys.hh"
|
|
#include "utils/rjson.hh"
|
|
#include "utils/big_decimal.hh"
|
|
|
|
class position_in_partition;
|
|
|
|
namespace alternator {
|
|
|
|
enum class alternator_type : int8_t {
|
|
S, B, BOOL, N, NOT_SUPPORTED_YET
|
|
};
|
|
|
|
struct type_info {
|
|
alternator_type atype;
|
|
data_type dtype;
|
|
};
|
|
|
|
struct type_representation {
|
|
std::string ident;
|
|
data_type dtype;
|
|
};
|
|
|
|
inline constexpr std::string_view scylla_paging_region(":scylla:paging:region");
|
|
inline constexpr std::string_view scylla_paging_weight(":scylla:paging:weight");
|
|
|
|
type_info type_info_from_string(std::string_view type);
|
|
type_representation represent_type(alternator_type atype);
|
|
|
|
bytes serialize_item(const rjson::value& item);
|
|
rjson::value deserialize_item(bytes_view bv);
|
|
std::optional<bytes> serialized_value_if_type(bytes_view bv, alternator_type expected_type);
|
|
|
|
std::string type_to_string(data_type type);
|
|
|
|
bytes get_key_column_value(const rjson::value& item, const column_definition& column);
|
|
bytes get_key_from_typed_value(const rjson::value& key_typed_value, const column_definition& column);
|
|
rjson::value json_key_column_value(bytes_view cell, const column_definition& column);
|
|
|
|
partition_key pk_from_json(const rjson::value& item, schema_ptr schema);
|
|
clustering_key ck_from_json(const rjson::value& item, schema_ptr schema);
|
|
position_in_partition pos_from_json(const rjson::value& item, schema_ptr schema);
|
|
|
|
// If v encodes a number (i.e., it is a {"N": [...]}, returns an object representing it. Otherwise,
|
|
// raises ValidationException with diagnostic.
|
|
big_decimal unwrap_number(const rjson::value& v, std::string_view diagnostic);
|
|
|
|
// try_unwrap_number is like unwrap_number, but returns an unset optional
|
|
// when the given v does not encode a number.
|
|
std::optional<big_decimal> try_unwrap_number(const rjson::value& v);
|
|
|
|
// unwrap_bytes decodes byte value, on decoding failure it either raises api_error::serialization
|
|
// iff from_query is true or returns unset optional iff from_query is false.
|
|
// Therefore it's safe to dereference returned optional when called with from_query equal true.
|
|
std::optional<bytes> unwrap_bytes(const rjson::value& value, bool from_query);
|
|
|
|
// Check if a given JSON object encodes a set (i.e., it is a {"SS": [...]}, or "NS", "BS"
|
|
// and returns set's type and a pointer to that set. If the object does not encode a set,
|
|
// returned value is {"", nullptr}
|
|
const std::pair<std::string, const rjson::value*> unwrap_set(const rjson::value& v);
|
|
|
|
// Check if a given JSON object encodes a list (i.e., it is a {"L": [...]}
|
|
// and returns a pointer to that list.
|
|
const rjson::value* unwrap_list(const rjson::value& v);
|
|
|
|
// Take two JSON-encoded numeric values ({"N": "thenumber"}) and return the
|
|
// sum, again as a JSON-encoded number.
|
|
rjson::value number_add(const rjson::value& v1, const rjson::value& v2);
|
|
rjson::value number_subtract(const rjson::value& v1, const rjson::value& v2);
|
|
// Take two JSON-encoded set values (e.g. {"SS": [...the actual set]}) and
|
|
// return the sum of both sets, again as a set value.
|
|
rjson::value set_sum(const rjson::value& v1, const rjson::value& v2);
|
|
// Take two JSON-encoded set values (e.g. {"SS": [...the actual list]}) and
|
|
// return the difference of s1 - s2, again as a set value.
|
|
// DynamoDB does not allow empty sets, so if resulting set is empty, return
|
|
// an unset optional instead.
|
|
std::optional<rjson::value> set_diff(const rjson::value& v1, const rjson::value& v2);
|
|
// Take two JSON-encoded list values (remember that a list value is
|
|
// {"L": [...the actual list]}) and return the concatenation, again as
|
|
// a list value.
|
|
// Returns a null value if one of the arguments is not actually a list.
|
|
rjson::value list_concatenate(const rjson::value& v1, const rjson::value& v2);
|
|
|
|
namespace internal {
|
|
struct magnitude_and_precision {
|
|
int magnitude;
|
|
int precision;
|
|
};
|
|
magnitude_and_precision get_magnitude_and_precision(std::string_view);
|
|
}
|
|
|
|
}
|