Compare commits

..

4 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
bb1ab98fc9 Remove unused unimplemented::cause enum values and document remaining ones
Co-authored-by: tgrabiec <283695+tgrabiec@users.noreply.github.com>
2025-12-05 22:26:58 +00:00
copilot-swe-agent[bot]
bdbc47a333 Restore unimplemented::cause::SUPER - still needed for error reporting
Co-authored-by: tgrabiec <283695+tgrabiec@users.noreply.github.com>
2025-12-05 21:01:46 +00:00
copilot-swe-agent[bot]
5407b6b43d Remove dead code for super table handling
Co-authored-by: tgrabiec <283695+tgrabiec@users.noreply.github.com>
2025-12-05 20:59:18 +00:00
copilot-swe-agent[bot]
8582156257 Initial plan 2025-12-05 20:48:35 +00:00
13 changed files with 135 additions and 246 deletions

View File

@@ -0,0 +1,109 @@
# Analysis of unimplemented::cause Enum Values
This document provides an analysis of the `unimplemented::cause` enum values after cleanup.
## Removed Unused Enum Values (20 values removed)
The following enum values had **zero usages** in the codebase and have been removed:
- `LWT` - Lightweight transactions
- `PAGING` - Query result paging
- `AUTH` - Authentication
- `PERMISSIONS` - Permission checking
- `COUNTERS` - Counter columns
- `MIGRATIONS` - Schema migrations
- `GOSSIP` - Gossip protocol
- `TOKEN_RESTRICTION` - Token-based restrictions
- `LEGACY_COMPOSITE_KEYS` - Legacy composite key handling
- `COLLECTION_RANGE_TOMBSTONES` - Collection range tombstones
- `RANGE_DELETES` - Range deletion operations
- `COMPRESSION` - Compression features
- `NONATOMIC` - Non-atomic operations
- `CONSISTENCY` - Consistency level handling
- `WRAP_AROUND` - Token wrap-around handling
- `STORAGE_SERVICE` - Storage service operations
- `SCHEMA_CHANGE` - Schema change operations
- `MIXED_CF` - Mixed column family operations
- `SSTABLE_FORMAT_M` - SSTable format M
## Remaining Enum Values (8 values kept)
### 1. `API` (4 usages)
**Impact**: REST API features that are not fully implemented.
**Usages**:
- `api/column_family.cc:1052` - Fails when `split_output` parameter is used in major compaction
- `api/compaction_manager.cc:100,146,216` - Warns when force_user_defined_compaction or related operations are called
**User Impact**: Some REST API endpoints for compaction management are stubs and will warn or fail.
### 2. `INDEXES` (6 usages)
**Impact**: Secondary index features not fully supported.
**Usages**:
- `api/column_family.cc:433,440,449,456` - Warns about index-related operations
- `cql3/restrictions/statement_restrictions.cc:1158` - Fails when attempting filtering on collection columns without proper indexing
- `cql3/statements/update_statement.cc:149` - Warns about index operations
**User Impact**: Some advanced secondary index features (especially filtering on collections) are not available.
### 3. `TRIGGERS` (2 usages)
**Impact**: Trigger support is not implemented.
**Usages**:
- `db/schema_tables.cc:2017` - Warns when loading trigger metadata from schema tables
- `service/storage_proxy.cc:4166` - Warns when processing trigger-related operations
**User Impact**: Cassandra triggers (stored procedures that execute on data changes) are not supported.
### 4. `METRICS` (1 usage)
**Impact**: Some query processor metrics are not collected.
**Usages**:
- `cql3/query_processor.cc:585` - Warns about missing metrics implementation
**User Impact**: Minor - some internal metrics may not be available.
### 5. `VALIDATION` (4 usages)
**Impact**: Schema validation checks are partially implemented.
**Usages**:
- `cql3/functions/token_fct.hh:38` - Warns about validation in token functions
- `cql3/statements/drop_keyspace_statement.cc:40` - Warns when dropping keyspace
- `cql3/statements/truncate_statement.cc:87` - Warns when truncating table
- `service/migration_manager.cc:750` - Warns during schema migrations
**User Impact**: Some schema validation checks are skipped (with warnings logged).
### 6. `REVERSED` (1 usage)
**Impact**: Reversed type support in CQL protocol.
**Usages**:
- `transport/server.cc:2085` - Fails when trying to use reversed types in CQL protocol
**User Impact**: Reversed types are not supported in the CQL protocol implementation.
### 7. `HINT` (1 usage)
**Impact**: Hint replaying is not implemented.
**Usages**:
- `db/batchlog_manager.cc:251` - Warns when attempting to replay hints
**User Impact**: Cassandra hints (temporary storage of writes when nodes are down) are not supported.
### 8. `SUPER` (2 usages)
**Impact**: Super column families are not supported.
**Usages**:
- `db/legacy_schema_migrator.cc:157` - Fails when encountering super column family in legacy schema
- `db/schema_tables.cc:2288` - Fails when encountering super column family in schema tables
**User Impact**: Super column families (legacy Cassandra feature) will cause errors if encountered in legacy data or schema migrations.
## Summary
- **Removed**: 20 unused enum values (76% reduction)
- **Kept**: 8 actively used enum values (24% remaining)
- **Total lines removed**: ~40 lines from enum definition and switch statement
The remaining enum values represent actual unimplemented features that users may encounter, with varying impacts ranging from warnings (TRIGGERS, METRICS, VALIDATION, HINT) to failures (API split_output, INDEXES on collections, REVERSED types, SUPER tables).

View File

@@ -716,7 +716,7 @@ batchStatement returns [std::unique_ptr<cql3::statements::raw::batch_statement>
auto attrs = std::make_unique<cql3::attributes::raw>();
}
: K_BEGIN
( K_UNLOGGED { type = btype::UNLOGGED; } | K_COUNTER { type = btype::COUNTER; } | K_GROUP0 { type = btype::GROUP0; } )?
( K_UNLOGGED { type = btype::UNLOGGED; } | K_COUNTER { type = btype::COUNTER; } )?
K_BATCH ( usingClause[attrs] )?
( s=batchStatementObjective ';'?
{
@@ -2374,7 +2374,6 @@ K_SCYLLA_CLUSTERING_BOUND: S C Y L L A '_' C L U S T E R I N G '_' B O U N D;
K_GROUP: G R O U P;
K_GROUP0: G R O U P '0';
K_LIKE: L I K E;

View File

@@ -186,10 +186,6 @@ void alter_table_statement::add_column(const query_options&, const schema& schem
if (!schema.is_compound()) {
throw exceptions::invalid_request_exception("Cannot use non-frozen collections with a non-composite PRIMARY KEY");
}
if (schema.is_super()) {
throw exceptions::invalid_request_exception("Cannot use non-frozen collections with super column families");
}
// If there used to be a non-frozen collection column with the same name (that has been dropped),
// we could still have some data using the old type, and so we can't allow adding a collection

View File

@@ -31,13 +31,9 @@ logging::logger batch_statement::_logger("BatchStatement");
timeout_config_selector
timeout_for_type(batch_statement::type t) {
if (t == batch_statement::type::COUNTER) {
return &timeout_config::counter_write_timeout;
} else if (t == batch_statement::type::GROUP0) {
return &timeout_config::other_timeout;
} else {
return &timeout_config::write_timeout;
}
return t == batch_statement::type::COUNTER
? &timeout_config::counter_write_timeout
: &timeout_config::write_timeout;
}
db::timeout_clock::duration batch_statement::get_timeout(const service::client_state& state, const query_options& options) const {
@@ -94,11 +90,6 @@ future<> batch_statement::check_access(query_processor& qp, const service::clien
});
}
bool batch_statement::needs_guard(query_processor& qp, service::query_state& state) const
{
return _type == type::GROUP0;
}
void batch_statement::validate()
{
if (_attrs->is_time_to_live_set()) {
@@ -113,22 +104,6 @@ void batch_statement::validate()
if (_type == type::COUNTER) {
throw exceptions::invalid_request_exception("Cannot provide custom timestamp for counter BATCH");
}
if (_type == type::GROUP0) {
throw exceptions::invalid_request_exception("Cannot provide custom timestamp for GROUP0 BATCH");
}
}
if (_type == type::GROUP0) {
if (_has_conditions) {
throw exceptions::invalid_request_exception("Cannot use conditions in GROUP0 BATCH");
}
// Validate that all statements target system keyspace tables managed by group0
for (auto& s : _statements) {
if (s.statement->keyspace() != "system") {
throw exceptions::invalid_request_exception("GROUP0 BATCH can only modify system keyspace tables");
}
}
return;
}
bool has_counters = std::ranges::any_of(_statements, [] (auto&& s) { return s.statement->is_counter(); });
@@ -260,9 +235,6 @@ static thread_local inheriting_concrete_execution_stage<
future<shared_ptr<cql_transport::messages::result_message>> batch_statement::execute(
query_processor& qp, service::query_state& state, const query_options& options, std::optional<service::group0_guard> guard) const {
if (_type == type::GROUP0) {
return execute_group0_batch(qp, state, options, std::move(guard));
}
return execute_without_checking_exception_message(qp, state, options, std::move(guard))
.then(cql_transport::messages::propagate_exception_as_future<shared_ptr<cql_transport::messages::result_message>>);
}
@@ -313,39 +285,6 @@ future<shared_ptr<cql_transport::messages::result_message>> batch_statement::do_
});
}
future<shared_ptr<cql_transport::messages::result_message>> batch_statement::execute_group0_batch(
query_processor& qp,
service::query_state& query_state, const query_options& options,
std::optional<service::group0_guard> guard) const
{
if (!guard) {
throw exceptions::invalid_request_exception("GROUP0 BATCH requires a guard");
}
auto timeout = db::timeout_clock::now() + get_timeout(query_state.get_client_state(), options);
// Create group0_batch and get the timestamp from it
service::group0_batch mc{std::move(guard)};
auto now = mc.write_timestamp();
// Get mutations from all statements
auto mutations = co_await get_mutations(qp, options, timeout, false, now, query_state);
// Add mutations to the group0_batch
mc.add_mutations(std::move(mutations), format("CQL GROUP0 BATCH: \"{}\"", raw_cql_statement));
// Announce the batch via group0
auto description = format("CQL GROUP0 BATCH: \"{}\"", raw_cql_statement);
auto [remote_, holder] = qp.remote();
auto [m, g] = co_await std::move(mc).extract();
if (!m.empty()) {
co_await remote_.get().mm.announce(std::move(m), std::move(g), description);
}
co_return make_shared<cql_transport::messages::result_message::void_message>();
}
future<coordinator_result<>> batch_statement::execute_without_conditions(
query_processor& qp,
utils::chunked_vector<mutation> mutations,

View File

@@ -95,8 +95,6 @@ public:
virtual future<> check_access(query_processor& qp, const service::client_state& state) const override;
virtual bool needs_guard(query_processor& qp, service::query_state& state) const override;
// Validates a prepared batch statement without validating its nested statements.
void validate();
@@ -132,11 +130,6 @@ private:
service::query_state& query_state, const query_options& options,
bool local, api::timestamp_type now) const;
future<shared_ptr<cql_transport::messages::result_message>> execute_group0_batch(
query_processor& qp,
service::query_state& query_state, const query_options& options,
std::optional<service::group0_guard> guard) const;
future<exceptions::coordinator_result<>> execute_without_conditions(
query_processor& qp,
utils::chunked_vector<mutation> mutations,

View File

@@ -23,7 +23,7 @@ class modification_statement;
class batch_statement : public raw::cf_statement {
public:
enum class type {
LOGGED, UNLOGGED, COUNTER, GROUP0
LOGGED, UNLOGGED, COUNTER
};
private:
type _type;

View File

@@ -152,8 +152,8 @@ public:
builder.with_version(sm.digest());
cf_type cf = sstring_to_cf_type(td.get_or("type", sstring("standard")));
if (cf == cf_type::super) {
auto type_str = td.get_or("type", sstring("standard"));
if (type_str == "Super") {
fail(unimplemented::cause::SUPER);
}
@@ -284,13 +284,8 @@ public:
if (kind_str == "compact_value") {
continue;
}
if (kind == column_kind::clustering_key) {
if (cf == cf_type::super && component_index != 0) {
continue;
}
if (cf != cf_type::super && !is_compound) {
continue;
}
if (kind == column_kind::clustering_key && !is_compound) {
continue;
}
}

View File

@@ -143,7 +143,7 @@ static computed_columns_map get_computed_columns(const schema_mutations& sm);
static std::vector<column_definition> create_columns_from_column_rows(
const query::result_set& rows, const sstring& keyspace,
const sstring& table, bool is_super, column_view_virtual is_view_virtual, const computed_columns_map& computed_columns,
const sstring& table, column_view_virtual is_view_virtual, const computed_columns_map& computed_columns,
const data_dictionary::user_types_storage& user_types);
@@ -1804,9 +1804,6 @@ static schema_mutations make_table_mutations(schema_ptr table, api::timestamp_ty
auto scylla_tables_mutation = make_scylla_tables_mutation(table, timestamp);
list_type_impl::native_type flags;
if (table->is_super()) {
flags.emplace_back("super");
}
if (table->is_dense()) {
flags.emplace_back("dense");
}
@@ -2280,7 +2277,6 @@ schema_ptr create_table_from_mutations(const schema_ctxt& ctxt, schema_mutations
auto id = table_id(table_row.get_nonnull<utils::UUID>("id"));
schema_builder builder{ks_name, cf_name, id};
auto cf = cf_type::standard;
auto is_dense = false;
auto is_counter = false;
auto is_compound = false;
@@ -2289,7 +2285,6 @@ schema_ptr create_table_from_mutations(const schema_ctxt& ctxt, schema_mutations
if (flags) {
for (auto& s : *flags) {
if (s == "super") {
// cf = cf_type::super;
fail(unimplemented::cause::SUPER);
} else if (s == "dense") {
is_dense = true;
@@ -2305,9 +2300,7 @@ schema_ptr create_table_from_mutations(const schema_ctxt& ctxt, schema_mutations
std::vector<column_definition> column_defs = create_columns_from_column_rows(
query::result_set(sm.columns_mutation()),
ks_name,
cf_name,/*,
fullRawComparator, */
cf == cf_type::super,
cf_name,
column_view_virtual::no,
computed_columns,
user_types);
@@ -2486,9 +2479,7 @@ static computed_columns_map get_computed_columns(const schema_mutations& sm) {
static std::vector<column_definition> create_columns_from_column_rows(
const query::result_set& rows,
const sstring& keyspace,
const sstring& table, /*,
AbstractType<?> rawComparator, */
bool is_super,
const sstring& table,
column_view_virtual is_view_virtual,
const computed_columns_map& computed_columns,
const data_dictionary::user_types_storage& user_types)
@@ -2565,12 +2556,12 @@ static schema_builder prepare_view_schema_builder_from_mutations(const schema_ct
}
auto computed_columns = get_computed_columns(sm);
auto column_defs = create_columns_from_column_rows(query::result_set(sm.columns_mutation()), ks_name, cf_name, false, column_view_virtual::no, computed_columns, user_types);
auto column_defs = create_columns_from_column_rows(query::result_set(sm.columns_mutation()), ks_name, cf_name, column_view_virtual::no, computed_columns, user_types);
for (auto&& cdef : column_defs) {
builder.with_column_ordered(cdef);
}
if (sm.view_virtual_columns_mutation()) {
column_defs = create_columns_from_column_rows(query::result_set(*sm.view_virtual_columns_mutation()), ks_name, cf_name, false, column_view_virtual::yes, computed_columns, user_types);
column_defs = create_columns_from_column_rows(query::result_set(*sm.view_virtual_columns_mutation()), ks_name, cf_name, column_view_virtual::yes, computed_columns, user_types);
for (auto&& cdef : column_defs) {
builder.with_column_ordered(cdef);
}

View File

@@ -10,7 +10,7 @@ Multiple ``INSERT``, ``UPDATE`` and ``DELETE`` can be executed in a single state
.. code-block::
batch_statement: BEGIN [ UNLOGGED | COUNTER | GROUP0 ] BATCH
batch_statement: BEGIN [ UNLOGGED | COUNTER ] BATCH
: [ USING `update_parameter` ( AND `update_parameter` )* ]
: `modification_statement` ( ';' `modification_statement` )*
: APPLY BATCH
@@ -67,29 +67,6 @@ used, a failed batch might leave the batch only partly applied.
Use the ``COUNTER`` option for batched counter updates. Unlike other
updates in ScyllaDB, counter updates are not idempotent.
``GROUP0`` batches
~~~~~~~~~~~~~~~~~~
Use the ``GROUP0`` option for batched modifications to system tables that are managed by group0
(e.g., ``system.topology``). GROUP0 batches execute mutations as a group0 command, ensuring they
are replicated through the Raft consensus protocol.
GROUP0 batches have the following restrictions:
- Can only modify tables in the ``system`` keyspace
- Cannot use custom timestamps (``USING TIMESTAMP`` is not allowed)
- Cannot use conditional statements (``IF EXISTS``, ``IF NOT EXISTS``, etc.)
- Requires a group0 guard to be taken before execution
Example:
.. code-block:: cql
BEGIN GROUP0 BATCH
INSERT INTO system.topology (key, value) VALUES ('node1', 'data1');
UPDATE system.topology SET value = 'data2' WHERE key = 'node2';
APPLY BATCH;
:doc:`Apache Cassandra Query Language (CQL) Reference </cql/index>`

View File

@@ -134,14 +134,11 @@ bool is_compatible(column_kind k1, column_kind k2);
enum class cf_type : uint8_t {
standard,
super,
};
inline sstring cf_type_to_sstring(cf_type t) {
if (t == cf_type::standard) {
return "Standard";
} else if (t == cf_type::super) {
return "Super";
}
throw std::invalid_argument(format("unknown type: {:d}\n", uint8_t(t)));
}
@@ -149,8 +146,6 @@ inline sstring cf_type_to_sstring(cf_type t) {
inline cf_type sstring_to_cf_type(sstring name) {
if (name == "Standard") {
return cf_type::standard;
} else if (name == "Super") {
return cf_type::super;
}
throw std::invalid_argument(format("unknown type: {}\n", name));
}
@@ -688,13 +683,13 @@ public:
}
bool is_cql3_table() const {
return !is_super() && !is_dense() && is_compound();
return !is_dense() && is_compound();
}
bool is_compact_table() const {
return !is_cql3_table();
}
bool is_static_compact_table() const {
return !is_super() && !is_dense() && !is_compound();
return !is_dense() && !is_compound();
}
const table_id& id() const {
@@ -711,10 +706,6 @@ public:
return _raw._type;
}
bool is_super() const {
return _raw._type == cf_type::super;
}
gc_clock::duration gc_grace_seconds() const {
auto seconds = std::chrono::seconds(_raw._props.gc_grace_seconds);
return std::chrono::duration_cast<gc_clock::duration>(seconds);

View File

@@ -1,63 +0,0 @@
# -*- coding: utf-8 -*-
# Copyright 2024-present ScyllaDB
#
# SPDX-License-Identifier: LicenseRef-ScyllaDB-Source-Available-1.0
#############################################################################
# Tests for GROUP0 BATCH operations
#############################################################################
from cassandra import InvalidRequest
import pytest
def test_group0_batch_syntax_error_for_non_system_table(cql, test_keyspace):
"""Verifies that GROUP0 BATCH can only be used with system keyspace tables"""
with pytest.raises(InvalidRequest, match="GROUP0 BATCH can only modify system keyspace tables"):
# Create a test table in a non-system keyspace
table_name = f"{test_keyspace}.test_table"
cql.execute(f"CREATE TABLE {table_name} (k int PRIMARY KEY, v int)")
try:
# Try to use GROUP0 BATCH with a non-system table
cql.execute(f"""
BEGIN GROUP0 BATCH
INSERT INTO {table_name} (k, v) VALUES (1, 1)
APPLY BATCH
""")
finally:
cql.execute(f"DROP TABLE IF EXISTS {table_name}")
def test_group0_batch_with_timestamp_error(cql):
"""Verifies that GROUP0 BATCH cannot have custom timestamp"""
with pytest.raises(InvalidRequest, match="Cannot provide custom timestamp for GROUP0 BATCH"):
cql.execute("""
BEGIN GROUP0 BATCH USING TIMESTAMP 12345
INSERT INTO system.topology (key) VALUES ('test')
APPLY BATCH
""")
def test_group0_batch_with_conditions_error(cql):
"""Verifies that GROUP0 BATCH cannot have conditions"""
with pytest.raises(InvalidRequest, match="Cannot use conditions in GROUP0 BATCH"):
cql.execute("""
BEGIN GROUP0 BATCH
INSERT INTO system.topology (key) VALUES ('test') IF NOT EXISTS
APPLY BATCH
""")
def test_group0_batch_basic_syntax(cql):
"""Verifies that GROUP0 BATCH has correct basic syntax"""
# This test just checks that the syntax is recognized
# The actual execution will fail if not properly set up with group0
# but the syntax should be accepted
try:
cql.execute("""
BEGIN GROUP0 BATCH
APPLY BATCH
""")
except Exception as e:
# Accept either success or group0-related errors, but not syntax errors
error_msg = str(e).lower()
assert "syntax" not in error_msg and "unexpected" not in error_msg, \
f"GROUP0 BATCH should be valid syntax, but got: {e}"

View File

@@ -21,33 +21,14 @@ static logging::logger ulogger("unimplemented");
std::string_view format_as(cause c) {
switch (c) {
case cause::API: return "API";
case cause::INDEXES: return "INDEXES";
case cause::LWT: return "LWT";
case cause::PAGING: return "PAGING";
case cause::AUTH: return "AUTH";
case cause::PERMISSIONS: return "PERMISSIONS";
case cause::TRIGGERS: return "TRIGGERS";
case cause::COUNTERS: return "COUNTERS";
case cause::METRICS: return "METRICS";
case cause::MIGRATIONS: return "MIGRATIONS";
case cause::GOSSIP: return "GOSSIP";
case cause::TOKEN_RESTRICTION: return "TOKEN_RESTRICTION";
case cause::LEGACY_COMPOSITE_KEYS: return "LEGACY_COMPOSITE_KEYS";
case cause::COLLECTION_RANGE_TOMBSTONES: return "COLLECTION_RANGE_TOMBSTONES";
case cause::RANGE_DELETES: return "RANGE_DELETES";
case cause::VALIDATION: return "VALIDATION";
case cause::REVERSED: return "REVERSED";
case cause::COMPRESSION: return "COMPRESSION";
case cause::NONATOMIC: return "NONATOMIC";
case cause::CONSISTENCY: return "CONSISTENCY";
case cause::HINT: return "HINT";
case cause::SUPER: return "SUPER";
case cause::WRAP_AROUND: return "WRAP_AROUND";
case cause::STORAGE_SERVICE: return "STORAGE_SERVICE";
case cause::API: return "API";
case cause::SCHEMA_CHANGE: return "SCHEMA_CHANGE";
case cause::MIXED_CF: return "MIXED_CF";
case cause::SSTABLE_FORMAT_M: return "SSTABLE_FORMAT_M";
}
abort();
}

View File

@@ -15,33 +15,14 @@
namespace unimplemented {
enum class cause {
API,
INDEXES,
LWT,
PAGING,
AUTH,
PERMISSIONS,
TRIGGERS,
COUNTERS,
METRICS,
MIGRATIONS,
GOSSIP,
TOKEN_RESTRICTION,
LEGACY_COMPOSITE_KEYS,
COLLECTION_RANGE_TOMBSTONES,
RANGE_DELETES,
VALIDATION,
REVERSED,
COMPRESSION,
NONATOMIC,
CONSISTENCY,
HINT,
SUPER,
WRAP_AROUND, // Support for handling wrap around ranges in queries on database level and below
STORAGE_SERVICE,
SCHEMA_CHANGE,
MIXED_CF,
SSTABLE_FORMAT_M,
API, // REST API features not implemented (force_user_defined_compaction, split_output in major compaction)
INDEXES, // Secondary index features (filtering on collections, clustering columns)
TRIGGERS, // Trigger support in schema tables and storage proxy
METRICS, // Query processor metrics
VALIDATION, // Schema validation in DDL statements (drop keyspace, truncate, token functions)
REVERSED, // Reversed types in CQL protocol
HINT, // Hint replaying in batchlog manager
SUPER, // Super column families (legacy Cassandra feature, never supported)
};
[[noreturn]] void fail(cause what);