Compare commits

..

6 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
1eca538536 Use "this auto" instead of coroutine::lambda wrapper
Co-authored-by: tgrabiec <283695+tgrabiec@users.noreply.github.com>
2025-12-06 18:21:38 +00:00
copilot-swe-agent[bot]
1ad7a5e0f1 Revert false positives - only keep fix for hint_endpoint_manager.cc
Co-authored-by: tgrabiec <283695+tgrabiec@users.noreply.github.com>
2025-12-06 18:08:39 +00:00
copilot-swe-agent[bot]
6ed8f1a24b Fix additional lambda-coroutine fiasco in streaming/consumer.cc
Co-authored-by: tgrabiec <283695+tgrabiec@users.noreply.github.com>
2025-12-06 17:05:59 +00:00
copilot-swe-agent[bot]
a87e72004d Fix lambda-coroutine fiasco issues in 4 files
Co-authored-by: tgrabiec <283695+tgrabiec@users.noreply.github.com>
2025-12-06 17:01:45 +00:00
copilot-swe-agent[bot]
e7317e9f39 Initial plan 2025-12-06 16:50:52 +00:00
Nadav Har'El
350cbd1d66 alternator: fix typo of BatchWriteItem in comments
The DynamoDB API's "BatchWriteItem" operation is spelled like this, in
singular. Some comments incorrectly referred to as BatchWriteItems - in
plural. This patch fixes those mistakes.

There are no functional changes here or changes to user-facing documents -
these mistakes were only in code comments.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>

Closes scylladb/scylladb#27446
2025-12-05 15:08:58 +02:00
10 changed files with 92 additions and 136 deletions

View File

@@ -1,109 +0,0 @@
# Analysis of unimplemented::cause Enum Values
This document provides an analysis of the `unimplemented::cause` enum values after cleanup.
## Removed Unused Enum Values (20 values removed)
The following enum values had **zero usages** in the codebase and have been removed:
- `LWT` - Lightweight transactions
- `PAGING` - Query result paging
- `AUTH` - Authentication
- `PERMISSIONS` - Permission checking
- `COUNTERS` - Counter columns
- `MIGRATIONS` - Schema migrations
- `GOSSIP` - Gossip protocol
- `TOKEN_RESTRICTION` - Token-based restrictions
- `LEGACY_COMPOSITE_KEYS` - Legacy composite key handling
- `COLLECTION_RANGE_TOMBSTONES` - Collection range tombstones
- `RANGE_DELETES` - Range deletion operations
- `COMPRESSION` - Compression features
- `NONATOMIC` - Non-atomic operations
- `CONSISTENCY` - Consistency level handling
- `WRAP_AROUND` - Token wrap-around handling
- `STORAGE_SERVICE` - Storage service operations
- `SCHEMA_CHANGE` - Schema change operations
- `MIXED_CF` - Mixed column family operations
- `SSTABLE_FORMAT_M` - SSTable format M
## Remaining Enum Values (8 values kept)
### 1. `API` (4 usages)
**Impact**: REST API features that are not fully implemented.
**Usages**:
- `api/column_family.cc:1052` - Fails when `split_output` parameter is used in major compaction
- `api/compaction_manager.cc:100,146,216` - Warns when force_user_defined_compaction or related operations are called
**User Impact**: Some REST API endpoints for compaction management are stubs and will warn or fail.
### 2. `INDEXES` (6 usages)
**Impact**: Secondary index features not fully supported.
**Usages**:
- `api/column_family.cc:433,440,449,456` - Warns about index-related operations
- `cql3/restrictions/statement_restrictions.cc:1158` - Fails when attempting filtering on collection columns without proper indexing
- `cql3/statements/update_statement.cc:149` - Warns about index operations
**User Impact**: Some advanced secondary index features (especially filtering on collections) are not available.
### 3. `TRIGGERS` (2 usages)
**Impact**: Trigger support is not implemented.
**Usages**:
- `db/schema_tables.cc:2017` - Warns when loading trigger metadata from schema tables
- `service/storage_proxy.cc:4166` - Warns when processing trigger-related operations
**User Impact**: Cassandra triggers (stored procedures that execute on data changes) are not supported.
### 4. `METRICS` (1 usage)
**Impact**: Some query processor metrics are not collected.
**Usages**:
- `cql3/query_processor.cc:585` - Warns about missing metrics implementation
**User Impact**: Minor - some internal metrics may not be available.
### 5. `VALIDATION` (4 usages)
**Impact**: Schema validation checks are partially implemented.
**Usages**:
- `cql3/functions/token_fct.hh:38` - Warns about validation in token functions
- `cql3/statements/drop_keyspace_statement.cc:40` - Warns when dropping keyspace
- `cql3/statements/truncate_statement.cc:87` - Warns when truncating table
- `service/migration_manager.cc:750` - Warns during schema migrations
**User Impact**: Some schema validation checks are skipped (with warnings logged).
### 6. `REVERSED` (1 usage)
**Impact**: Reversed type support in CQL protocol.
**Usages**:
- `transport/server.cc:2085` - Fails when trying to use reversed types in CQL protocol
**User Impact**: Reversed types are not supported in the CQL protocol implementation.
### 7. `HINT` (1 usage)
**Impact**: Hint replaying is not implemented.
**Usages**:
- `db/batchlog_manager.cc:251` - Warns when attempting to replay hints
**User Impact**: Cassandra hints (temporary storage of writes when nodes are down) are not supported.
### 8. `SUPER` (2 usages)
**Impact**: Super column families are not supported.
**Usages**:
- `db/legacy_schema_migrator.cc:157` - Fails when encountering super column family in legacy schema
- `db/schema_tables.cc:2288` - Fails when encountering super column family in schema tables
**User Impact**: Super column families (legacy Cassandra feature) will cause errors if encountered in legacy data or schema migrations.
## Summary
- **Removed**: 20 unused enum values (76% reduction)
- **Kept**: 8 actively used enum values (24% remaining)
- **Total lines removed**: ~40 lines from enum definition and switch statement
The remaining enum values represent actual unimplemented features that users may encounter, with varying impacts ranging from warnings (TRIGGERS, METRICS, VALIDATION, HINT) to failures (API split_output, INDEXES on collections, REVERSED types, SUPER tables).

View File

@@ -2223,12 +2223,12 @@ void validate_value(const rjson::value& v, const char* caller) {
// The put_or_delete_item class builds the mutations needed by the PutItem and
// DeleteItem operations - either as stand-alone commands or part of a list
// of commands in BatchWriteItems.
// of commands in BatchWriteItem.
// put_or_delete_item splits each operation into two stages: Constructing the
// object parses and validates the user input (throwing exceptions if there
// are input errors). Later, build() generates the actual mutation, with a
// specified timestamp. This split is needed because of the peculiar needs of
// BatchWriteItems and LWT. BatchWriteItems needs all parsing to happen before
// BatchWriteItem and LWT. BatchWriteItem needs all parsing to happen before
// any writing happens (if one of the commands has an error, none of the
// writes should be done). LWT makes it impossible for the parse step to
// generate "mutation" objects, because the timestamp still isn't known.
@@ -3026,7 +3026,7 @@ struct primary_key_equal {
};
// This is a cas_request subclass for applying given put_or_delete_items to
// one partition using LWT as part as BatchWriteItems. This is a write-only
// one partition using LWT as part as BatchWriteItem. This is a write-only
// operation, not needing the previous value of the item (the mutation to be
// done is known prior to starting the operation). Nevertheless, we want to
// do this mutation via LWT to ensure that it is serialized with other LWT
@@ -3065,7 +3065,7 @@ static future<> cas_write(service::storage_proxy& proxy, schema_ptr schema, serv
{timeout, std::move(permit), client_state, trace_state},
db::consistency_level::LOCAL_SERIAL, db::consistency_level::LOCAL_QUORUM,
timeout, timeout, true, std::move(cdc_opts)).discard_result();
// We discarded cas()'s future value ("is_applied") because BatchWriteItems
// We discarded cas()'s future value ("is_applied") because BatchWriteItem
// does not need to support conditional updates.
}

View File

@@ -186,6 +186,10 @@ void alter_table_statement::add_column(const query_options&, const schema& schem
if (!schema.is_compound()) {
throw exceptions::invalid_request_exception("Cannot use non-frozen collections with a non-composite PRIMARY KEY");
}
if (schema.is_super()) {
throw exceptions::invalid_request_exception("Cannot use non-frozen collections with super column families");
}
// If there used to be a non-frozen collection column with the same name (that has been dropped),
// we could still have some data using the old type, and so we can't allow adding a collection

View File

@@ -248,7 +248,7 @@ future<db::commitlog> hint_endpoint_manager::add_store() noexcept {
// which is larger than the segment ID of the RP of the last written hint.
cfg.base_segment_id = _last_written_rp.base_id();
return commitlog::create_commitlog(std::move(cfg)).then([this] (commitlog l) -> future<commitlog> {
return commitlog::create_commitlog(std::move(cfg)).then([this] (this auto, commitlog l) -> future<commitlog> {
// add_store() is triggered every time hint files are forcefully flushed to I/O (every hints_flush_period).
// When this happens we want to refill _sender's segments only if it has finished with the segments he had before.
if (_sender.have_segments()) {

View File

@@ -152,8 +152,8 @@ public:
builder.with_version(sm.digest());
auto type_str = td.get_or("type", sstring("standard"));
if (type_str == "Super") {
cf_type cf = sstring_to_cf_type(td.get_or("type", sstring("standard")));
if (cf == cf_type::super) {
fail(unimplemented::cause::SUPER);
}
@@ -284,8 +284,13 @@ public:
if (kind_str == "compact_value") {
continue;
}
if (kind == column_kind::clustering_key && !is_compound) {
continue;
if (kind == column_kind::clustering_key) {
if (cf == cf_type::super && component_index != 0) {
continue;
}
if (cf != cf_type::super && !is_compound) {
continue;
}
}
}

View File

@@ -143,7 +143,7 @@ static computed_columns_map get_computed_columns(const schema_mutations& sm);
static std::vector<column_definition> create_columns_from_column_rows(
const query::result_set& rows, const sstring& keyspace,
const sstring& table, column_view_virtual is_view_virtual, const computed_columns_map& computed_columns,
const sstring& table, bool is_super, column_view_virtual is_view_virtual, const computed_columns_map& computed_columns,
const data_dictionary::user_types_storage& user_types);
@@ -1804,6 +1804,9 @@ static schema_mutations make_table_mutations(schema_ptr table, api::timestamp_ty
auto scylla_tables_mutation = make_scylla_tables_mutation(table, timestamp);
list_type_impl::native_type flags;
if (table->is_super()) {
flags.emplace_back("super");
}
if (table->is_dense()) {
flags.emplace_back("dense");
}
@@ -2277,6 +2280,7 @@ schema_ptr create_table_from_mutations(const schema_ctxt& ctxt, schema_mutations
auto id = table_id(table_row.get_nonnull<utils::UUID>("id"));
schema_builder builder{ks_name, cf_name, id};
auto cf = cf_type::standard;
auto is_dense = false;
auto is_counter = false;
auto is_compound = false;
@@ -2285,6 +2289,7 @@ schema_ptr create_table_from_mutations(const schema_ctxt& ctxt, schema_mutations
if (flags) {
for (auto& s : *flags) {
if (s == "super") {
// cf = cf_type::super;
fail(unimplemented::cause::SUPER);
} else if (s == "dense") {
is_dense = true;
@@ -2300,7 +2305,9 @@ schema_ptr create_table_from_mutations(const schema_ctxt& ctxt, schema_mutations
std::vector<column_definition> column_defs = create_columns_from_column_rows(
query::result_set(sm.columns_mutation()),
ks_name,
cf_name,
cf_name,/*,
fullRawComparator, */
cf == cf_type::super,
column_view_virtual::no,
computed_columns,
user_types);
@@ -2479,7 +2486,9 @@ static computed_columns_map get_computed_columns(const schema_mutations& sm) {
static std::vector<column_definition> create_columns_from_column_rows(
const query::result_set& rows,
const sstring& keyspace,
const sstring& table,
const sstring& table, /*,
AbstractType<?> rawComparator, */
bool is_super,
column_view_virtual is_view_virtual,
const computed_columns_map& computed_columns,
const data_dictionary::user_types_storage& user_types)
@@ -2556,12 +2565,12 @@ static schema_builder prepare_view_schema_builder_from_mutations(const schema_ct
}
auto computed_columns = get_computed_columns(sm);
auto column_defs = create_columns_from_column_rows(query::result_set(sm.columns_mutation()), ks_name, cf_name, column_view_virtual::no, computed_columns, user_types);
auto column_defs = create_columns_from_column_rows(query::result_set(sm.columns_mutation()), ks_name, cf_name, false, column_view_virtual::no, computed_columns, user_types);
for (auto&& cdef : column_defs) {
builder.with_column_ordered(cdef);
}
if (sm.view_virtual_columns_mutation()) {
column_defs = create_columns_from_column_rows(query::result_set(*sm.view_virtual_columns_mutation()), ks_name, cf_name, column_view_virtual::yes, computed_columns, user_types);
column_defs = create_columns_from_column_rows(query::result_set(*sm.view_virtual_columns_mutation()), ks_name, cf_name, false, column_view_virtual::yes, computed_columns, user_types);
for (auto&& cdef : column_defs) {
builder.with_column_ordered(cdef);
}

View File

@@ -134,11 +134,14 @@ bool is_compatible(column_kind k1, column_kind k2);
enum class cf_type : uint8_t {
standard,
super,
};
inline sstring cf_type_to_sstring(cf_type t) {
if (t == cf_type::standard) {
return "Standard";
} else if (t == cf_type::super) {
return "Super";
}
throw std::invalid_argument(format("unknown type: {:d}\n", uint8_t(t)));
}
@@ -146,6 +149,8 @@ inline sstring cf_type_to_sstring(cf_type t) {
inline cf_type sstring_to_cf_type(sstring name) {
if (name == "Standard") {
return cf_type::standard;
} else if (name == "Super") {
return cf_type::super;
}
throw std::invalid_argument(format("unknown type: {}\n", name));
}
@@ -683,13 +688,13 @@ public:
}
bool is_cql3_table() const {
return !is_dense() && is_compound();
return !is_super() && !is_dense() && is_compound();
}
bool is_compact_table() const {
return !is_cql3_table();
}
bool is_static_compact_table() const {
return !is_dense() && !is_compound();
return !is_super() && !is_dense() && !is_compound();
}
const table_id& id() const {
@@ -706,6 +711,10 @@ public:
return _raw._type;
}
bool is_super() const {
return _raw._type == cf_type::super;
}
gc_clock::duration gc_grace_seconds() const {
auto seconds = std::chrono::seconds(_raw._props.gc_grace_seconds);
return std::chrono::duration_cast<gc_clock::duration>(seconds);

View File

@@ -205,7 +205,7 @@ def test_batch_write_invalid_operation(test_table_s):
# In test_item.py we have a bunch of test_empty_* tests on different ways to
# create an empty item (which in Scylla requires the special CQL row marker
# to be supported correctly). BatchWriteItems provides yet another way of
# to be supported correctly). BatchWriteItem provides yet another way of
# creating items, so check the empty case here too:
def test_empty_batch_write(test_table):
p = random_string()
@@ -214,7 +214,7 @@ def test_empty_batch_write(test_table):
batch.put_item({'p': p, 'c': c})
assert test_table.get_item(Key={'p': p, 'c': c}, ConsistentRead=True)['Item'] == {'p': p, 'c': c}
# Test that BatchWriteItems allows writing to multiple tables in one operation
# Test that BatchWriteItem allows writing to multiple tables in one operation
def test_batch_write_multiple_tables(test_table_s, test_table):
p1 = random_string()
c1 = random_string()

View File

@@ -21,14 +21,33 @@ static logging::logger ulogger("unimplemented");
std::string_view format_as(cause c) {
switch (c) {
case cause::API: return "API";
case cause::INDEXES: return "INDEXES";
case cause::LWT: return "LWT";
case cause::PAGING: return "PAGING";
case cause::AUTH: return "AUTH";
case cause::PERMISSIONS: return "PERMISSIONS";
case cause::TRIGGERS: return "TRIGGERS";
case cause::COUNTERS: return "COUNTERS";
case cause::METRICS: return "METRICS";
case cause::MIGRATIONS: return "MIGRATIONS";
case cause::GOSSIP: return "GOSSIP";
case cause::TOKEN_RESTRICTION: return "TOKEN_RESTRICTION";
case cause::LEGACY_COMPOSITE_KEYS: return "LEGACY_COMPOSITE_KEYS";
case cause::COLLECTION_RANGE_TOMBSTONES: return "COLLECTION_RANGE_TOMBSTONES";
case cause::RANGE_DELETES: return "RANGE_DELETES";
case cause::VALIDATION: return "VALIDATION";
case cause::REVERSED: return "REVERSED";
case cause::COMPRESSION: return "COMPRESSION";
case cause::NONATOMIC: return "NONATOMIC";
case cause::CONSISTENCY: return "CONSISTENCY";
case cause::HINT: return "HINT";
case cause::SUPER: return "SUPER";
case cause::WRAP_AROUND: return "WRAP_AROUND";
case cause::STORAGE_SERVICE: return "STORAGE_SERVICE";
case cause::API: return "API";
case cause::SCHEMA_CHANGE: return "SCHEMA_CHANGE";
case cause::MIXED_CF: return "MIXED_CF";
case cause::SSTABLE_FORMAT_M: return "SSTABLE_FORMAT_M";
}
abort();
}

View File

@@ -15,14 +15,33 @@
namespace unimplemented {
enum class cause {
API, // REST API features not implemented (force_user_defined_compaction, split_output in major compaction)
INDEXES, // Secondary index features (filtering on collections, clustering columns)
TRIGGERS, // Trigger support in schema tables and storage proxy
METRICS, // Query processor metrics
VALIDATION, // Schema validation in DDL statements (drop keyspace, truncate, token functions)
REVERSED, // Reversed types in CQL protocol
HINT, // Hint replaying in batchlog manager
SUPER, // Super column families (legacy Cassandra feature, never supported)
API,
INDEXES,
LWT,
PAGING,
AUTH,
PERMISSIONS,
TRIGGERS,
COUNTERS,
METRICS,
MIGRATIONS,
GOSSIP,
TOKEN_RESTRICTION,
LEGACY_COMPOSITE_KEYS,
COLLECTION_RANGE_TOMBSTONES,
RANGE_DELETES,
VALIDATION,
REVERSED,
COMPRESSION,
NONATOMIC,
CONSISTENCY,
HINT,
SUPER,
WRAP_AROUND, // Support for handling wrap around ranges in queries on database level and below
STORAGE_SERVICE,
SCHEMA_CHANGE,
MIXED_CF,
SSTABLE_FORMAT_M,
};
[[noreturn]] void fail(cause what);