Commit Graph

57 Commits

Author SHA1 Message Date
Pavel Emelyanov
60a834d9fa cql3: Add cql_config parameter to parsed_statement::prepare()
Pass cql_config to prepare() so that statement preparation can use
CQL-specific configuration rather than reaching into db::config
directly.

Callers that use default_cql_config:
- db/view/view.cc: builds a SELECT statement internally to compute view
  restrictions, not in response to a user query
- cql3/statements/create_view_statement.cc: same -- parses the view's
  WHERE clause as a synthetic SELECT to extract restrictions
- tools/schema_loader.cc: offline schema loading tool, no runtime
  config available
- tools/scylla-sstable.cc: offline sstable inspection tool, no runtime
  config available

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-16 07:57:25 +03:00
Pavel Emelyanov
debfb147f5 describe_statement: Get cluster info from storage_service
Update cluster_describe_statement::describe() to retrieve cluster metadata
from storage_service::describe_cluster() instead of directly from db::config
or gossiper.

The storage_service provides a centralized API for accessing cluster metadata
(cluster_name, partitioner, snitch_name) that works in both normal and
maintenance modes, improving separation of concerns.

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-14 19:33:06 +03:00
Avi Kivity
0ae22a09d4 LICENSE: Update to version 1.1
Updated terms of non-commercial use (must be a never-customer).
2026-04-12 19:46:33 +03:00
Michał Chojnowski
ff60a5f1e5 cql3: suggest ALTER MATERIALIZED VIEW to users trying to use ALTER TABLE on a view
When a user tries to use ALTER TABLE on a materialized view,
the resulting error message is `Cannot use ALTER TABLE on Materialized View`.

The intention behind this error is that ALTER MATERIALIZED VIEW should
be used instead.

But we observed that some users interpret this error message as a general
"You cannot do any ALTER on this thing".

This patch enhances the error message (and others similar to it)
to prevent the confusion.

Closes scylladb/scylladb#28831
2026-03-09 15:07:21 +01:00
Dario Mirovic
6a1edab2ac client_state: add has_superuser method
Encapsulate the superuser check in client_state so that it
respects _bypass_auth_checks. Connections that bypass auth
(internal callers and the maintenance socket) are always
considered superusers.

Migrate existing call sites from auth::has_superuser(service, user)
to client_state.has_superuser(). Also add _bypass_auth_checks
handling to ensure_not_anonymous().

Refs SCYLLADB-409
2026-03-03 22:31:35 +01:00
Michał Jadwiszczak
f89a8c4ec4 cql3/statements/describe_statement: hide paxos state tables
Paxos state tables are internal tables fully managed by Scylla
and they shouldn't be exposed to the user nor they shouldn't be backed up.

This commit hides those kind of tables from all listings and if such table
is directly described with `DESC ks."tbl$paxos"`, the description is generated
withing a comment and a note for the user is added.

Fixes scylladb/scylladb#28183
2026-01-20 15:58:08 +01:00
Nadav Har'El
c4a9d7eb3e cql: fix DESC KEYSPACES when a "USE" is in effect
If a CQL session USEs a keyspace and then calls DESC TABLES, the user
expects to see only the tables in the chosen keyspace. However, calling
DESC KEYSPACES should still return list all the keyspaces - returning
just the USEd one is not useful - and also not what Cassandra does.
We had an xfailing test test_describe.py::test_keyspaces_with_use which
reproduces this bug (and passes on Cassandra).

In this patch we fix this bug. The fix is simple - USE should affect
DESC statements, but be ignored for DESC KEYSPACES. We can then remove
the xfail marker from the test.

The patch also includes a new test for the DESC TABLES case, where the
USE *does* have an affect. And I wanted to make sure the patch doesn't
break this case. As usual, the new test passes on both Cassandra and
ScyllaDB.

Fixes #26334

Signed-off-by: Nadav Har'El <nyh@scylladb.com>

Closes scylladb/scylladb#27971
2026-01-04 22:01:12 +02:00
Dawid Mędrek
d2c5268196 cql3: Produce CREATE MATERIALIZED VIEW statement when describing MV of index
Before this change, executing `DESCRIBE MATERIALIZED VIEW` on the underlying
materialized view of a secondary index would produce a `CREATE INDEX` statement.
It was not only confusing, but it also prevented from learning about
the definition of the view. The only way to do so was to query system tables.

We change that behavior and produce a `CREATE MATERIALIZED VIEW` statement
instead. The statement is printed as a comment to implicitly convey that
the user should not attempt to execute it to restore the view. A short comment
is provided to make it clearer.

Before this commit:

```
cqlsh> CREATE TABLE ks.t(p int PRIMARY KEY, v int);
cqlsh> CREATE INDEX i ON ks.t(v);
cqlsh> DESCRIBE MATERIALIZED VIEW ks.i;

CREATE INDEX i ON ks.t(v);
```

After this commit:

```
cqlsh> CREATE TABLE ks.t(p int PRIMARY KEY, v int);
cqlsh> CREATE INDEX i ON ks.t(v);
cqlsh> DESCRIBE MATERIALIZED VIEW ks.i;

/* Do NOT execute this statement! It's only for informational purposes.
   This materialized view is the underlying materialized view of a secondary
   index. It can be restored via restoring the index.

CREATE MATERIALIZED VIEW ks.i_index [...];

*/
```

Note that describing the base table has not been affected and still works
as follows:

```
cqlsh> CREATE TABLE ks.t(p int PRIMARY KEY, v int);
cqlsh> CREATE INDEX i ON ks.t(v);
cqlsh> DESCRIBE TABLE ks.t;

CREATE TABLE ks.t (
    p int,
    v int,
    PRIMARY KEY (p)
) WITH bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
    AND comment = ''
    AND compaction = {'class': 'IncrementalCompactionStrategy'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND speculative_retry = '99.0PERCENTILE'
    AND tombstone_gc = {'mode': 'timeout', 'propagation_delay_in_seconds': '3600'};

CREATE INDEX i ON ks.t(v);
```

We also provide two reproducers of scylladb/scylladb#24610.

Fixes scylladb/scylladb#24610

Closes scylladb/scylladb#25697
2025-09-03 15:21:37 +02:00
Dawid Pawlik
35b82e6d2f describe, cdc: adjust describe for cdc log tables
Make CDC log table describe mention that it can be created
by creating the vector index on base table's vector column.
2025-08-20 12:38:52 +02:00
Nadav Har'El
04b263b51a Merge 'vector_index: do not create a view when creating a vector index' from Michał Hudobski
This PR adds a way for custom indexes to decide whether a view should be created for them, as for the vector_index the view is not needed, because we store it in the external service. To allow this, custom logic for describing indexes using custom classes was added (as it used to depend on the view corresponding to an index).

Fixes: VECTOR-10

Closes scylladb/scylladb#24438

* github.com:scylladb/scylladb:
  custom_index: do not create view when creating a custom index
  custom_index: refactor describe for custom indexes
  custom_index: remove unneeded duplicate of a static string
2025-07-17 13:48:49 +03:00
Marcin Maliszkiewicz
19bc6ffcb0 replica: make truncate_table_on_all_shards get whole schema from table_shards
Before for views and indexes it was fetching base schema from db (and
couple other properties). This is a problem once we introduce atomic
tables and views deletion (in the following commit).
Because once we delete table it can no longer be fetched from db object,
and truncation is performed after atomically deleting all relevant
tables/views/indexes.

Now the whole relevant schema will be fetched via global_table_ptr
(table_shards) object.
2025-07-10 10:40:43 +02:00
Michał Hudobski
d4002b61dd custom_index: refactor describe for custom indexes
Currently, to describe an index we look at
a corresponding view. However for custom indexes
the view may not exist (as we are removing the views
from vector indexes). This commit adds a way for a custom
index class to override the default describing logic
and provides such an override for the vector_index
class.
2025-07-07 13:47:07 +02:00
Dawid Mędrek
9d03dcd28e cql3/description: Serialize only rvalues of description
We discard instances of `cql3::description` right after serializing them,
so let's change the signature of the function to save some work.
2025-07-01 12:58:11 +02:00
Dawid Mędrek
ac9062644f cql3: Represent create_statement using managed_string
When describing a table, we need to do it carefully: if some
columns were dropped, we must specify that explicitly by

```
ALTER TABLE {table} DROP {column} USING TIMESTAMP ...
```

in the result of the DESCRIBE statement. Failing to do so
could lead to data resurrection.

However, if a table has been altered many, many times,
we might end up with a huge create statement. Constructing
it could, in turn, trigger an oversized allocation.
Some tests ran into that very problem in fact.

In this commit, we want to mitigate the problem: instead of
allocating a contiguous chunk of memory for the create
statement, we use `fragmented_ostringstream` and `managed_string`
to possibly keep data scattered in memory. It makes handling
`cql3::description` less convenient in the code, but since
the struct is pretty much immediately serialized after
creating it, it's a very good trade-off.

We provide a reproducer. It consistently passes with this commit,
while having about 50% chance of failure before it (based on my
own experiments). Playing with the parameters of the test
doesn't seem to improve that chance, so let's keep it as-is.

Fixes scylladb/scylladb#24018
2025-07-01 12:58:02 +02:00
Dawid Mędrek
d4315e4fae cql3/statements/describe_statement.cc: Don't copy descriptions
In an upcoming commit, `cql3::description` is going to become
a move-only type. These changes are a prerequisite for it:
we get rid of all places in the file where we copy its instances
and start moving them instead.
2025-06-30 19:12:14 +02:00
Dawid Mędrek
9472da3220 cql3: Use managed_bytes instead of bytes in DESCRIBE
This is a prerequiste for a following commit. We want
to move towards using non-contiguous memory chunks
to avoid making large allocations.

This commit does NOT change the behavior of Scylla
at all. The rows corresponding to the result of a DESCRIBE
statement are represented by an instance of `result_set`.
Before these changes, we encoded descriptions using `bytes`
and then passed them into a `result_set` using its method
`add_row`. What it does is turn the instances of `bytes`
into instances of `managed_bytes` and append them at the
end of its internal vector. In these changes, we do it
on our own and use another overload of the method.
2025-06-30 19:12:14 +02:00
Avi Kivity
cd79a8fc25 Revert "Merge 'Atomic in-memory schema changes application' from Marcin Maliszkiewicz"
This reverts commit 0b516da95b, reversing
changes made to 30199552ac. It breaks
cluster.random_failures.test_random_failures.test_random_failures
in debug mode (at least).

Fixes #24513
2025-06-16 22:38:12 +03:00
Marcin Maliszkiewicz
a27776b4ff replica: make truncate_table_on_all_shards get whole schema from table_shards
Before for views and indexes it was fetching base schema from db (and
couple other properties). This is a problem once we introduce atomic
tables and views deletion (in the following commit).
Because once we delete table it can no longer be fetched from db object,
and truncation is performed after atomically deleting all relevant
tables/views/indexes.

Now the whole relevant schema will be fetched via global_table_ptr
(table_shards) object.
2025-06-06 08:50:33 +02:00
Paweł Zakrzewski
98f5e49ea8 audit: Add support to CQL statements
Integrates audit functionality into CQL statement processing to enable tracking of database operations. Key changes:

- Add audit_info and statement_category to all CQL statements
- Implement audit categories for different statement types:
  - DDL: Schema altering statements (CREATE/ALTER/DROP)
  - DML: Data manipulation (INSERT/UPDATE/DELETE/TRUNCATE/USE)
  - DCL: Access control (GRANT/REVOKE/CREATE ROLE)
  - QUERY: SELECT statements
  - ADMIN: Service level operations

- Add audit inspection points in query processing:
  - Before statement execution
  - After access checks
  - After statement completion
  - On execution failures

- Add password sanitization for role management statements
  - Mask plaintext passwords in audit logs
  - Handle both direct password parameters and options maps
  - Preserve query structure while hiding sensitive data

- Modify prepared statement lifecycle to carry audit context
  - Pass audit info during statement preparation
  - Track audit info through statement execution
  - Support batch statement auditing

This change enables comprehensive auditing of CQL operations while ensuring sensitive data is properly masked in audit logs.
2025-01-15 11:10:36 +01:00
Avi Kivity
eb62593f2c treewide: use angle brackets when including seastar headers
We treat Seastar as a "system" library, and those are included
with angle brackets.

Closes scylladb/scylladb#21959
2024-12-20 16:16:28 +02:00
Avi Kivity
f3eade2f62 treewide: relicense to ScyllaDB-Source-Available-1.0
Drop the AGPL license in favor of a source-available license.
See the blog post [1] for details.

[1] https://www.scylladb.com/2024/12/18/why-were-moving-to-a-source-available-license/
2024-12-18 17:45:13 +02:00
Kefu Chai
bab12e3a98 treewide: migrate from boost::adaptors::transformed to std::views::transform
now that we are allowed to use C++23. we now have the luxury of using
`std::views::transform`.

in this change, we:

- replace `boost::adaptors::transformed` with `std::views::transform`
- use `fmt::join()` when appropriate where `boost::algorithm::join()`
  is not applicable to a range view returned by `std::view::transform`.
- use `std::ranges::fold_left()` to accumulate the range returned by
  `std::view::transform`
- use `std::ranges::fold_left()` to get the maximum element in the
  range returned by `std::view::transform`
- use `std::ranges::min()` to get the minimal element in the range
  returned by `std::view::transform`
- use `std::ranges::equal()` to compare the range views returned
  by `std::view::transform`
- remove unused `#include <boost/range/adaptor/transformed.hpp>`
- use `std::ranges::subrange()` instead of `boost::make_iterator_range()`,
  to feed `std::views::transform()` a view range.

to reduce the dependency to boost for better maintainability, and
leverage standard library features for better long-term support.

this change is part of our ongoing effort to modernize our codebase
and reduce external dependencies where possible.

limitations:

there are still a couple places where we are still using
`boost::adaptors::transformed` due to the lack of a C++23 alternative
for `boost::join()` and `boost::adaptors::uniqued`.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#21700
2024-12-03 09:41:32 +02:00
Kefu Chai
a5ee0c896b treewide: migrate from boost::adaptors::filtered to std::views::filter
Modernize the codebase by replacing Boost range adaptors with C++23 standard library views,
reducing external dependencies and leveraging modern C++ language features.

Key Changes:
- Replace `boost::adaptors::filtered` with `std::views::filter`
- Remove `#include <boost/range/adaptor/filtered.hpp>`
- Utilize standard library range views

Motivation:
- Reduce project's external dependency footprint
- Leverage standard library's range and view capabilities
- Improve long-term code maintainability
- Align with modern C++ best practices

Implementation Challenges and Considerations:
1. Range Conversion and Move Semantics
   - `std::ranges::to` adaptor requires rvalue references
   - Necessitated updates to variable and parameter constness
   - Example: `cql3/restrictions/statement_restrictions.cc` modified to remove `const`
     from `common` to enable efficient range conversion

2. Range Iteration and Mutation
   - Range views may mutate internal state during iteration
   - Cannot pass ranges by const reference in some scenarios
   - Solution: Pass ranges by rvalue reference to explicitly indicate
     state invalidation

Limitations:
- One instance of `boost::adaptors::filtered` temporarily preserved
  due to lack of a C++23 alternative for `boost::join()`
- A comprehensive replacement will be addressed in a follow-up change

This change is part of our ongoing effort to modernize the codebase,
reducing external dependencies and adopting modern C++ practices.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#21648
2024-11-26 14:26:50 +02:00
Nikita Kurashkin
3032d8ccbf add check to refuse usage of DESC TABLE on a materialized view
Fixes #21026

Closes scylladb/scylladb#21500
2024-11-11 10:23:30 +02:00
Piotr Dulikowski
7f17894c88 Merge 'cql3: Allow for describing CDC log tables' from Dawid Mędrek
In the past, DESC SCHEMA would produce create statements for both the base
and the log table. That was incorrect as the log table is automatically
created alongside the base one. That was solved in scylladb/scylladb@9ab57b1
(scylladb/scylladb#18467).

The mentioned changes implemented the following solution:

* DESC SCHEMA/KEYSPACE/TABLE would still print a create statement for the
  CDC base table,
* DESC SCHEMA/KEYSPACE would start printing an alter statement for the
  CDC log table. That statement would ensure that the restored log table
  has the same parameters as the original one,
* DESC TABLE <base table> would behave as DESC SCHEMA/KEYSPACE, i.e.
  it would print a create statement for the base table and an alter
  statement for the log table,
* DESC TABLE <log table> would result in an error.

While that solution was good and behaved correctly in the context of
restoring the schema, it had one flaw: describe statement aren't only
used as a means for producing a backup; they also serve an informative
purpose to learn about the schema, e.g. to learn what parameters a specific
table uses. Because we didn't allow for describing CDC log tables, the user
couldn't look them up directly via a describe statement -- they had to
describe the base table for that.

Attempting to describe a log table ended with an error, e.g.:

```
$ DESC TABLE ks.t_scylla_cdc_log;

ks.t_scylla_cdc_log is a cdc log table and it cannot be described directly. Try `DESC TABLE ks.t` to describe cdc base table and it's log table.
```

In these changes, we allow for describing CDC log tables again. The
semantics of the first three bullets above remains unchanged, but
we impose new behavior for DESC TABLE <log table>:

* When the user executes DESC TABLE <log table>, a create statement
  will be returned, treating the table as if it were a regular one,
* The create statement will be wrapped in CQL comment markers.

The rationale for the second bullet is that although we want to give the
user a means to look into the structure and options of a CDC log table,
the returned statement is not supposed to be ever executed by them. We
want to minimize the risk of that.

An example of the behavior after the change:

```
$ DESC TABLE ks.t_scylla_cdc_log;

/* Do NOT execute this statement! It's only for informational purposes.
   A CDC log table is created automatically when the base is created.

CREATE TABLE ks.t_scylla_cdc_log (
    "cdc$stream_id" blob,
    "cdc$time" timeuuid,
    "cdc$batch_seq_no" int,
    "cdc$end_of_batch" boolean,
    "cdc$operation" tinyint,
    "cdc$ttl" bigint,
    p int,
    PRIMARY KEY ("cdc$stream_id", "cdc$time", "cdc$batch_seq_no")
) WITH CLUSTERING ORDER BY ("cdc$time" ASC, "cdc$batch_seq_no" ASC)
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'enabled': 'false', 'keys': 'NONE', 'rows_per_partition': 'NONE'}
    AND comment = 'CDC log for ks.t'
    AND compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '60', 'compaction_window_unit': 'MINUTES', 'expired_sstable_check_frequency_seconds': '1800'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 0
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND speculative_retry = '99.0PERCENTILE';

*/
```

We also extend the developer documentation regarding DESCRIBE
statements on CDC tables.

Fixes scylladb/scylladb#21235

Backport: these changes are an enhancement, so not needed.

Closes scylladb/scylladb#21228

* github.com:scylladb/scylladb:
  docs/dev: Document semantics of describing CDC tables
  cql3: Allow for describing CDC log tables
2024-11-05 10:06:13 +01:00
Kefu Chai
ee2a9419b3 lang: remove unused "#includes"
these unused includes are identified by clang-include-cleaner. after
auditing the source files, all of the reports have been confirmed.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-11-05 10:01:04 +08:00
Dawid Mędrek
39e0513e1b cql3: Allow for describing CDC log tables
In the past, DESC SCHEMA would produce create statements for both the base
and the log table. That was incorrect as the log table is automatically
created alongside the base one. That was solved in scylladb/scylladb@9ab57b1
(scylladb/scylladb#18467).

The mentioned changes implemented the following solution:

* DESC SCHEMA/KEYSPACE/TABLE would still print a create statement for the
  CDC base table,
* DESC SCHEMA/KEYSPACE would start printing an alter statement for the
  CDC log table. That statement would ensure that the restored log table
  has the same parameters as the original one,
* DESC TABLE <base table> would behave as DESC SCHEMA/KEYSPACE, i.e.
  it would print a create statement for the base table and an alter
  statement for the log table,
* DESC TABLE <log table> would result in an error.

While that solution was good and behaved correctly in the context of
restoring the schema, it had one flaw: describe statement aren't only
used as a means for producing a backup; they also serve an informative
purpose to learn about the schema, e.g. to learn what parameters a specific
table uses. Because we didn't allow for describing CDC log tables, the user
couldn't look them up directly via a describe statement -- they had to
describe the base table for that.

Attempting to describe a log table ended with an error, e.g.:

```
$ DESC TABLE ks.t_scylla_cdc_log;

ks.t_scylla_cdc_log is a cdc log table and it cannot be described directly. Try `DESC TABLE ks.t` to describe cdc base table and it's log table.
```

In these changes, we allow for describing CDC log tables again. The
semantics of the first three bullets above remains unchanged, but
we impose new behavior for DESC TABLE <log table>:

* When the user executes DESC TABLE <log table>, a create statement
  will be returned, treating the table as if it were a regular one,
* The create statement will be wrapped in CQL comment markers.

The rationale for the second bullet is that although we want to give the
user a means to look into the structure and options of a CDC log table,
the returned statement is not supposed to be ever executed by them. We
want to minimize the risk of that.

An example of the behavior after the change:

```
$ DESC TABLE ks.t_scylla_cdc_log;

/* Do NOT execute this statement! It's only for informational purposes.
   A CDC log table is created automatically when the base is created.

CREATE TABLE ks.t_scylla_cdc_log (
    "cdc$stream_id" blob,
    "cdc$time" timeuuid,
    "cdc$batch_seq_no" int,
    "cdc$end_of_batch" boolean,
    "cdc$operation" tinyint,
    "cdc$ttl" bigint,
    p int,
    PRIMARY KEY ("cdc$stream_id", "cdc$time", "cdc$batch_seq_no")
) WITH CLUSTERING ORDER BY ("cdc$time" ASC, "cdc$batch_seq_no" ASC)
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'enabled': 'false', 'keys': 'NONE', 'rows_per_partition': 'NONE'}
    AND comment = 'CDC log for ks.t'
    AND compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '60', 'compaction_window_unit': 'MINUTES', 'expired_sstable_check_frequency_seconds': '1800'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 0
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND speculative_retry = '99.0PERCENTILE';

*/
```

Fixes scylladb/scylladb#21235
2024-10-31 11:25:19 +01:00
Dawid Mędrek
b984488552 cql3: Rename SALTED HASH to HASHED PASSWORD
Cassandra 4.1 announced a new option to create a role with:
`HASHED PASSWORD`. Example:

```
CREATE ROLE bob WITH HASHED PASSWORD = 'hashed_password';
```

We've already introduced another option following the same
semantics: `SALTED HASH`; example:

```
CREATE ROLE bob WITH SALTED HASH = 'salted_hash';
```

The change hasn't made it to any release yet, so in this commit
we rename it to `HASHED PASSWORD` to be compatible with Cassandra.

Additionally, we adjust existing tests to work against Cassandra too.

Fixes scylladb/scylladb#21350

Closes scylladb/scylladb#21352
2024-10-30 14:07:58 +02:00
Botond Dénes
86fd9ce8fd schema/schema: break circular dependency with replica::database
The schema module (everything in schema/) is supposed to be towards the
leafs in the ScyllaDB inter-module dependency graph. In other words, it
should not depend on many other modules. On the other hand, almost the
entire codebase depends on the schema module itself.
Currently there is a circular dependency between schema and
replica::database, as the latter is a required argument for
schema::describe(). This is bad, not just because of the dependency mess
it introduces, but also because now schema::describe() can only be used
by code which has a reference to the database handy.

This patch breaks this circular dependency, by introducing the
schema_describe_helper interface and providing an implementation for it
in database.hh.

There is another circular dependency: schema <-> replica::table. This is
not addressed by this patch.

Closes scylladb/scylladb#20893
2024-10-10 10:07:26 +03:00
Benny Halevy
5a0f3889e0 treewide: use std::ranges sort functions rather than boost
Using the standard library is preffered over boost.

In cql3/expr/expression.cc to_sorted_vector got more of a
face-list and was modernized to use also std::unique
and while at it, to move its input range in the uniquely sorted
result vector.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2024-10-01 14:19:05 +03:00
Dawid Mędrek
be851cef10 cql3: Implement DESCRIBE SCHEMA WITH INTERNALS AND PASSWORDS
When executing `DESC SCHEMA WITH INTERNALS`, Scylla now also returns
statements that can be used to recreate service levels and restore
the state of auth. That encompasses granting roles and permissions
as well as attaching service levels to roles.

If the additional parameter `WITH PASSWORDS` is provided,
the statements corresponding to recreating roles in the system
will also contain the stored salted hashes.
2024-09-24 14:18:01 +02:00
Dawid Mędrek
b357307406 data_dictionary: Remove keyspace_element.hh
The interface is not used anywhere anymore, so we can
remove it safely. It has been replaced by custom
functions for each keyspace element and `cql3::description`.
2024-09-20 14:24:54 +02:00
Dawid Mędrek
7b4f9c806c treewide: Start using new overloads of describe
We continue removing `data_dictionary::keyspace_element`.
In this commit, we start using the overloads returning
`cql3::description` in places where the methods specified
by `data_dictionary::keyspace_element` were used.
2024-09-20 14:24:54 +02:00
Dawid Mędrek
1844c71f9a cql3: Refactor description
In these changes, we describe the purpose of the type
and make it reusable for other parts of the code.
That includes ditching the existing constructors,
leaving the formatting of its fields to the user
of the interface.

The removed constructors have been replaced by
free functions so that existing code can still
use them the way it did before.
2024-09-20 14:24:53 +02:00
Dawid Mędrek
05d6794e65 cql3: Move description to dedicated files
We move the declaration of `description` to dedicated files
to be able to create instances of it from other parts of
the code.

`describe_statement.cc` has been functioning as an
intermediary between objects that can be described
and the end user. It will still perform that duty,
but we want to let other modules be able to generate
descriptions on their own, without having to share
an additional layer of abstraction in form of types
inheriting from `data_dictionary::keyspace_element`.
Those types may not perform any other function than
that and thus may be redundant.

Adjusting `description` to its new purpose will happen
in an upcoming commit.
2024-09-20 14:24:53 +02:00
Marcin Maliszkiewicz
16b770ff1a cql3: functions: make functions class non-static
This is done to ease code reuse in the following commit.
It'd also help should we ever want properly mount functions
class to schema object instead of static storage.
2024-07-04 10:24:57 +02:00
Calle Wilund
dfd996e7c1 describe_statement: Filter out "extension internal" keyspaces in DESC SCHEMA
Fixes /scylladb/scylla-enterprise#4168

Unless listing all (including system) keyspaces, filter out "extension internal"
keyspaces. These are to be considered "system" for the purposes of exposing to
end user.

Closes scylladb/scylladb#19214
2024-06-11 10:01:42 +03:00
Pavel Emelyanov
bebd121936 code: Enlighten wasm headers usage
Now when function context creation is encapsulated in lang::manager,
some .cc files can stop using wasm-specific headers and just go with the
lang/manager.hh one.

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
2024-06-07 13:07:05 +03:00
Michał Jadwiszczak
f29820fb27 cql3/statements/describe_statement: UDTs topological sorting
User-defined types can depend on each other, creating directed acyclic
graph.
In order to support restoring schema from `DESC SCHEMA`, UDTs should be
ordered topologically, not alphabetically as it was till now.
2024-05-16 13:30:03 +02:00
Michał Jadwiszczak
7be938192b cql3/statements/describe_statement: allow to skip alphabetical sorting
In a next commit, we are going to introduce topological sorting of
user-defined types, so alphabetical sorting must be skipped not to
interfere.
2024-05-16 13:30:03 +02:00
Michał Jadwiszczak
f12edbdd95 cql3/statements/describe_statement: hide cdc tables
Tables with `_scylla_cdc_log` suffix are internal tables used by cdc.

We want to hide those tables in all describe statements, as they
shouldn't be created by user but created by Scylla when user creates a
table with cdc enabled.

Instead, we include `ALTER TABLE <cdc log table> WITH <all table properties>`
to the description of cdc base table, so all changes to cdc log table's
properties are preserved in backup.
2024-05-13 16:11:13 +02:00
Botond Dénes
7f17d3bb0e replica/database: keyspace: add uses_tablets()
Mirroring table::uses_tablets(), provides a convenient and -- more
importabtly -- easily discoverable way to determine whether the keyspace
uses tablets or not.
This information is of course already available via the abstract
replication strategy, but as seen in a few examples, this is not easily
discoverable and sometimes people resorted to enumerating the keyspace's
tables to be able to invoke table::uses_tablets().
2024-02-15 01:51:26 -05:00
Kefu Chai
2dbf044b91 cql3: do not include unused headers
these unused includes were identified by clangd. see
https://clangd.llvm.org/guides/include-cleaner#unused-include-warning
for more details on the "Unused include" warning.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#16791
2024-01-16 16:43:17 +02:00
Botond Dénes
66bef6e961 cql3: cluster_describe_statement: don't produce range ownership for tablet keyspaces
Tablet keyspaces have per/table range ownership, which cannot currently
be expressed in a DESC CLUSTER statement, which describes range
ownership in the current keyspace (if set). Until we figure out how to
represent range ownership (tablets) of all tables of a keyspace, we
disable range ownership for tablet keyspaces.

Fixes: #16483

Closes scylladb/scylladb#16713
2024-01-15 14:03:54 +01:00
Yaniv Kaul
c658bdb150 Typos: fix typos in comments
Fixes some typos as found by codespell run on the code.
In this commit, I was hoping to fix only comments, not user-visible alerts, output, etc.
Follow-up commits will take care of them.

Refs: https://github.com/scylladb/scylladb/issues/16255
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
2023-12-02 22:37:22 +02:00
Nadav Har'El
4e1e7568d8 Merge 'cql3:statements:describe_statement: include UDT/UDF/UDA in generic describe' from Michał Jadwiszczak
So far generic describe (`DESC <name>`) followed Cassandra implementation and it only described keyspace/table/view/index.

This commit adds UDT/UDF/UDA to generic describe.

Fixes: #14170

Closes scylladb/scylladb#14334

* github.com:scylladb/scylladb:
  docs:cql: add information  about generic describe
  cql-pytest:test_describe: add test for generic UDT/UDF/UDA desc
  cql3:statements:describe_statement: include UDT/UDF/UDA in generic describe
2023-09-24 13:03:04 +03:00
Gleb Natapov
4ffc39d885 cql3: Extend the scope of group0_guard during DDL statement execution
Currently we hold group0_guard only during DDL statement's execute()
function, but unfortunately some statements access underlying schema
state also during check_access() and validate() calls which are called
by the query_processor before it calls execute. We need to cover those
calls with group0_guard as well and also move retry loop up. This patch
does it by introducing new function to cql_statement class take_guard().
Schema altering statements return group0 guard while others do not
return any guard. Query processor takes this guard at the beginning of a
statement execution and retries if service::group0_concurrent_modification
is thrown. The guard is passed to the execute in query_state structure.

Fixes: #13942

Message-ID: <ZNsynXayKim2XAFr@scylladb.com>
2023-08-17 15:52:48 +03:00
Avi Kivity
d57a951d48 Revert "cql3: Extend the scope of group0_guard during DDL statement execution"
This reverts commit 70b5360a73. It generates
a failure in group0_test .test_concurrent_group0_modifications in debug
mode with about 4% probability.

Fixes #15050
2023-08-15 00:26:45 +03:00
Gleb Natapov
70b5360a73 cql3: Extend the scope of group0_guard during DDL statement execution
Currently we hold group0_guard only during DDL statement's execute()
function, but unfortunately some statements access underlying schema
state also during check_access() and validate() calls which are called
by the query_processor before it calls execute. We need to cover those
calls with group0_guard as well and also move retry loop up. This patch
does it by introducing new function to cql_statement class take_guard().
Schema altering statements return group0 guard while others do not
return any guard. Query processor takes this guard at the beginning of a
statement execution and retries if service::group0_concurrent_modification
is thrown. The guard is passed to the execute in query_state structure.

Fixes: #13942

Message-ID: <ZNSWF/cHuvcd+g1t@scylladb.com>
2023-08-13 14:19:39 +03:00
Michał Jadwiszczak
d5748fd895 cql3:statements:describe_statement: include UDT/UDF/UDA in generic describe
So far generic describe (`DESC <name>`) followed Cassandra
implementation and it only described keyspace/table/view/index.

This commit adds UDT/UDF/UDA to generic describe.

Fixes: #14170
2023-06-30 14:38:22 +02:00