Adding Scylla internal headers to the PCH changes which transitive
includes are available. Add explicit includes where needed:
- group0_fwd.hh: timer, lowres_clock, variant, vector
- discovery.hh: unordered_set
- Various test files: result_message.hh, result_set.hh, selection.hh,
gossiper.hh, seastar net/api and core/seastar headers
Small test binaries with partial link sets cannot satisfy symbol
references injected by -fpch-instantiate-templates. Exclude source
files used by tests with fewer than 50 dependencies from PCH
compilation to avoid linker failures.
replica/database.hh is included by 112 translation units and is one of
the heaviest headers in the codebase. Adding it to the PCH provides a
major compile-time reduction as its large transitive include tree is
parsed only once.
Clean dev build time drops from ~14m to ~6m20s (with previous PCH
commits; ~22m33s baseline without any PCH changes).
The specialization is unused and conflicts with PCH template
pre-instantiation. scheduling_group already has operator==, so
std::equal_to works via the default template.
Move the declaration of hist_key<segment_descriptor> specialization
into compaction.hh so it is visible before the primary template gets
instantiated via log_heap. This prevents -fpch-instantiate-templates
from instantiating the primary template in the PCH, which would
conflict with the explicit specialization in the .cc file.
Inside namespace utils, unqualified coroutine:: resolves to the
utils::coroutine class (utils/coroutine.hh) rather than the
seastar::coroutine namespace. This causes build failures when
replica/database.hh is added to the precompiled header, because
utils/coroutine.hh becomes transitively visible in all TUs.
Qualify all coroutine:: references with seastar:: in affected files
under utils/ and db/.
Add to stdafx.hh: locator/token_metadata.hh, gms/gossiper.hh,
db/system_keyspace.hh, service/topology_state_machine.hh,
cql3/query_options.hh, service/client_state.hh, cql3/query_processor.hh,
db/config.hh, service/storage_proxy.hh, schema/schema_builder.hh,
exceptions/exceptions.hh, gms/feature_service.hh,
service/migration_manager.hh, sstables/sstables.hh,
service/storage_service.hh, transport/messages/result_message.hh.
These headers are included by 40-140 translation units each. Adding them
to the PCH avoids redundant parsing across the build. Combined with the
previous PCH commit, clean dev build time drops from 22m33s to ~14m23s
(-36.2%).
Convert config_file.cc read_from_file() from continuation-style to
coroutines, avoiding a template instantiation conflict with
-fpch-instantiate-templates when heavy Scylla headers are in the PCH.
Qualify input_stream<char> in object_storage.cc lambda parameter with
seastar:: to resolve the same PCH template parsing issue.
Add schema/schema.hh, types/types.hh, mutation/mutation_partition.hh,
mutation/mutation_fragment.hh and their dependencies (bytes.hh, keys.hh,
dht/token.hh, locator types, etc.) to the PCH. These are included by
the vast majority of translation units and benefit greatly from being
precompiled once rather than parsed ~400 times.
Reduces clean dev build time from ~22m to ~18m (~19% faster).
Create raft/raft_fwd.hh with lightweight type aliases (server_id, group_id,
term_t, index_t) backed only by raft/internal.hh, avoiding the heavy
raft/raft.hh (832 lines with futures, abort_source, bytes_ostream).
Replace raft/raft.hh with raft/raft_fwd.hh in headers that only need the
basic ID types: tablets.hh, topology_state_machine.hh,
topology_coordinator.hh, storage_service.hh, group0_fwd.hh,
view_building_coordinator.hh, view_building_worker.hh.
Also remove gossiper.hh and tablet_allocator.hh from storage_service.hh
(forward declarations suffice), and remove unused reactor.hh from
tablets.hh. Add explicit includes in .cc files that lost transitive
availability.
Extract loaded_endpoint_state into a standalone lightweight header to
avoid pulling the heavy gossiper.hh (and transitively query-result-set.hh)
into every includer of system_keyspace.hh. Add explicit includes where
the full definitions are actually needed.
Reduces clean dev build time by ~2 minutes (-8%).
storage_proxy.hh included storage_service.hh but never referenced any
symbol from it. storage_service.hh costs 3.7s to parse per file, and
storage_proxy.hh has 75 direct includers. While most of those also
include database.hh (which shares transitive deps), removing this
unnecessary include still reduces total parse work.
Speedup: part of a series measured at -5.8% wall-clock improvement
(same-session A/B: 16m14s -> 15m17s at -j16, 16 cores).
Add explicit includes that were previously available transitively through
service/storage_proxy.hh -> service/storage_service.hh.
This prepares for removing the unused storage_service.hh include from
storage_proxy.hh in a follow-up commit.
Speedup: prerequisite for storage_proxy.hh include chain reduction
(measured -5.8% wall-clock combined with all changes in this series,
same-session A/B: 16m14s -> 15m17s at -j16).
Extract replication_strategy_type enum from locator/abstract_replication_strategy.hh
into a new lightweight header locator/replication_strategy_type.hh, and use it in
db/config.hh instead of the full abstract_replication_strategy.hh.
abstract_replication_strategy.hh pulls in a large transitive dependency tree
(schema.hh, mutation serializers, etc.) costing ~1.7s per file. With this change,
config.hh's incremental parse cost drops from 1.7s to 0.6s. Since ~85 files
include config.hh without also including database.hh (which would bring in these
deps anyway), this saves ~93s total CPU.
Speedup: part of a series measured at -5.8% wall-clock improvement
(same-session A/B: 16m14s -> 15m17s at -j16, 16 cores).
Add explicit #include for utils/exceptions.hh which was previously
available transitively through db/config.hh -> abstract_replication_strategy.hh.
This prepares for removing the heavy abstract_replication_strategy.hh
include from db/config.hh in a follow-up commit.
Speedup: prerequisite for config.hh include chain reduction
(measured -5.8% wall-clock combined with all changes in this series,
same-session A/B: 16m14s -> 15m17s at -j16).
The api_docs.hh header contains inline method bodies (api_registry::handle)
that call seastar::json::formatter::to_json(), forcing the compiler to
instantiate seastar::json template specializations (json_list_template,
formatter::write, do_with, etc.) in every compilation unit — even files
that never use any HTTP/JSON API types.
Measured ~6s of wasted template instantiation per file × ~620 files =
~3,700s total CPU. Only 2 files outside the PCH include api_docs.hh
directly, so removing it has no impact on code that actually uses these
types.
Wall-clock build time (-j16, Seastar/Abseil cached):
Before (with loading_cache fix): avg 23m29s
After: avg 23m04s (-1.8%)
vs original baseline: avg 24m01s (-4.0%)
utils/loading_cache.hh is an expensive template header that costs
~2,494 seconds of aggregate CPU time across 133 files that include it.
88 of those files include it only transitively via query_processor.hh
through the chain: query_processor.hh -> prepared_statements_cache.hh
-> loading_cache.hh, costing ~1,690s of template instantiation.
Break the chain by:
- Replacing #include of prepared_statements_cache.hh and
authorized_prepared_statements_cache.hh in query_processor.hh with
forward declarations and the lightweight prepared_cache_key_type.hh
- Replacing #include of result_message.hh with result_message_base.hh
(which doesn't pull in prepared_statements_cache.hh)
- Changing prepared_statements_cache and authorized_prepared_statements_cache
members to std::unique_ptr (PImpl) since forward-declared types
cannot be held by value
- Moving get_prepared(), execute_prepared(), execute_direct(), and
execute_batch() method bodies from the header to query_processor.cc
- Updating transport/server.cc to use the concrete type instead of the
no-longer-visible authorized_prepared_statements_cache::value_type
Per-file measurement: files including query_processor.hh now show zero
loading_cache template instantiation events (previously 20-32s each).
Wall-clock measurement (clean build, -j16, 16 cores, Seastar cached):
Baseline (origin/master): avg 24m01s (24m03s, 23m59s)
With loading_cache chain break: avg 23m29s (23m32s, 23m29s, 23m27s)
Improvement: ~32s, ~2.2%
Move prepared_cache_key_type class and its std::hash / fmt::formatter
specializations from prepared_statements_cache.hh into a new header
cql3/prepared_cache_key_type.hh.
The new header only depends on bytes.hh, utils/hash.hh, and
cql3/dialect.hh -- it does NOT include utils/loading_cache.hh.
This allows code that needs the cache key type (e.g. for function
signatures) without pulling in the expensive loading_cache template
machinery.
prepared_statements_cache.hh now includes prepared_cache_key_type.hh,
so existing includers are unaffected.
No functional change. Prepares for breaking the loading_cache include
chain from query_processor.hh.
Add explicit #include directives for headers that are currently
available transitively through cql3/query_processor.hh but will stop
being available after a subsequent refactoring that removes the
loading_cache include chain.
Files changed:
- cql3/statements/drop_keyspace_statement.cc: add unimplemented.hh
- cql3/statements/truncate_statement.cc: add unimplemented.hh
- cql3/statements/batch_statement.cc: add result_message.hh
- cql3/statements/broadcast_modification_statement.cc: add result_message.hh
- service/paxos/paxos_state.cc: add result_message.hh
- test/lib/cql_test_env.cc: add result_message.hh
- table_helper.cc: add result_message.hh
No functional change. Prepares for subsequent query_processor.hh cleanup.
implement tablet split, tablet merge and tablet migration for tables that use the experimental logstor storage engine.
* tablet merge simply merges the histograms of segments of one compaction group with another.
* for tablet split we take the segments from the source compaction group, read them and write all live records to separate segments according to the split classifier, and move separated segments to the target compaction groups.
* for tablet migration we use stream_blob, similarly to file streaming of sstables. we add a new op type for streaming a logstor segment. on the source we take a snapshot of the segments with an input stream that reads the segment, and on the target we create a sink that allocates a new segment on the target shard and writes to it.
* we also do some improvements for recovery and loading of segments. we add a segment header that contains useful information for non-mixed segments, such as the table and token range.
Refs SCYLLADB-770
no backport - still a new and experimental feature
Closesscylladb/scylladb#29207
* github.com:scylladb/scylladb:
test: logstor: additional logstor tests
docs/dev: add logstor on-disk format section
logstor: add version and crc to buffer header
test: logstor: tablet split/merge and migration
logstor: enable tablet balancing
logstor: streaming of logstor segments using stream_blob
logstor: add take_logstor_snapshot
logstor: segment input/output stream
logstor: implement compaction_group::cleanup
logstor: tablet split
logstor: tablet merge
logstor: add compaction reenabler
logstor: add segment header
logstor: serialize writes to active segment
replica: extend compaction_group functions for logstor
replica: add compaction_group_for_logstor_segment
logstor: code cleanup
Since we do no longer support upgrade from versions that do not support
v2 of "view building status" code (building status is managed by raft) we can remove v1 code and upgrade code and make sure we do not boot with old "builder status" version.
v2 version was introduced by 8d25a4d678 which is included in scylla-2025.1.0.
No backport needed since this is code removal.
Closesscylladb/scylladb#29105
* github.com:scylladb/scylladb:
view: drop unused v1 builder code
view: remove upgrade to raft code
Replace the range scan in read_verify_workload() with individual
single-partition queries, using the keys returned by
prepare_write_workload() instead of hard-coding them.
The range scan was previously observed to time out in debug mode after
a hard cluster restart. Single-partition reads are lighter on the
cluster and less likely to time out under load.
The new verification is also stricter: instead of merely checking that
the expected number of rows is returned, it verifies that each written
key is individually readable, catching any data-loss or key-identity
mismatch that the old count-only check would have missed.
This is the second attemp at stabilizing this test, after the recent
854c374ebf. That fix made sure that the
cluster has converged on topology and nodes see each other before running
the verify workload.
Fixes: SCYLLADB-1331
Closesscylladb/scylladb#29313
The supergroup replaces streaming (a.k.a. maintenance as well) group, inherits 200 shares from it and consists of four sub-groups (all have equal shares of 200 withing the new supergroup)
* maintenance_compaction. This group configures `compaction_manager::maintenance_sg()` group. User-triggered compaction runs in it
* backup. This group configures `snapshot_ctl::config::backup_sched_group`. Native backup activity runs there
* maintenance. It's a new "visible" name, everything that was called "maintenance" in the code ran in "streaming" group. Now it will run in "maintenance". The activities include those that don't communicate over RPC (see below why)
* `tablet_allocator::balance_tablets()`
* `sstables_manager::components_reclaim_reload_fiber()`
* `tablet_storage_group_manager::merge_completion_fiber()`
* metrics exporting http server altogether
* streaming. This is purely existing streaming group that just moves under the new supergroup. Everything else that was run there, continues doing so, including
* hints sender
* all view building related components (update generator, builder, workers)
* repair
* stream_manager
* messaging service (except for verb handlers that switch groups)
* join_cluster() activity
* REST API
* ... something else I forgot
The `--maintenance_io_throughput_mb_per_sec` option is introduced. It controls the IO throughput limit applied to the maintenance supergroup. If not set, the `--stream_io_throughput_mb_per_sec` option is used to preserve backward compatibility.
All new sched groups inherit `request_class::maintenance` (however, "backup" seem not to make any requests yet).
Moving more activities from "streaming" into "maintenance" (or its own group) is possible, but one will need to take care of RPC group switching. The thing is that when a client makes an RPC call, the server may switch to one of pre-negotiated scheduling groups. Verbs for existing activities that run in "streaming" group are routed through RPC index that negotiates "streaming" group on the server side. If any of that client code moves to some other group, server will still run the handlers in "streaming" which is not quite expected. That's one of the main reasons why only the selected fibers were moved to their own "maintenance" group. Similar for backup -- this code doesn't use RPC, so it can be moved. Restoring code uses load-and-stream and corresponding RPCs, so it cannot be just moved into its own new group.
Fixes SCYLLADB-351
New feature, not backporting
Closesscylladb/scylladb#28542
* github.com:scylladb/scylladb:
code: Add maintenance/maintenance group
backup: Add maintenance/backup group
compaction: Add maintenance/maintenance_compaction group
main: Introduce maintenance supergroup
main: Move all maintenance sched group into streaming one
database: Use local variable for current_scheduling_group
code: Live-update IO throughputs from main
shared_tombstone_gc_state::update_repair_time() uses copy-on-write
semantics: each call copies the entire per_table_history_maps and the
per-table repair_history_map. repair_service::load_history() called
this once per history entry, making the load O(N²) in both time and
memory.
Introduce batch_update_repair_time() which performs a single
copy-on-write for any number of entries belonging to the same table.
Restructure load_history() to collect entries into batches of up to
1000 and flush each batch in one call, keeping peak memory bounded.
The batch size limit is intentional: the repair history table currently
has no bound on the number of entries and can grow large. Note that
this does not cause a problem in the in-memory history map itself:
entries are coalesced internally and only the latest repair time is
kept per range. The unbounded entry count only makes the batched
update during load expensive.
Fixes: SCYLLADB-104
Closesscylladb/scylladb#29326
Include non-primary key restrictions (e.g. regular column filters) in
the filter JSON sent to the Vector Store service. Previously only
partition key and clustering column restrictions were forwarded, so
filtering on regular columns was silently ignored.
Add get_nonprimary_key_restrictions() getter to statement_restrictions.
Add unit tests for non-primary key equality, range, and bind marker
restrictions in filter_test.
Fixes: SCYLLADB-970
Closesscylladb/scylladb#29019
For counter updates, use a counter ID that is constructed from the
node's rack instead of the node's host ID.
A rack can have at most two active tablet replicas at a time: a single
normal tablet replica, and during tablet migration there are two active
replicas, the normal and pending replica. Therefore we can have two
unique counter IDs per rack that are reused by all replicas in the rack.
We construct the counter ID from the rack UUID, which is constructed
from the name "dc:rack". The pending replica uses a deterministic
variation of the rack's counter ID by negating it.
This improves the performance and size of counter cells by having less
unique counter IDs and less counter shards in a counter cell.
Previously the number of counter shards was the number of different
host_id's that updated the counter, which can be typically the number of
nodes in the cluster and continue growing indefinitely when nodes are
replaced. with the rack-based counter id the number of counter shards
will be at most twice the number of different racks (including removed
racks, which should not be significant).
Fixes SCYLLADB-356
backport not needed - an enhancement
Closesscylladb/scylladb#28901
* github.com:scylladb/scylladb:
docs/dev: add counters doc
counters: reuse counter IDs by rack
Replace move_to_shard()/move_to_host() with as_bounce()/target_shard()/
target_host() to clarify the interface after bounce was extended to
support cross-node bouncing.
- Add virtual as_bounce() returning const bounce* to the base class
(nullptr by default, overridden in bounce to return this), replacing
the virtual move_to_shard() which conflated bounce detection with
shard access
- Rename move_to_shard() -> target_shard() (now non-virtual, returns
unsigned directly) and move_to_host() -> target_host() on bounce
- Replace dynamic_pointer_cast with static_pointer_cast at call sites
that already checked as_bounce()
- Move forward declarations of message types before the virtual
methods so as_bounce() can reference bounce
Fixes: SCYLLADB-1066
Closesscylladb/scylladb#29367
Motivation
----------
Since strongly consistent tables are based on the concept of Raft
groups, operations on them can get stuck for indefinite amounts of
time. That may be problematic, and so we'd like to implement a way
to cancel those operations at suitable times.
Description of solution
-----------------------
The situations we focus on are the following:
* Timed-out queries
* Leader changes
* Tablet migrations
* Table drops
* Node shutdowns
We handle each of them and provide validation tests.
Implementation strategy
-----------------------
1. Auxiliary commits.
2. Abort operations on timeout.
3. Abort operations on tablet removal.
4. Extend `client_state`.
5. Abort operation on shutdown.
6. Help `state_machine` be aborted as soon as possible.
Tests
-----
We provide tests that validate the correctness of the solution.
The total time spent on `test_strong_consistency.py`
(measured on my local machine, dev mode):
Before:
```
real 0m31.809s
user 1m3.048s
sys 0m21.812s
```
After:
```
real 0m34.523s
user 1m10.307s
sys 0m27.223s
```
The incremental differences in time can be found in the commit messages.
Fixes SCYLLADB-429
Backport: not needed. This is an enhancement to an experimental feature.
Closesscylladb/scylladb#28526
* github.com:scylladb/scylladb:
service: strong_consistency: Abort state_machine::apply when aborting server
service: strong_consistency: Abort ongoing operations when shutting down
service: client_state: Extend with abort_source
service: strong_consistency: Handle abort when removing Raft group
service: strong_consistency: Abort Raft operations on timeout
service: strong_consistency: Use timeout when mutating
service: strong_consistency: Fix indentation
service: strong_consistency: Enclose coordinator methods with try-catch
service: strong_consistency: Crash at unexpected exception
test: cluster: Extract default config & cmdline in test_strong_consistency.py
This reverts commit 8b4a91982b.
Two commits independently added rolling_max_tracker_test to test/boost/CMakeLists.txt:
8b4a919 cmake: add missing rolling_max_tracker_test and symmetric_key_test
f3a91df test/cmake: add missing tests to boost test suite
The second was merged two days after the first. They didn't conflict on
code-level and applied cleanly resulting in a duplicate add_scylla_test()
entries that breaks the CMake build:
CMake Error: add_executable cannot create target
"test_boost_rolling_max_tracker_test" because another target
with the same name already exists.
Remove the duplicate.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Reported-by: Łukasz Paszkowski <lukasz.paszkowski@scylladb.com>
The PR contains more code cleanups, mostly in gossiper. Dropping more gossiper state leaving only NORMAL and SHUTDOWN. All other states are checked against topology state. Those two are left because SHUTDOWN state is propagated through gossiper only and when the node is not in SHUTDOWN it should be in some other state.
No need to backport. Cleanups.
Closesscylladb/scylladb#29129
* https://github.com/scylladb/scylladb:
storage_service: cleanup unused code
storage_service: simplify get_peer_info_for_update
gossiper: send shutdown notifications in parallel
gms: remove unused code
virtual_tables: no need to call gossiper if we already know that the node is in shutdown
gossiper: print node state from raft topology in the logs
gossiper: use is_shutdown instead of code it manually
gossiper: mark endpoint_state(inet_address ip) constructor as explicit
gossiper: remove unused code
gossiper: drop last use of LEFT state and drop the state
gossiper: drop unused STATUS_BOOTSTRAPPING state
gossiper: rename is_dead_state to is_left since this is all that the function checks now.
gossiper: use raft topology state instead of gossiper one when checking node's state
storage_service: drop check_for_endpoint_collision function
storage_service: drop is_first_node function
gossiper: remove unused REMOVED_TOKEN state
gossiper: remove unused advertise_token_removed function
Cassandra's native vector index type is StorageAttachedIndex (SAI). Libraries such as CassIO, LangChain, and LlamaIndex generate `CREATE CUSTOM INDEX` statements using the SAI class name. Previously, ScyllaDB rejected these with "Non-supported custom class".
This PR adds compatibility so that SAI-style CQL statements work on ScyllaDB without modification.
1. **test: enable SAI_VECTOR_ALLOW_CUSTOM_PARAMETERS for Cassandra tests**
Enables the `SAI_VECTOR_ALLOW_CUSTOM_PARAMETERS` Cassandra system property so that `search_beam_width` tests pass against Cassandra 5.0.7.
2. **test: modernize vector index test comments and fix xfail**
Updates test comments from "Reproduces" to "Validates fix for" for clarity, and converts the `test_ann_query_with_pk_restriction` xfail into a stripped-down CREATE INDEX syntax test (removing unused INSERT/SELECT lines). Removes the redundant `test_ann_query_with_non_pk_restriction` test.
3. **cql: add Cassandra SAI (StorageAttachedIndex) compatibility**
Core implementation: the SAI class name is detected and translated to ScyllaDB's native `vector_index`. The fully-qualified class name (`org.apache.cassandra.index.sai.StorageAttachedIndex`) requires exact case; short names (`StorageAttachedIndex`, `sai`) are matched case-insensitively — matching Cassandra's behavior. Non-vector and multi-column SAI targets are rejected with clear errors. Adds `skip_on_scylla_vnodes` fixture, SAI compatibility docs, and the Cassandra compatibility table entry (split into "SAI general" vs "SAI for vector search").
4. **cql: accept source_model option for Cassandra SAI compatibility**
The `source_model` option is a Cassandra SAI property used by Cassandra libraries (e.g., CassIO) to tag vector indexes with the name of the embedding model. ScyllaDB accepts it for compatibility but does not use it — the validator is a no-op lambda. The option is preserved in index metadata and returned in DESCRIBE INDEX output.
- `cql3/statements/create_index_statement.cc`: SAI class detection and rewriting logic
- `index/secondary_index_manager.cc`: case-insensitive class name lookup (lowercasing restored before `classes.find()`)
- `index/vector_index.cc`: `source_model` accepted as a valid option with no-op validator
- `docs/cql/secondary-indexes.rst`: SAI compatibility documentation with `source_model` table row
- `docs/using-scylla/cassandra-compatibility.rst`: SAI entry split into general (not supported) and vector search (supported)
- `test/cqlpy/conftest.py`: `scylla_with_tablets` renamed to `skip_on_scylla_vnodes`
- `test/cqlpy/test_vector_index.py`: SAI tests inlined (no constants), `check_bad_option()` helper for numeric validation, uppercase class name test, merged `source_model` tests with DESCRIBE check
| Backend | Passed | Skipped | Failed |
|--------------------|--------|---------|--------|
| ScyllaDB (dev) | 42 | 0 | 0 |
| Cassandra 5.0.7 | 16 | 26 | 0 |
None: new feature.
Fixes: SCYLLADB-239
Closesscylladb/scylladb#28645
* github.com:scylladb/scylladb:
cql: accept source_model option and show options in DESCRIBE
cql: add Cassandra SAI (StorageAttachedIndex) compatibility
test: modernize vector index test comments and fix xfail
test: enable SAI_VECTOR_ALLOW_CUSTOM_PARAMETERS for Cassandra tests
Add system.tablets to the set of system resources that can be
accessed with the VECTOR_SEARCH_INDEXING permission.
Fixes: VECTOR-605
Closesscylladb/scylladb#29397
Accept the Cassandra SAI 'source_model' option for vector indexes.
This option is used by Cassandra libraries (e.g., CassIO, LangChain)
to tag vector indexes with the name of the embedding model that
produced the vectors.
ScyllaDB does not use the source_model value but stores it and
includes it in the DESCRIBE INDEX output for Cassandra compatibility.
Additionally, extend vector_index::describe() to emit a
WITH OPTIONS = {...} clause containing all user-provided index options
(filtering out system keys: target, class_name, index_version).
This makes options like similarity_function, source_model, etc.
visible in DESCRIBE output.
Libraries such as CassIO, LangChain, and LlamaIndex create vector
indexes using Cassandra's StorageAttachedIndex (SAI) class name.
This commit lets ScyllaDB accept these statements without modification.
When a CREATE CUSTOM INDEX statement specifies an SAI class name on a
vector column, ScyllaDB automatically rewrites it to the native
vector_index implementation. Accepted class names (case-insensitive):
- org.apache.cassandra.index.sai.StorageAttachedIndex
- StorageAttachedIndex
- sai
SAI on non-vector columns is rejected with a clear error directing
users to a secondary index instead.
The SAI detection and rewriting logic is extracted into a dedicated
static function (maybe_rewrite_sai_to_vector_index) to keep the
already-long validate_while_executing method manageable.
Multi-column (local index) targets and nonexistent columns are
skipped with continue — the former are treated as filtering columns
by vector_index::check_target(), and the latter are caught later by
vector_index::validate().
Tests that exercise features common to both backends (basic creation,
similarity_function, IF NOT EXISTS, bad options, etc.) now use the
SAI class name with the skip_on_scylla_vnodes fixture so they run
against both ScyllaDB and Cassandra. ScyllaDB-specific tests continue
to use USING 'vector_index' with scylla_only.
- Change 'Reproduces' to 'Validates fix for' in test comments to
reflect that the referenced issues are already fixed.
- Condense the VECTOR-179 comment to two lines.
- Replace the xfailed test_ann_query_with_restriction_works_only_on_pk
with a focused test (test_ann_query_with_pk_restriction) that creates
a vector index on a table with a PK column restriction, validating
the VECTOR-374 fix.
Every time someone modifies the build system — adding a source file, changing a compilation flag, or wiring a new test — the change tends to land in only one of our two build systems (configure.py or CMake). Over time this causes three classes of problems:
1. **CMake stops compiling entirely.** Missing defines, wrong sanitizer flags, or misplaced subdirectory ordering cause hard build failures that are only discovered when someone tries to use CMake (e.g. for IDE integration).
2. **Missing build targets.** Tests or binaries present in configure.py are never added to CMake, so `cmake --build` silently skips them. This PR fixes several such cases (e.g. `symmetric_key_test`, `auth_cache_test`, `sstable_tablet_streaming`).
3. **Missing compilation units in targets.** A `.cc` file is added to a test binary in one system but not the other, causing link errors or silently omitted test coverage.
To fix the existing drift and prevent future divergence, this series:
**Adds a build-system comparison script**
(`scripts/compare_build_systems.py`) that configures both systems into a temporary directory, parses their generated `build.ninja` files, and compares per-file compilation flags, link target sets, and per-target libraries. configure.py is treated as the baseline; CMake must match it. The script supports a `--ci` mode suitable for gating PRs that touch
build files.
**Fixes all current mismatches** found by the script:
- Mode flag alignment in `mode.common.cmake` and `mode.Coverage.cmake`
(sanitizer flags, `-fno-lto`, stack-usage warnings, coverage defines).
- Global define alignment (`SEASTAR_NO_EXCEPTION_HACK`, `XXH_PRIVATE_API`,
`BOOST_ALL_DYN_LINK`, `SEASTAR_TESTING_MAIN` placement).
- Seastar build configuration (shared vs static per mode, coverage
sanitizer link options).
- Abseil sanitizer flags (`-fno-sanitize=vptr`).
- Missing test targets in `test/boost/CMakeLists.txt`.
- Redundant per-test flags now covered by global settings.
- Lua library resolution via a custom `cmake/FindLua.cmake` using
pkg-config, matching configure.py's approach.
**Adds documentation** (`docs/dev/compare-build-systems.md`) describing how to run the script and interpret its output.
No backport needed — this is build infrastructure improvement only.
Closesscylladb/scylladb#29273
* github.com:scylladb/scylladb:
scripts: remove lua library rename workaround from comparison script
cmake: add custom FindLua using pkg-config to match configure.py
test/cmake: add missing tests to boost test suite
test/cmake: remove per-test LTO disable
cmake: add BOOST_ALL_DYN_LINK and strip per-component defines
cmake: move SEASTAR_TESTING_MAIN after seastar and abseil subdirs
cmake: add -fno-sanitize=vptr for abseil sanitizer flags
cmake: align Seastar build configuration with configure.py
cmake: align global compile defines and options with configure.py
cmake: fix Coverage mode in mode.Coverage.cmake
cmake: align mode.common.cmake flags with configure.py
configure.py: add sstable_tablet_streaming to combined_tests
docs: add compare-build-systems.md
scripts: add compare_build_systems.py to compare ninja build files
Add .set_skip_when_empty() to all error-path metrics in the tracing
module. Tracing itself is not a commonly used feature, making all of
these metrics almost always zero:
Tier 1 (very rare - corruption/schema issues):
- tracing_keyspace_helper::bad_column_family_errors: tracing schema
missing or incompatible, should never happen post-bootstrap
- tracing::trace_errors: internal error building trace parameters
Tier 2 (overload - tracing backend saturated):
- tracing::dropped_sessions: too many pending sessions
- tracing::dropped_records: too many pending records
Tier 3 (general tracing write errors):
- tracing_keyspace_helper::tracing_errors: errors during writes to
system_traces keyspace
Since tracing is an opt-in feature that most deployments rarely use,
all five metrics are almost always zero and create unnecessary
reporting overhead.
AI-Assisted: yes
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
Closesscylladb/scylladb#29346
Add a documentation of the counters feature implementation in
docs/dev/counters.md.
The documentation is taken from the wiki and updated according to the
current state of the code - legacy details are removed, and a section
about the counter id is added.
For counter updates, use a counter ID that is constructed from the
node's rack instead of the node's host ID.
A rack can have at most two active tablet replicas at a time: a single
normal tablet replica, and during tablet migration there are two active
replicas, the normal and pending replica. Therefore we can have two
unique counter IDs per rack that are reused by all replicas in the rack.
We construct the counter ID from the rack UUID, which is constructed
from the name "dc:rack". The pending replica uses a deterministic
variation of the rack's counter ID by negating it.
This improves the performance and size of counter cells by having less
unique counter IDs and less counter shards in a counter cell.
Previously the number of counter shards was the number of different
host_id's that updated the counter, which can be typically the number of
nodes in the cluster and continue growing indefinitely when nodes are
replaced. with the rack-based counter id the number of counter shards
will be at most twice the number of different racks (including removed
racks, which should not be significant).
Fixes SCYLLADB-356
Add .set_skip_when_empty() to four metrics in replica/database.cc that
are only incremented on very rare error paths and are almost always zero:
- database::dropped_view_updates: view updates dropped due to overload.
NOTE: this metric appears to never be incremented in the current
codebase and may be a candidate for removal.
- database::multishard_query_failed_reader_stops: documented as a 'hard
badness counter' that should always be zero. NOTE: no increment site
was found in the current codebase; may be a candidate for removal.
- database::multishard_query_failed_reader_saves: documented as a 'hard
badness counter' that should always be zero.
- database::total_writes_rejected_due_to_out_of_space_prevention: only
fires when disk utilization is critical and user table writes are
disabled, a very rare operational state.
These metrics create unnecessary reporting overhead when they are
perpetually zero. set_skip_when_empty() suppresses them from metrics
output until they become non-zero.
AI-Assisted: yes
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
Closesscylladb/scylladb#29345
After obtaining the CQL response, check if its actual size exceeds the initially acquired memory permit. If so, acquire additional semaphore units and adopt them into the permit, ensuring accurate memory accounting for large responses.
Additionally, move the permit into a .then() continuation so that the semaphore units are kept alive until write_message finishes, preventing premature release of memory permit. This is especially important with slow networks and big responses when buffers can accumulate and deplete a node's memory.
Fixes: https://scylladb.atlassian.net/browse/SCYLLADB-1306
Related https://scylladb.atlassian.net/browse/SCYLLADB-740
Backport: all supported versions
Closesscylladb/scylladb#29288
* github.com:scylladb/scylladb:
transport: add per-service-level pending response memory metric
transport: hold memory permit until response write completes
transport: account for response size exceeding initial memory estimate
Add .set_skip_when_empty() to four metrics in the db module that are
only incremented on very rare error paths and are almost always zero:
- cache::pinned_dirty_memory_overload: described as 'should sit
constantly at 0, nonzero is indicative of a bug'
- corrupt_data::entries_reported: only fires on actual data corruption
- hints::corrupted_files: only fires on on-disk hint file corruption
- rate_limiter::failed_allocations: only fires when the rate limiter
hash table is completely full and gives up allocating, requiring
extreme cardinality pressure
These metrics create unnecessary reporting overhead when they are
perpetually zero. set_skip_when_empty() suppresses them from metrics
output until they become non-zero.
AI-Assisted: yes
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
Closesscylladb/scylladb#29344
get_live_members function called is_shutdown which inet_address
argument, which caused temporary endpoint_state to be created. Fix
it by prohibiting implicit conversion and calling the correct
is_shutdown function instead.
The decommission sets left gossiper state only to prevent shutdown
notification be issued by the node during shutdown. Since the
notification code now checks the state in raft topology this is no
longer needed.
The state machine used by strongly consistent tablets may block on a
read barrier if the local schema is insufficient to resolve pending
mutations [1]. To deal with that, we perform a read barrier that may
block for a long time.
When a strongly consistent tablet is being removed, we'd like to cancel
all ongoing executions of `state_machine::apply`: the shard is no
longer responsible for the tablet, so it doesn't matter what the outcome
is.
---
In the implementation, we abort the operations by simply throwing
an exception from `state_machine::apply` and not doing anything.
That's a red flag considering that it may lead to the instance
being killed on the spot [2].
Fortunately for us, strongly consistent tables use the default Raft
server implementation, i.e. `raft::server_impl`, which actually
handles one type of an exception thrown by the method: namely,
`abort_requested_exception`, which is the default exception thrown
by `seastar::abort_source` [3]. We leverage this property.
---
Unfortunately, `raft::server_impl::abort` isn't perfectly suited for
us. If we look into its code, we'll see that the relevant portion of
the procedure boils down to three steps:
1. Prevent scheduling adding new entries.
2. Wait for the applier fiber.
3. Abort the state machine.
Since aborting the state machine happens only after the applier fiber
has already finished, there will no longer be anything to abort. Either
all executions of `state_machine::apply` have already finished, or they
are hanging and we cannot do anything.
That's a pre-existing problem that we won't be solving here (even
though it's possible). We hope the problem will be solved, and it seems
likely: the code suggests that the behavior is not intended. For more
details, see e.g. [4].
---
We provide two validation tests. They simulate the abortion of
`state_machine::apply` in two different scenarios:
* when the table is dropped (which should also cover the case of tablet
migration),
* when the node is shutting down.
The value of the tests isn't high since they don't ensure that the
state of the group is still valid (though it should be), nor do they
perform any other check. Instead, we rely on the testing framework to
spot any anomalies or errors. That's probably the best we can do at
the moment.
Unfortunately, both tests are marked as skipped becuause of the current
limitations of `raft::server_impl::abort` described above and in [4].
References:
[1] 4c8dba1
[2] See the description of `raft::state_machine` in `raft/raft.hh`.
[3] See `server_impl::applier_fiber` in `raft/server.cc`.
[4] SCYLLADB-1056
These changes are complementary to those from a recent commit where we
handled aborting ongoing operations during tablet events, such as
tablet migration. In this commit, we consider the case of shutting down
a node.
When a node is shutting down, we eventually close the connections. When
the client can no longer get a response from the server, it makes no
sense to continue with the queries. We'd like to cancel them at that
point.
We leverage the abort source passed down via `client_state` down to
the strongly consistent coordinator. This way, the transport layer can
communicate with it and signal that the queries should be canceled.
The abort source is triggered by the CQL server (cf.
`generic_server::server::{stop,shutdown}`).
---
Note that this is not an optional change. In fact, if we don't abort
those requests, we might hang for an indefinite amount of time when
executing the following code in `main.cc`:
```
// Register at_exit last, so that storage_service::drain_on_shutdown will be called first
auto do_drain = defer_verbose_shutdown("local storage", [&ss] {
ss.local().drain_on_shutdown().get();
});
```
The problem boils down to the fact that `generic_server::server::stop`
will wait for all connections to be closed, but that won't happen until
all ongoing operations (at least those to strongly consistent tables)
are finished.
It's important to highlight that even though we hang on this, the
client can no longer get any response. Thus, it's crucial that at that
point we simply abort ongoing operations to proceed with the rest of
shutdown.
---
Two tests are added to verify that the implementation is correct:
one focusing on local operations, the other -- on a forwarded write.
Difference in time spent on the whole test file
`test_strong_consistency.py` on my local machine, in dev mode:
Before:
```
real 0m31.775s
user 1m4.475s
sys 0m22.615s
```
After:
```
real 0m32.024s
user 1m10.751s
sys 0m23.871s
```
Individual runs of the added tests:
test_queries_when_shutting_down:
```
real 0m12.818s
user 0m36.726s
sys 0m4.577s
```
test_abort_forwarded_write_upon_shutdown:
```
real 0m12.930s
user 0m36.622s
sys 0m4.752s
```
We make `client_state` store a pointer to an `abort_source`. This will
be useful in the following commit that will implement aborting ongoing
requests to strongly consistent tables upon connection shutdowns.
It might also be useful in some other places in the code in the future.
We set the abort source for client states in relevant places.
When a strongly consistent Raft group is being removed, it means one of
the following cases:
(A) The node is shutting down and it's simply part of the the shutdown
procedure.
(B) The tablet is somehow leaving the replica. For example, due to:
- Tablet migration
- Tablet split/merge
- Tablet removal (e.g. because the table is dropped)
In this commit, we focus on case (A). Case (B) will be handled in the
following one.
---
The changes in the code are literally none, and there's a reason to it.
First, let's note that we've already implemented abortion of timed-out
requests. There is a limit to how long a query can run and sooner or
later it will finish, regardless of what we do.
Second, we need to ask ourselves if the cases we're considering in this
commit (i.e. case (B)) is a situation where we'd like to speed up the
process. The answer is no.
Tablet migrations are effectively internal operations that are invisible
to the users. User requests are, quite obviously, the opposite of that.
Because of that, we want to patiently wait for the queries to finish or
time out, even though it's technically possible to lead to an abort
earlier.
Lastly, the changes in the code that actually appear in this commit are
not completely irrelevant either. We consider the important case of
the `leader_info_updater` fiber and argue that it's safe to not pass
any abort source to the Raft methods used by it.
---
Unfortunately, we don't have tablet migrations implemented yet [1],
so our testing capabilities are limited. Still, we provide a new test
that corresponds to case (B) described above. We simulate a tablet
migration by dropping a table and observe how reads and writes behave
in such a situation. There's no extremely careful validation involved
there, but that's what we can have for the time being.
Difference in time spent on the whole test file
`test_strong_consistency.py` on my local machine, in dev mode:
Before:
```
real 0m30.841s
user 1m3.294s
sys 0m21.091s
```
After:
```
real 0m31.775s
user 1m4.475s
sys 0m22.615s
```
The time spent on the new test only:
```
real 0m5.264s
user 0m34.646s
sys 0m3.374s
```
References:
[1] SCYLLADB-868
If a query, either a write, or a read to a strongly consistent table,
times out, we immediately abort the operation and throw an exception.
Unfortunately, due to the inconsistency in exception types thrown
on timeout by the many methods we use in the code, it results in
pretty messy `try-catch` clauses. Perhaps there's a better alternative
to this, but it's beyond the scope of this work, so we leave it as-is.
We provide a validation test that consists of three cases corresponding
to reads, writes, and waiting for the leader. They verify that the code
works as expected in all affected places.
A comparison of time spent on the whole `test_strong_consistency.py` on
my local machine, in dev mode:
Before:
```
real 0m32.185s
user 0m55.391s
sys 0m15.745s
```
After:
```
real 0m30.841s
user 1m3.294s
sys 0m21.091s
```
The time spent on the new test only:
```
real 0m7.077s
user 0m35.359s
sys 0m3.717s
```
- Document Alternator (DynamoDB-compatible API) auditing support in
the operator-facing auditing guide (docs/operating-scylla/security/auditing.rst)
- Cover operation-to-category mapping, operation field format,
keyspace/table filtering, and audit log examples
- Document the audit_tables=alternator.<table> shorthand format
- Minor wording improvements throughout (Scylla -> ScyllaDB,
clarify default audit backend)
Closesscylladb/scylladb#29231
We remove the inconsistency between reads and writes to strongly
consistent tables. Before the commit, only reads used a timeout.
Now, writes do as well.
Although the parameter isn't used yet, that will change in the following
commit. This is a prerequisite for it.
We enclose `coordinator::{mutate,query}` with `try-catch` clauses. They
do nothing at the moment, but we'll use them later. We do this now to
avoid noise in the upcoming commits.
We'll fix the indentation in the following commit.
The loop shouldn't throw any other exception than the ones already
covered by the `catch` claues. Crash, at least when
`abort_on_internal_error` is set, if we catch any other type since
that may be a sign of a bug.
All used configs and cmdlines share the same values. Let's extract them
to avoid repeating them every time a new test is written. Those options
should be enabled for all tests in the file anyway.
Fixes#29043 with the following docs changes:
- docs/dev/system-keyspaces.md: Added a new file that documents all keyspaces created internally
Closesscylladb/scylladb#29044
Spreading db::config around and making all services depend on it is not nice. Most other service that need configuration provide their own config that's populated from db::config in main.cc/cql_test_env.cc and use it, not the global config.
This PR does the same for repair_service.
Enhancing components dependencies, not backporting
Closesscylladb/scylladb#29153
* github.com:scylladb/scylladb:
repair: Remove db/config.hh from repair/*.cc files
repair: Move repair_multishard_reader options onto repair_service::config
repair: Move critical_disk_utilization_level onto repair_service::config
repair: Move repair_partition_count_estimation_ratio onto repair_service::config
repair: Move repair_hints_batchlog_flush_cache_time_in_ms onto repair_service::config
repair: Move enable_small_table_optimization_for_rbno onto repair_service::config
repair: Introduce repair_service::config
While working on benchmarks for strong consistency we noticed that the raft logic attempted to take snapshots during the benchmark. Snapshot transfer is not implemented for strong consistency yet and the methods that take or transfer snapshots throw exceptions. This causes the raft groups to stop working completely.
While implementing snapshot transfers is out of scope, we can implement some mitigations now to stop the tests from breaking:
- The first commit adjusts the configuration options. First, it disables periodic snapshotting (i.e. creating a snapshot every X log entries). Second, it increases the memory threshold for the raft log before which a snapshot is created from 2MB to 10MB.
- The second commit relaxes the take snapshot / drop snapshot methods and makes it possible to actually use them - they are no-ops. It is still forbidden to transfer snapshots.
I am including both commits because applying only the first one didn't completely prevent the issue from occurring when testing locally.
Refs: SCYLLADB-1115
Strong consistency is experimental, no need for backport.
Closesscylladb/scylladb#29189
* github.com:scylladb/scylladb:
strong_consistency: fake taking and dropping snapshots
strong_consistency: adjust limits for snapshots
This commit removes references ScyllaDB versions ("Since x.y")
from the ScyllaDB documentation on Docker Hub, as they are
redundant and confusing (some versions are super ancient).
Fixes SCYLLADB-1212
Closesscylladb/scylladb#29204
The code responds ealry with READY message, but lack some necessary set up, namely:
* update_scheduling_group(): without it, the connection runs under the default scheduling group instead of the one mapped to the user's service level.
* on_connection_ready(): without it, the connection never releases its slot in the uninitialized-connections concurrency semaphore (acquired at connection creation), leaking one unit per cert-authenticated connection for the lifetime of the connection.
* _authenticating = false / _ready = true: without them, system.clients reports connection_stage = AUTHENTICATING forever instead of READY (not critical, but not nice either)
The PR fixes it and adds a regression test, that (for sanity) also covers AllowAll and Password authrticators
Fixes SCYLLADB-1226
Present since 2025.1, probably worth backporting
Closesscylladb/scylladb#29220
* github.com:scylladb/scylladb:
transport: fix process_startup cert-auth path missing connection-ready setup
transport: test that connection_stage is READY after auth via all process_startup paths
During incremental repair, each tablet replica holds three SSTable views:
UNREPAIRED, REPAIRING, and REPAIRED. The repair lifecycle is:
1. Replicas snapshot unrepaired SSTables and mark them REPAIRING.
2. Row-level repair streams missing rows between replicas.
3. mark_sstable_as_repaired() runs on all replicas, rewriting the
SSTables with repaired_at = sstables_repaired_at + 1 (e.g. N+1).
4. The coordinator atomically commits sstables_repaired_at=N+1 and
the end_repair stage to Raft, then broadcasts
repair_update_compaction_ctrl which calls clear_being_repaired().
The bug lives in the window between steps 3 and 4. After step 3, each
replica has on-disk SSTables with repaired_at=N+1, but sstables_repaired_at
in Raft is still N. The classifier therefore sees:
is_repaired(N, sst{repaired_at=N+1}) == false
sst->being_repaired == null (lost on restart, or not yet set)
and puts them in the UNREPAIRED view. If a new write arrives and is
flushed (repaired_at=0), STCS minor compaction can fire immediately and
merge the two SSTables. The output gets repaired_at = max(N+1, 0) = N+1
because compaction preserves the maximum repaired_at of its inputs.
Once step 4 commits sstables_repaired_at=N+1, the compacted output is
classified REPAIRED on the affected replica even though it contains data
that was never part of the repair scan. Other replicas, which did not
experience this compaction, classify the same rows as UNREPAIRED. This
divergence is never healed by future repairs because the repaired set is
considered authoritative. The result is data resurrection: deleted rows
can reappear after the next compaction that merges unrepaired data with the
wrongly-promoted repaired SSTable.
The fix has two layers:
Layer 1 (in-memory, fast path): mark_sstable_as_repaired() now also calls
mark_as_being_repaired(session) on the new SSTables it writes. This keeps
them in the REPAIRING view from the moment they are created until
repair_update_compaction_ctrl clears the flag after step 4, covering the
race window in the normal (no-restart) case.
Layer 2 (durable, restart-safe): a new is_being_repaired() helper on
tablet_storage_group_manager detects the race window even after a node
restart, when being_repaired has been lost from memory. It checks:
sst.repaired_at == sstables_repaired_at + 1
AND tablet transition kind == tablet_transition_kind::repair
Both conditions survive restarts: repaired_at is on-disk in SSTable
metadata, and the tablet transition is persisted in Raft. Once the
coordinator commits sstables_repaired_at=N+1 (step 4), is_repaired()
returns true and the SSTable naturally moves to the REPAIRED view.
The classifier in make_repair_sstable_classifier_func() is updated to call
is_being_repaired(sst, sstables_repaired_at) in place of the previous
sst->being_repaired.uuid().is_null() check.
A new test, test_incremental_repair_race_window_promotes_unrepaired_data,
reproduces the bug by:
- Running repair round 1 to establish sstables_repaired_at=1.
- Injecting delay_end_repair_update to hold the race window open.
- Running repair round 2 so all replicas complete mark_sstable_as_repaired
(repaired_at=2) but the coordinator has not yet committed step 4.
- Writing post-repair keys to all replicas and flushing servers[1] to
create an SSTable with repaired_at=0 on disk.
- Restarting servers[1] so being_repaired is lost from memory.
- Waiting for autocompaction to merge the two SSTables on servers[1].
- Asserting that the merged SSTable contains post-repair keys (the bug)
and that servers[0] and servers[2] do not see those keys as repaired.
NOTE FOR MAINTAINER: Copilot initially only implemented Layer 1 (the
in-memory being_repaired guard), missing the restart scenario entirely.
I pointed out that being_repaired is lost on restart and guided Copilot
to add the durable Layer 2 check. I also polished the implementation:
moving is_being_repaired into tablet_storage_group_manager so it can
reuse the already-held _tablet_map (avoiding an ERM lookup and try/catch),
passing sstables_repaired_at in from the classifier to avoid re-reading it,
and using compaction_group_for_sstable inside the function rather than
threading a tablet_id parameter through the classifier.
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1239.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29244
This series fixes two related inconsistencies around secondary-index
names.
1. `DESCRIBE INDEX ... WITH INTERNALS` returned the backing
materialized-view name in the `name` column instead of the logical
index name.
2. The snapshot REST API accepted backing table names for MV-backed
secondary indexes, but not the logical index names exposed to users.
The snapshot side now resolves logical secondary-index names to backing
table names where applicable, reports logical index names in snapshot
details, rejects vector index names with HTTP 400, and keeps multi-keyspace
DELETE atomic by resolving all keyspaces before deleting anything.
The tests were also extended accordingly, and the snapshot test helper
was fixed to clean up multi-table snapshots using one DELETE per table.
Fixes: SCYLLADB-1122
Minor bugfix, no need to backport.
Closesscylladb/scylladb#29083
* github.com:scylladb/scylladb:
cql3: fix DESCRIBE INDEX WITH INTERNALS name
test: add snapshot REST API tests for logical index names
test: fix snapshot cleanup helper
api: clarify snapshot REST parameter descriptions
api: surface no_such_column_family as HTTP 400
db: fix clear_snapshot() atomicity and use C++23 lambda form
db: normalize index names in get_snapshot_details()
db: add resolve_table_name() to snapshot_ctl
To create `process_staging` view building tasks, we firstly need to collect informations about them on shard0, create necessary mutations, commit them to group0 and move staging sstables objects to their original shards.
But there is a possible race after committing the group0 command and before moving the staging sstables to their shards. Between those two events, the coordinator may schedule freshly created tasks and dispatch them to the worker but the worker won't have the sstables objects because they weren't moved yet.
This patch fixes the race by holding `_staging_sstables_mutex` locks from all necessary shards when executing `create_staging_sstable_tasks()`. With this, even if the task will be scheduled and dispatched quickly, the worker will wait with executing it until the sstables objects are moved and the locks are released.
Fixes SCYLLADB-816
This PR should be backported to all versions containing view building coordinator (2025.4 and newer).
Closesscylladb/scylladb#29174
* github.com:scylladb/scylladb:
db/view/view_building_worker: fix indentation
db/view/view_building_worker: lock staging sstables mutex for necessary shards when creating tasks
DESCRIBE INDEX ... WITH INTERNALS returned the name of
the backing materialized view in the name column instead
of the logical index name.
Return the logical index name from schema::describe()
for index schemas so all callers observe the
user-facing name consistently.
Fixes: SCYLLADB-1122
Add focused REST coverage for logical secondary-index
names in snapshot creation, deletion, and details
output.
Also cover vector-index rejection and verify
multi-keyspace delete resolves all keyspaces before
deleting anything so mixed index kinds cannot cause
partial removal.
The snapshot REST helper cleaned up multi-table
snapshots with a single DELETE request that passed a
comma-separated cf filter, but the API accepts only one
table name there.
Delete each table snapshot separately so existing tests
that snapshot multiple tables use the API as
documented.
Document the current /storage_service/snapshots behavior
more accurately.
For DELETE, cf is a table filter applied independently
in each keyspace listed in kn. If cf is omitted or
empty, snapshots for all tables are eligible, and
secondary indexes can be addressed by their logical
index name.
Snapshot requests that name a non-existent table or a
non-snapshotable logical index currently surface an
internal server error.
Translate no_such_column_family into a bad request so
callers get a client-facing error that matches the
invalid input.
clear_snapshot() applies a table filter independently in
each keyspace, so logical index names must be resolved
per keyspace on the delete path as well.
Resolve all keyspaces before deleting anything so a later
failure cannot partially remove a snapshot, and use the
explicit-object-parameter coroutine lambda form for the
asynchronous implementation.
Snapshot details exposed backing secondary-index view
names instead of logical index names.
Normalize index entries in get_snapshot_details() so the
REST API reports the user-facing name, and update the
existing REST test to assert that behavior directly.
The snapshot REST API accepted backing secondary-index
table names, but not logical index names.
Introduce resolve_table_name() so snapshot creation can
translate a logical index name to the backing table when
the index is materialized as a view.
Currently we don't support 'local' consistency, which would
imply maintaining separate raft group for each dc. What we
support is actually 'global' consistency -- one raft group
per tablet replica set. We don't plan to support local
consistency for the first GA.
Closesscylladb/scylladb#29221
Hi, thanks for Scylla!
We found a small issue in tracker::set_configuration() during joint consensus and put together a fix.
When a server is demoted from voter to non-voter, set_configuration processes the current config first (can_vote=false), then the previous config. But when it finds the server already in the progress map (tracker.cc:118), it hits `continue` without updating can_vote. So the server's follower_progress::can_vote stays false even though it's still a voter in the previous config.
This causes broadcast_read_quorum (fsm.cc:1055) to skip the demoted server, reducing the pool of responders. Since committed() correctly includes the server in _previous_voters for quorum calculation, read barriers can stall if other servers are slow.
The fix is to use configuration::can_vote() in tracker::set_configuration.
We included a reproduction unit test (test_tracker_voter_demotion_joint_config) that extracts the set_configuration algorithm and demonstrates the mismatch. We weren't able to build the full Scylla test suite to add an in-tree test, so we kept it as a standalone file for reference.
No backport: the bug is non-critical and the change needs some soak time in master.
Closesscylladb/scylladb#29226
* https://github.com/scylladb/scylladb:
fix: use is_voter::yes instead of true in test assertions
test: add tracker voter demotion test to fsm_test.cc
fix: use configuration::can_vote() in tracker::set_configuration
The test_create_index_synchronous_updates test in test_secondary_index_properties.py
was intermittently failing with 'assert found_wanted_trace' because the expected
trace event 'Forcing ... view update to be synchronous' was missing from the
trace events returned by get_query_trace().
Root cause: trace events are written asynchronously to system_traces.events.
The Python driver's populate() method considers a trace complete once the
session row in system_traces.sessions has duration IS NOT NULL, then reads
events exactly once. Since the session row and event rows are written as
separate mutations with no transactional guarantee, the driver can read an
incomplete set of events.
Evidence from the failed CI run logs:
- The entire test (CREATE TABLE through DROP TABLE) completed in ~300ms
(01:38:54,859 - 01:38:55,157)
- The INSERT with tracing happened in a ~50ms window between the second
CREATE INDEX completing (01:38:55,108) and DROP TABLE starting
(01:38:55,157)
- The 'Forcing ... synchronous' trace message is generated during the
INSERT write path (db/view/view.cc:2061), so it was produced, but
not yet flushed to system_traces.events when the driver read them
- This matches the known limitation documented in test/alternator/
test_tracing.py: 'we have no way to know whether the tracing events
returned is the entire trace'
Fix: replace the single-shot trace.events read with a retry loop that
directly queries system_traces.events until the expected event appears
(with a 30s timeout). Use ConsistencyLevel.ONE since system_traces has
RF=2 and cqlpy tests run on a single-node cluster.
The same race condition pattern exists in test_mv_synchronous_updates in
test_materialized_view.py (which this test was modeled after), so the
same fix is proactively applied there as well.
Fixes SCYLLADB-1314
Closesscylladb/scylladb#29374
Previously, the timestamp decoded from a timeuuid was printed using the
local timezone via datetime.fromtimestamp(), which produces different
output depending on the machine's locale settings.
ScyllaDB logs are emitted in UTC by default. Printing the decoded date
in UTC makes it straightforward to correlate SSTable identifiers with
log entries without having to mentally convert timezones.
Also fix the embedded pytest assertion, which was accidentally correct
only on machines in UTC+8 — it now uses an explicit UTC-aware datetime.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29253
Add explicit permissions block (contents: read, pull-requests: write,
statuses: write) matching the requirements of the called reusable
workflow which checks out code, posts PR comments, and sets commit
statuses. Fixes code scanning alert #172.
Closesscylladb/scylladb#29183
Replace strict case-sensitive '== "True"' check with strcasecmp(..., "true")
so that Python's str(True) -> "True" is properly recognized. Accepts any
case variation of "true" ("True", "TRUE", etc.), with empty string
defaulting to false.
Maintains backward compatibility with out-of-tree tests that rely on
Python's bool stringification.
The goal is to reduce the number of distinct ways API handlers use to
convert string http query parameters into bool variables. This place is the
only one that simply compares param to "True".
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29236
Switch to branch lts_2026_01_07, which is exactly equal to
upstream now.
There were no notable changes in the release notes, but the
new versions are more friendly to newer compilers (specifically,
in include hygiene).
configure.py needs a few library updates; cmake works without
change.
scylla-gdb.py updated for new hash table layout (by Claude Opus 4.6).
* abseil d7aaad83...255c84da (1179):
> Abseil LTS branch, Jan 2026, Patch 1 (#2007)
> Cherry-picks for LTS 20260107 (#1990)
> Apply LTS transformations for 20260107 LTS branch (#1989)
> Mark legacy Mutex methods and MutexLock pointer constructors as deprecated
> `cleanup`: specify that it's safe to use the class in a signal handler.
> Suppress bugprone-use-after-move in benign cases
> StrFormat: format scientific notation without heap allocation
> Introduce a legacy copy of GetDebugStackTraceHook API.
> Report 1ns instead of 0ns for probe_benchmarks. Some tools incorrectly assume that benchmark was not run if 0ns reported.
> Add absl::chunked_queue
> `CRC32` version of `CombineContiguous` for length <= 32.
> Add `absl::down_cast`
> Fix FixedArray iterator constructor, which should require input_iterator, not forward_iterator
> Add a latency benchmark for hashing a pair of integers.
> Delete absl::strings_internal::STLStringReserveAmortized()
> As IsAtLeastInputIterator helper
> Use StringAppendAndOverwrite() in CEscapeAndAppendInternal()
> Add support for absl::(u)int128 in FastIntToBuffer()
> absl/strings: Prepare helper for printing objects to string representations.
> Use SimpleAtob() for parsing bool flags
> No-op changes to relative timeout support code.
> Adjust visibility of heterogeneous_lookup_testing.h
> Remove -DUNORDERED_SET_CXX17 since the macro no longer exists
> [log] Prepare helper for streaming container contents to strings.
> Restrict the visibility of some internal testing utilities
> Add absl::linked_hash_set and absl::linked_hash_map
> [meta] Add constexpr testing helper.
> BUILD file reformatting.
> `absl/meta`: Add C++17 port of C++20 `requires` expression for internal use
> Remove the implementation of `absl::string_view`, which was only needed prior to C++17. `absl::string_view` is now an alias for `std::string_view`. It is recommended that clients simply use `std::string_view`.
> No public description
> absl:🎏 Stop echoing file content in flagfile parsing errors Modified ArgsList::ReadFromFlagfile to redact the content of unexpected lines from error messages. \
> Refactor the declaration of `raw_hash_set`/`btree` to omit default template parameters from the subclasses.
> Import of CCTZ from GitHub.
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND to Flag help generator
> Correct `Mix4x16Vectors` comment.
> Special implementation for string hash with sizes greater than 64.
> Reorder function parameters so that hash state is the first argument.
> Search more aggressively for open slots in absl::internal_stacktrace::BorrowedFixupBuffer
> Implement SpinLockHolder in terms of std::lock_guard.
> No public description
> Avoid discarding test matchers.
> Import of CCTZ from GitHub.
> Automated rollback of commit 9f40d6d6f3cfc1fb0325dd8637eb65f8299a4b00.
> Enable clang-specific warnings on the clang-cl build instead of just trying to be MSVC
> Enable clang-specific warnings on the clang-cl build instead of just trying to be MSVC
> Make AnyInvocable remember more information
> Add further diagnostics under clang for string_view(nullptr)
> Import of CCTZ from GitHub.
> Document the differing trimming behavior of absl::Span::subspan() and std::span::subspan()
> Special implementation for string hash with sizes in range [33, 64].
> Add the deleted string_view(std::nullptr_t) constructor from C++23
> CI: Use a cached copy of GoogleTest in CMake builds if possible to minimize the possibility of errors downloading from GitHub
> CI: Enable libc++ hardening in the ASAN build for even more checks https://libcxx.llvm.org/Hardening.html
> Call the common case of AllocateBackingArray directly instead of through the function pointer.
> Change AlignedType to have a void* array member so that swisstable backing arrays end up in the pointer-containing partition for heap partitioning.
> base: Discourage use of ABSL_ATTRIBUTE_PACKED
> Revert: Add an attribute to HashtablezInfo which performs a bitwise XOR on all hashes. The purposes of this attribute is to identify if identical hash tables are being created. If we see a large number of identical tables, it's likely the code can be improved by using a common table as opposed to keep rebuilding the same one.
> Import of CCTZ from GitHub.
> Record insert misses in hashtable profiling.
> Add absl::StatusCodeToStringView.
> Add a missing dependency on str_format that was being pulled in transitively
> Pico-optimize `SkipWhitespace` to use `StripLeadingAsciiWhitespace`.
> absl::string_view: Upgrade the debug assert on the single argument char* constructor to ABSL_HARDENING_ASSERT
> Use non-stack storage for stack trace buffers
> Fixed incorrect include for ABSL_NAMESPACE_BEGIN
> Add ABSL_REFACTOR_INLINE to separate the inliner directive from the deprecated directive so that we can give users a custom deprecation message.
> Reduce stack usage when unwinding without fixups
> Reduce stack usage when unwinding from 170 to 128 on x64
> Rename RecordInsert -> RecordInsertMiss.
> PR #1968: Use std::move_backward within InlinedVector's Storage::Insert
> Use the new absl::StringResizeAndOverwrite() in CUnescape()
> Explicitly instantiate common `raw_hash_set` backing array functions.
> Rollback reduction of maximum load factor. Now it is back to 28/32.
> Export Mutex::Dtor from shared libraries in NDEBUG mode
> Allow `IsOkAndHolds` to rely on duck typing for matching `StatusOr` like types instead of uniquely `absl::StatusOr`, e.g. `google::cloud::StatusOr`.
> Fix typo in macro and add missing static_cast for WASM builds.
> windows(cmake): add abseil_test_dll to target link libraries when required
> Handle empty strings in `SimpleAtof` after stripping whitespace
> Avoid using a thread_local in an inline function since this causes issues on some platforms.
> (Roll forward) Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes.
> Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes.
> Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes.
> Fixes for String{Resize|Append}AndOverwrite - StringAppendAndOverwrite() should always call StringResizeAndOverwrite() with at least capacity() in case the standard library decides to shrink the buffer (Fixes#1965) - Small refactor to make the minimum growth an addition for clarity and to make it easier to test 1.5x growth in the future - Turn an ABSL_HARDENING_ASSERT into a ThrowStdLengthError - Add a missing std::move
> Correct the supported features of Status Matchers
> absl/time: Use "memory order acquire" for loads, which would allow for the safe removal of the data memory barrier.
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> Add an internal-only helper StringAppendAndOverwrite() similar to StringResizeAndOverwrite() but optimized for repeated appends, using exponential growth to ensure amortized complexity of increasing a string size by a small amount is O(1).
> Release `ABSL_EXPECT_OK` and `ABSL_ASSERT_OK`.
> Fix the CHECK_XX family of macros to not print `char*` arguments as C-strings if the comparison happened as pointers. Printing as pointers is more relevant to the result of the comparison.
> Rollback StringAppendAndOverwrite() - the problem is that StringResizeAndOverwrite has MSAN testing of the entire string. This causes quadratic MSAN verification on small appends.
> Add an internal-only helper StringAppendAndOverwrite() similar to StringResizeAndOverwrite() but optimized for repeated appends, using exponential growth to ensure amortized complexity of increasing a string size by a small amount is O(1).
> PR #1961: Fix Clang warnings on powerpc
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> macOS CI: Move the Bazel vendor_dir to ${HOME} to workaround a Bazel issue where it does not work when it is in ${TMP} and also fix the quoting which was causing it to incorrectly receive the argument
> Use __msan_check_mem_is_initialized for detailed MSan report
> Optimize stack unwinding by reducing `AddressIsReadable` calls.
> Add internal API to allow bypassing stack trace fixups when needed
> absl::StrFormat: improve test coverage with scientific exponent test cases
> Add throughput and latency benchmarks for `absl::ToDoubleXYZ` functions.
> CordzInfo: Use absl::NoDestructor to remove a global destructor. Chromium requires no global destructors.
> string_view: Enable std::view and std::borrowed_range
> cleanup: s/logging_internal/log_internal/ig for consistency
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> Use the new absl::StringResizeAndOverwrite() in absl::AsciiStrTo{Lower|Upper}
> Use the new absl::StringResizeAndOverwrite() in absl::StrJoin()
> Use the new absl::StringResizeAndOverwrite() in absl::StrCat()
> string_view: Fix include order
> Don't pass nullptr as the 1st arg of `from_chars`
> absl/types: format code with clang-format.
> Validate absl::StringResizeAndOverwrite op has written bytes as expected.
> Skip the ShortStringCollision test on WASM.
> Rollback `absl/types`: format code with clang-format.
> Remove usage of the WasmOffsetConverter for Wasm / Emscripten stack-traces.
> Use the new absl::StringResizeAndOverwrite() in absl::CordCopyToString()
> Remove an undocumented behavior of --vmodule and absl::SetVLogLevel that could set a module_pattern to defer to the global vlog threshold.
> Update to rules_cc 0.2.9
> Avoid redefine warnings with ntstatus constants
> PR #1944: Use same element-width for non-temporal loads and stores on Arm
> absl::StringResizeAndOverwrite(): Add the requirement that the only value that can be written to buf[size] is the terminator character.
> absl/types: format code with clang-format.
> Minor formatting changes.
> Remove `IntIdentity` and `PtrIdentity` from `raw_hash_set_probe_benchmark`.
> Automated rollback of commit cad60580dba861d36ed813564026d9774d9e4e2b.
> FlagStateInterface implementors need only support being restored once.
> Clarify the post-condition of `reserve()` in Abseil hash containers.
> Clarify the post-condition of `reserve()` in Abseil hash containers.
> Represent dropped samples in hashtable profile.
> Add lifetimebound to absl::implicit_cast and make it work for rvalue references as it already does with lvalue references
> Clean up a doc example where we had `absl_nonnull` and `= nullptr;`
> Change Cordz to synchronize tracked cords with Snapshots / DeleteQueue
> Minor refactor to `num_threads` in deadlock test
> Rename VLOG macro parameter to match other uses of this pseudo type.
> `time`: Fix indentation
> Automated Code Change
> Adds `absl::StringResizeAndOverwrite` as a polyfill for C++23's `std::basic_string<CharT,Traits,Allocator>::resize_and_overwrite`
> Internal-only change
> absl/time: format code with clang-format.
> No public description
> Expose typed releasers of externally appended memory.
> Fix __declspec support for ABSL_DECLARE_FLAG()
> Annotate absl::AnyInvocable as an owner type via [[gsl::Owner]] and absl_internal_is_view = std::false_type
> Annotate absl::FunctionRef as a view type via [[gsl::Pointer]] and absl_internal_is_view
> Remove unnecessary dep on `core_headers` from the `nullability` cc_library
> type_traits: Add type_identity and type_traits_t backfills
> Refactor raw_hash_set range insertion to call private insert_range function.
> Fix bug in absl::FunctionRef conversions from non-const to const
> PR #1937: Simplify ConvertSpecialToEmptyAndFullToDeleted
> Improve absl::FunctionRef compatibility with C++26
> Add a workaround for unused variable warnings inside of not-taken if-constexpr codepaths in older versions of GCC
> Annotate ABSL_DIE_IF_NULL's return type with `absl_nonnull`
> Move insert index computation into `PrepareInsertLarge` in order to reduce inlined part of insert/emplace operations.
> Automated Code Change
> PR #1939: Add missing rules_cc loads
> Expose (internally) a LogMessage constructor taking file as a string_view for (internal, upcoming) FFI integration.
> Fixed up some #includes in mutex.h
> Make absl::FunctionRef support non-const callables, aligning it with std::function_ref from C++26
> Move capacity update in `Grow1To3AndPrepareInsert` after accessing `common.infoz()` to prevent assertion failure in `control()`.
> Fix check_op(s) compilation failures on gcc 8 which eagerly tries to instantiate std::underlying_type for non-num types.
> Use `ABSL_ATTRIBUTE_ALWAYS_INLINE`for lambda in `find_or_prepare_insert_large`.
> Mark the implicit floating operators as constexpr for `absl::int128` and `absl::uint128`
> PR #1931: raw_hash_set: fix instantiation for recursive types on MSVC with /Zc:__cplusplus
> Add std::pair specializations for IsOwner and IsView
> Cast ABSL_MIN_LOG_LEVEL to absl::LogSeverityAtLeast instead of absl::LogSeverity.
> Fix a corner case in the aarch64 unwinder
> Fix inconsistent nullability annotation in ReleasableMutexLock
> Remove support for Native Client
> Rollback f040e96b93dba46e8ed3ca59c0444cbd6c0a0955
> When printing CHECK_XX failures and both types are unprintable, don't bother printing " (UNPRINTABLE vs. UNPRINTABLE)".
> PR #1929: Fix shorten-64-to-32 warning in stacktrace_riscv-inl.inc
> Refactor `find_or_prepare_insert_large` to use a single return statement using a lambda.
> Use possible CPUs to identify NumCPUs() on Linux.
> Fix incorrect nullability annotation of `absl::Cord::InlineRep::set_data()`.
> Move SetCtrl* family of functions to cc file.
> Change absl::InlinedVector::clear() so that it does not deallocate any allocated space. This allows allocations to be reused and matches the behavior specification of std::vector::clear().
> Mark Abseil container algorithms as `constexpr` for C++20.
> Fix `CHECK_<OP>` ambiguous overload for `operator<<` in older versions of GCC when C-style strings are compared
> stacktrace_test: avoid spoiling errno in the test signal handler.
> Optimize `CRC32AcceleratedX86ARMCombinedMultipleStreams::Extend` by interleaving the `CRC32_u64` calls at a lower level.
> stacktrace_test: avoid spoiling errno in the test signal handler.
> stacktrace_test: avoid spoiling errno in the test signal handler.
> std::multimap::find() is not guaranteed to return the first entry with the requested key. Any may be returned if many exist.
> Mark `/`, `%`, and `*` operators as constexpr when intrinsics are available.
> Add the C++20 string_view contructor that uses iterators
> Implement absl::erase_if for absl::InlinedVector
> Adjust software prefetch to fetch 5 cachelines ahead, as benchmarking suggests this should perform better.
> Reduce maximum load factor to 27/32 (from 28/32).
> Remove unused include
> Remove unused include statement
> PR #1921: Fix ABSL_BUILD_DLL mode (absl_make_dll) with mingw
> PR #1922: Enable mmap for WASI if it supports the mman header
> Rollback C++20 string_view constructor that uses iterators due to broken builds
> Add the C++20 string_view contructor that uses iterators
> Bump versions of dependencies in MODULE.bazel
> Automated Code Change
> PR #1918: base: add musl + ppc64le fallback for UnscaledCycleClock::Frequency
> Optimize crc32 Extend by removing obsolete length alignment.
> Fix typo in comment of `ABSL_ATTRIBUTE_UNUSED`.
> Mark AnyInvocable as being nullability compatible.
> Ensure stack usage remains low when unwinding the stack, to prevent stack overflows
> Shrink #if ABSL_HAVE_ATTRIBUTE_WEAK region sizes in stacktrace_test.cc
> <filesystem> is not supported for XTENSA. Disable it in //absl/hash/internal/hash.h.
> Use signal-safe dynamic memory allocation for stack traces when necessary
> PR #1915: Fix SYCL Build Compatibility with Intel LLVM Compiler on Windows for abseil
> Import of CCTZ from GitHub.
> Tag tests that currently fail on ios_sim_arm64 with "no_test_ios_sim_arm64"
> Automated Code Change
> Automated Code Change
> Import of CCTZ from GitHub.
> Move comment specific to pointer-taking MutexLock variant to its definition.
> Add lifetime annotations to MutexLock, SpinLockHolder, etc.
> Add lifetimebound annotations to absl::MakeSpan and absl::MakeConstSpan to detect dangling references
> Remove comment mentioning deferenceability.
> Add referenceful MutexLock with Condition overload.
> Mark SpinLock camel-cased methods as ready for inlining.
> Whitespace change
> In logging tests that write expectations against `ScopedMockLog::Send`, suppress the default behavior that forwards to `ScopedMockLog::Log` so that unexpected logs are printed with full metadata. Many of these tests are poking at those metadata, and a failure message that doesn't include them is unhelpful.
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::ClippedSubstr
> Inline internal usages of Mutex::Lock, etc. in favor of lock.
> Inline internal usages of pointerful SpinLockHolder/MutexLock.
> Remove wrong comment in Cord::Unref
> Update the crc32 dynamic dispatch table with newer platforms.
> PR #1914: absl/base/internal/poison.cc: Minor build fix
> Accept references on SpinLockHolder/MutexLock
> Import of CCTZ from GitHub.
> Fix typos in comments.
> Inline SpinLock Lock->lock, Unlock->unlock internal to Abseil.
> Rename Mutex methods to use the typical C++ lower case names.
> Rename SpinLock methods to use the typical C++ lower case names.
> Add an assert that absl::StrSplit is not called with a null char* argument.
> Fix sign conversion warning
> PR #1911: Fix absl_demangle_test on ppc64
> Disallow using a hash function whose return type is smaller than size_t.
> Optimize CRC-32C extension by zeroes
> Deduplicate stack trace implementations in stacktrace.cc
> Align types of location_table_ and mapping_table_ keys (-Wshorten-64-to-32).
> Move SigSafeArena() out to absl/base/internal/low_level_alloc.h
> Allow CHECK_<OP> variants to be used with unprintable types.
> Import of CCTZ from GitHub.
> Adds required load statements for C++ rules to BUILD and bzl files.
> Disable sanitizer bounds checking in ComputeZeroConstant.
> Roll back NDK weak symbol mode for backtrace() due to internal test breakage
> Add converter for extracting SwissMap profile information into a https://github.com/google/pprof suitable format for inspection.
> Allocate memory for frames and sizes during stack trace fix-up when no memory is provided
> Support NDK weak symbol mode for backtrace() on Android.
> Change skip_empty_or_deleted to not use groups.
> Fix bug of dereferencing invalidated iterator in test case.
> Refactor: split erase_meta_only into large and small versions.
> Fix a TODO to use std::is_nothrow_swappable when it became available.
> Clean up the testing of alternate options that were removed in previous changes
> Only use generic stacktrace when ABSL_HAVE_THREAD_LOCAL.
> Automated Code Change
> Add triviality tests for absl::Span
> Loosen the PointerAlignment test to allow up to 5 stuck bits to avoid flakiness.
> Prevent conversion constructions from absl::Span to itself
> Skip flaky expectations in waiter_test for MSVC.
> Refactor: call AssertIsFull from iterator::assert_is_full to avoid passing the same arguments repeatedly.
> In AssertSameContainer, remove the logic checking for whether the iterators are from SOO tables or not since we don't use it to generate a more informative debug message.
> Remove unused NonIterableBitMask::HighestBitSet function.
> Refactor: move iterator unchecked_* members before data members to comply with Google C++ style guide.
> Mix pointers once instead of twice now that we've improved mixing on 32-bit platforms and improved the kMul constant.
> Remove unused utility functions/constants.
> Revert a change for breaking downstream third party libs
> Remove unneeded include from cord_rep_btree_navigator.h
> Refactor: move find_first_non_full into raw_hash_set.cc.
> Perform stronger mixing on 32-bit platforms and enable the LowEntropyStrings test.
> Include deallocated caller-provided size in delete hooks.
> Roll back one more time: In debug mode, assert that the probe sequence isn't excessively long.
> Allow a `std::move` of `delimiter_` to happen in `ByString::ByString(ByString&&)`. Right now the move ctor is making a copy because the source object is `const`.
> Assume that control bytes don't alias CommonFields.
> Consistently use [[maybe_unused]] in raw_hash_set.h for better compiler warning compatibility.
> Roll forward: In debug mode, assert that the probe sequence isn't excessively long.
> Add a new test for hash collisions for short strings when PrecombineLengthMix has low quality.
> Refactor: define CombineRawImpl for repeated `Mix(state ^ value, kMul)` operations.
> Automated Code Change
> Mark hash_test as large so that the timeout is increased.
> Change the value of kMul to have higher entropy and prevent collisions when keys are aligned integers or pointers.
> Fix LIFETIME annotations for op*/op->/value operators for reference types.
> Update StatusOr to support lvalue reference value types.
> Rollback debug assertion that the probe sequence isn't excessively long.
> AnyInvocable: Fix operator==/!= comments
> In debug mode, assert that the probe sequence isn't excessively long.
> Improve NaN handling in absl::Duration arithmetic.
> Change PrecombineLengthMix to sample data from kStaticRandomData.
> Fix includes and fuse constructors of SpinLock.
> Enable `operator==` for `StatusOr` only if the contained type is equality-comparable
> Enable SIMD memcpy-crc on ARM cores.
> Improve mixing on 32-bit platforms.
> Change DurationFromDouble to return -InfiniteDuration() for all NaNs.
> Change return type of hash internal `Seed` to `size_t` from `uint64_t`
> CMake: Add a fatal error when the compiler defaults to or is set to a C++ language standard prior to C++17.
> Make bool true hash be ~size_t{} instead of 1 so that all bits are different between true/false instead of only one.
> Automated Code Change
> Pass swisstable seed as seed to absl::Hash so we can save an XOR in H1.
> Add support for scoped enumerations in CHECK_XX().
> Revert no-inline on Voidify::operator&&() -- caused unexpected binary size growth
> Mark Voidify::operator&&() as no-inline. This improves stack trace for `LOG(FATAL)` with optimization on.
> Refactor long strings hash computations and move `len <= PiecewiseChunkSize()` out of the line to keep only one function call in the inlined hash code.
> rotr/rotl: Fix undefined behavior when passing INT_MIN as the number of positions to rotate by
> Reorder members of MixingHashState to comply with Google C++ style guide ordering of type declarations, static constants, ctors, non-ctor functions.
> Delete unused function ShouldSampleHashtablezInfoOnResize.
> Remove redundant comments that just name the following symbol without providing additional information.
> Remove unnecessary modification of growth info in small table case.
> Suppress CFI violation on VDSO call.
> Replace WeakMix usage with Mix and change H2 to use the most significant 7 bits - saving 1 cycle in H1.
> Fix -Wundef warning
> Fix conditional constexpr in ToInt64{Nano|Micro|Milli}seconds under GCC7 and GCC8 using an else clause as a workaround
> Enable CompressedTupleTest.NestedEbo test case.
> Lift restriction on using EBCO[1] for nested CompressedTuples. The current implementation of CompressedTuple explicitly disallows EBCO for cases where CompressedTuples are nested. This is because the implentation for a tuple with EBCO-compatible element T inherits from Storage<T, I>, where I is the index of T in the tuple, and
> absl::string_view: assert against (data() == nullptr && size() != 0)
> Fix a false nullability warning in [Q]CHECK_OK by replacing nullptr with an empty char*
> Make `combine_contiguous` to mix length in a weak way by adding `size << 24`, so that we can avoid a separate mixing of size later. The empty range is mixing 0x57 byte.
> Add a test case that -1.0 and 1.0 have different hashes.
> Update CI to a more recent Clang on Linux x86-64
> `absl::string_view`: Add a debug assert to the single-argument constructor that the argument is not `nullptr`.
> Fix CI on macOS Sequoia
> Use Xcode 16.3 for testing
> Use a proper fix instead of a workaround for a parameter annotated absl_nonnull since the latest Clang can see through the workaround
> Assert that SetCtrl isn't called on small tables - there are no control bytes in such cases.
> Use `MaskFullOrSentinel` in `skip_empty_or_deleted`.
> Reduce flakiness in MockDistributions.Examples test case.
> Rename PrepareInsertNonSoo to PrepareInsertLarge now that it's no longer used in all non-SOO cases.
> PR #1895: use c++17 in podspec
> Avoid hashing the key in prefetch() for small tables.
> Remove template alias nullability annotations.
> Add `Group::MaskFullOrSentinel` implementation without usage.
> Move `hashtable_control_bytes` tests into their own file.
> Simplify calls to `EqualElement` by introducing `equal_to` helper function.
> Do `common.increment_size()` directly in SmallNonSooPrepareInsert if inserting to reserved 1 element table.
> Import of CCTZ from GitHub.
> Small cleanup of `infoz` processing to get the logic out of the line or removed.
> Extract the entire PrepareInsert to Small non SOO table out of the line.
> Take `get_hash` implementation out of the SwissTable class to minimize number of instantiations.
> Change kEmptyGroup to kDefaultIterControl now that it's only used for default-constructed iterators.
> [bits] Add tests for return types
> Avoid allocating control bytes in capacity==1 swisstables.
> PR #1888: Adjust Table.GrowExtremelyLargeTable to avoid OOM on i386
> Avoid mixing after `Hash64` calls for long strings by passing `state` instead of `Seed` to low level hash.
> Indent absl container examples consistently
> Revert- Doesn't actually work because SWIG doesn't use the full preprocessor
> Add tags to skip some tests under UBSAN.
> Avoid subtracting `it.control()` and `table.control()` in single element table during erase.
> Remove the `salt` parameter from low level hash and use a global constant. That may potentially remove some loads.
> In SwissTable, don't hash the key when capacity<=1 on insertions.
> Remove the "small" size designation for thread_identity_test, which causes the test to timeout after 60s.
> Add comment explaining math behind expressions.
> Exclude SWIG from ABSL_DEPRECATED and ABSL_DEPRECATE_AND_INLINE
> stacktrace_x86: Handle nested signals on altstack
> Import of CCTZ from GitHub.
> Simplify MixingHashState::Read9To16 to not depend on endianness.
> Delete deprecated `absl::Cord::Get` and its remaining call sites.
> PR #1884: Remove duplicate dependency
> Remove relocatability test that is no longer useful
> Import of CCTZ from GitHub.
> Fix a bug of casting sizeof(slot_type) to uint16_t instead of uint32_t.
> Rewrite `WideToUtf8` for improved readability.
> Avoid requiring default-constructability of iterator type in algorithms that use ContainerIterPairType
> Added test cases for invalid surrogates sequences.
> Use __builtin_is_cpp_trivially_relocatable to implement absl::is_trivially_relocatable in a way that is compatible with PR2786 in the upcoming C++26.
> Remove dependency on `wcsnlen` for string length calculation.
> Stop being strict about validating the "clone" part of mangled names
> Add support for logging wide strings in `absl::log`.
> Deprecate `ABSL_HAVE_STD_STRING_VIEW`.
> Change some nullability annotations in absl::Span to absl_nullability_unknown to workaround a bug that makes nullability checks trigger in foreach loops, while still fixing the -Wnullability-completeness warnings.
> Linux CI update
> Fix new -Wnullability-completeness warnings found after upgrading the Clang version used in the Linux ARM CI to Clang 19.
> Add __restrict for uses of PolicyFunctions.
> Use Bazel vendor mode to cache external dependencies on Windows and macOS
> Move PrepareInsertCommon from header file to cc file.
> Remove the explicit from the constructor to a test allocator in hash_policy_testing.h. This is rejected by Clang when using the libstdc++ that ships with GCC15
> Extract `WideToUtf8` helper to `utf8.h`.
> Updates the documentation for `CHECK` to make it more explicit that it is used to require that a condition is true.
> Add PolicyFunctions::soo_capacity() so that the compiler knows that soo_capacity() is always 0 or 1.
> Expect different representations of pointers from the Windows toolchain.
> Add set_no_seed_for_testing for use in GrowExtremelyLargeTable test.
> Update GoogleTest dependency to 1.17.0 to support GCC15
> Assume that frame pointers inside known stack bounds are readable.
> Remove fallback code in absl/algorithm/container.h
> Fix GCC15 warning that <ciso646> is deprecated in C++17
> Fix misplaced closing brace
> Remove unused include.
> Automated Code Change
> Type erase copy constructor.
> Refactor to use hash_of(key) instead of hash_ref()(key).
> Create Table.Prefetch test to make sure that it works.
> Remove NOINLINE on the constructor with buckets.
> In SwissTable, don't hash the key in find when capacity<=1.
> Use 0x57 instead of Seed() for weakly mixing of size.
> Use absl::InsecureBitGen in place of std::random_device in Abseil tests.
> Remove unused include.
> Use large 64 bits kMul for 32 bits platforms as well.
> Import of CCTZ from GitHub.
> Define `combine_weakly_mixed_integer` in HashSelect::State in order to allow `friend auto AbslHashValue` instead of `friend H AbslHashValue`.
> PR #1878: Fix typos in comments
> Update Abseil dependencies in preparation for release
> Use weaker mixing for absl::Hash for types that mix their sizes.
> Update comments on UnscaledCycleClock::Now.
> Use alignas instead of the manual alignment for the Randen entropy pool.
> Document nullability annotation syntax for array declarations (not many people may know the syntax).
> Import of CCTZ from GitHub.
> Release tests for ABSL_RAW_DCHECK and ABSL_RAW_DLOG.
> Adjust threshold for stuck bits to avoid flaky failures.
> Deprecate template type alias nullability annotations.
> Add more probe benchmarks
> PR #1874: Simplify detection of the powerpc64 ELFv1 ABI
> Make `absl::FunctionRef` copy-assignable. This brings it more in line with `std::function_ref`.
> Remove unused #includes from absl/base/internal/nullability_impl.h
> PR #1870: Retry SymInitialize on STATUS_INFO_LENGTH_MISMATCH
> Prefetch from slots in parallel with reading from control.
> Migrate template alias nullability annotations to macros.
> Improve dependency graph in `TryFindNewIndexWithoutProbing` hot path evaluation.
> Add latency benchmarks for Hash for strings with size 3, 5 and 17.
> Exclude UnwindImpl etc. from thread sanitizer due to false positives.
> Use `GroupFullEmptyOrDeleted` inside of `transfer_unprobed_elements_to_next_capacity_fn`.
> PR #1863: [minor] Avoid variable shadowing for absl btree
> Extend stack-frame walking functionality to allow dynamic fixup
> Fix "unsafe narrowing" in absl for Emscripten
> Roll back change to address breakage
> Extend stack-frame walking functionality to allow dynamic fixup
> Introduce `absl::Cord::Distance()`
> Avoid aliasing issues in growth information initialization.
> Make `GrowSooTableToNextCapacityAndPrepareInsert` in order to initialize control bytes all at once and avoid two function calls on growth right after SOO.
> Simplify `SingleGroupTableH1` since we do not need to mix all bits anymore. Per table seed has a good last bit distribution.
> Use `NextSeed` instead of `NextSeedBaseNumber` and make the result type to be `uint16_t`. That avoids unnecessary bit twiddling and simplify the code.
> Optimize `GrowthToLowerBoundCapacity` in order to avoid division.
> [base] Make :endian internal to absl
> Fully qualify absl names in check macros to avoid invalid name resolution when the user scope has those names defined.
> Fix memory sanitization in `GrowToNextCapacityAndPrepareInsert`.
> Define and use `ABSL_SWISSTABLE_ASSERT` in cc file since a lot of logic moved there.
> Remove `ShouldInsertBackwards` functionality. It was used for additional order randomness in debug mode. It is not necessary anymore with introduction of separate per table `seed`.
> Fast growing to the next capacity based on carbon hash table ideas.
> Automated Code Change
> Refactor CombinePiecewiseBuffer test case to (a) call PiecewiseChunkSize() to get the chunk size and (b) use ASSERT for expectation in a loop.
> PR #1867: Remove global static in stacktrace_win32-inl.inc
> Mark Abseil hardening assert in AssertIsValidForComparison as slow.
> Roll back a problematic change.
> Add absl::FastTypeId<T>()
> Automated Code Change
> Update TestIntrinsicInt128 test to print the indices with the conflicting hashes.
> Code simplification: we don't need XOR and kMul when mixing large string hashes into hash state.
> Refactor absl::CUnescape() to use direct string output instead of pointer/size.
> Rename `policy.transfer` to `policy.transfer_n`.
> Optimize `ResetCtrl` for small tables with `capacity < Group::KWidth * 2` (<32 if SSE enabled and <16 if not).
> Use 16 bits of per-table-seed so that we can save an `and` instruction in H1.
> Fully annotate nullability in headers where it is partially annotated.
> Add note about sparse containers to (flat|node)_hash_(set|map).
> Make low_level_alloc compatible with -Wthread-safety-pointer
> Add missing direct includes to enable the removal of unused includes from absl/base/internal/nullability_impl.h.
> Add tests for macro nullability annotations analogous to existing tests for type alias annotations.
> Adds functionality to return stack frame pointers during stack walking, in addition to code addresses
> Use even faster reduction algorithm in FinalizePclmulStream()
> Add nullability annotations to some very-commonly-used APIs.
> PR #1860: Add `unsigned` to character buffers to ensure they can provide storage (https://eel.is/c++draft/intro.object#3)
> Release benchmarks for absl::Status and absl::StatusOr
> Use more efficient reduction algorithm in FinalizePclmulStream()
> Add a test case to make it clear that `--vmodule=foo/*=1` does match any children and grandchildren and so on under `foo/`.
> Gate use of clang nullability qualifiers through absl nullability macros on `nullability_on_classes`.
> Mark `absl::StatusOr::status()` as ABSL_MUST_USE_RESULT
> Cleanups related to benchmarks * Fix many benchmarks to be cc_binary instead of cc_test * Add a few benchmarks for StrFormat * Add benchmarks for Substitute * Add benchmarks for Damerau-Levenshtein distance used in flags
> Add a log severity alias `DO_NOT_$UBMIT` intended for logging during development
> Avoid relying on true and false tokens in the preprocessor macros used in any_invocable.h
> Avoid relying on true and false tokens in the preprocessor macros used in absl/container
> Refactor to make it clear that H2 computation is not repeated in each iteration of the probe loop.
> Turn on C++23 testing for GCC and Clang on Linux
> Fix overflow of kSeedMask on 32 bits platform in `generate_new_seed`.
> Add a workaround for std::pair not being trivially copyable in C++23 in some standard library versions
> Refactor WeakMix to include the XOR of the state with the input value.
> Migrate ClearPacBits() to a more generic implementation and location
> Annotate more Abseil container methods with [[clang::lifetime_capture_by(...)]] and make them all forward to the non-captured overload
> Make PolicyFunctions always be the second argument (after CommonFields) for type-erased functions.
> Move GrowFullSooTableToNextCapacity implementation with some dependencies to cc file.
> Optimize btree_iterator increment/decrement to avoid aliasing issues by using local variables instead of repeatedly writing to `this`.
> Add constexpr conversions from absl::Duration to int64_t
> PR #1853: Add support for QCC compiler
> Fix documentation for key requirements of flat_hash_set
> Use `extern template` for `GrowFullSooTableToNextCapacity` since we know the most common set of paramenters.
> C++23: Fix log_format_test to match the stream format for volatile pointers
> C++23: Fix compressed_tuple_test.
> Implement `btree::iterator::+=` and `-=`.
> Stop calling `ABSL_ANNOTATE_MEMORY_IS_INITIALIZED` for threadlocal counter.
> Automated Code Change
> Introduce seed stored in the hash table inside of the size.
> Replace ABSL_ATTRIBUTE_UNUSED with [[maybe_unused]]
> Minor consistency cleanups to absl::BitGen mocking.
> Restore the empty CMake targets for bad_any_cast, bad_optional_access, and bad_variant_access to allow clients to migrate.
> bits.h: Add absl::endian and absl::byteswap polyfills
> Use absl::NoDestructor an absl::Mutex instance in the flags library to prevent some exit-time destructor warnings
> Add thread GetEntropyFromRandenPool test
> Update nullability annotation documentation to focus on macro annotations.
> Simplify some random/internal types; expose one function to acquire entropy.
> Remove pre-C++17 workarounds for lack of std::launder
> UBSAN: Use -fno-sanitize-recover
> int128_test: Avoid testing signed integer overflow
> Remove leading commas in `Describe*` methods of `StatusIs` matcher.
> absl::StrFormat: Avoid passing null to memcpy
> str_cat_test: Avoid using invalid enum values
> hash_generator_testing: Avoid using invalid enum values
> absl::Cord: Avoid passing null to memcpy and memset
> graphcycles_test: Avoid applying a non-zero offset to a null pointer
> Make warning about wrapping empty std::function in AnyInvocable stronger.
> absl/random: Convert absl::BitGen / absl::InsecureBitGen to classes from aliases.
> Fix buffer overflow the internal demangling function
> Avoid calling `ShouldRehashForBugDetection` on the first two inserts to the table.
> Remove the polyfill implementations for many type traits and alias them to their std equivalents. It is recommended that clients now simple use the std equivalents.
> ROLLBACK: Limit slot_size to 2^16-1 and maximum table size to 2^43-1.
> Limit `slot_size` to `2^16-1` and maximum table size to `2^43-1`.
> Use C++17 [[nodiscard]] instead of the deprecated ABSL_MUST_USE_RESULT
> Remove the polyfills for absl::apply and absl::make_from_tuple, which were only needed prior to C++17. It is recommended that clients simply use std::apply and std::make_from_tuple.
> PR #1846: Fix build on big endian
> Bazel: Move environment variables to --action_env
> Remove the implementation of `absl::variant`, which was only needed prior to C++17. `absl::variant` is now an alias for `std::variant`. It is recommended that clients simply use `std::variant`.
> MSVC: Fix warnings c4244 and c4267 in the main library code
> Update LowLevelHashLenGt16 to be LowLevelHashLenGt32 now that the input is guaranteed to be >32 in length.
> Xtensa does not support thread_local. Disable it in absl/base/config.h.
> Add support for 8-bit and 16-bit integers to absl::SimpleAtoi
> CI: Update Linux ARM latest container
> Add time hash tests
> `any_invocable`: Update comment that refer to C++17 and C++11
> `check_test_impl.inc`: Use C++17 features unconditionally
> Remove the implementation of `absl::optional`, which was only needed prior to C++17. `absl::optional` is now an alias for `std::optional`. It is recommended that clients simply use `std::optional`.
> Move hashtable control bytes manipulation to a separate file.
> Fix a use-after-free bug in which the string passed to `AtLocation` may be referenced after it is destroyed. While the string does live until the end of the full statement, logging (previously occurred) in the destructor of the `LogMessage` which may be constructed before the temporary string (and thus destroyed after the temporary string's destructor).
> `internal/layout`: Delete pre-C++17 out of line definition of constexpr class member
> Extract slow path for PrepareInsertNonSoo to a separate function `PrepareInsertNonSooSlow`.
> Minor code cleanups
> `internal/log_message`: Use `if constexpr` instead of SFINAE for `operator<<`
> [absl] Use `std::min` in `constexpr` contexts in `absl::string_view`
> Remove the implementation of `absl::any`, which was only needed prior to C++17. `absl::any` is now an alias for `std::any`. It is recommended that clients simply use `std::any`.
> Remove ABSL_INTERNAL_NEED_REDUNDANT_CONSTEXPR_DECL which is longer needed with the C++17 floor
> Make `OptimalMemcpySizeForSooSlotTransfer` ready to work with MaxSooSlotSize upto `3*sizeof(size_t)`.
> `internal/layout`: Replace SFINAE with `if constexpr`
> PR #1830: C++17 improvement: use if constexpr in internal/hash.h
> `absl`: Deprecate `ABSL_HAVE_CLASS_TEMPLATE_ARGUMENT_DEDUCTION`
> Add a verification for access of being destroyed table. Also enabled access after destroy check in ASAN optimized mode.
> Store `CharAlloc` in SwissTable in order to simplify type erasure of functions accepting allocator as `void*`.
> Introduce and use `SetCtrlInLargeTable`, when we know that table is at least one group. Similarly to `SetCtrlInSingleGroupTable`, we can save some operations.
> Make raw_hash_set::slot_type private.
> Delete absl/utility/internal/if_constexpr.h
> `internal/any_invocable`: Use `if constexpr` instead of SFINAE when initializing storage accessor
> Depend on string_view directly
> Optimize and slightly simplify `PrepareInsertNonSoo`.
> PR #1833: Make ABSL_INTERNAL_STEP_n macros consistent in crc code
> `internal/any_invocable`: Use alias `RawT` consistently in `InitializeStorage`
> Move the implementation of absl::ComputeCrc32c to the header file, to facilitate inlining.
> Delete absl/base/internal/inline_variable.h
> Add lifetimebound to absl::StripAsciiWhitespace
> Revert: Random: Use target attribute instead of -march
> Add return for opt mode in AssertNotDebugCapacity to make sure that code is not evaluated in opt mode.
> `internal/any_invocable`: Delete TODO, improve comment and simplify pragma in constructor
> Split resizing routines and type erase similar instructions.
> Random: Use target attribute instead of -march
> `internal/any_invocable`: Use `std::launder` unconditionally
> `internal/any_invocable`: Remove suppresion of false positive -Wmaybe-uninitialized on GCC 12
> Fix feature test for ABSL_HAVE_STD_OPTIONAL
> Support C++20 iterators in raw_hash_map's random-access iterator detection
> Fix mis-located test dependency
> Disable the DestroyedCallsFail test on GCC due to flakiness.
> `internal/any_invocable`: Implement invocation using `if constexpr` instead of SFINAE
> PR #1835: Bump deployment_target version and add visionos to podspec
> PR #1828: Fix spelling of pseudorandom in README.md
> Make raw_hash_map::key_arg private.
> `overload`: Delete obsolete macros for undefining `absl::Overload` when C++ < 17
> `absl/base`: Delete `internal/invoke.h` and `invoke_test.cc`
> Remove `WORKSPACE.bazel`
> `absl`: Replace `base_internal::{invoke,invoke_result_t,is_invocable_r}` with `std` equivalents
> Allow C++20 forward iterators to use fast paths
> Factor out some iterator traits detection code
> Type erase IterateOverFullSlots to decrease code size.
> `any_invocable`: Delete pre-C++17 workarounds for `noexcept` and guaranteed copy elision
> Make raw_hash_set::key_arg private.
> Rename nullability macros to use new lowercase spelling.
> Fix bug where ABSL_REQUIRE_EXPLICIT_INIT did not actually result in a linker error
> Make Randen benchmark program use runtime CPU detection.
> Add CI for the C++20/Clang/libstdc++ combination
> Move Abseil to GoogleTest 1.16.0
> `internal/any_invocable`: Use `if constexpr` instead of SFINAE in `InitializeStorage`
> More type-erasing of InitializeSlots by removing the Alloc and AlignOfSlot template parameters.
> Actually use the hint space instruction to strip PAC bits for return addresses in stack traces as the comment says
> `log/internal`: Replace `..._ATTRIBUTE_UNUSED_IF_STRIP_LOG` with C++17 `[[maybe_unused]]`
> `attributes`: Document `ABSL_ATTRIBUTE_UNUSED` as deprecated
> `internal/any_invocable`: Initialize using `if constexpr` instead of ternary operator, enum, and templates
> Fix flaky tests due to sampling by introducing utility to refresh sampling counters for the current thread.
> Minor reformatting in raw_hash_set: - Add a clear_backing_array member to declutter calls to ClearBackingArray. - Remove some unnecessary `inline` keywords on functions. - Make PoisonSingleGroupEmptySlots static.
> Update CI for linux_gcc-floor to use GCC9, Bazel 7.5, and CMake 3.31.5.
> `internal/any_invocable`: Rewrite `IsStoredLocally` type trait into a simpler constexpr function
> Add ABSL_REQUIRE_EXPLICIT_INIT to Abseil to enable enforcing explicit field initializations
> Require C++17
> Minimize number of `InitializeSlots` with respect to SizeOfSlot.
> Leave the call to `SampleSlow` only in type erased InitializeSlots.
> Update comments for Read4To8 and Read1To3.
> PR #1819: fix compilation with AppleClang
> Move SOO processing inside of InitializeSlots and move it once.
> PR #1816: Random: use getauxval() via <sys/auxv.h>
> Optimize `InitControlBytesAfterSoo` to have less writes and make them with compile time known size.
> Remove stray plus operator in cleanup_internal::Storage
> Include <cerrno> to fix compilation error in chromium build.
> Adjust internal logging namespacing for consistency s/ABSL_LOGGING_INTERNAL_/ABSL_LOG_INTERNAL_/
> Rewrite LOG_EVERY_N (et al) docs to clarify that the first instance is logged. Also, deliberately avoid giving exact numbers or examples since IRL behavior is not so exact.
> ABSL_ASSUME: Use a ternary operator instead of do-while in the implementations that use a branch marked unreachable so that it is usable in more contexts.
> Simplify the comment for raw_hash_set::erase.
> Remove preprocessors for now unsupported compilers.
> `absl::ScopedMockLog`: Explicitly document that it captures logs emitted by all threads
> Fix potential integer overflow in hash container create/resize
> Add lifetimebound to StripPrefix/StripSuffix.
> Random: Rollforward support runtime dispatch on AArch64 macOS
> Crc: Only test non_temporal_store_memcpy_avx on AVX targets
> Provide information about types of all flags.
> Deprecate the precomputed hash find() API in swisstable.
> Import of CCTZ from GitHub.
> Adjust whitespace
> Expand documentation for absl::raw_hash_set::erase to include idiom example of iterator post-increment.
> Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower()
> Crc: Remove the __builtin_cpu_supports path for SupportsArmCRC32PMULL
> Use absl::NoDestructor for some absl::Mutex instances in the flags library to prevent some exit-time destructor warnings
> Update the WORKSPACE dependency of rules_cc to 0.1.0
> Rollback support runtime dispatch on AArch64 macOS for breaking some builds
> Downgrade to rules_cc 0.0.17 because 0.1.0 was yanked
> Use unused set in testing.
> Random: Support runtime dispatch on AArch64 macOS
> crc: Use absl::nullopt when returning absl::optional
> Annotate absl::FixedArray to warn when unused.
> PR #1806: Fix undefined symbol: __android_log_write
> Move ABSL_HAVE_PTHREAD_CPU_NUMBER_NP to the file where it is needed
> Use rbit instruction on ARM rather than rev.
> Debugging: Report the CPU we are running on under Darwin
> Add a microbenchmark for very long int/string tuples.
> Crc: Detect support for pmull and crc instructions on Apple AArch64 With a newer clang, we can use __builtin_cpu_supports which caches all the feature bits.
> Add special handling for hashing integral types so that we can optimize Read1To3 and Read4To8 for the strings case.
> Use unused FixedArray instances.
> Minor reformatting
> Avoid flaky expectation in WaitDurationWoken test case in MSVC.
> Use Bazel rules_cc for many compiler-specific rules instead of our custom ones from before the Bazel rules existed.
> Mix pointers twice in absl::Hash.
> New internal-use-only classes `AsStructuredLiteralImpl` and `AsStructuredValueImpl`
> Annotate some Abseil container methods with [[clang::lifetime_capture_by(...)]]
> Faster copy from inline Cords to inline Strings
> Add new benchmark cases for hashing string lengths 1,2,4,8.
> Move the Arm implementation of UnscaledCycleClock::Now() into the header file, like the x86 implementation, so it can be more easily inlined.
> Minor include cleanup in absl/random/internal
> Import of CCTZ from GitHub.
> Use Bazel Platforms to support AES-NI compile options for Randen
> In HashState::Create, require that T is a subclass of HashStateBase in order to discourage users from defining their own HashState types.
> PR #1801: Remove unncessary <iostream> includes
> New class StructuredProtoField
> Mix pointers twice in TSan and MSVC to avoid flakes in the PointerAlignment test.
> Add a test case that type-erased absl::HashState is consistent with absl::HashOf.
> Mix pointers twice in build modes in which the PointerAlignment test is flaky if we mix once.
> Increase threshold for stuck bits in PointerAlignment test on android.
> Use hashing ideas from Carbon's hashtable in absl hashing: - Use byte swap instead of mixing pointers twice. - Change order of branches to check for len<=8 first. - In len<=16 case, do one multiply to mix the data instead of using the logic from go/absl-hash-rl (reinforcement learning was used to optimize the instruction sequence). - Add special handling for len<=32 cases in 64-bit architectures.
> Test that using a table that was moved-to from a moved-from table fails in sanitizer mode.
> Remove a trailing comma causing an issue for an OSS user
> Add missing includes in hash.h.
> Use the public implementation rule for "@bazel_tools//tools/cpp:clang-cl"
> Import of CCTZ from GitHub.
> Change the definition of is_trivially_relocatable to be a bit less conservative.
> Updates to CI to support newer versions of tools
> Check if ABSL_HAVE_INTRINSIC_INT128 is defined
> Print hash expansions in the hash_testing error messages.
> Avoid flakiness in notification_test on MSVC.
> Roll back: Add more debug capacity validation checks on moves.
> Add more debug capacity validation checks on moves.
> Add macro versions of nullability annotations.
> Improve fork-safety by opening files with `O_CLOEXEC`.
> Move ABSL_HARDENING_ASSERTs in constexpr methods to their own lines.
> Add test cases for absl::Hash: - That hashes are consistent for the same int value across different int types. - That hashes of vectors of strings are unequal even when their concatenations are equal. - That FragmentedCord hashes works as intended for small Cords.
> Skip the IterationOrderChangesOnRehash test case in ASan mode because it's flaky.
> Add missing includes in absl hash.
> Try to use file descriptors in the 2000+ range to avoid mis-behaving client interference.
> Add weak implementation of the __lsan_is_turned_off in Leak Checker
> Fix a bug where EOF resulted in infinite loop.
> static_assert that absl::Time and absl::Duration are trivially destructible.
> Move Duration ToInt64<unit> functions to be inline.
> string_view: Add defaulted copy constructor and assignment
> Use `#ifdef` to avoid errors when `-Wundef` is used.
> Strip PAC bits for return addresses in stack traces
> PR #1794: Update cpu_detect.cc fix hw crc32 and AES capability check, fix undefined
> PR #1790: Respect the allocator's .destroy method in ~InlinedVector
> Cast away nullability in the guts of CHECK_EQ (et al) where Clang doesn't see that the nullable string returned by Check_EQImpl is statically nonnull inside the loop.
> string_view: Correct string_view(const char*, size_type) docs
> Add support for std::string_view in StrCat even when absl::string_view != std::string_view.
> Misc. adjustments to unit tests for logging.
> Use local_config_cc from rules_cc and make it a dev dependency
> Add additional iteration order tests with reservation. Reserved tables have a different way of iteration randomization compared to gradually resized tables (at least for small tables).
> Use all the bits (`popcount`) in `FindFirstNonFullAfterResize` and `PrepareInsertAfterSoo`.
> Mark ConsumePrefix, ConsumeSuffix, StripPrefix, and StripSuffix as constexpr since they are all pure functions.
> PR #1789: Add missing #ifdef pp directive to the TypeName() function in the layout.h
> PR #1788: Fix warning for sign-conversion on riscv
> Make StartsWith and EndsWith constexpr.
> Simplify logic for growing single group table.
> Document that absl::Time and absl::Duration are trivially destructible.
> Change some C-arrays to std::array as this enables bounds checking in some hardened standard library builds
> Replace outdated select() on --cpu with platform API equivalent.
> Take failure_message as const char* instead of string_view in LogMessageFatal and friends.
> Mention `c_any_of` in the function comment of `absl::c_linear_search`.
> Import of CCTZ from GitHub.
> Rewrite some string_view methods to avoid a -Wunreachable-code warning
> IWYU: Update includes and fix minor spelling mistakes.
> Add comment on how to get next element after using erase.
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND and a doc note about absl::LogAsLiteral to clarify its intended use.
> Import of CCTZ from GitHub.
> Reduce memory consumption of structured logging proto encoding by passing tag value
> Remove usage of _LIBCPP_HAS_NO_FILESYSTEM_LIBRARY.
> Make Span's relational operators constexpr since C++20.
> distributions: support a zero max value in Zipf.
> PR #1786: Fix typo in test case.
> absl/random: run clang-format.
> Add some nullability annotations in logging and tidy up some NOLINTs and comments.
> CMake: Change the default for ABSL_PROPAGATE_CXX_STD to ON
> Delete UnvalidatedMockingBitGen
> PR #1783: [riscv][debugging] Fix a few warnings in RISC-V inlines
> Add conversion operator to std::array for StrSplit.
> Add a comment explaining the extra comparison in raw_hash_set::operator==. Also add a small optimization to avoid the extra comparison in sets that use hash_default_eq as the key_equal functor.
> Add benchmark for absl::HexStringToBytes
> Avoid installing options.h with the other headers
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::Span constructors.
> Annotate absl::InlinedVector to warn when unused.
> Make `c_find_first_of`'s `options` parameter a const reference to allow temporaries.
> Disable Elf symbols for Xtensa
> PR #1775: Support symbolize only on WINAPI_PARTITION_DESKTOP
> Require through an internal presubmit that .h|.cc|.inc files contain either the string ABSL_NAMESPACE_BEGIN or SKIP_ABSL_INLINE_NAMESPACE_CHECK
> Xtensa supports mmap, enable it in absl/base/config.h
> PR #1777: Avoid std::ldexp in `operator double(int128)`.
> Marks absl::Span as view and borrowed_range, like std::span.
> Mark inline functions with only a simple comparison in strings/ascii.h as constexpr.
> Add missing Abseil inline namespace and fix includes
> Fix bug where the high bits of `__int128_t`/`__uint128_t` might go unused in the hash function. This fix increases the hash quality of these types.
> Add a test to verify bit casting between signed and unsigned int128 works as expected
> Add suggestions to enable sanitizers for asserts when doing so may be helpful.
> Add nullability attributes to nullability type aliases.
> Refactor swisstable moves.
> Improve ABSL_ASSERT performance by guaranteeing it is optimized away under NDEBUG in C++20
> Mark Abseil hardening assert in AssertSameContainer as slow.
> Add workaround for q++ 8.3.0 (QNX 7.1) compiler by making sure MaskedPointer is trivially copyable and copy constructible.
> Small Mutex::Unlock optimization
> Optimize `CEscape` and `CEscapeAndAppend` by up to 40%.
> Fix the conditional compilation of non_temporal_store_memcpy_avx to verify that AVX can be forced via `gnu::target`.
> Delete TODOs to move functors when moving hashtables and add a test that fails when we do so.
> Fix benchmarks in `escaping_benchmark.cc` by properly calling `benchmark::DoNotOptimize` on both inputs and outputs and by removing the unnecessary and wrong `ABSL_RAW_CHECK` condition (`check != 0`) of `BM_ByteStringFromAscii_Fail` benchmark.
> It seems like commit abc9b916a94ebbf251f0934048295a07ecdbf32a did not work as intended.
> Fix a bug in `absl::SetVLogLevel` where a less generic pattern incorrectly removed a more generic one.
> Remove the side effects between tests in vlog_is_on_test.cc
> Attempt to fix flaky Abseil waiter/sleep tests
> Add an explicit tag for non-SOO CommonFields (removing default ctor) and add a small optimization for early return in AssertNotDebugCapacity.
> Make moved-from swisstables behave the same as empty tables. Note that we may change this in the future.
> Tag tests that currently fail on darwin_arm64 with "no_test_darwin_arm64"
> add gmock to cmake defs for no_destructor_test
> Optimize raw_hash_set moves by allowing some members of CommonFields to be uninitialized when moved-from.
> Add more debug capacity validation checks on iteration/size.
> Add more debug capacity validation checks on copies.
> constinit -> constexpr for DisplayUnits
> LSC: Fix null safety issues diagnosed by Clang’s `-Wnonnull` and `-Wnullability`.
> Remove the extraneous variable creation in Match().
> Import of CCTZ from GitHub.
> Add more debug capacity validation checks on merge/swap.
> Add `absl::` namespace to c_linear_search implementation in order to avoid ADL
> Distinguish the debug message for the case of self-move-assigned swiss tables.
> Update LowLevelHash comment regarding number of hash state variables.
> Add an example for the `--vmodule` flag.
> Remove first prefetch.
> Add moved-from validation for the case of self-move-assignment.
> Allow slow and fast abseil hardening checks to be enabled independently.
> Update `ABSL_RETIRED_FLAG` comment to reflect `default_value` is no longer used.
> Add validation against use of moved-from hash tables.
> Provide file-scoped pragma behind macro ABSL_POINTERS_DEFAULT_NONNULL to indicate the default nullability. This is a no-op for now (not understood by checkers), but does communicate intention to human readers.
> Add stacktrace config for android using the generic implementation
> Fix nullability annotations in ABSL code.
> Replace CHECKs with ASSERTs and EXPECTs -- no reason to crash on failure.
> Remove ABSL_INTERNAL_ATTRIBUTE_OWNER and ABSL_INTERNAL_ATTRIBUTE_VIEW
> Migrate ABSL_INTERNAL_ATTRIBUTE_OWNER and ABSL_INTERNAL_ATTRIBUTE_VIEW to ABSL_ATTRIBUTE_OWNER and ABSL_ATTRIBUTE_VIEW
> Disable ABSL_ATTRIBUTE_OWNER and ABSL_ATTRIBUTE_VIEW prior to Clang-13 due to false positives.
> Make ABSL_ATTRIBUTE_VIEW and ABSL_ATTRIBUTE_OWNER public
> Optimize raw_hash_set::AssertHashEqConsistent a bit to avoid having as much runtime overhead.
> PR #1728: Workaround broken compilation against NDK r25
> Add validation against use of destroyed hash tables.
> Do not truncate `ABSL_RAW_LOG` output at null bytes
> Use several unused cord instances in tests and benchmarks.
> Add comments about ThreadIdentity struct allocation behavior.
> Refactoring followup for reentrancy validation in swisstable.
> Add debug mode checks that element constructors/destructors don't make reentrant calls to raw_hash_set member functions.
> Add tagging for cc_tests that are incompatible with Fuchsia
> Add GetTID() implementation for Fuchsia
> PR #1738: Fix shell option group handling in pkgconfig files
> Disable weak attribute when absl compiled as windows DLL
> Remove `CharIterator::operator->`.
> Mark non-modifying container algorithms as constexpr for C++20.
> PR #1739: container/internal: Explicitly include <cstdint>
> Don't match -Wnon-virtual-dtor in the "flags are needed to suppress warnings in headers". It should fall through to the "don't impose our warnings on others" case. Do this by matching on "-Wno-*" instead of "-Wno*".
> PR #1732: Fix build on NVIDIA Jetson board. Fix#1665
> Update GoogleTest dependency to 1.15.2
> Enable AsciiStrToLower and AsciiStrToUpper overloads for rvalue references.
> PR #1735: Avoid `int` to `bool` conversion warning
> Add `absl::swap` functions for `*_hash_*` to avoid calling `std::swap`
> Change internal visibility
> Remove resolved issue.
> Increase test timeouts to support running on Fuchsia emulators
> Add tracing annotations to absl::Notification
> Suppress compiler optimizations which may break container poisoning.
> Disable ABSL_INTERNAL_HAVE_DEBUGGING_STACK_CONSUMPTION for Fuchsia
> Add tracing annotations to absl::BlockingCounter
> Add absl_vlog_is_on and vlog_is_on to ABSL_INTERNAL_DLL_TARGETS
> Update swisstable swap API comments to no longer guarantee that we don't move/swap individual elements.
> PR #1726: cmake: Fix RUNPATH when using BUILD_WITH_INSTALL_RPATH=True
> Avoid unnecessary copying when upper-casing or lower-casing ASCII string_view
> Add weak internal tracing API
> Fix LINT.IfChange syntax
> PR #1720: Fix spelling mistake: occurrance -> occurrence
> Add missing include for Windows ASAN configuration in poison.cc
> Delete absl/strings/internal/has_absl_stringify.h now that the GoogleTest version we depend on uses the public file
> Update versions of dependencies in preparation for release
> PR #1699: Add option to build with MSVC static runtime
> Remove unneeded 'be' from comment.
> PR #1715: Generate options.h using CMake only once
> Small type fix in absl/log/internal/log_impl.h
> PR #1709: Handle RPATH CMake configuration
> PR #1710: fixup! PR #1707: Fixup absl_random compile breakage in Apple ARM64 targets
> PR #1695: Fix time library build for Apple platforms
> Remove cyclic cmake dependency that breaks in cmake 3.30.0
> Roll forward poisoned pointer API and fix portability issues.
> Use GetStatus in IsOkAndHoldsMatcher
> PR #1707: Fixup absl_random compile breakage in Apple ARM64 targets
> PR #1706: Require CMake version 3.16
> Add an MSVC implementation of ABSL_ATTRIBUTE_LIFETIME_BOUND
> Mark c_min_element, c_max_element, and c_minmax_element as constexpr in C++17.
> Optimize the absl::GetFlag cost for most non built-in flag types (including string).
> Encode some additional metadata when writing protobuf-encoded logs.
> Replace signed integer overflow, since that's undefined behavior, with unsigned integer overflow.
> Make mutable CompressedTuple::get() constexpr.
> vdso_support: support DT_GNU_HASH
> Make c_begin, c_end, and c_distance conditionally constexpr.
> Add operator<=> comparison to absl::Time and absl::Duration.
> Deprecate `ABSL_ATTRIBUTE_NORETURN` in favor of the `[[noreturn]]` standardized in C++11
> Rollback new poisoned pointer API
> Static cast instead of reinterpret cast raw hash set slots as casting from void* to T* is well defined
> Fix absl::NoDestructor documentation about its use as a global
> Declare Rust demangling feature-complete.
> Split demangle_internal into a tree of smaller libraries.
> Decode Rust Punycode when it's not too long.
> Add assertions to detect reentrance in `IterateOverFullSlots` and `absl::erase_if`.
> Decoder for Rust-style Punycode encodings of bounded length.
> Add `c_contains()` and `c_contains_subrange()` to `absl/algorithm/container.h`.
> Three-way comparison spaceship <=> operators for Cord.
> internal-only change
> Remove erroneous preprocessor branch on SGX_SIM.
> Add an internal API to get a poisoned pointer.
> optimization.h: Add missing <utility> header for C++
> Add a compile test for headers that require C compatibility
> Fix comment typo
> Expand documentation for SetGlobalVLogLevel and SetVLogLevel.
> Roll back 6f972e239f668fa29cab43d7968692cd285997a9
> PR #1692: Add missing `<utility>` include
> Remove NOLINT for `#include <new>` for __cpp_lib_launder
> Remove not used after all kAllowRemoveReentrance parameter from IterateOverFullSlots.
> Create `absl::container_internal::c_for_each_fast` for SwissTable.
> Disable flaky test cases in kernel_timeout_internal_test.
> Document that swisstable and b-tree containers are not exception-safe.
> Add `ABSL_NULLABILITY_COMPATIBLE` attribute.
> LSC: Move expensive variables on their last use to avoid copies.
> Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to more types in Abseil
> Drop std:: qualification from integer types like uint64_t.
> Increase slop time on MSVC in PerThreadSemTest.Timeouts again due to continued flakiness.
> Turn on validation for out of bounds MockUniform in MockingBitGen
> Use ABSL_UNREACHABLE() instead of equivalent
> If so configured, report which part of a C++ mangled name didn't parse.
> Sequence of 1-to-4 values with prefix sum to support Punycode decoding.
> Add the missing inline namespace to the nullability files
> Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to types in Abseil
> Disallow reentrance removal in `absl::erase_if`.
> Fix implicit conversion of temporary bitgen to BitGenRef
> Use `IterateOverFullSlots` in `absl::erase_if` for hash table.
> UTF-8 encoding library to support Rust Punycode decoding.
> Disable negative NaN float ostream format checking on RISC-V
> PR #1689: Minor: Add missing quotes in CMake string view library definition
> Demangle template parameter object names, TA <template-arg>.
> Demangle sr St <simple-id> <simple-id>, a dubious encoding found in the wild.
> Try not to lose easy type combinators in S::operator const int*() and the like.
> Demangle fixed-width floating-point types, DF....
> Demangle _BitInt types DB..., DU....
> Demangle complex floating-point literals.
> Demangle <extended-qualifier> in types, e.g., U5AS128 for address_space(128).
> Demangle operator co_await (aw).
> Demangle fully general vendor extended types (any <template-args>).
> Demangle transaction-safety notations GTt and Dx.
> Demangle C++11 user-defined literal operator functions.
> Demangle C++20 constrained friend names, F (<source-name> | <operator-name>).
> Demangle dependent GNU vector extension types, Dv <expression> _ <type>.
> Demangle elaborated type names, (Ts | Tu | Te) <name>.
> Add validation that hash/eq functors are consistent, meaning that `eq(k1, k2) -> hash(k1) == hash(k2)`.
> Demangle delete-expressions with the global-scope operator, gs (dl | da) ....
> Demangle new-expressions with braced-init-lists.
> Demangle array new-expressions, [gs] na ....
> Demangle object new-expressions, [gs] nw ....
> Demangle preincrement and predecrement, pp_... and mm_....
> Demangle throw and rethrow (tw... and tr).
> Remove redundant check of is_soo() while prefetching heap blocks.
> Demangle ti... and te... expressions (typeid).
> Demangle nx... syntax for noexcept(e) as an expression in a dependent signature.
> Demangle alignof expressions, at... and az....
> Demangle C++17 structured bindings, DC...E.
> Demangle modern _ZGR..._ symbols.
> Remove redundant check of is_soo() while prefetching heap blocks.
> Demangle sizeof...(pack captured from an alias template), sP ... E.
> Demangle types nested under vendor extended types.
> Demangle il ... E syntax (braced list other than direct-list-initialization).
> Avoid signed overflow for Ed <number> _ manglings with large <number>s.
> Remove redundant check of is_soo() while prefetching heap blocks.
> Remove obsolete TODO
> Clarify function comment for `erase` by stating that this idiom only works for "some" standard containers.
> Move SOVERSION to global CMakeLists, apply SOVERSION to DLL
> Set ABSL_HAVE_THREAD_LOCAL to 1 on all platforms
> Demangle constrained auto types (Dk <type-constraint>).
> Parse <discriminator> more accurately.
> Demangle lambdas in class member functions' default arguments.
> Demangle unofficial <unresolved-qualifier-level> encodings like S0_IT_E.
> Do not make std::filesystem::path hash available for macOS <10.15
> Include flags in DLL build (non-Windows only)
> Enable building monolithic shared library on macOS and Linux.
> Demangle Clang's last-resort notation _SUBSTPACK_.
> Demangle C++ requires-expressions with parameters (rQ ... E).
> Demangle Clang's encoding of __attribute__((enable_if(condition, "message"))).
> Demangle static_cast and friends.
> Demangle decltype(expr)::nested_type (NDT...E).
> Optimize GrowIntoSingleGroupShuffleControlBytes.
> Demangle C++17 fold-expressions.
> Demangle thread_local helper functions.
> Demangle lambdas with explicit template arguments (UlTy and similar forms).
> Demangle &-qualified function types.
> Demangle valueless literals LDnE (nullptr) and LA<number>_<type>E ("foo").
> Correctly demangle the <unresolved-name> at the end of dt and pt (x.y, x->y).
> Add missing targets to ABSL_INTERNAL_DLL_TARGETS
> Build abseil_test_dll with ABSL_BUILD_TESTING
> Demangle C++ requires-expressions without parameters (rq ... E).
> overload: make the constructor constexpr
> Update Abseil CI Docker image to use Clang 19, GCC 14, and CMake 3.29.3
> Workaround symbol resolution bug in Clang 19
> Workaround bogus GCC14 -Wmaybe-uninitialized warning
> Silence a bogus GCC14 -Warray-bounds warning
> Forbid absl::Uniform<absl::int128>(gen)
> Use IN_LIST to replace list(FIND) + > -1
> Recognize C++ vendor extended expressions (e.g., u9__is_same...E).
> `overload_test`: Remove a few unnecessary trailing return types
> Demangle the C++ this pointer (fpT).
> Stop eating an extra E in ParseTemplateArg for some L<type><value>E literals.
> Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to Abseil.
> Demangle C++ direct-list-initialization (T{1, 2, 3}, tl ... E).
> Demangle the C++ spaceship operator (ss, operator<=>).
> Demangle C++ sZ encodings (sizeof...(pack)).
> Demangle C++ so ... E encodings (typically array-to-pointer decay).
> Recognize dyn-trait-type in Rust demangling.
> Rework casting in raw_hash_set's IsFull().
> Remove test references to absl::SharedBitGen, which was never part of the open source release. This was only used in tests that never ran as part in the open source release.
> Recognize fn-type and lifetimes in Rust demangling.
> Support int128/uint128 in validated MockingBitGen
> Recognize inherent-impl and trait-impl in Rust demangling.
> Recognize const and array-type in Rust mangled names.
> Remove Asylo from absl.
> Recognize generic arguments containing only types in Rust mangled names.
> Fix missing #include <random> for std::uniform_int_distribution
> Move `prepare_insert` out of the line as type erased `PrepareInsertNonSoo`.
> Revert: Add -Wdead-code-aggressive to ABSL_LLVM_FLAGS
> Add (unused) validation to absl::MockingBitGen
> Support `AbslStringify` with `DCHECK_EQ`.
> PR #1672: Optimize StrJoin with tuple without user defined formatter
> Give ReturnAddresses and N<uppercase> namespaces separate stacks for clarity.
> Demangle Rust backrefs.
> Use Nt for struct and trait names in Rust demangler test inputs.
> Allow __cxa_demangle on MIPS
> Add a `string_view` overload to `absl::StrJoin`
> Demangle Rust's Y<type><path> production for passably simple <type>s.
> `convert_test`: Delete obsolete condition around ASSERT_EQ in TestWithMultipleFormatsHelper
> `any_invocable`: Clean up #includes
> Resynchronize absl/functional/CMakeLists.txt with BUILD.bazel
> `any_invocable`: Add public documentation for undefined behavior when invoking an empty AnyInvocable
> `any_invocable`: Delete obsolete reference to proposed standard type
> PR #1662: Replace shift with addition in crc multiply
> Doc fix.
> `convert_test`: Extract loop over tested floats from helper function
> Recognize some simple Rust mangled names in Demangle.
> Use __builtin_ctzg and __builtin_clzg in the implementations of CountTrailingZeroesNonzero16 and CountLeadingZeroes16 when they are available.
> Remove the forked absl::Status matchers implementation in statusor_test
> Add comment hack to fix copybara reversibility
> Add GoogleTest matchers for absl::Status
> [random] LogUniform: Document as a discrete distribution
> Enable Cord tests with Crc.
> Fix order of qualifiers in `absl::AnyInvocable` documentation.
> Guard against null pointer dereference in DumpNode.
> Apply ABSL_MUST_USE_RESULT to try lock functions.
> Add public aliases for default hash/eq types in hash-based containers
> Import of CCTZ from GitHub.
> Remove the hand-rolled CordLeaker and replace with absl::NoDestructor to test the after-exit behavior
> `convert_test`: Delete obsolete `skip_verify` parameter in test helper
> overload: allow using the underlying type with CTAD directly.
> PR #1653: Remove unnecessary casts when calling CRC32_u64
> PR #1652: Avoid C++23 deprecation warnings from float_denorm_style
> Minor cleanup for `absl::Cord`
> PR #1651: Implement ABSL_INTERNAL_DISABLE_DEPRECATED_DECLARATION_WARNING for MSVC compiler
> Add `operator<=>` support to `absl::int128` and `absl::uint128`
> [absl] Re-use the existing `std::type_identity` backfill instead of redefining it again
> Add `absl::AppendCordToString`
> `str_format/convert_test`: Delete workaround for [glibc bug](https://sourceware.org/bugzilla/show_bug.cgi?id=22142)
> `absl/log/internal`: Document conditional ABSL_ATTRIBUTE_UNUSED, add C++17 TODO
> `log/internal/check_op`: Add ABSL_ATTRIBUTE_UNUSED to CHECK macros when STRIP_LOG is enabled
> log_benchmark: Add VLOG_IS_ON benchmark
> Restore string_view detection check
> Remove an unnecessary ABSL_ATTRIBUTE_UNUSED from a logging macro
< Abseil LTS Branch, Jan 2024, Patch 2 (#1650)
> In example code, add missing template parameter.
> Optimize crc32 V128_From2x64 on Arm
> Annotate that Mutex should warn when unused.
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND to Cord::Flatten/TryFlat
> Deprecate `absl::exchange`, `absl::forward` and `absl::move`, which were only useful before C++14.
> Temporarily revert dangling std::string_view detection until dependent is fixed
> Use _decimal_ literals for the CivilDay example.
> Fix bug in BM_EraseIf.
> Add internal traits to absl::string_view for lifetimebound detection
> Add internal traits to absl::StatusOr for lifetimebound detection
> Add internal traits to absl::Span for lifetimebound detection
> Add missing dependency for log test build target
> Add internal traits for lifetimebound detection
> Use local decoding buffer in HexStringToBytes
> Only check if the frame pointer is inside a signal stack with known bounds
> Roll forward: enable small object optimization in swisstable.
> Optimize LowLevelHash by breaking dependency between final loads and previous len/ptr updates.
> Fix the wrong link.
> Optimize InsertMiss for tables without kDeleted slots.
> Use GrowthInfo without applying any optimizations based on it.
> Disable small object optimization while debugging some failing tests.
> Adjust conditonal compilation in non_temporal_memcpy.h
> Reformat log/internal/BUILD
> Remove deprecated errno constants from the absl::Status mapping
> Introduce GrowthInfo with tests, but without usage.
> Enable small object optimization in swisstable.
> Refactor the GCC unintialized memory warning suppression in raw_hash_set.h.
> Respect `NDEBUG_SANITIZER`
> Revert integer-to-string conversion optimizations pending more thorough analysis
> Fix a bug in `Cord::{Append,Prepend}(CordBuffer)`: call `MaybeRemoveEmptyCrcNode()`. Otherwise appending a `CordBuffer` an empty Cord with a CRC node crashes (`RemoveCrcNode()` which increases the refcount of a nullptr child).
> Add `BM_EraseIf` benchmark.
> Record sizeof(key_type), sizeof(value_type) in hashtable profiles.
> Fix ClangTidy warnings in btree.h.
> LSC: Move expensive variables on their last use to avoid copies.
> PR #1644: unscaledcycleclock: remove RISC-V support
> Reland: Make DLOG(FATAL) not understood as [[noreturn]]
> Separate out absl::StatusOr constraints into statusor_internal.h
> Use Layout::WithStaticSizes in btree.
> `layout`: Delete outdated comments about ElementType alias not being used because of MSVC
> Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower()
> `layout_benchmark`: Replace leftover comment with intended call to MyAlign
> Remove absl::aligned_storage_t
> Delete ABSL_ANNOTATE_MEMORY_IS_INITIALIZED under Thread Sanitizer
> Remove vestigial variables in the DumpNode() helper in absl::Cord
> Do hashtablez sampling on the first insertion into an empty SOO hashtable.
> Add explicit #include directives for <tuple>, "absl/base/config.h", and "absl/strings/string_view.h".
> Add a note about the cost of `VLOG` in non-debug builds.
> Fix flaky test failures on MSVC.
> Add template keyword to example comment for Layout::WithStaticSizes.
> PR #1643: add xcprivacy to all subspecs
> Record sampling stride in cord profiling to facilitate unsampling.
> Fix a typo in a comment.
> [log] Correct SetVLOGLevel to SetVLogLevel in comments
> Add a feature to container_internal::Layout that lets you specify some array sizes at compile-time as template parameters. This can make offset and size calculations faster.
> `layout`: Mark parameter of Slices with ABSL_ATTRIBUTE_UNUSED, remove old workaround
> `layout`: Use auto return type for functions that explicitly instantiate std::tuple in return statements
> Remove redundant semicolons introduced by macros
> [log] Make :vlog_is_on/:absl_vlog_is_on public in BUILD.bazel
> Add additional checks for size_t overflows
> Replace //visibility:private with :__pkg__ for certain targets
> PR #1603: Disable -Wnon-virtual-dtor warning for CommandLineFlag implementations
> Add several missing includes in crc/internal
> Roll back extern template instatiations in swisstable due to binary size increases in shared libraries.
> Add nodiscard to SpinLockHolder.
> Test that rehash(0) reduces capacity to minimum.
> Add extern templates for common swisstable types.
> Disable ubsan for benign unaligned access in crc_memcpy
> Make swisstable SOO support GDB pretty printing and still compile in OSS.
> Fix OSX support with CocoaPods and Xcode 15
> Fix GCC7 C++17 build
> Use UnixEpoch and ZeroDuration
> Make flaky failures much less likely in BasicMocking.MocksNotTriggeredForIncorrectTypes test.
> Delete a stray comment
> Move GCC uninitialized memory warning suppression into MaybeInitializedPtr.
> Replace usages of absl::move, absl::forward, and absl::exchange with their std:: equivalents
> Fix the move to itself
> Work around an implicit conversion signedness compiler warning
> Avoid MSan: use-of-uninitialized-value error in find_non_soo.
> Fix flaky MSVC test failures by using longer slop time.
> Add ABSL_ATTRIBUTE_UNUSED to variables used in an ABSL_ASSUME.
> Implement small object optimization in swisstable - disabled for now.
> Document and test ability to use absl::Overload with generic lambdas.
> Extract `InsertPosition` function to be able to reuse it.
> Increase GraphCycles::PointerMap size
> PR #1632: inlined_vector: Use trivial relocation for `erase`
> Create `BM_GroupPortable_Match`.
> [absl] Mark `absl::NoDestructor` methods with `absl::Nonnull` as appropriate
> Automated Code Change
> Rework casting in raw_hash_set's `IsFull()`.
> Adds ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::BitGenRef
> Workaround for NVIDIA C++ compiler being unable to parse variadic expansions in range of range-based for loop
> Rollback: Make DLOG(FATAL) not understood as [[noreturn]]
> Make DLOG(FATAL) not understood as [[noreturn]]
> Optimize `absl::Duration` division and modulo: Avoid repeated redundant comparisons in `IDivFastPath`.
> Optimize `absl::Duration` division and modulo: Allow the compiler to inline `time_internal::IDivDuration`, by splitting the slow path to a separate function.
> Fix typo in example code snippet.
> Automated Code Change
> Add braces for conditional statements in raw_hash_map functions.
> Optimize `prepare_insert`, when resize happens. It removes single unnecessary probing before resize that is beneficial for small tables the most.
> Add noexcept to move assignment operator and swap function
> Import of CCTZ from GitHub.
> Minor documentation updates.
> Change find_or_prepare_insert to return std::pair<iterator, bool> to match return type of insert.
> PR #1618: inlined_vector: Use trivial relocation for `SwapInlinedElements`
> Improve raw_hash_set tests.
> Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower()
> Use const_cast to avoid duplicating the implementation of raw_hash_set::find(key).
> Import of CCTZ from GitHub.
> Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower()
> Annotate that SpinLock should warn when unused.
> PR #1625: absl::is_trivially_relocatable now respects assignment operators
> Introduce `Group::MaskNonFull` without usage.
> `demangle`: Parse template template and C++20 lambda template param substitutions
> PR #1617: fix MSVC 32-bit build with -arch:AVX
> Minor documentation fix for `absl::StrSplit()`
> Prevent overflow in `absl::CEscape()`
> `demangle`: Parse optional single template argument for built-in types
> PR #1412: Filter out `-Xarch_` flags from pkg-config files
> `demangle`: Add complexity guard to `ParseQRequiresExpr`
< Prepare 20240116.1 patch for Apple Privacy Manifest (#1623)
> Remove deprecated symbol absl::kuint128max
> Add ABSL_ATTRIBUTE_WARN_UNUSED.
> `demangle`: Parse `requires` clauses on template params, before function return type
> On Apple, implement absl::is_trivially_relocatable with the fallback.
> `demangle`: Parse `requires` clauses on functions
> Make `begin()` to return `end()` on empty tables.
> `demangle`: Parse C++20-compatible template param declarations, except those with `requires` expressions
> Add the ABSL_DEPRECATE_AND_INLINE() macro
> Span: Fixed comment referencing std::span as_writable_bytes() as as_mutable_bytes().
> Switch rank structs to be consistent with written guidance in go/ranked-overloads
> Avoid hash computation and `Group::Match` in small tables copy and use `IterateOverFullSlots` for iterating for all tables.
> Optimize `absl::Hash` by making `LowLevelHash` faster.
> Add -Wdead-code-aggressive to ABSL_LLVM_FLAGS
< Backport Apple Privacy Manifest (#1613)
> Stop using `std::basic_string<uint8_t>` which relies on a non-standard generic `char_traits<>` implementation, recently removed from `libc++`.
> Add absl_container_hash-based HashEq specialization
> `demangle`: Implement parsing for simplest constrained template arguments
> Roll forward 9d8588bfc4566531c4053b5001e2952308255f44 (which was rolled back in 146169f9ad357635b9cd988f976b38bcf83476e3) with fix.
> Add a version of absl::HexStringToBytes() that returns a bool to validate that the input was actually valid hexadecimal data.
> Enable StringLikeTest in hash_function_defaults_test
> Fix a typo.
> Minor changes to the BUILD file for absl/synchronization
> Avoid static initializers in case of ABSL_FLAGS_STRIP_NAMES=1
> Rollback 9d8588bfc4566531c4053b5001e2952308255f44 for breaking the build
> No public description
> Decrease the precision of absl::Now in x86-64 debug builds
> Optimize raw_hash_set destructor.
> Add ABSL_ATTRIBUTE_UNINITIALIZED macros for use with clang and GCC's `uninitialized`
> Optimize `Cord::Swap()` for missed compiler optimization in clang.
> Type erased hash_slot_fn that depends only on key types (and hash function).
> Replace `testonly = 1` with `testonly = True` in abseil BUILD files.
> Avoid extra `& msbs` on every iteration over the mask for GroupPortableImpl.
> Missing parenthesis.
> Early return from destroy_slots for trivially destructible types in flat_hash_{*}.
> Avoid export of testonly target absl::test_allocator in CMake builds
> Use absl::NoDestructor for cordz global queue.
> Add empty WORKSPACE.bzlmod
> Introduce `RawHashSetLayout` helper class.
> Fix a corner case in SpyHashState for exact boundaries.
> Add nullability annotations
> Use absl::NoDestructor for global HashtablezSampler.
> Always check if the new frame pointer is readable.
> PR #1604: Add privacy manifest
< Disable ABSL_ATTRIBUTE_TRIVIAL_ABI in open-source builds (#1606)
> Remove code pieces for no longer supported GCC versions.
> Disable ABSL_ATTRIBUTE_TRIVIAL_ABI in open-source builds
> Prevent brace initialization of AlphaNum
> Remove code pieces for no longer supported MSVC versions.
> Added benchmarks for smaller size copy constructors.
> Migrate empty CrcCordState to absl::NoDestructor.
> Add protected copy ctor+assign to absl::LogSink, and clarify thread-safety requirements to apply to the interface methods.
< Apply LTS transformations for 20240116 LTS branch (#1599)
Closesscylladb/scylladb#28756
The rack option was fully implemented in the code but omitted from
both docs/operating-scylla/admin.rst and conf/scylla.yaml comments.
Closesscylladb/scylladb#29239
In a multi-declarator declaration, the & ref-qualifier is part of each
individual declarator, not the shared type specifier. So:
const auto& a = x(), b = y();
declares 'a' as a reference but 'b' as a value, silently copying y().
The same applies to:
const T& a = v[i], b = v[j];
Both operator== lines had this pattern, causing an unnecessary copy of
the column vector and an unnecessary copy of each entry on every call.
Fix by repeating & on the second declarator in both lines.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29213
clear_gently() (introduced in 322aa2f8b5) clears all token_metadata_impl
members using co_await to avoid reactor stalls on large data structures.
_topology_change_info (introduced in 10bf8c7901) was added later and not
included in clear_gently().
update_topology_change_info() already uses utils::clear_gently() when
replacing the value, so it looks reasonable to apply the same pattern
in clear_gently().
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29210
Some on_internal_error() calls have the selector argument to a format
string with no placeholder for it in the format string.
"While at it", disambiguate selector type in the message text.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Closesscylladb/scylladb#29208
There's only one test left that uses it, and it can be patched to use standard ks/cf creation helpers from pylib. This patch does so and drops the lengthy create_dataset() helper
Tests improvements, no need to backport
Closesscylladb/scylladb#29176
* github.com:scylladb/scylladb:
test/backup: drop create_dataset helper
test/backup: use new_test_keyspace in test_restore_primary_replica
The enable_tablets(false) was added when LWT wasn't supported for tablets, now it's, so no need in this attribute are more.
The test covers behavior which should work in similar way for both vnodes and tablets -> it doesn't seem it would benefit much from running it in both enable_tablets(true) and enable_tablets(false) modes.
Closesscylladb/scylladb#29167
In order to apply fsult-injected delay, there's the inject(duration)
overload. Results in shorter code
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Closesscylladb/scylladb#29168
The endpoint in question has some places worth fixing, in particular
- the keyspace parameter is not validated
- the validated table name is resolved into table_id, but the id is unused
- two ugly static helpers to stream obtained token ranges into json
Improving the API code flow, not backporting
Closesscylladb/scylladb#29154
* github.com:scylladb/scylladb:
api: Inline describe_ring JSON handling
storage_service: Make describe_ring_for_table() take table_id
Add skip_reason_plugin.py — a framework-agnostic pytest plugin that
provides typed skip markers (skip_bug, skip_not_implemented, skip_slow,
skip_env) so that the reason a test is skipped is machine-readable in
JUnit XML and Allure reports. Bare untyped pytest.mark.skip now
triggers a warning (to become an error after full migration). Runtime
skips via skip() are also enriched by parsing the [type] prefix from
the skip message.
The plugin is a class (SkipReasonPlugin) that receives the concrete
SkipType enum and an optional report_callback from conftest.py, keeping
it decoupled from allure and project-specific types.
Extract SkipType enum and convenience runtime skip wrappers (skip_bug,
skip_env, etc.) into test/pylib/skip_types.py so callers only need a
single import instead of importing both SkipType and skip() separately.
conftest.py imports SkipType from the new module and registers the
plugin instance unconditionally (for all test runners).
New files:
- test/pylib/skip_reason_plugin.py: core plugin — typed marker
processing, bare-skip warnings, JUnit/Allure report enrichment
(including runtime skip() parsing via _parse_skip_type helper)
- test/pylib/skip_types.py: SkipType enum and convenience wrappers
(skip_bug, skip_not_implemented, skip_slow, skip_env)
- test/pylib_test/test_skip_reason_plugin.py: 17 pytester-based
test functions (51 cases across 3 build modes) covering markers,
warnings, reports, callbacks, and skip_mode interaction
Infrastructure changes:
- test/conftest.py: import SkipType from skip_types, register
SkipReasonPlugin with allure report callback
- test/pylib/runner.py: set SKIP_TYPE_KEY/SKIP_REASON_KEY stash keys
for skip_mode so the report hook can enrich JUnit/Allure with
skip_type=mode without longrepr parsing
- test/pytest.ini: register typed marker definitions (required for
--strict-markers even when plugin is not loaded)
Migrated test files (representative samples):
- test/cluster/test_tablet_repair_scheduler.py:
skip -> skip_bug (#26844), skip -> skip_not_implemented
- test/cqlpy/.../timestamp_test.py: skip -> skip_slow
- test/cluster/dtest/schema_management_test.py: skip -> skip_not_implemented
- test/cluster/test_change_replication_factor_1_to_0.py: skip -> skip_bug (#20282)
- test/alternator/conftest.py: skip -> skip_env
- test/alternator/test_https.py: use skip_env() wrapper
Fixes SCYLLADB-79
Closesscylladb/scylladb#29235
This patch series implements `object_storage_base::clone`, which was previously a stub that aborted at runtime. Clone creates a copy of an sstable under a new generation and is used during compaction.
The implementation uses server-side object copies (S3 CopyObject / GCS Objects: rewrite) and mirrors the filesystem clone semantics: TemporaryTOC is written first to mark the operation as in-progress, component objects are copied, and TemporaryTOC is removed to commit (unless the caller requested the destination be left unsealed).
The first two patches fix pre-existing bugs in the underlying storage clients that were exposed by the new clone code path:
- GCS `copy_object` used the wrong HTTP method (PUT instead of POST) and sent an invalid empty request body.
- S3 `copy_object` silently ignored the abort_source parameter.
1. **gcp_client: fix copy_object request method and body** — Fix two bugs in the GCS rewrite API call.
2. **s3_client: pass through abort_source in copy_object** — Stop ignoring the abort_source parameter.
3. **object_storage: add copy_object to object_storage_client** — New interface method with S3 and GCS implementations.
4. **storage: add make_object_name overload with generation** — Helper for building destination object names with a different generation.
5. **storage: make delete_object const** — Needed by the const clone method.
6. **storage: implement object_storage_base::clone** — The actual clone implementation plus a copy_object wrapper.
7. **test/boost: enable sstable clone tests for S3 and GCS** — Re-enable the previously skipped tests.
A test similar to `sstable_clone_leaving_unsealed_dest_sstable` was added to properly test the sealed/unsealed states for object storage. Works for both S3 and GCS.
Fixes: https://scylladb.atlassian.net/browse/SCYLLADB-1045
Prerequisite: https://github.com/scylladb/scylladb/pull/28790
No need to backport since this code targets future feature
Closesscylladb/scylladb#29166
* github.com:scylladb/scylladb:
compaction_test: enable sstable clone tests for S3 and GCS
storage: implement object_storage_base::clone
storage: make delete_object const in object_storage_base
storage: add make_object_name overload with generation
sstables: add get_format() accessor to sstable
object_storage: add copy_object to object_storage_client
s3_client: pass through abort_source in copy_object
gcp_client: fix copy_object request method and body
The vector-search feature introduced the somewhat confusing feature of
enabling CDC without explicitly enabling CDC: When a vector index is
enabled on a table, CDC is "enabled" for it even if the user didn't
ask to enable CDC.
For this, write-path code began to use a new cdc_enabled() function
instead of checking schema.cdc_options.enabled() directly. This
cdc_enabled() function checks if either this enabled() is true, or
has_vector_index() is true.
Unfortunately, LWT writes continued to use cdc_options.enabled() instead
of the new cdc_enabled(). This means that if a vector index is used and
a vector is written using an LWT write, the new value is not indexed.
This patch fixes this bug. It also adds a regression test that fails
before this patch and passes afterwards - the new test verifies that
when a table has a vector index (but no explicit CDC enabled), the CDC
log is updated both after regular writes and after successful LWT writes.
This patch was also tested in the context of the upcoming vector-search-
for-Alternator pull request, which has a test reproducing this bug
(Alternator uses LWT frequently, so this is very important there).
It will also be tested by the vector-store test suite ("validator").
Fixes SCYLLADB-1342
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Closesscylladb/scylladb#29300
In the unregistered-ID branch, ldap_msgfree() was called on a result
already owned by an RAII ldap_msg_ptr, causing a double-free on scope
exit. Remove the redundant manual free.
Fixes: SCYLLADB-1344
Backport: 2026.1, 2025.4, 2025.1 - it's a memory corruption, with a one-line fix, so better backport it everywhere.
Closesscylladb/scylladb#29302
* github.com:scylladb/scylladb:
test: ldap: add regression test for double-free on unregistered message ID
ldap: fix double-free of LDAPMessage in poll_results()
Now that object_storage_base::clone is implemented,
remove the early-return skips and re-enable the
sstable_clone_leaving_unsealed_dest_sstable tests for
both S3 and GCS storage backends.
Implement the clone method for object_storage_base, which creates
a copy of an sstable with a new generation using server-side object
copies. Also add a const copy_object convenience wrapper, similar
to the existing put_object and delete_object wrappers.
A dedicated test for the new object storage clone path will be
added in the following commit. The preexisting local-filesystem
clone is already covered by the sstable_clone_leaving_unsealed_dest_sstable
test.
Add a make_object_name overload that accepts a target
generation parameter for constructing object names with
a generation different from the source sstable's own.
Refactor the original make_object_name to delegate to
the new overload, eliminating code duplication.
This is needed by clone to build destination object
names for the new generation.
Add a public get_format() accessor for the _format member, following
the same pattern as the existing get_version(). This allows storage
implementations to access the sstable format without reaching into
private members, and is needed by the upcoming object_storage_base::clone
to construct entry_descriptor for the sstables registry.
Add a copy_object method to the object_storage_client
interface for server-side object copies, with
implementations for both S3 and GCS wrappers.
The S3 wrapper delegates to s3::client::copy_object.
The GCS wrapper delegates to gcp::storage::client's
cross-bucket copy_object overload.
This is a prerequisite for implementing sstable clone
on object storage.
The abort_source parameter in s3::client::copy_object
was ignored — the function accepted it but always passed
nullptr to the underlying copy_s3_object. Forward it
properly so callers can cancel in-progress copies.
The GCP copy_object (rewrite API) had two bugs:
1. The request body was an empty string, but the GCP
rewrite endpoint always parses it as JSON metadata.
An empty string is not valid JSON, resulting in
400 "Metadata in the request couldn't decode".
Fix: send "{}" (empty JSON object) as the body.
2. The HTTP method was PUT, but the GCP Objects: rewrite
API requires POST per the documentation.
Fix: use POST.
Test coverage in a follow-up patch
When `BatchWriteItem` operates on multiple items sharing the same partition key in `always_use_lwt` write isolation mode, all CDC log entries are emitted under a single timestamp. The previous `get_records` parsing algorithm in `alternator/streams.cc` assumed that all CDC log entries sharing the same timestamp correspond to a single DynamoDB item change. As a result, it would incorrectly squash multiple distinct item changes into a single Streams record — producing wrong event data (e.g., one INSERT instead of four, with mismatched key/attribute values).
Note: the bug is specific to `always_use_lwt` mode because only in LWT mode does the entire batch share a single timestamp. In non-LWT modes, each item in the batch receives a separate timestamp, so the entries naturally stay separate.
**Commit 1: alternator: add BatchWriteItem Streams test**
- Adds new tests `test_streams_batchwrite_no_clustering_deletes_non_existing_items` and `test_streams_batchwrite_no_clustering_deletes_existing_items` that cover the corner cases of batch-deleting a existing and non-existing item in a table without a clustering key. CDC tables without clustering keys are handled differently, and this path was previously untested for delete operations.
- Adds a new test `test_streams_batchwrite_into_the_same_partition_will_report_wrong_stream_data`, that is a simple way to trigger a bug.
- Adds a new test `test_streams_batchwrite_into_the_same_partition_deletes_existing_items`, that validates various combinations of puts and deletes in a single BatchWrite against the same partition.
- Adds a new `test_table_ss_new_and_old_images_write_isolation_always` fixture and extends `create_table_ss` to accept `additional_tags`, enabling tests with a specific write isolation mode.
**Commit 2: alternator: fix BatchWriteItem squashed Streams entries**
The core fix rewrites the CDC log entry parsing in `get_records` to distinguish items by their clustering key:
- Introduces `managed_bytes_ptr_hash` and `managed_bytes_ptr_equal` helper structs for pointer-based hash map lookups on `managed_bytes`.
- Replaces the single `record`/`dynamodb` pair with a `std::unordered_map<const managed_bytes*, Record, ...>` (`records_map`) keyed by the base table's clustering key value from each CDC log row. For tables without a clustering key, all entries map to a single sentinel key.
- Adds a validation that Alternator tables have at most one clustering key column (as required by the DynamoDB data model).
- On end-of-record (`eor`), flushes all accumulated per-clustering-key records into the output, each with a unique `eventID` (the `event_id` format now includes an index suffix).
- Adjusts the limit check: since a single CDC timestamp bucket can now produce multiple output records, the limit may be slightly exceeded to avoid breaking mid-batch.
Fixes#28439
Fixes: SCYLLADB-540
Closesscylladb/scylladb#28452
* github.com:scylladb/scylladb:
alternator/test: explain why 'always' write isolation mode is used in tests
alternator/test: add scylla_only to always write isolation fixture
alternator: fix BatchWriteItem squashed Streams entries
alternator: add BatchWriteItem test (failing)
Queries against local vector indexes were failing with the error:
```ANN ordering by vector requires the column to be indexed using 'vector_index'```
This was a regression introduced by 15788c3734, which incorrectly
assumed the first column in the targets list is always the vector column.
For local vector indexes, the first column is the partition key, causing
the failure.
Previously, serialization logic for the target index option was shared
between vector and secondary indexes. This is no longer viable due to
the introduction of local vector indexes and vector indexes with filtering
columns, which have different target format.
This commit introduces a dedicated JSON-based serialization format for
vector index targets, identifying the target column (tc), filtering
columns (fc), and partition key columns (pk). This ensures unambiguous
serialization and deserialization for all vector index types.
This change is backward compatible for regular vector indexes. However,
it breaks compatibility for local vector indexes and vector indexes with
filtering columns created in version 2026.1.0. To mitigate this, usage
of these specific index types will be blocked in the 2026.1.0 release
by failing ANN queries against them in vector-store service.
Fixes: SCYLLADB-895
Backport to 2026.1 is required as this issue occurs also on this branch.
Closesscylladb/scylladb#28862
* github.com:scylladb/scylladb:
index: fix DESC INDEX for vector index
vector_search: test: refactor boilerplate setup
vector_search: fix SELECT on local vector index
index: test: vector index target option serialization test
index: test: secondary index target option serialization test
To create `process_staging` view building tasks, we firstly need to
collect informations about them on shard0, create necessary mutations,
commit them to group0 and move staging sstables objects to their
original shards.
But there is a possible race after committing the group0 command
and before moving the staging sstables to their shards.
Between those two events, the coordinator may schedule freshly created
tasks and dispatch them to the worker but the worker won't have the
sstables objects because they weren't moved yet.
This patch fixes the race by holding `_staging_sstables_mutex` locks
from necessary shards when executing `create_staging_sstable_tasks()`.
With this, even if the task will be scheduled and dispatched quickly,
the worker will wait with executing it until the sstables objects are
moved and the locks are released.
Fixes SCYLLADB-816
Add a test that verifies filesystem_storage::clone preserves the sstable
state: an sstable in staging is cloned to a new generation, the clone is
re-loaded from the staging directory, and its state is asserted to still
be staging.
The change proves that https://scylladb.atlassian.net/browse/SCYLLADB-1205
is invalid, and can be closed.
* No functional change and no backport needed
Closesscylladb/scylladb#29209
* github.com:scylladb/scylladb:
test: add test_sstable_clone_preserves_staging_state
test: derive sstable state from directory in test_env::make_sstable
sstables: log debug message in filesystem_storage::clone
`data_value::to_parsable_string()` crashes with a null pointer dereference when called on a `null` data_value. Return `"null"` instead.
Added tests after the fix. Manually checked that tests fail without the fix.
Fixes SCYLLADB-1350
This is a fix that prevents format crash. No known occurrence in production, but backport is desirable.
Closesscylladb/scylladb#29262
* github.com:scylladb/scylladb:
test: boost: test null data value to_parsable_string
cql3: fix null handling in data_value formatting
Extend test_basic to run with both RF=1 and RF=3 to verify that
object storage works correctly with multiple replicas. The test now
starts one server per replica (each on its own rack), flushes all
nodes, validates tablet replica counts for RF>1, and restarts all
servers before verifying data is still readable.
Fixes: SCYLLADB-546
Closesscylladb/scylladb#28583
As noted in issue #5027 and issue #29138, Alternator's support for
ReturnConsumedCapacity is lacking in a two areas:
1. While ReturnConsumedCapacity is supported for most relevant
operations, it's not supported in two operations: Query and Scan.
2. While ReturnConsumedCapacity=TOTAL is supported, INDEXES is not
supported at all.
This patch adds extensive tests for all these cases. All these tests
pass on DynamoDB but fail on Alternator, so are marked with "xfail".
The tests for ReturnConsumedCapacity=INDEXES are deliberately split
into two: First, we test the case where the table has no indexes, so
INDEXES is almost the same as TOTAL and should be very easy to
implement. A second test checks the cases where there are indexes,
and different operations increment the capacity of the base table
and/or indexes differently - it will require significantly more work
to make the second test pass.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Closesscylladb/scylladb#29188
When a table has a vector index, cdc::cdc_enabled() returns true because
vector index writes are implemented via the CDC augmentation path. However,
register_cdc_operation_result_tracker() was checking only
cdc_options().enabled(), which is false for tables that have a vector index
but not traditional CDC.
As a result, the operation_result_tracker was never attached to write
response handlers for vector-indexed tables. This tracker was added in
commit 1b92cbe, and its job is to update metrics of CDC operations,
and since vector search really does use CDC under the hood, these
metrics could be useful when diagnosing problems.
Fix by using cdc::cdc_enabled() instead of cdc_options().enabled(), which
covers both traditional CDC and vector-indexed tables.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Closesscylladb/scylladb#29343
Clang 22 verifies [[nodiscard]] for co_await,
causing compilation failures where return values of expected<> were
silently discarded.
These call sites were discarding the return value of client::request()
and vector_store_client::ann(), both of which return expected<> types
marked [[nodiscard]]. Rather than suppressing the warning with (void)
casts, properly check the return values using the established test
patterns: BOOST_CHECK(result) where the call is expected to succeed,
and BOOST_CHECK(!result) where the call is expected to fail.
Closesscylladb/scylladb#29297
This issue adds the upgrade guide for all patch releases within 2026.x major release.
In addition, it fixes the link to Upgrade Policy in the 2025.x-to-2026.1 upgrade guide.
Fixes SCYLLADB-1247
Closesscylladb/scylladb#29307
This PR introduces the vnodes-to-tablets migration procedure, which enables converting an existing vnode-based keyspace to tablets.
The migration is implemented as a manual, operator-driven process executed in several stages. The core idea is to first create tablet maps with the same token boundaries and replica hosts as the vnodes, and then incrementally convert the storage of each node to the tablets layout. At a high level, the procedure is the following:
1. Create tablet maps for all tables in the keyspace.
2. Sequentially upgrade all nodes from vnodes to tablets:
1. Mark a node for upgrade in the topology state.
2. Restart the node. During startup, while the node is offline, it reshards the SSTables on vnode boundaries and switches to a tablet ERM.
3. Wait for the node to return online before proceeding to the next node.
4. Finalize the migration:
1. Update the keyspace schema to mark it as tablet-based.
2. Clear the group0 state related to the migration.
From the client's perspective, the migration is online; the cluster can still serve requests on that keyspace, although performance may be temporarily degraded.
During the migration, some nodes use vnode ERMs while others use tablet ERMs. Cluster-level algorithms such as load balancing will treat the keyspace's tables as vnode-based. Once migration is finalized, the keyspace is permanently switched to tablets and cannot be reverted back to vnodes. However, a rollback procedure is available before finalization.
The patch series consists of:
* Load balancer adjustments to ignore tablets belonging to a migrating keyspace.
* A new vnode-based resharding mode, where SSTables are segregated on vnode boundaries rather than with the static sharder.
* A new per-node `intended_storage_mode` column in `system.topology`. Represents migration intent (whether migration should occur on restart) and direction.
* Four new REST endpoints for driving the migration (start, node upgrade/downgrade, finalize, status), along with `nodetool` wrappers. The finalization is implemented as a global topology request.
* Wiring of the migration process into the startup logic: the `distributed_loader` determines a migrating table's ERM flavor from the `intended_storage_mode` and the ERM flavor determines the `table_populator`'s resharding mode. Token metadata changes have been adjusted to preserve the ERM flavor.
* Cluster tests for the migration process.
Fixes SCYLLADB-722.
Fixes SCYLLADB-723.
Fixes SCYLLADB-725.
Fixes SCYLLADB-779.
Fixes SCYLLADB-948.
New feature, no backport is needed.
Closesscylladb/scylladb#29065
* github.com:scylladb/scylladb:
docs: Add ops guide for vnodes-to-tablets migration
test: cluster: Add test for migration of multiple keyspaces
test: cluster: Add test for error conditions
test: cluster: Add vnodes->tablets migration test (rollback)
test: cluster: Add vnodes->tablets migration test (1 table, 3 nodes)
test: cluster: Add vnodes->tablets migration test (1 table, 1 node)
scylla-nodetool: Add migrate-to-tablets subcommand
api: Add REST endpoint for vnode-to-tablet migration status
api: Add REST endpoint for migration finalization
topology_coordinator: Add `finalize_migration` request
database: Construct migrating tables with tablet ERMs
api: Add REST endpoint for upgrading nodes to tablets
api: Add REST endpoint for starting vnodes-to-tablets migration
topology_state_machine: Add intended_storage_mode to system.topology
distributed_loader: Wire vnode-based resharding into table populator
replica: Pick any compaction group for resharding
compaction: resharding_compaction: add vnodes_resharding option
storage_service: Preserve ERM flavor of migrating tables
tablet_allocator: Exclude migrating tables from load balancing
feature_service: Add vnodes_to_tablets_migrations feature
The estimate() function in the size_estimates virtual reader only
considered sstables local to the shard that happened to own the
keyspace's partition key token. Since sstables are distributed across
shards, this caused partition count estimates to be approximately
1/smp_count of the actual value.
This bug has been present since the virtual reader was introduced in
225648780d.
Use db.container().map_reduce0() to aggregate sstable estimates
across all shards. Each shard contributes its local count and
estimated_histogram, which are then merged to produce the correct
total.
Also fix the `test_partitions_estimate_full_overlap` test which becomes
flaky (xpassing ~1% of runs) because autocompaction could merge the
two overlapping sstables before the size estimate was read. Wrap the
test body in nodetool.no_autocompaction_context to prevent this race.
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1179
Refs https://github.com/scylladb/scylladb/issues/9083Closesscylladb/scylladb#29286
As discussed with @ScyllaPiotr in
https://github.com/scylladb/scylladb/pull/29232, the doc about to be
removed is just:
> Looking at history, I think this audit.md is a design doc: scylladb/scylla-enterprise@87a5c19, for which the feature has been implemented differently, eventually, and was created around the time when design docs, apparently, where stored within the repository itself. So for me it's some trash (sorry for strong language) that can be safely removed.
Closesscylladb/scylladb#29316
Python warns that the sequence "\(" is an invalid escape and
might be rejected in the future. Protect against that by using
a raw string.
Closesscylladb/scylladb#29334
Convert auth_test.cc to coroutines for improved readability. Each test is converted in its own commit. Some
are trivial.
Indentation is left broken in some commits to reduce the diff, then fixed up in the last commit.
Code cleanup, so no backport.
Closesscylladb/scylladb#29336
* github.com:scylladb/scylladb:
auth_test: fix whitespace
auth_test: coroutinize test_try_describe_schema_with_internals_and_passwords_as_anonymous_user
auth_test: coroutinize test_try_login_after_creating_roles_with_hashed_password
auth_test: coroutinize test_create_roles_with_hashed_password_and_log_in
auth_test: coroutinize test_try_create_role_with_hashed_password_as_anonymous_user
auth_test: coroutinize test_try_to_create_role_with_password_and_hashed_password
auth_test: coroutinize test_try_to_create_role_with_hashed_password_and_password
auth_test: coroutinize test_alter_with_workload_type
auth_test: coroutinize test_alter_with_timeouts
auth_test: coroutinize role_permissions_table_is_protected
auth_test: coroutinize role_members_table_is_protected
auth_test: coroutinize roles_table_is_protected
auth_test: coroutinize test_password_authenticator_operations
auth_test: coroutinize test_password_authenticator_attributes
auth_test: coroutinize test_default_authenticator
When an SSTable was encrypted with a KMS host that is not present in
scylla.yaml, the error thrown was:
std::invalid_argument (No such host: <host-name>)
This message is very obscure in general, and especially confusing when
encountered while using the scylla-sstable tool: it gives no indication
that the SSTable is encrypted, that a KMS host lookup is involved, or
what the user needs to do to fix the problem.
Replace it with a message that names the missing host and points
directly to the relevant scylla.yaml section:
Encryption host "<host-name>" is not defined in scylla.yaml.
Make sure it is listed under the "kmip_hosts" section.
The wording is intentionally kept neutral (not framed as an SSTable tool
problem) because the same code path is exercised by production ScyllaDB
when a node's configuration no longer contains a host referenced by an
existing data file (e.g. after a config rollback or when restoring data
from a different cluster). The production use-case takes precedence, but
the message is equally actionable from the tool.
Closesscylladb/scylladb#29228
start - end will result in negative length, rejected by the python
runtime. Use the correct end - start to calculate length.
Closesscylladb/scylladb#29249
data_dictionary::database was converted to replica::database in two
places, just to call find_keyspace(), then call
get_replication_strategy() on the returned keyspace. This is not
necessary, data_dictionary::database already has find_keyspace() and the
returned data_dictionary::keyspace also has get_replication_strategy().
This patch removes a small layering violation but more importantly, it
is necessary for the sstable tool to be able to load schemas from disk,
when said schema has tombstone_gc props.
Closesscylladb/scylladb#29279
This series updates the storage abstraction and extends the compaction tests to support object‑storage backends (S3 and GCS), while tightening several parts of the test environment.
The changes include:
- New exists/object_exists helpers across storage backends and clock fixes in the S3 client to make signature generation stable under test conditions.
- A new get_storage_for_tests accessor and adjustments to the test environment to avoid premature teardown of the sstable registry.
- Refactoring of compaction tests to remove direct sstable access, ensure proper schema setup, and avoid use of moved‑from objects.
- Extraction of test_env‑based logic into reusable functions and addition of S3/GCS variants of the compaction tests.
Not all tests were converted to be backend‑agnostic yet, and a few require further investigation before they can run cleanly against S3/GCS backends. These will be addressed in follow‑up work.
Fixes: https://scylladb.atlassian.net/browse/SCYLLADB-704 however, followup is needed
No backport needed since this change targeting future feature
Closesscylladb/scylladb#28790
* github.com:scylladb/scylladb:
compaction_test: fix formatting after previous patches
compaction_test: add S3/GCS variations to tests
compaction_test: extract test_env-based tests into functions
compaction_test: replace file_exists with storage::exists
compaction_test: initialize tables with schema via make_table_for_tests
compaction_test: use sstable APIs to manipulate component files
compaction_test: fix use-after-move issue
sstable_utils: add `get_storage` and `open_file` helpers
test_env: delay unplugging sstable registry
storage: add `exists` method to storage abstraction
s3_client: use lowres_system_clock for aws_sigv4
s3_client: add `object_exists` helper
gcs_client: add `object_exists` helper
View building worker was breaking semaphores without holding their locks.
This lead to races like SCYLLADB-844 and SCYLLADB-543,
where a new batch was started after `view_building_worker::state` was cleared in the `drain()` process.
This patch fix the race by:
- taking a lock of the mutex before breaking it
- distinguishing between `state::clear()`(can happen multiple times) and `state::drain()`(can be called only once during shutdown)
- asserting that the state is not doing any new work after it was drained
Fixes SCYLLADB-844
Fixes SCYLLADB-543
This PR should be backported to all versions containing view building coordinator (2025.4 and newer).
Closesscylladb/scylladb#29303
* github.com:scylladb/scylladb:
view_building_worker: extract starting a new batch to state's method
view_building_worker: distinguish between state's `clear()` and `drain()`
view_building_worker: lock mutexes before breaking them in `drain()`
view_building_worker: execute drain() once
The fix for SCYLLADB-1373 (b4f652b7c1) changed get_session() to use
the default timeout=30 for the retry loop in patient_*_cql_connection
(previously timeout=0.1). This correctly allowed retrying transient
NoHostAvailable errors during node startup, but introduced a new
flakiness in test_login and other auth tests.
The failure chain:
1. test_login connects with bad credentials (e.g. user="doesntexist")
2. get_session() calls patient_exclusive_cql_connection(), which calls
retry_till_success() with bypassed_exception=NoHostAvailable
3. The first attempt correctly fails: the server rejects the credentials
with AuthenticationFailed, wrapped in NoHostAvailable
4. retry_till_success() catches NoHostAvailable indiscriminately and
retries, not distinguishing between transient errors (node not ready)
and permanent errors (bad credentials)
5. A subsequent retry attempt times out (connect_timeout=5), producing
OperationTimedOut wrapped in NoHostAvailable
6. After 30 seconds, the last NoHostAvailable is raised -- now wrapping
OperationTimedOut instead of the original AuthenticationFailed
7. The assertion `isinstance(..., AuthenticationFailed)` fails
With the old timeout=0.1, the deadline was already exceeded after the
first attempt, so the original AuthenticationFailed propagated.
Fix: Add a `should_retry` predicate parameter to retry_till_success()
and use it in patient_cql_connection() and
patient_exclusive_cql_connection() to immediately re-raise
NoHostAvailable when it wraps AuthenticationFailed. Retrying
authentication failures is never useful since the credentials won't
change between attempts.
Fixes: SCYLLADB-1382
Closesscylladb/scylladb#29348
Following the previous commit, a new batch cannot be started if the
state was already drained.
This commit also adds a check that only one batch is running at a time.
While both of this methods do the same (abort current batch, clear
data), we can clear the state multiple times during view_building_worker
lifetime (for instance when processing base table is changed) but
`view_building_worker::state::drain()` should be called only once and
after this no other work on the state should be done.
Not doing this may lead to races like SCYLLADB-844.
If some consumer is holding a lock of a mutex and `drain()`
is just braking the mutex without locking it beforehand,
then the consumer may process its code which should be aborted.
An example of the race is SCYLLADB-844, where `work_on_tasks()` is
holding `_state._mutex` while it is broken by `drain()`.
This causes a new batch is started after the `_state` is cleared.
get_session() was passing timeout=0.1 to patient_exclusive_cql_connection
and patient_cql_connection, leaving only 0.1 seconds for the retry loop
in retry_till_success(). Since each connection attempt can take up to 5
seconds (connect_timeout=5), the retry loop effectively got only one
attempt with no chance to retry on transient NoHostAvailable errors.
Use the default timeout=30 seconds, consistent with all other callers.
Fixes: SCYLLADB-1373
Closesscylladb/scylladb#29332
Flatten continuation chains (.then()) into linear thread-style code
with .get() calls for improved readability. Remove the now-unused
require_throws helper template.
When a Scylla node starts, the scylla-image-setup.service invokes the
`scylla_swap_setup` script to provision swap. This script allocates a
swap file and creates a swap systemd unit to delegate control to
systemd. By default, systemd injects a Before=swap.target dependency
into every swap unit, allowing other services to use swap.target to wait
for swap to be enabled.
On Azure, this doesn't work so well because we store the swap file on
the ephemeral disk [1] which has network dependencies (`_netdev` mount
option, configured by cloud-init [2]). This makes the swap.target
indirectly depend on the network, leading to dependency cycles such as:
swap.target -> mnt-swapfile.swap -> mnt.mount -> network-online.target
-> network.target -> systemd-resolved.service -> tmp.mount -> swap.target
This patch breaks the cycle by removing the swap unit from swap.target
using DefaultDependencies=no. The swap unit will still be activated via
WantedBy=multi-user.target, just not during early boot.
Although this problem is specific to Azure, this patch applies the fix
to all clouds to keep the code simple.
Fixes#26519.
Fixes SCYLLADB-1257
[1] https://github.com/scylladb/scylla-machine-image/pull/426
[2] https://github.com/canonical/cloud-init/pull/1213#issuecomment-1026065501
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
Closesscylladb/scylladb#28504
Replace direct filesystem checks (file_exists) with the
storage-agnostic exists() method in unsealed_sstable_compaction,
sstable_clone_leaving_unsealed_dest_sstable, and
failure_when_adding_new_sstable tests, making them compatible
with object-storage backends (S3, GCS).
Start using `table_for_tests::make_default_schema` so test tables are
created with a real schema. This is required for object-storage
backends, which cannot operate correctly without proper schema
initialization.
Switch tests to use sstable member functions for file manipulation
instead of opening files directly on the filesystem. This affects the
helpers that emulate sstable corruption: we now overwrite the entire
component file rather than just the first few kilobytes, which is
sufficient for producing a corrupted sstable.
Add a non-const `get_storage` accessor to expose underlying storage,
and an `open_file` helper to access sstable component files directly.
These are needed so compaction tests can read and write sstable
components.
Unplugging the mock sstable_registry happened too early in the test
environment. During sstable destruction, components may still need
access to the registry, so the unplugging is moved to a later stage.
Add an `exists` method to the storage abstraction to allow S3, GCS,
and local storage implementations to check whether an sstable
component is present.
Switch aws_sigv4 to lowres_system_clock since it is not affected by
time offsets often introduced in tests, which can skew db_clock. S3
requests cannot represent time shifts greater than 15 minutes from
server time, so a stable clock is required.
When test_exception_safety_of_update_from_memtable was converted from
manual fail_after()/catch to with_allocation_failures() in 74db08165d,
the populate_range() call ended up inside the failure injection scope
without a scoped_critical_alloc_section guard. The other two tests
converted in the same commit (test_exception_safety_of_transitioning...
and test_exception_safety_of_partition_scan) were correctly guarded.
Without the guard, the allocation failure injector can sometimes
target an allocation point inside the cleanup path of populate_range().
In a rare corner case, this triggers a bad_alloc in a noexcept context
(reader_concurrency_semaphore::stop()), causing std::terminate.
Fixes SCYLLADB-1346
Closesscylladb/scylladb#29321
Verify that upgrading from 2025.1 to master does not silently drop DDL
auditing for table-scoped audit configurations (SCYLLADB-1155).
Test time in dev: 4s
Refs: SCYLLADB-1155
Fixes: SCYLLADB-1305
The old execute_and_validate_audit_entry required every caller to
pass audit_settings so it could decide internally whether to expect
an entry. A test added later in this series needs to simply assert
an entry was produced, without specifying audit_settings at all.
Split into two methods:
- execute_and_validate_new_audit_entry: unconditionally expects an
audit entry.
- execute_and_validate_if_category_enabled: checks audit_settings
to decide whether to expect an entry or assert absence.
Local wrapper functions and **kwargs forwarding are removed in favor
of explicit arguments at each call site, and expected-error cases are
handled inline with assert_invalid + assert_entries_were_added.
AuditTester uses self.manager throughout but never declares it.
The attribute is only assigned in the CQLAuditTester subclass
__init__, so the type checker reports 'Attribute "manager" is
unknown' on every self.manager reference in the base class.
Add an __init__ to AuditTester that accepts and stores the manager
instance, and update CQLAuditTester to forward it via super().__init__
instead of assigning self.manager directly.
Fixes: SCYLLADB-1106
* Small fix in scylla_cluster - remove debug print
* Fix GSServer::unpublish so it does not except if publish was not called beforehand
* Improve dockerized_server so mock server logs echo to the test log to help diagnose CI failures (because we don't collect log files from mocks etc, and in any case correlation will be much easier).
No backport needed.
Closesscylladb/scylladb#29112
* github.com:scylladb/scylladb:
dockerized_service: Convert log reader to pipes and push to test log
test::cluster::conftest::GSServer: Fix unpublish for when publish was not called
scylla_cluster: Use thread safe future signalling
scylla_cluster: Remove left-over debug printout
The test was failing because the call to:
await log.wait_for('Stopping.*ongoing compactions')
was missing the 'from_mark=log_mark' argument. The log mark was updated
(line: log_mark = await log.mark()) immediately after detecting
'splitting_mutation_writer_switch_wait: waiting', and just before
launching the shutdown task. However, the wait_for call on the following
line was scanning from the beginning of the log, not from that mark.
As a result, the search immediately matched old 'Stopping N tasks for N
ongoing compactions for table system.X due to table removal' messages
emitted during initial server bootstrap (for system.large_partitions,
system.large_rows, system.large_cells), rather than waiting for the
shutdown to actually stop the user-table split compaction.
This caused the test to prematurely send the message to the
'splitting_mutation_writer_switch_wait' injection. The split compaction
was unblocked before the shutdown had aborted it, so it completed
successfully. Since the split succeeded, 'Failed to complete splitting
of table' was never logged.
Meanwhile, 'storage_service_drain_wait' was blocking do_drain() waiting
for a message. With the split already done, the test was stuck waiting
for the expected failure log that would never come (600s timeout). At
the same time, after 60s the 'storage_service_drain_wait' injection
timed out internally, triggering on_internal_error() which -- with
--abort-on-internal-error=1 -- crashed the server (exit code -6).
Fix: pass from_mark=log_mark to the wait_for('Stopping.*ongoing
compactions') call so it only matches messages that appear after the
shutdown has started, ensuring the test correctly synchronizes with the
shutdown aborting the user-table split compaction before releasing the
injection.
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1319.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29311
Replace the random port selection with an OS-assigned port. We open
a temporary TCP socket, bind it to (ip, 0) with SO_REUSEADDR, read back
the port number the OS selected, then close the socket before launching
rest_api_mock.py.
Add reuse_address=True and reuse_port=True to TCPSite in rest_api_mock.py
so the server itself can also reclaim a TIME_WAIT port if needed.
Fixes: SCYLLADB-1275
Closesscylladb/scylladb#29314
On slow/overloaded CI machines the lowres_clock timer may not have
fired after the fixed 2x sleep, causing the assertion on
get_abort_exception() to fail. Replace the fixed sleep with
sleep(1x) + eventually_true() which retries with exponential backoff,
matching the pattern already used in test_time_based_cache_eviction.
Fixes: SCYLLADB-1311
Closesscylladb/scylladb#29299
Track the total memory consumed by responses waiting to be
written to the socket, exposed as a per-scheduling-group gauge
(cql_pending_response_memory). This complements the response
memory accounting added in the previous commits by giving
visibility into how much memory each service level is holding
in unsent response buffers.
make sure the driver is stopped even though cluster
teardown throws and avoid potential stale driver
connections entering infinite reconnect loops which
exhaust cpu resources.
Fixes: SCYLLADB-1189
Signed-off-by: Robert Bindar <robert.bindar@scylladb.com>
Closesscylladb/scylladb#29230
Debug mode shuffles task position in the queue. So the following is possible:
1) shard 1 calls manual_clock::advance(). This expires timers on shard 1 and queues a background smp call to shard 0 which will expire timers there
2) the smp::submit_to(0, ...) from shard 1 called by the test sumbits the call
3) shard 0 creates tasks for both calls, but (2) is run first, and preempts the reactor
4) shard 1 sees the completion, completes m_svc.invoke_on(1, ..)
5) shard 0 inserts the completion from (4) before task from (1)
6) the check on shard 0: m.find(id1) fails because the timer is not expired yet
To fix that, wait for timer expiration on shard 0, so that the test
doesn't depend on task execution order.
Note: I was not able to reproduce the problem locally using test.py --mode
debug --repeat 1000.
It happens in jenkins very rarely. Which is expected as the scenario which
leads to this is quite unlikely.
Fixes SCYLLADB-1265
Closesscylladb/scylladb#29290
The test exercises all five node operations (bootstrap, replace, rebuild,
removenode, decommission) and by the end only one node out of four
remains alive. The CQL driver session, however, still holds stale
references to the dead hosts in its connection pool and load-balancing
policy state.
When the new_test_keyspace context manager exits and attempts
DROP KEYSPACE, the driver routes the query to the dead hosts first,
gets ConnectionShutdown from each, and throws NoHostAvailable before
ever trying the single live node.
Fix by calling driver_connect() after the decommission step, which
closes the old session and creates a fresh one connected only to the
servers the test manager reports as running.
Fixes: https://scylladb.atlassian.net/browse/SCYLLADB-1313.
Closesscylladb/scylladb#29306
Sends a search via the raw LDAP handle (bypassing _msgid_to_promise
registration), then triggers poll_results() through the public API
to exercise the unregistered-ID branch.
Refs: SCYLLADB-1344
The multishard_query_test/fuzzy_test was timing out (SIGKILL after
15 minutes) in release mode CI.
In release mode the test generates up to 64 partitions with up to
1000 clustering rows and 1000 range tombstones each. With deeply
nested randomly-generated types (e.g. frozen<map<varint,
frozen<map<frozen<tuple<...>>>>>>), this volume of data can exceed
the 15-minute CI timeout.
Reduce the release-mode clustering-row and range-tombstone
distributions from 0-1000 to 0-200. This caps the worst case at
~12,800 rows -- still 2x the devel-mode maximum (0-100) and
sufficient to exercise multi-partition paged scanning with many
pages.
Fixes: SCYLLADB-1270
No need to backport for now, only appeared on master.
Closesscylladb/scylladb#29293
* github.com:scylladb/scylladb:
test: clean up fuzzy_test_config and add comments
test: fix fuzzy_test timeout in release mode
In the unregistered-ID branch, ldap_msgfree() was called on a result
already owned by an RAII ldap_msg_ptr, causing a double-free on scope
exit. Remove the redundant manual free.
Fixes: SCYLLADB-1344
This commit improves how test.py chohoses the default number of
parallele jobs.
This update keeps logic of selecting number of jobs from memory and cpu limits
but simplifies the heuristic so it is smoother, easier to reason about.
This avoids discontinuities such as neighboring machine sizes producing
unexpectedly different job counts, and behaves more predictably on asymmetric
machines where CPU and RAM do not scale together.
Compared to the current threshold-based version, this approach:
- avoids hard jumps around memory cutoffs
- avoids bucketed debug scaling based on CPU count
- keeps CPU and memory as separate constraints and combines them in one place
- avoids double-penalizing debug mode
- is easier to tune later by adjusting a few constants instead of rewriting branching logic
Closesscylladb/scylladb#28904
get_audit_partitions_for_operation() returns None when no audit log
rows are found. In _test_insert_failure_doesnt_report_success_assign_nodes,
this None is passed to set(), causing TypeError: 'NoneType' object is
not iterable.
The audit log entry may not yet be visible immediately after executing
the INSERT, so use wait_for() from test.pylib.util with exponential
backoff to poll until the entry appears. Import it as wait_for_async
to avoid shadowing the existing wait_for from test.cluster.dtest.dtest_class,
which has a different signature (timeout vs deadline).
Fixes SCYLLADB-1330
Closesscylladb/scylladb#29289
implement tablet migration for logstor tables by streaming segments
using stream_blob, similar to file streaming of sstables.
take a snapshot of the logstor segments and create a stream_blob_info
vector with entry for each segment with the input stream that reads the
segment and an op of type file_ops::stream_logstor_segments.
the stream_blob_handler creates a logstor sink that allocates a segment
on the target shard and creates an output stream that writes to it. when
the sink is closed it loads the segment.
add the function table::take_logstor_snapshot that is similar to
take_storage_snapshot for sstables.
given a token range, for each storage group in the range, it flushes the
separator buffers and then makes a snapshot of all segments in the sg's
compaction groups while disabling compaction.
the segment snapshot holds a reference to the segment so that it won't
be freed by compaction, and it provides an input stream for reading the
segment.
this will be used for tablet migration to stream the segments.
add functions for creating segment input and output streams, that will
be used for segment streaming.
the segment input stream creates a file input stream that reads a given
segment.
the segment output stream allocates a new local segment and creates an
output stream that writes to the segment, and when closed it loads the
segment and adds it to the compaction group.
implement compaction group cleanup by clearing the range in the index
and discarding the segments of the compaction group.
segments are discarded by overwriting the segment header to indicate the
segment is empty while preserving the segment generation number in order
to not resurrect old data in the segment.
implement tablet split for logstor.
flush the separator and then perform split as a new type of compaction:
take a batch of segments from the source compaction group, read them and
write all live records into left/right write buffers according to the
split classifier, flush them to the compaction group, and free the old
segments. segments that fit in a single target compaction group are
removed from the source and added to the correct target group.
implement tablet merge with logstor.
disable compaction for the new compaction group, then merge the merging
compaction groups by merging their logstor segments set into the new cg
- simply merging the segment histogram.
add a function that stops and disabled compaction for a compaction group
and returns a compaction reenabler object, similarly to the normal
compaction manager.
this will be useful for disabling compaction while doing operations on
the compaction group's logstor segment set.
we have two types of segments. the active segment is "mixed" because we
can write to it multiple write_buffers, each write buffer having records
from different tables and tablets. in constrast, the separator and
compaction write "full" segments - they write a single write_buffer that
has records from a single tablet and storage group.
for "full" segments, we add a segment header the contains additional
useful metadata such as the table and token range in the segment.
the write buffer header contains the type of the buffer, mixed or full.
if it's full then it has a segment header placed after the write buffer
header.
previously when writing to the active segment, the allocation was
serialized but multiple writes could proceed concurrently to different
offsets. change it instead to serialize the entire write.
we prefer to write larger buffers sequentially instead of multiple
buffers concurrently. it is also better that we don't have "holes" in
the segment.
we also change the buffered_writer to send a single flushing buffer at a
time. it has a ring of buffers, new writes are written to the head
buffer, and a single consumer flushes the tail buffer.
extend compaction_group functions such as disk size calculation and
empty() to account also for the logstor segments that the compaction
group owns.
reuse the sstable_add_gate when there is a write in process to a
compaction group, in order for the compaction group to be considered not
empty.
add the function table::compaction_group_for_logstor_segment that we use
when recovering a segment to find the compaction group for a segment
based on its token range, similarly to compaction_group_for_sstable for
sstables.
extract the common logic from compaction_group_for_sstable to a common
function compaction_group_for_token_range that finds a compaction group
for a token range.
Switch _promoted_indexes storage in partition_index_page from
managed_vector to chunked_managed_vector to avoid large contiguous
allocations.
Avoid allocation failure (or crashes with --abort-on-internal-error)
when large partitions have enough promoted index entries to trigger a
large allocation with managed_vector.
Fixes: SCYLLADB-1315
Closesscylladb/scylladb#29283
Remove the unused timeout field from fuzzy_test_config. It was
declared, initialized per build mode, and logged, but never actually
enforced anywhere.
Document the intentionally small max_size (1024 bytes) passed to
read_partitions_with_paged_scan in run_fuzzy_test_scan: it forces
many pages per scan to stress the paging and result-merging logic.
The multishard_query_test/fuzzy_test was timing out (SIGKILL after
15 minutes) in release mode CI.
In release mode the test generates up to 64 partitions with up to
1000 clustering rows and 1000 range tombstones each. With deeply
nested randomly-generated types (e.g. frozen<map<varint,
frozen<map<frozen<tuple<...>>>>>>), this volume of data can exceed
the 15-minute CI timeout.
Reduce the release-mode clustering-row and range-tombstone
distributions from 0-1000 to 0-200. This caps the worst case at
~12,800 rows -- still 2x the devel-mode maximum (0-100) and
sufficient to exercise multi-partition paged scanning with many
pages.
Fixes: SCYLLADB-1270
The test was flaky because it stopped dc2_node immediately after an
LWT write, before cross-DC replication could complete. The LWT commit
uses LOCAL_QUORUM, which only guarantees persistence in the
coordinator's DC. Replication to the remote DC is async background
work, and CAS mutations don't store hints. Stopping dc2_node could
drop in-flight RPCs, leaving DC1 without the mutation.
Fix by polling both live DC1 nodes after the write to confirm
cross-DC replication completed before stopping dc2_node. Both nodes
must have the data so that the later ConsistentRead=True
(LOCAL_QUORUM) read on restarted node1 is guaranteed to succeed.
Fixes SCYLLADB-1267
Closesscylladb/scylladb#29287
The approach taken in 1ae2ae50a6 turned
out to be incorrect. The Raft member requesting a read barrier could
incorrectly advance its commit_idx and break linearizability. We revert that
commit in this PR.
We also remake the read barrier optimization with a completely new approach.
We make the leader replicate to the non-voting requester of a read barrier if
its `commit_idx` is behind.
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-998
No backport: the issue is present only in master.
Closesscylladb/scylladb#29216
* github.com:scylladb/scylladb:
raft: speed up read barrier requested by non-voters
Revert "raft: read_barrier: update local commit_idx to read_idx when it's safe"
Capture the memory permit in the leave lambda's .finally()
continuation so that the semaphore units are kept alive until
write_response finishes, preventing premature release of
memory accounting.
This is especially important with slow network and big responses
when buffers can accumulate and deplete node's memory.
Fix two independent race conditions in the syslog audit test that cause intermittent `assert 2 <= 1` failures in `assert_entries_were_added`.
**Datagram ordering race:**
`UnixSockerListener` used `ThreadingUnixDatagramServer`, where each datagram spawns a new thread. The notification barrier in `get_lines()` assumes FIFO handling, but the notification thread can win the lock before an audit entry thread, so `clear_audit_logs()` misses entries that arrive moments later. Fix: switch to sequential `UnixDatagramServer`.
**Config reload race:**
The live-update path used `wait_for_config` (REST API poll on shard 0) which can return before `broadcast_to_all_shards()` completes. Fix: wait for `"completed re-reading configuration file"` in the server log after each SIGHUP, which guarantees all shards have the new config.
Fixes SCYLLADB-1277
This is CI improvement for the latest code. No need for backport.
Closesscylladb/scylladb#29282
* github.com:scylladb/scylladb:
test: cluster: wait for full config reload in audit live-update path
test: cluster: fix syslog listener datagram ordering race
`test_limit_concurrent_requests` could create far more tables than intended
because worker threads looped indefinitely and only the probe path terminated
the test. In practice, workers often hit `RequestLimitExceeded` first, but the
test kept running and creating tables, increasing memory pressure and causing
flakiness due to bad_alloc errors in logs.
Fix by replacing the old probe-driven termination with worker-driven
termination. Workers now run until any worker sees
`RequestLimitExceeded`.
Fixes SCYLLADB-1181
Closesscylladb/scylladb#29270
A joining node hung forever if the topology coordinator added it to the
group 0 configuration before the node reached `post_server_start`. In
that case, `server->get_configuration().contains(my_id)` returned true
and the node broke out of the join loop early, skipping
`post_server_start`. `_join_node_group0_started` was therefore never set,
so the node's `join_node_response` RPC handler blocked indefinitely.
Meanwhile the topology coordinator's `respond_to_joining_node` call
(which has no timeout) hung forever waiting for the reply that never came.
Fix by only taking the early-break path when not starting as a follower
(i.e. when the node is the discovery leader or is restarting). A joining
node must always reach `post_server_start`.
We also provide a regression test. It takes 6s in dev mode.
Fixes SCYLLADB-959
Closesscylladb/scylladb#29266
After obtaining the CQL response, check if its actual size exceeds
the initially acquired memory permit. If so, take semaphore units
and adopt them into the permit (non blocking).
This doesn't fully prevent from allocating too much memory as
size is known when buffer is already allocated but improves
memory accounting for big responses.
_apply_config_to_running_servers used wait_for_config (REST API poll)
to confirm live config updates. The REST API reads from shard 0 only,
so it can return before broadcast_to_all_shards() completes — other
shards may still have stale audit config, generating unexpected entries.
Additionally, server_remove_config_option for absent keys sent separate
SIGHUPs before server_update_config, and the single wait_for_config at
the end could match a completion from an earlier SIGHUP.
Wait for "completed re-reading configuration file" in the server log
after each SIGHUP-producing operation. This message is logged only
after both read_config() and broadcast_to_all_shards() finish,
guaranteeing all shards have the new config. Each operation gets its
own mark+wait so no stale completion is matched.
Fixes SCYLLADB-1277
UnixSockerListener used ThreadingUnixDatagramServer, which spawns a
new thread per datagram. The notification barrier in get_lines() relies
on all prior datagrams being handled before the notification. With
threading, the notification handler can win the lock before an audit
entry handler, so get_lines() returns before the entry is appended.
clear_audit_logs() then clears an incomplete buffer, and the late
entry leaks into the next test's before/after diff.
Switch to sequential UnixDatagramServer. The server thread now handles
datagrams in kernel FIFO order, so the notification is always processed
after all preceding audit entries.
Refs SCYLLADB-1277
The `DESC INDEX` command returned incorrect results for local vector
indexes and for vector indexes that included filtering columns.
This patch corrects the implementation to ensure `DESCRIBE INDEX`
accurately reflects the index configuration.
This was a pre-existing issue, not a regression from recent
serialization schema changes for vector index target options.
Queries against local vector indexes were failing with the error:
"ANN ordering by vector requires the column to be indexed using 'vector_index'"
This was a regression introduced by 15788c3734, which incorrectly
assumed the first column in the targets list is always the vector column.
For local vector indexes, the first column is the partition key, causing
the failure.
Previously, serialization logic for the target index option was shared
between vector and secondary indexes. This is no longer viable due to
the introduction of local vector indexes and vector indexes with filtering
columns, which have different target format.
This commit introduces a dedicated JSON-based serialization format for
vector index targets, identifying the target column (tc), filtering
columns (fc), and partition key columns (pk). This ensures unambiguous
serialization and deserialization for all vector index types.
This change is backward compatible for regular vector indexes. However,
it breaks compatibility for local vector indexes and vector indexes with
filtering columns created in version 2026.1.0. To mitigate this, usage
of these specific index types will be blocked in the 2026.1.0 release
by failing ANN queries against them in vector-store service.
Fixes: SCYLLADB-895
This test ensures that the serialization format for vector index target
options remains stable. Maintaining backward compatibility is critical
because the index is restored from this property on startup.
Any unintended changes to the serialization schema could break existing
indexes after an upgrade.
This option is also an interface for the vector-store service,
which uses it to identify the indexed column.
Target option serialization must remain stable for backward compatibility.
The index is restored from this property on startup, so unintentional
changes to the serialization schema can break indexes after upgrade.
We achieve this by making the leader replicate to the non-voting requester
of a read barrier if its commit_idx is behind.
There are some corner cases where the new `replicate_to(*opt_progress, true);`
call will be a no-op, while the corresponding call in `tick_leader()` would
result in sending the AppendEntries RPC to the follower. These cases are:
- `progress.state == follower_progress::state::PROBE && progress.probe_sent`,
- `progress.state == follower_progress::state::PIPELINE
&& progress.in_flight == follower_progress::max_in_flight`.
We could try to improve the optimization by including some of the cases above,
but it would only complicate the code without noticeable benefits (at least
for group0).
Note: this is the second attempt for this optimization. The first approach
turned out to be incorrect and was reverted in the previous commit. The
performance improvement is the same as in the previous case.
Use get_cql_exclusive(node1) so the driver only connects to node1 and
never attempts to contact the stopped node2. The test was flaky because
the driver received `Host has been marked down or removed` from node2.
Fixes: SCYLLADB-1227
Closesscylladb/scylladb#29268
The test was using time.sleep(1) (a blocking call) to wait after
scheduling the stop_compaction task, intending to let it register on
the server before releasing the sstable_cleanup_wait injection point.
However, time.sleep() blocks the asyncio event loop entirely, so the
asyncio.create_task(stop_compaction) task never gets to run during the
sleep. After the sleep, the directly-awaited message_injection() runs
first, releasing the injection point before stop_compaction is even
sent. By the time stop_compaction reaches Scylla, the cleanup has
already completed successfully -- no exception is raised and the test
fails.
Fix by replacing time.sleep(1) with await asyncio.sleep(1), which
yields control to the event loop and allows the stop_compaction task
to actually send its HTTP request before message_injection is called.
Fixes: SCYLLADB-834
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29202
The vnodes-to-tablets migration is a manual procedure, so instructions
need to be provided to the users.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
Now that cmake/FindLua.cmake uses pkg-config (matching configure.py),
both build systems resolve to the same 'lua' library name. Remove the
lua/lua-5.4 entries from _KNOWN_LIB_ASYMMETRIES and add 'm' (math
library) as a known transitive dependency that configure.py gets via
pkg-config for lua.
CMake's built-in FindLua resolves to the versioned library file
(e.g. liblua-5.4.so) instead of the unversioned symlink (liblua.so),
causing a library name mismatch between the two build systems.
Add a custom cmake/FindLua.cmake that uses pkg-config — matching
configure.py's approach — and find_library(NAMES lua) to find the
unversioned symlink. This also mirrors the pattern used by other
Find modules in cmake/ (FindxxHash, Findlz4, etc.).
Add symmetric_key_test (standalone, links encryption library) and
auth_cache_test to the combined_tests binary. These tests already
exist in configure.py; this aligns the CMake build.
The per-test -fno-lto link option is now redundant since -fno-lto
was added globally in mode.common.cmake. LTO-enabled targets
(the scylla binary in RelWithDebInfo) override it via enable_lto().
Match configure.py's Boost handling:
- Add BOOST_ALL_DYN_LINK when using shared Boost libraries.
- Strip per-component defines (BOOST_UNIT_TEST_FRAMEWORK_DYN_LINK,
BOOST_REGEX_DYN_LINK, etc.) that CMake's Boost package config
adds on imported targets. configure.py only uses the umbrella
BOOST_ALL_DYN_LINK define.
Place add_compile_definitions(SEASTAR_TESTING_MAIN) after both
add_subdirectory(seastar) and add_subdirectory(abseil) are processed.
This matches configure.py's global define without leaking into
seastar's subdirectory build (which would cause a duplicate main
symbol in seastar_testing).
Remove the now-redundant per-test SEASTAR_TESTING_MAIN compile
definition from test/CMakeLists.txt.
Match configure.py line 2192: abseil gets sanitizer flags with
-fno-sanitize=vptr to exclude vptr checks which are incompatible
with abseil's usage of type-punning patterns.
- Set BUILD_SHARED_LIBS based on build type to match configure.py's
build_seastar_shared_libs: Debug and Dev build Seastar as a shared
library, all other modes build it static.
- Add sanitizer link options on the seastar target for Coverage
mode. Seastar's CMake only activates sanitizer targets for
Debug/Sanitize configs, but Coverage mode needs them too since
configure.py's seastar_libs_coverage carries -fsanitize flags.
- Disable CMake's automatic -fcolor-diagnostics injection for
Clang+Ninja (CMake 3.24+), matching configure.py which does not
add any color diagnostics flags.
- Add SEASTAR_NO_EXCEPTION_HACK and XXH_PRIVATE_API as global
defines (previously SEASTAR_NO_EXCEPTION_HACK was only on the
seastar target as PRIVATE; it needs to be project-wide).
- Add -fpch-validate-input-files-content to check precompiled
header content when timestamps don't match.
Fix multiple deviations from configure.py's coverage mode:
- Remove -fprofile-list from CMAKE_CXX_FLAGS_COVERAGE. That flag
belongs in COVERAGE_INST_FLAGS applied to other modes, not to
coverage mode itself.
- Replace incorrect defines (DEBUG, SANITIZE, DEBUG_LSA_SANITIZER,
SCYLLA_ENABLE_ERROR_INJECTION) with the correct Seastar debug
defines (SEASTAR_DEBUG, SEASTAR_DEFAULT_ALLOCATOR, etc.) that
configure.py's pkg-config query produces for coverage mode.
- Add sanitizer and stack-clash-protection compile flags for
Coverage config, matching the flags that Seastar's pkg-config
--cflags output includes for debug builds.
- Change CMAKE_STATIC_LINKER_FLAGS_COVERAGE to
CMAKE_EXE_LINKER_FLAGS_COVERAGE. Coverage flags need to reach
the executable linker, not the static archiver.
Add three flag-alignment changes:
- -Wno-error=stack-usage= alongside the stack-usage threshold flag,
preventing hard errors from stack-usage warnings (matching
configure.py behavior).
- -fno-lto global link option. configure.py adds -fno-lto to all
binaries; LTO-enabled targets override it via enable_lto().
- Sanitizer link flags (-fsanitize=address, -fsanitize=undefined) for
Debug/Sanitize configs, matching configure.py's cxx_ld_flags.
Document the purpose, usage, and examples for
scripts/compare_build_systems.py which compares the configure.py
and CMake build systems by parsing their ninja build files.
Add a script that compares configure.py and CMake build systems by
parsing their generated build.ninja files. The script checks:
- Per-file compilation flags (defines, warnings, optimization)
- Link target sets (detect missing/extra targets)
- Per-target linker flags and libraries
configure.py is treated as the baseline. CMake should match it.
Both systems are always configured into a temporary directory so the
user's build tree is never touched.
Usage:
scripts/compare_build_systems.py -m dev # single mode
scripts/compare_build_systems.py # all modes
scripts/compare_build_systems.py --ci # CI mode (strict)
Fix the ordering of the concurrency limit check in the Alternator HTTP server so it happens before memory acquisition, and reduce test pressure to avoid LSA exhaustion on the memory-constrained test node.
The patch moves the concurrency check to right after the content-length early-out, before any memory acquisition or I/O. The check was originally placed before memory acquisition but was inadvertently moved after it during a refactoring. This allowed unlimited requests to pile up consuming memory, reading bodies, verifying signatures, and decompressing — all before being rejected. Restores the original ordering and mirrors the CQL transport (`transport/server.cc`).
Lowers `concurrent_requests_limit` from 5 to 3 and the thread multiplier from 5 to 2 (6 threads instead of 25). This is still sufficient to reliably trigger RequestLimitExceeded, while keeping flush pressure within what 512MB per shard can sustain.
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1248
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1181
The test started to fail quite recently. It affects master only. No backport is needed. We might want to consider backporting a commit moving the concurrency check earlier.
Closesscylladb/scylladb#29272
* github.com:scylladb/scylladb:
test: reduce concurrent-request-limit test pressure to avoid LSA exhaustion
alternator: check concurrency limit before memory acquisition
The test_limit_concurrent_requests dtest uses concurrent CreateTable
requests to verify Alternator's concurrency limiting. Each admitted
CreateTable triggers Raft consensus, schema mutations, and memtable
flushes—all of which consume LSA memory. On the 1 GB test node
(2 SMP × 512 MB), the original settings (limit=5, 25 threads) created
enough flush pressure to exhaust the LSA emergency reserve, producing
logalloc::bad_alloc errors in the node log. The test was always
marginal under these settings and became flaky as new system tables
increased baseline LSA usage over time.
Lower concurrent_requests_limit from 5 to 3 and the thread multiplier
from 5 to 2 (6 threads total). This is still well above the limit and
sufficient to reliably trigger RequestLimitExceeded, while keeping flush
pressure within what 512 MB per shard can sustain.
The concurrency limit check in the Alternator server was positioned after
memory acquisition (get_units), request body reading (read_entire_stream),
signature verification, and decompression. This allowed unlimited requests
to pile up consuming memory before being rejected, exhausting LSA memory
and causing logalloc::bad_alloc errors that cascade into Raft applier
and topology coordinator failures, breaking subsequent operations.
Without this fix, test_limit_concurrent_requests on a 1GB node produces
50 logalloc::bad_alloc errors and cascading failures: reads from
system.scylla_local fail, the Raft applier fiber stops, the topology
coordinator stops, and all subsequent CreateTable operations fail with
InternalServerError (500). With this fix, the cascade is eliminated --
admitted requests may still cause LSA pressure on a memory-constrained
node, but the server remains functional.
Move the concurrency check to right after the content-length early-out,
before any memory acquisition or I/O. This mirrors the CQL transport
which correctly checks concurrency before memory acquisition
(transport/server.cc).
The concurrency check was originally added in 1b8c946ad7 (Sep 2020)
*before* memory acquisition, which at the time lived inside with_gate
(after the concurrency gate). The ordering was inverted by f41dac2a3a
(Mar 2021, "avoid large contiguous allocation for request body"), which
moved get_units() earlier in the function to reserve memory before
reading the newly-introduced content stream -- but inadvertently also
moved it before the concurrency check. c3593462a4 (Mar 2025) further
worsened the situation by adding a 16MB fallback reservation for
requests without Content-Length and ungzip/deflate decompression steps
-- all before the concurrency check -- greatly increasing the memory
consumed by requests that would ultimately be rejected.
**The Bug**
Assertion failure: `SCYLLA_ASSERT(res.second)` in `raft/server.cc`
when creating a snapshot transfer for a destination that already had a
stale in-flight transfer.
**Root Cause**
If a node loses leadership and later becomes leader again before the next
`io_fiber` iteration, the old transfer from the previous term can remain
in `_snapshot_transfers` while `become_leader()` resets progress state.
When the new term emits `install_snapshot(dst)`, `send_snapshot(dst)`
tries to create a new entry for the same destination and can hit the
assertion.
**The Fix**
Abort all in-flight snapshot transfers in `process_fsm_output()` when
`term_and_vote` is persisted. A term/vote change marks existing transfers
as stale, so we clean them up before dispatching messages from that batch
and before any new snapshot transfer is started.
With cross-term cleanup moved to the term-change path, `send_snapshot()`
now asserts the within-term invariant that there is at most one in-flight
transfer per destination.
Fixes: SCYLLADB-862
Backport: The issue is reproducible in master, but is present in all
active branches.
Closesscylladb/scylladb#29092
This reverts commit c30607d80b.
With the default configuration, enabling DDL has no effect because
no `audit_keyspaces` or `audit_tables` are specified. Including DDL
in the default categories can be misleading for some customers, and
ideally we would like to avoid it.
However, DDL has been one of the default audit categories for years,
and removing it risks silently breaking existing deployments that
depend on it. Therefore, the recent change to disable DDL by default
is reverted.
Fixes: SCYLLADB-1155
Closesscylladb/scylladb#29169
test_reboot uses a custom restart function that SIGKILLs and restarts
nodes sequentially. After all nodes are back up, the test proceeded
directly to reads after wait_for_cql_and_get_hosts(), which only
confirms CQL reachability.
While a node is restarted, other nodes might execute global token
metadata barriers, which advance the topology fence version. The
restarted node has to learn about the new version before it can send
reads/writes to the other nodes. The test issues reads as soon as the
CQL port is opened, which might happen before the last restarted node
learns of the latest topology version. If this node acts as a
coordinator for reads/write before this happens, these will fail as the
other nodes will reject the ops with the outdated topology fence
version.
Fix this by replacing wait_for_cql_and_get_hosts() on the abrupt-restart
path with the more robus get_ready_cql(), which makes sure servers see
each other before refreshing the cql connection. This should ensure that
nodes have exchanged gossip and converged on topology state before any
reads are executed. The rolling_restart() path is unaffected as it
handles this internally.
Fixes: SCYLLADB-557
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29211
`test_crashed_node_substitution` intermittently failed:
```python
assert len(gossiper_eps) == (len(server_eps) + 1)
```
The test crashed the node right after a single ACK2 handshake (`finished do_send_ack2_msg`), assuming the node state was visible to all peers. However, since gossip is eventually consistent, the update may not have propagated yet, so some nodes did not see the failed node.
This change: Wait until the gossiper state is visible on peers before continuing the test and asserting.
Fixes: [SCYLLADB-1256](https://scylladb.atlassian.net/browse/SCYLLADB-1256).
backport: this issue may affect CI for all branches, so should be backported to all versions.
[SCYLLADB-1256]: https://scylladb.atlassian.net/browse/SCYLLADB-1256?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQClosesscylladb/scylladb#29254
* github.com:scylladb/scylladb:
test: test_crashed_node_substitution: add docstring and fix whitespace
test: fix race condition in test_crashed_node_substitution
If lwt_workload() sends an update immediately after a
rolling restart, the coordinator might still see a replica as
down due to gossip lagging behind. Concurrently restarting another
node leaves only one available replica, failing the
LOCAL_QUORUM requirement for learn or eventually consistent
sp::query() in sp::cas() and resulting in
a mutation_write_failure_exception.
We fix this problem by waiting for the restarted server
to see 2 other peers. The server_change_version
doesn't do that by default -- it passes
wait_others=0 to server_start().
Fixes SCYLLADB-1136
Closesscylladb/scylladb#29234
`test_crashed_node_substitution` intermittently failed:
```
assert len(gossiper_eps) == (len(server_eps) + 1)
```
The test crashed the node right after a single ACK2 handshake
("finished do_send_ack2_msg"), assuming the node state was
visible to all peers. However, since gossip is eventually
consistent, the update may not have propagated yet, so some
nodes did not see the failed node.
This change: Wait until the gossiper state is visible on
peers before continuing the test and asserting.
Fixes: SCYLLADB-1256.
SSTable unlinking is async, so in some cases it may happen that
the upload dir is not empty immediately after refresh is done.
This patch adjusts test_refresh_deletes_uploaded_sstables so
it waits with a timeout till the upload dir becomes empty
instead of just assuming the API will sync on sstables being
gone.
Fixes SCYLLADB-1190
Signed-off-by: Robert Bindar <robert.bindar@scylladb.com>
Closesscylladb/scylladb#29215
This test runs the vnodes-to-tablets migration for a single table on a
single-node cluster. The node has multiple shards and multiple
power-of-two aligned vnodes, so resharding is triggered.
More details in the docstring.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
The vnodes-to-tablets migration is a manual procedure, so orchestration
must be done via nodetool.
This patch adds the following new commands:
* nodetool migrate-to-tablets start {ks}
* nodetool migrate-to-tablets upgrade
* nodetool migrate-to-tablets downgrade
* nodetool migrate-to-tablets status {ks}
* nodetool migrate-to-tablets finalize {ks}
The commands are just wrappers over the REST API.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
If the keyspace is migrating, it reports the intended and actual storage
mode for each node.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
Remove unused `pytest.mark.single_node` marker from `TestCQLAudit`.
Rename `TestCQLAudit` to `CQLAuditTester` to reflect that it is a test helper, not a test class. This avoids accidental pytest collection and subsequent warning about `__init__`.
Logs before the fixes:
```
test/cluster/test_audit.py:514: 14 warnings
/home/dario/dev/scylladb/test/cluster/test_audit.py:514: PytestCollectionWarning: cannot collect test class 'TestCQLAudit' because it has a __init__ constructor (from: cluster/test_audit.py)
@pytest.mark.single_node
```
Fixes SCYLLADB-1237
This is an addition to the latest master code. No backport needed.
Closesscylladb/scylladb#29237
* github.com:scylladb/scylladb:
test: audit: rename TestCQLAudit to CQLAuditTester
test: audit: remove unused pytest.mark.single_node
Improve test comments for test_streams_batchwrite_into_the_same_partition_deletes_existing_items
and test_streams_batchwrite_into_the_same_partition_will_report_wrong_stream_data to explain why
'always' write isolation mode is required: in always_use_lwt mode all items in a batch get the
same CDC timestamp, which triggers the squashing bug. In other modes each item gets a separate
timestamp so the bug doesn't manifest.
Also fix the example in the second test comment to use cleaner key values and correct event type
(INSERT, not MODIFY, since items are inserted into an empty table), and fix the issue reference
from #28452 (the PR) to #28439 (the issue).
pytest tries to collect tests for execution in several ways.
One is to pick all classes that start with 'Test'. Those classes
must not have custom '__init__' constructor. TestCQLAudit does.
TestCQLAudit after migration from test/cluster/dtest is not a test
class anymore, but rather a helper class. There are two ways to fix
this:
1. Add __init__ = False to the TestCQLAudit class
2. Rename it to not start with 'Test'
Option 2 feels better because the new name itself does not convey
the wrong message about its role.
Fixes SCYLLADB-1237
Remove unused pytest.mark.single_node in TestCQLAudit class.
This is a leftover from audit tests migration from
test/cluster/dtest to test/cluster.
Refs SCYLLADB-1237
Add scylla_only fixture dependency to the
test_table_ss_new_and_old_images_write_isolation_always fixture.
This ensures all tests using the 'always' write isolation mode
are skipped when running against DynamoDB (--aws), since the
system:write_isolation tag is a Scylla-only feature.
BatchWriteItem with items for the same partition (and write isolation
set to always) will trigger LWT and run different cdc code path, which
will result in wrong Streams data being returned to the user -
changes will be randomly squashed together.
For example batch write:
batch.put_item(Item={'p': 'p', 'c': 'c0'})
batch.put_item(Item={'p': 'p', 'c': 'c1'})
batch.put_item(Item={'p': 'p', 'c': 'c2'})
instead of producing 3 modify / insert events will produce one:
type=INSERT, key={'c': {'S': 'c0'}, 'p': {'S': 'p'}},
old_image=None, new_image={'c': {'S': 'c2'}, 'p': {'S': 'p'}}
with `new_image` having different `c` key from `key` field.
This happens because BatchWriteItem (when using LWT) emits it's changes
to cdc under the same timestamp. This results in in all log entries
being put in single cdc "bucket" (under the same cdc$timestamp key).
Previous parsing algorithm would interpret those changes as a change
to a single item and squash them together.
The patch rewrites algorithm to use `std::unordered_map` for records
based on value of clustering key, that is added to every cdc log entry.
This allows rebuilding all item modifications.
Fixes#28439
Fixes: SCYLLADB-540
Add additional BatchWriteItem tests (some failing):
- `test_streams_batchwrite_no_clustering_deletes_non_existing_items`
`test_streams_batchwrite_no_clustering_deletes_existing_items` -
those tests pass, we add it here for completness, as non clustering
tables trigger different paths.
- `test_streams_batchwrite_into_the_same_partition_deletes_existing_items` -
failing test, that checks combinations of puts and deletes in a single
batch write (so for example 3 items, 2 puts and 1 delete).
- `test_streams_batchwrite_into_the_same_partition_will_report_wrong_stream_data` -
failing simple test.
Tests fail, because current implementation, when writing cdc log
entries will squash all changes done to the same partition together.
The data is still there, but when GetRecords is called and we parse
cdc log entries, we don't correctly recover it (see issue #28439 for
more details).
Previously, the result of when_all was discarded. when_all stores
exceptions in the returned futures rather than throwing, so the outer
catch(in_use&) could never trigger. Now we capture the when_all result
and inspect each future individually to properly detect in_use from
either stream.
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1216Closesscylladb/scylladb#29219
Change wait_for() defaults from period=1s/no backoff to period=0.1s
with 1.5x backoff capped at 1.0s. This catches fast conditions in
100ms instead of 1000ms, benefiting ~100 call sites automatically.
Add completion logging with elapsed time and iteration count.
Tested local with test/cluster/test_fencing.py::test_fence_hints (dev mode),
log output:
wait_for(at_least_one_hint_failed) completed in 0.83s (4 iterations)
wait_for(exactly_one_hint_sent) completed in 1.34s (5 iterations)
Fixes SCYLLADB-738
Closesscylladb/scylladb#29173
The test was flaky. The scenario looked like this:
1. Stop server 1.
2. Set its rf_rack_valid_keyspaces configuration option to true.
3. Create an RF-rack-invalid keyspace.
4. Start server 1 and expect a failure during start-up.
It was wrong. We cannot predict when the Raft mutation corresponding to
the newly created keyspace will arrive at the node or when it will be
processed. If the check of the RF-rack-valid keyspaces we perform at
start-up was done before that, it won't include the keyspace. This will
lead to a test failure.
Unfortunately, it's not feasible to perform a read barrier during
start-up. What's more, although it would help the test, it wouldn't be
useful otherwise. Because of that, we simply fix the test, at least for
now.
The new scenario looks like this:
1. Disable the rf_rack_valid_keyspaces configuration option on server 1.
2. Start the server.
3. Create an RF-rack-invalid keyspace.
4. Perform a read barrier on server 1. This will ensure that it has
observed all Raft mutations, and we won't run into the same problem.
5. Stop the node.
6. Set its rf_rack_valid_keyspaces configuration option to true.
7. Try to start the node and observe a failure.
This will make the test perform consistently.
---
I ran the test (in dev mode, on my local machine) three times before
these changes, and three times with them. I include the time results
below.
Before:
```
real 0m47.570s
user 0m41.631s
sys 0m8.634s
real 0m50.495s
user 0m42.499s
sys 0m8.607s
real 0m50.375s
user 0m41.832s
sys 0m8.789s
```
After:
```
real 0m50.509s
user 0m43.535s
sys 0m9.715s
real 0m50.857s
user 0m44.185s
sys 0m9.811s
real 0m50.873s
user 0m44.289s
sys 0m9.737s
```
Fixes SCYLLADB-1137
Backport: The test is present on all supported branches, and so we
should backport these changes to them.
Closesscylladb/scylladb#29218
* github.com:scylladb/scylladb:
test: cluster: Deflake test_startup_with_keyspaces_violating_rf_rack_valid_keyspaces
test: cluster: Mark test with @pytest.mark.asyncio in test_multidc.py
This PR contains two small improvements to `test_incremental_repair.py`
motivated by the sporadic failure of
`test_tablet_incremental_repair_and_scrubsstables_abort`.
The test fails with `assert 3 == 2` on `len(sst_add)` in the second
repair round. The extra SSTable has `repaired_at=0`, meaning scrub
unexpectedly produced more unrepaired SSTables than anticipated. Since
scrub (and compaction in general) logs at DEBUG level and the test did
not enable debug logging, the existing logs do not contain enough
information to determine the root cause.
**Commit 1** fixes a long-standing typo in the helper function name
(`preapre` -> `prepare`).
**Commit 2** enables `compaction=debug` for the Scylla nodes started by
`do_tablet_incremental_repair_and_ops`, which covers all
`test_tablet_incremental_repair_and_*` variants. This will capture full
compaction/scrub activity on the next reproduction, making the failure
diagnosable.
Refs: SCYLLADB-1086
Backport: test improvement, no backport
Closesscylladb/scylladb#29175
* https://github.com/scylladb/scylladb:
test/cluster/test_incremental_repair.py: enable compaction DEBUG logs in do_tablet_incremental_repair_and_ops
test/cluster/test_incremental_repair.py: fix typo preapre -> prepare
When authenticate() returns a user directly (certificate-based auth,
introduced in 20e9619bb1), process_startup was missing the same
post-authentication bookkeeping that the no-auth and SASL paths perform:
- update_scheduling_group(): without it, the connection runs under the
default scheduling group instead of the one mapped to the user's
service level.
- _authenticating = false / _ready = true: without them,
system.clients reports connection_stage = AUTHENTICATING forever
instead of READY.
- on_connection_ready(): without it, the connection never releases its
slot in the uninitialized-connections concurrency semaphore (acquired
at connection creation), leaking one unit per cert-authenticated
connection for the lifetime of the connection.
The omission was introduced when on_connection_ready() was added to the
else and SASL branches in 474e84199c but the cert-auth branch was missed.
Fixes: 20e9619bb1 ("auth: support certificate-based authentication")
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The cert-auth path in process_startup (introduced in 20e9619bb1) was
missing _ready = true, _authenticating = false, update_scheduling_group()
and on_connection_ready(). The result is that connections authenticated
via certificate show connection_stage = AUTHENTICATING in system.clients
forever, run under the wrong service-level scheduling group, and hold
the uninitialized-connections semaphore slot for the lifetime of the
connection.
Add a parametrized cluster test that verifies all three process_startup
branches result in connection_stage = READY:
- allow_all: AllowAllAuthenticator (no-auth path)
- password: PasswordAuthenticator (SASL/process_auth_response path)
- cert_bypass: CertificateAuthenticator with transport_early_auth_bypass
error injection (cert-auth path -- the buggy one)
The injection is added to certificate_authenticator::authenticate() so
tests can bypass actual TLS certificate parsing while still exercising
the cert-auth code path in process_startup.
The cert_bypass case is marked xfail until the bug is fixed.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Instead of always passing sstable_state::normal, infer the state from
the last component of the directory path by comparing against the known
state subdirectory constants (staging_dir, upload_dir, quarantine_dir).
Any unrecognized path component (the common case for normal-state
sstables) maps to sstable_state::normal.
When a non-normal state is detected, strip the state subdirectory from
dir so that the base table directory is passed to storage.
Two issues prevented the precompiled header from compiling
successfully when using CMake directly (rather than the
configure.py + ninja build system):
a) Propagate build flags to Rust binding targets reusing the
PCH. The wasmtime_bindings and inc targets reuse the PCH
from scylla-precompiled-header, which is compiled with
Seastar's flags (including sanitizer flags in
Debug/Sanitize modes). Without matching compile options,
the compiler rejects the PCH due to flag mismatch (e.g.,
-fsanitize=address). Link these targets against
Seastar::seastar to inherit the required compile options.
Closesscylladb/scylladb#28941
The test was flaky. The scenario looked like this:
1. Stop server 1.
2. Set its rf_rack_valid_keyspaces configuration option to true.
3. Create an RF-rack-invalid keyspace.
4. Start server 1 and expect a failure during start-up.
It was wrong. We cannot predict when the Raft mutation corresponding to
the newly created keyspace will arrive at the node or when it will be
processed. If the check of the RF-rack-valid keyspaces we perform at
start-up was done before that, it won't include the keyspace. This will
lead to a test failure.
Unfortunately, it's not feasible to perform a read barrier during
start-up. What's more, although it would help the test, it wouldn't be
useful otherwise. Because of that, we simply fix the test, at least for
now.
The new scenario looks like this:
1. Disable the rf_rack_valid_keyspaces configuration option on server 1.
2. Start the server.
3. Create an RF-rack-invalid keyspace.
4. Perform a read barrier on server 1. This will ensure that it has
observed all Raft mutations, and we won't run into the same problem.
5. Stop the node.
6. Set its rf_rack_valid_keyspaces configuration option to true.
7. Try to start the node and observe a failure.
This will make the test perform consistently.
---
I ran the test (in dev mode, on my local machine) three times before
these changes, and three times with them. I include the time results
below.
Before:
```
real 0m47.570s
user 0m41.631s
sys 0m8.634s
real 0m50.495s
user 0m42.499s
sys 0m8.607s
real 0m50.375s
user 0m41.832s
sys 0m8.789s
```
After:
```
real 0m50.509s
user 0m43.535s
sys 0m9.715s
real 0m50.857s
user 0m44.185s
sys 0m9.811s
real 0m50.873s
user 0m44.289s
sys 0m9.737s
```
Fixes SCYLLADB-1137
ERMs created in `calculate_vnode_effective_replication_map` have RF computed based
on the old token metadata during a topology change. The reading replicas, however,
are computed based on the new token metadata (`target_token_metadata`) when
`read_new` is true. That can create a mismatch for EverywhereStrategy during some
topology changes - RF can be equal to the number of reading replicas +-1. During
bootstrap, this can cause the
`everywhere_replication_strategy::sanity_check_read_replicas` check to fail in
debug mode.
We fix the check in this commit by allowing one more reading replica when
`read_new` is true.
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1147Closesscylladb/scylladb#29150
Before these changes, we would send mutations to the node and
immediately query the metrics to see how many hints had been written.
However, that could lead to random failures of the test: even if the
mutations have finished executing, hints are stored asynchronously, so
we don't have a guarantee they have already been processed.
To prevent such failures, we rewrite the check: we will perform multiple
checks against the metrics until we have confirmed that the hints have
indeed been written or we hit the timeout.
We're generous with the timeout: we give the test 60 seconds. That
should be enough time to avoid flakiness even on super slow machines,
and if the test does fail, we will know something is really wrong.
As a bonus, we improve the test in general too. We explicitly express
the preconditions we rely on, as well as bump the log level. If the
test fails in the future, it might be very difficult do debug it
without this additional information.
Fixes SCYLLADB-1133
Backport: The test is present on all supported branches. To avoid
running into more failures, we should backport these changes
to them.
Closesscylladb/scylladb#29191
* github.com:scylladb/scylladb:
test: cluster: Increase log level in test_write_cl_any_to_dead_node_generates_hints
test: cluster: Await all mutations concurrently in test_write_cl_any_to_dead_node_generates_hints
test: cluster: Specify min_tablet_count in test_write_cl_any_to_dead_node_generates_hints
test: cluster: Use new_test_table in test_write_cl_any_to_dead_node_generates_hints
test: cluster: Introduce auxiliary function keyspace_has_tablets
test: cluster: Deflake test_write_cl_any_to_dead_node_generates_hints
The endpoint is the following:
POST /storage_service/vnode_tablet_migrations/keyspaces/{keyspace}/finalization
When called, it issues a `finalize_migration` topology request and waits
for its completion.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
Vnodes-to-tablets migration needs a finalization step to finish or
rollback the migration. Finishing the migration involves switching the
keyspace schema to tablets and clearing the `intended_storage_mode` from
system.topology. Rolling back the migration involves deleting the tablet
maps and clearing the `intended_storage_mode`.
The finalization needs to be done as a topology request to exclude with
other operations such as repair and TRUNCATE.
This patch introduces the `finalize_migration` global topology request
for this purpose. The request takes a keyspace name as an argument.
The direction of the finalization (i.e., forward path vs rollback) is
inferred from the `intended_storage_mode` of all nodes (not ideal,
should be made explicit).
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
Extend `database::add_column_family()` with a `storage_mode` argument.
If the table is under vnodes-to-tablets migration and the storage mode
is "tablets", create a tablet ERM.
Make the distributed loader determine the storage mode from topology
(`intended_storage_mode` column in system.topology).
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
The endpoint is the following:
POST /storage_service/vnode_tablet_migrations/node/storage_mode?intended_mode={tablets,vnodes}
This endpoint is part of the vnodes-to-tablets migration process and
controls a node's intended_storage_mode in system.topology. The storage
mode represents the node-local data distribution model, i.e., how data
are organized across shards. The node will apply the intended storage
mode to migrating tables upon next restart by resharding their SSTables
(either on vnode boundaries if intended_mode=tablets, or with the static
sharder if intended_mode=vnodes).
Note that this endpoint controls the intended_storage_mode of the local
node only. This has the nice benefit that once the API call returns, the
change has not only been committed to group0 but also applied to the
local node's state machine. This guarantees that the change is part of
the node's local copy upon next restart; no additional read barrier is
needed.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
The endpoint is the following:
POST /storage_service/vnode_tablet_migrations/keyspaces/{keyspace}
Its purpose is to start the migration of a whole keyspace from vnodes to
tablets.
When called, Scylla will synchronously create a tablet map for each
table in the specified keyspace. The tablet maps of all tables are
identical and they mirror the vnode layout; they contain one tablet per
vnode and each tablet uses the same replica hosts and token boundaries
as the corresponding vnode.
The only difference from vnodes lies in the sharding approach. Tablets
are assigned to a single shard - using a round-robin strategy in this
patch - whereas vnodes are distributed evenly across all shards. If the
tablet count per shard is low and tablet sizes are uneven, or some
shards have more tablets than others, performance may degrade during the
migration process. For example, a cluster with i8g.48xlarge (192 vCPUs),
256 vnodes per node and RF=3 will have 256 * 3 / 192 vCPUs = 4 tablet
replicas per shard during the migration. One additional tablet or a
double-sized tablet would cause 25% overcommit.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
Reduce the timeout for one test to 60 minutes. The longest test we had
so far was ~10-15 minutes. So reducing this timeout is pretty safe and
should help with hanging tests.
Closesscylladb/scylladb#29212
Part of the vnodes-to-tablets migration is to reshard the SSTables of
each node on vnode boundaries. Resharding is a heavy operation that
runs on startup while the node is offline. Since nodes can restart
for unexpected reasons, we need a flag to do it in a controllable way.
We also need the ability to roll back the migration, which requires
resharding in the opposite direction. This means a node must be aware of
the intended migration direction.
To address both requirements, this patch introduces a new column,
intended_storage_mode, in system.topology. A non-null value indicates
that a node should perform a migration and specifies the migration
direction.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
Make the table populator migration-aware. If a table is migrating to
tablets, switch from normal resharding to vnode-based resharding.
Vnode-based resharding requires passing a vector of "owned ranges" upon
which resharding will segregate the SSTables. Compute it from the tablet
map. We could also compute them from the vnodes, since tablets are
identical to vnodes during the migration, but in the future we may
switch to a different model (multiple tablets per vnode).
Let the distributed loader decide if a table is migrating or not and
communicate that to the table populator. A table is migrating if the
keyspace replication strategy uses vnodes but the table replication
strategy uses tablets.
Currently, tables cannot enter this "migrating" state; support for this
will be introduced in the next patches.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
In the previous patch, reshard compaction was extended with a special
operation mode where SSTables from vnode-based tables are segregated on
vnode boundaries and not with the static sharder. This will later be
wired into vnodes-to-tablets migration.
The problem is that resharding requires a compaction group. With a
vnode-based table, there is only one compaction group per shard, and
this is what the current code utilizes
(`try_get_compaction_group_view_with_static_sharding()`). But the new
operation mode will apply to migrating tables, which use a
`tablet_storage_group_manager`, which creates one compaction group for
each tablet. Some compaction group needs to be selected.
Pick any compaction group that is available on the current shard.
Reshard compaction is an operation that happens early in the startup
process; compaction groups do not own any SSTables yet, so all
compaction groups are equivalent.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
In this mode, the output sstables generated by resharding
compaction are segregated by token range, based on the keyspace
vnode-based owned token ranges vector.
A basic unit test was also added to sstable_directory_test.
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
When a table is migrating from vnodes to tablets, the cluster is in a
mixed state where some nodes use vnode ERMs and others use tablet ERMs.
The ERM flavor is a node-local property that expresses the node's
storage organization.
Preserve the flavor across token metadata changes. The flavor needs to
be on par with storage, but the storage can change only on startup, as
it requires resharding all SSTables to conform with the flavor.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
The tablet load balancer operates on all tablet-based tables that appear
in the tablet metadata.
With the introduction of the vnodes-to-tablets migration procedure later
in this series, migrating tables will also appear in the tablet
metadata, but they need to be treated as vnode tables until migration is
finished. This patch excludes such tables from load balancing.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
Vnodes-to-tablets migrations require cluster-level support: the REST API
and the group0 state need to be supported by all nodes.
Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
We increase the log level of `hints_manager` to TRACE in the test.
If it fails, it may be incredibly difficult to debug it without any
additional information.
The test relies on the assumption that mutations will be distributed
more or less uniformly over the nodes. Although in practice this should
not be possible, theoretically it's possible that there's only one
tablet allocated for the table.
To clearly indicate this precondition, we explicitly set the property
`min_tablet_count` when creating the table. This way, we have a gurantee
that the table has multiple tablets. The load balancer should now take
care of distributing them over the nodes equally. Thanks to that,
`servers[1]` will have some tablets, and so it'll be the target for some
of the mutations we perform.
The context manager is the de-facto standard in the test suite. It will
also allow us for a prettier way to conditionally enable per-table
tablet options in the following commit.
The function is adapted from its counterpart in the cqlpy test suite:
cqlpy/util.py::keyspace_has_tablets. We will use it in a commit in this
series to conditionally set tablet properties when creating a table.
It might also be useful in general.
Before these changes, we would send mutations to the node and
immediately query the metrics to see how many hints had been written.
However, that could lead to random failures of the test: even if the
mutations have finished executing, hints are stored asynchronously, so
we don't have a guarantee they have already been processed.
To prevent such failures, we rewrite the check: we will perform multiple
checks against the metrics until we have confirmed that the hints have
indeed been written or we hit the timeout.
We're generous with the timeout: we give the test 60 seconds. That
should be enough time to avoid flakiness even on super slow machines,
and if the test does fail, we will know something is really wrong.
Fixes SCYLLADB-1133
Snapshots are not implemented yet for strong consistency - attempting to
take, transfer or drop a snapshot results in an exception. However, the
logic of our state machine forces snapshot transfer even if there are no
lagging replicas - every raft::server::configuration::snapshot_threshold
log entries. We have actually encountered an issue in our benchmarks
where snapshots were being taken even though the cluster was not under
any disruption, and this is one of the possible causes.
It turns out that we can safely allow for taking snapshots right now -
we can just implement it as a no-op and return a random UUID.
Conversely, dropping a snapshot can also be a no-op. This is safe
because snapshot transfer still throws an exception - as long as the
taken/recovered snapshots are never attempted to be transferred.
Raft snapshots are not implemented yet for strong consistency. Adjust
the current raft group config to make them much less likely to occur:
- snapshot_threshold config option decides how many log entries need to
be applied after the last snapshot. Set it to the maximum value for
size_t in order to effectively disable it.
- snapshot_threshold_log_size defines a threshold for the log memory
usage over which a snapshot is created. Increase it from the default
2MB to 10MB.
- max_log_size defines the threshold for the log memory usage over which
requests are stopped to be admitted until the log is shrunk back by a
snapshot. Set it to 20MB, as this option is recommended to be at least
twice as much as snapshot_threshold_log_size.
Refs: SCYLLADB-1115
Replace create_dataset + manual DROP/CREATE KEYSPACE with two sequential
new_test_keyspace context manager blocks, matching the pattern used by
do_test_streaming_scopes. The first block covers backup, the second
covers restore. Keyspace lifecycle is now automatic.
The streaming directions validation loop is moved outside of the second
context block, since it only parses logs and has no dependency on the
keyspace being alive.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The test sporadically fails because scrub produces an unexpected number
of SSTables. Compaction logs are needed to diagnose why, but were not
captured since scrub runs at DEBUG level. Enable compaction=debug for
the servers started by do_tablet_incremental_repair_and_ops so the next
reproduction provides enough information to root-cause the issue.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
And move some activities from streaming group into it, namely
- tablet_allocator background group
- sstables_manager-s components reclaimer
- tablet storage group manager merge completion fiber
- prometheus
All other activity that was in streaming group remains there, but can be
moved to this group (or to new maintenance subgroup) later.
All but prometheus are patched here, prometheus still uses the
maintenance_sched_group variable in main.cc, so it transparently
moves into new group
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
The snapshot_ctl::backup_task_impl runs in configured scheduling group.
Now it's streaming one. This patch introduces the maintenance/backup
group and re-configures backup task with it.
The group gets its --backup_io_throughput_mb_per_sec option that
controls bandwidth limit for this sub-group only.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Compaction manager tells compaction_sched_group from
maintenance_compaction_sched_group. The latter, however, is set to be
"streaming" group. This patch adds real maintenance_compaction group
under the maintenance supergroup and makes compaction manager use it.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
And just move streaming group inside it. Next patches will populate this
supergroup further.
The new supergroup gets its --maintenance-io-throughput-mb-per-sec
option that controls supergroup-wide IO bandwidth applied to it. If not
configured, the supergroup gets the throughput from streaming to be
backward compatible.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
The main.cc code uses two variables to reference streaming scheduling.
This patch stops using the maintenance_sched_group one, because it's in
fact streaming group, and real "maintenance" will appear later in this
set.
One place is deliberately not patched -- prometheus code starts before
dbcfg.streaming_scheduling_group appears, so it still sits uses the
maintenance_sched_group variable. This fact will be used in one of the
next patches.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
The classify_request() helper captures current scheduling group into
local variable and compares it with groups from db_config to decide
which "class" it belongs to.
One if uses current_scheduling_group(), while it could use the local
variable.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Currently we have two live-updateable IO-throughput options -- one for
streaming and one for compaction. Both are observed and the changed
value is applied to the corresponding scheduling_group by the relevant
serice -- respectively, stream_manager and compaction_manager.
Both observe/react/apply places use pretty heavy boilerplate code for
such simple task. Next patches will make things worse by adding two more
options to control IO throughput of some other groups.
Said that, the proposal is to hold the updating code in main.cc with the
help of a wrapper class. In there all the needed bits are at hand, and
classes can get their IO updates applied easily.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
There are two helpers for describe_ring endpoint. Both can be squashed
together for code brevity.
Also, while at it, the "keyspace" parameter is not properly validated by
the endpoint.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
All callers already have it. It makes no difference for the method
itself with which table identifier to work, but will help to simplify
the flow in API handler (next patch)
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This actually uses two interconnected options:
repair_multishard_reader_buffer_hint_size and
repair_multishard_reader_enable_read_ahead.
Both are propagated through repair_service::config and pass their
values to repair_reader/make_reader at construction time.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Most other services have their configs, rpair still uses global
db::config.
Add an empty config struct to repair_service to carry db::config options
the repair service needs.
Subsequent patches will populate the struct with options.
The config is created in main.cc as sharded_parameter because all future
options are live-updateable and should capture theirs source from
db::config on correct shard.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Since we do no longer support upgrade from versions that do not support
v2 of view building code we can remove upgrade code and make sure we do
not boot with old builder version.
2026-03-18 17:45:40 +02:00
2688 changed files with 19795 additions and 9147 deletions
@@ -12,20 +12,48 @@ The terms "**You**" or "**Licensee**" refer to any individual accessing or using
**Grant of License**
* **Software Definitions:** Software means the ScyllaDB software provided by Licensor, including the source code, object code, and any accompanying documentation or tools, or any part thereof, as made available under this Agreement.
* **Grant of License:** Subject to the terms and conditions of this Agreement, Licensor grants You a limited, non-exclusive, revocable, non-sublicensable, non-transferable, royalty free license to Use the Software, in each case solely for the purposes of:
* **Definitions:**
1.**Software:** Software means the ScyllaDB software provided by Licensor, including the source code, object code, and any accompanying documentation or tools, or any part thereof, as made available under this Agreement.
2.**Commercial Customer**: means any legal entity (including its Affiliates) that has entered into a transaction with Licensor, or an authorized reseller/distributor, for the provision of any ScyllaDB products or services. This includes, without limitation: (a) Scope of Service: Any paid subscription, enterprise license, "BYOA" or Database-as-a-Service (DBaaS) offering, technical support, professional services, consulting, or training. (b) Scale and Volume: Any deployment regardless of size, capacity, or performance metrics (c) Payment Method: Any compensation model, including but not limited to, fixed-fee, consumption-based (On-Demand), committed spend, third-party marketplace credits (e.g., AWS, GCP, Azure), or promotional credits and discounts.
* **Grant of License:** Subject to the terms and conditions of this Agreement, including the Eligibility and Exclusive Use Restrictions clause, Licensor grants You a limited, non-exclusive, revocable, non-sublicensable, non-transferable, royalty free license to Use the Software, in each case solely for the purposes of:
1) Copying, distributing, evaluating (including performing benchmarking or comparative tests or evaluations , subject to the limitations below) and improving the Software and ScyllaDB; and
2) create a modified version of the Software (each, a "**Licensed Work**"); provided however, that each such Licensed Work keeps all or substantially all of the functions and features of the Software, and/or using all or substantially all of the source code of the Software. You hereby agree that all the Licensed Work are, upon creation, considered Licensed Work of the Licensor, shall be the sole property of the Licensor and its assignees, and the Licensor and its assignees shall be the sole owner of all rights of any kind or nature, in connection with such Licensed Work. You hereby irrevocably and unconditionally assign to the Licensor all the Licensed Work and any part thereof. This License applies separately for each version of the Licensed Work, which shall be considered "Software" for the purpose of this Agreement.
* **Eligibility and Exclusive Use Restrictions**
**License Limitations, Restrictions and Obligations:** The license grant above is subject to the following limitations, restrictions, and obligations. If Licensee’s Use of the Software does not comply with the above license grant or the terms of this section (including exceeding the Usage Limit set forth below), Licensee must: (i) refrain from any Use of the Software; and (ii) purchase a [commercial paid license](https://www.scylladb.com/scylladb-proprietary-software-license-agreement/) from the Licensor.
i. Restricted to "Never Customers" Only. The license granted under this Agreement is strictly limited to Never Customers. For purposes of this Agreement, a "Never Customer" is an entity (including its Affiliates) that does not have, and has not had within the previous twelve (12) months, a paid commercial subscription, professional services agreement, or any other commercial relationship with Licensor. Satisfaction of the Never Customer criteria is a strict condition precedent to the effectiveness of this License.
ii. Total Prohibition for Existing Commercial Customers. If You (or any of Your Affiliates) are an existing Commercial Customer of Licensor within the last twelve (12) months, no license is deemed to have been offered or extended to You, and any download or installation of the Software by You is unauthorized. This prohibition applies to all deployments, including but not limited to:
(a) existing commercial workloads;
(b) any new use cases, new applications, or new workloads
iii. **No "Hybrid" Usage**. Licensee is expressly prohibited from combining free tier usage under this Agreement with paid commercial units.
If You are a Commercial Customer, all use of the Software across Your entire organization (and any of your Affiliates) must be governed by a valid, paid commercial agreement. Use of the Software under this license by a Commercial Customer (which is not a "Never Customer") shall:
(a) Void this license *ab initio*;
(b) Be deemed a material breach of both this Agreement and any existing commercial terms; and
(c) Entitle Licensor to invoice Licensee for such unauthorized usage at Licensor's standard list prices, retroactive to the date of first use.
Notwithstanding anything to the contrary in the Eligibility or License Limitations sections above a Commercial Customer may use the Software exclusively for non-production purposes, including Continuous Integration (CI), automated testing, and quality assurance environments, provided that such use at all times remains compliant with the Usage Limit.
iv. **Verification**. Licensor reserves the right to audit Licensee's environment and corporate identity to ensure compliance with these eligibility criteria.
For the purposes of this Agreement an "**Affiliate**" means any entity that directly or indirectly controls, is controlled by, or is under common control with a party, where "control" means ownership of more than 50% of the voting stock or decision-making authority
**License Limitations, Restrictions and Obligations:** The license grant above is subject to the following limitations, restrictions, and obligations. If Licensee’s Use of the Software does not comply with the above license grant or the terms of this section (including exceeding the Usage Limit set forth below), Licensee must: (i) refrain from any Use of the Software; and (ii) unless Licensee is a Never Customer, purchase a [commercial paid license](https://www.scylladb.com/scylladb-proprietary-software-license-agreement/) from the Licensor.
* **Updates:** You shall be solely responsible for providing all equipment, systems, assets, access, and ancillary goods and services needed to access and Use the Software. Licensor may modify or update the Software at any time, without notification, in its sole and absolute discretion. After the effective date of each such update, Licensor shall bear no obligation to run, provide or support legacy versions of the Software.
* **"Usage Limit":** Licensee's total overall available storage across all deployments and clusters of the Software and the Licensed Work under this License shall not exceed 10TB and/or an upper limit of 50 VCPUs (hyper threads).
* **IP Markings:** Licensee must retain all copyright, trademark, and other proprietary notices contained in the Software. You will not modify, delete, alter, remove, or obscure any intellectual property, including without limitations licensing, copyright, trademark, or any other notices of Licensor in the Software.
* **License Reproduction:** You must conspicuously display this Agreement on each copy of the Software. If You receive the Software from a third party, this Agreement still applies to Your Use of the Software. You will be responsible for any breach of this Agreement by any such third-party.
* Distribution of any Licensed Works is permitted, provided that: (i) You must include in any Licensed Work prominent notices stating that You have modified the Software, (ii) You include a copy of this Agreement with the Licensed Work, and (iii) You clearly identify all modifications made in the Licensed Work and provides attribution to the Licensor as the original author(s) of the Software.
* **Commercial Use Restrictions:** Licensee may not offer the Software as a software-as-a-service (SaaS) or commercial database-as-as-service (dBaaS) offering. Licensee may not use the Software to compete with Licensor's existing or future products or services. If your Use of the Software does not comply with the requirements currently in effect as described in this License, you must purchase a commercial license from the Licensor, its affiliated entities, or you must refrain from using the Software and all Licensed Work. Furthermore, if You make any written claim of patent infringement relating to the Software, Your patent license for the Software granted under this Agreement terminates immediately.
* **Commercial Use Restrictions:** Licensee may not offer the Software as a software-as-a-service (SaaS) or commercial database-as-as-service (dBaaS) offering. Licensee may not use the Software to compete with Licensor's existing or future products or services. If your Use of the Software does not comply with the requirements currently in effect as described in this License, you must purchase a commercial license from the Licensor, its Affiliated entities, or you must refrain from using the Software and all Licensed Work. Furthermore, if You make any written claim of patent infringement relating to the Software, Your patent license for the Software granted under this Agreement terminates immediately.
* Notwithstanding anything to the contrary, under the License granted hereunder, You shall not and shall not permit others to: (i) transfer the Software or any portions thereof to any other party except as expressly permitted herein; (ii) attempt to circumvent or overcome any technological protection measures incorporated into the Software; (iii) incorporate the Software into the structure, machinery or controls of any aircraft, other aerial device, military vehicle, hovercraft, waterborne craft or any medical equipment of any kind; or (iv) use the Software or any part thereof in any unlawful, harmful or illegal manner, or in a manner which infringes third parties’ rights in any way, including intellectual property rights.
**Monitoring; Audit**
@@ -41,14 +69,14 @@ The terms "**You**" or "**Licensee**" refer to any individual accessing or using
**Indemnity; Disclaimer; Limitation of Liability**
* **Indemnity:** Licensee hereby agrees to indemnify, defend and hold harmless Licensor and its affiliates from any losses or damages incurred due to a third party claim arising out of: (i) Licensee’s breach of this Agreement; (ii) Licensee’s negligence, willful misconduct or violation of law, or (iii) Licensee’s products or services.
* **Indemnity:** Licensee hereby agrees to indemnify, defend and hold harmless Licensor and its Affiliates from any losses or damages incurred due to a third party claim arising out of: (i) Licensee’s breach of this Agreement; (ii) Licensee’s negligence, willful misconduct or violation of law, or (iii) Licensee’s products or services.
* DISCLAIMER OF WARRANTIES: LICENSEE AGREES THAT LICENSOR HAS MADE NO EXPRESS WARRANTIES REGARDING THE SOFTWARE AND THAT THE SOFTWARE IS BEING PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. LICENSOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THE SOFTWARE, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE; TITLE; MERCHANTABILITY; OR NON-INFRINGEMENT OF THIRD PARTY RIGHTS. LICENSOR DOES NOT WARRANT THAT THE SOFTWARE WILL OPERATE UNINTERRUPTED OR ERROR FREE, OR THAT ALL ERRORS WILL BE CORRECTED. LICENSOR DOES NOT GUARANTEE ANY PARTICULAR RESULTS FROM THE USE OF THE SOFTWARE, AND DOES NOT WARRANT THAT THE SOFTWARE IS FIT FOR ANY PARTICULAR PURPOSE.
* LIMITATION OF LIABILITY: TO THE FULLEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW, IN NO EVENT WILL LICENSOR AND/OR ITS AFFILIATES, EMPLOYEES, OFFICERS AND DIRECTORS BE LIABLE TO LICENSEE FOR (I) ANY LOSS OF USE OR DATA; INTERRUPTION OF BUSINESS; OR ANY INDIRECT; SPECIAL; INCIDENTAL; OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING LOST PROFITS); AND (II) ANY DIRECT DAMAGES EXCEEDING THE TOTAL AMOUNT OF ONE THOUSAND US DOLLARS ($1,000). THE FOREGOING PROVISIONS LIMITING THE LIABILITY OF LICENSOR SHALL APPLY REGARDLESS OF THE FORM OR CAUSE OF ACTION, WHETHER IN STRICT LIABILITY, CONTRACT OR TORT.
**Proprietary Rights; No Other Rights**
* **Ownership:** Licensor retains sole and exclusive ownership of all rights, interests and title in the Software and any scripts, processes, techniques, methodologies, inventions, know-how, concepts, formatting, arrangements, visual attributes, ideas, database rights, copyrights, patents, trade secrets, and other intellectual property related thereto, and all derivatives, enhancements, modifications and improvements thereof. Except for the limited license rights granted herein, Licensee has no rights in or to the Software and/ or Licensor’s trademarks, logo, or branding and You acknowledge that such Software, trademarks, logo, or branding is the sole property of Licensor.
* **Feedback:** Licensee is not required to provide any suggestions, enhancement requests, recommendations or other feedback regarding the Software ("Feedback"). If, notwithstanding this policy, Licensee submits Feedback, Licensee understands and acknowledges that such Feedback is not submitted in confidence and Licensor assumes no obligation, expressed or implied, by considering it. All right in any trademark or logo of Licensor or its affiliates and You shall make no claim of right to the Software or any part thereof to be supplied by Licensor hereunder and acknowledges that as between Licensor and You, such Software is the sole proprietary, title and interest in and to Licensor.such Feedback shall be assigned to, and shall become the sole and exclusive property of, Licensor upon its creation.
* **Feedback:** Licensee is not required to provide any suggestions, enhancement requests, recommendations or other feedback regarding the Software ("Feedback"). If, notwithstanding this policy, Licensee submits Feedback, Licensee understands and acknowledges that such Feedback is not submitted in confidence and Licensor assumes no obligation, expressed or implied, by considering it. All right in any trademark or logo of Licensor or its Affiliates and You shall make no claim of right to the Software or any part thereof to be supplied by Licensor hereunder and acknowledges that as between Licensor and You, such Software is the sole proprietary, title and interest in and to Licensor.such Feedback shall be assigned to, and shall become the sole and exclusive property of, Licensor upon its creation.
* Except for the rights expressly granted to You under this Agreement, You are not granted any other licenses or rights in the Software or otherwise. This Agreement constitutes the entire agreement between You and the Licensor with respect to the subject matter hereof and supersedes all prior or contemporaneous communications, representations, or agreements, whether oral or written.
* **Third-Party Software:** Customer acknowledges that the Software may contain open and closed source components (“OSS Components”) that are governed separately by certain licenses, in each case as further provided by Company upon request. Any applicable OSS Component license is solely between Licensee and the applicable licensor of the OSS Component and Licensee shall comply with the applicable OSS Component license.
* If any provision of this Agreement is held to be invalid or unenforceable, such provision shall be struck and the remaining provisions shall remain in full force and effect.
@@ -56,7 +84,7 @@ The terms "**You**" or "**Licensee**" refer to any individual accessing or using
**Miscellaneous**
* **Miscellaneous:** This Agreement may be modified at any time by Licensor, and constitutes the entire agreement between the parties with respect to the subject matter hereof. Licensee may not assign or subcontract its rights or obligations under this Agreement. This Agreement does not, and shall not be construed to create any relationship, partnership, joint venture, employer-employee, agency, or franchisor-franchisee relationship between the parties.
* **Modifications**: Licensor reserves the right to modify this Agreement at any time. Changes will be effective upon posting to the Website or within the Software repository. Continued use of the Software after such changes constitutes acceptance.
* **Governing Law & Jurisdiction:** This Agreement shall be governed and construed in accordance with the laws of Israel, without giving effect to their respective conflicts of laws provisions, and the competent courts situated in Tel Aviv, Israel, shall have sole and exclusive jurisdiction over the parties and any conflict and/or dispute arising out of, or in connection to, this Agreement
co_returnapi_error::request_limit_exceeded(format("too many in-flight requests (configured via max_concurrent_requests_per_shard): {}",_pending_requests.get_count()));
co_returnapi_error::request_limit_exceeded(format("too many in-flight requests (configured via max_concurrent_requests_per_shard): {}",_pending_requests.get_count()));
"description":"The snapshot tag to delete. If omitted, all snapshots are removed.",
"required":false,
"allowMultiple":false,
"type":"string",
@@ -751,7 +751,7 @@
},
{
"name":"kn",
"description":"Comma-separated keyspaces name that their snapshot will be deleted",
"description":"Comma-separated list of keyspace names to delete snapshots from. If omitted, snapshots are deleted from all keyspaces.",
"required":false,
"allowMultiple":false,
"type":"string",
@@ -759,7 +759,7 @@
},
{
"name":"cf",
"description":"an optional table name that its snapshot will be deleted",
"description":"A table name used to filter which table's snapshots are deleted. If omitted or empty, snapshots for all tables are eligible. When provided together with 'kn', the table is looked up in each listed keyspace independently. For secondary indexes, the logical index name (e.g. 'myindex') can be used and is resolved automatically.",
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
