Include non-primary key restrictions (e.g. regular column filters) in
the filter JSON sent to the Vector Store service. Previously only
partition key and clustering column restrictions were forwarded, so
filtering on regular columns was silently ignored.
Add get_nonprimary_key_restrictions() getter to statement_restrictions.
Add unit tests for non-primary key equality, range, and bind marker
restrictions in filter_test.
Fixes: SCYLLADB-970
Closesscylladb/scylladb#29019
For counter updates, use a counter ID that is constructed from the
node's rack instead of the node's host ID.
A rack can have at most two active tablet replicas at a time: a single
normal tablet replica, and during tablet migration there are two active
replicas, the normal and pending replica. Therefore we can have two
unique counter IDs per rack that are reused by all replicas in the rack.
We construct the counter ID from the rack UUID, which is constructed
from the name "dc:rack". The pending replica uses a deterministic
variation of the rack's counter ID by negating it.
This improves the performance and size of counter cells by having less
unique counter IDs and less counter shards in a counter cell.
Previously the number of counter shards was the number of different
host_id's that updated the counter, which can be typically the number of
nodes in the cluster and continue growing indefinitely when nodes are
replaced. with the rack-based counter id the number of counter shards
will be at most twice the number of different racks (including removed
racks, which should not be significant).
Fixes SCYLLADB-356
backport not needed - an enhancement
Closesscylladb/scylladb#28901
* github.com:scylladb/scylladb:
docs/dev: add counters doc
counters: reuse counter IDs by rack
Replace move_to_shard()/move_to_host() with as_bounce()/target_shard()/
target_host() to clarify the interface after bounce was extended to
support cross-node bouncing.
- Add virtual as_bounce() returning const bounce* to the base class
(nullptr by default, overridden in bounce to return this), replacing
the virtual move_to_shard() which conflated bounce detection with
shard access
- Rename move_to_shard() -> target_shard() (now non-virtual, returns
unsigned directly) and move_to_host() -> target_host() on bounce
- Replace dynamic_pointer_cast with static_pointer_cast at call sites
that already checked as_bounce()
- Move forward declarations of message types before the virtual
methods so as_bounce() can reference bounce
Fixes: SCYLLADB-1066
Closesscylladb/scylladb#29367
Motivation
----------
Since strongly consistent tables are based on the concept of Raft
groups, operations on them can get stuck for indefinite amounts of
time. That may be problematic, and so we'd like to implement a way
to cancel those operations at suitable times.
Description of solution
-----------------------
The situations we focus on are the following:
* Timed-out queries
* Leader changes
* Tablet migrations
* Table drops
* Node shutdowns
We handle each of them and provide validation tests.
Implementation strategy
-----------------------
1. Auxiliary commits.
2. Abort operations on timeout.
3. Abort operations on tablet removal.
4. Extend `client_state`.
5. Abort operation on shutdown.
6. Help `state_machine` be aborted as soon as possible.
Tests
-----
We provide tests that validate the correctness of the solution.
The total time spent on `test_strong_consistency.py`
(measured on my local machine, dev mode):
Before:
```
real 0m31.809s
user 1m3.048s
sys 0m21.812s
```
After:
```
real 0m34.523s
user 1m10.307s
sys 0m27.223s
```
The incremental differences in time can be found in the commit messages.
Fixes SCYLLADB-429
Backport: not needed. This is an enhancement to an experimental feature.
Closesscylladb/scylladb#28526
* github.com:scylladb/scylladb:
service: strong_consistency: Abort state_machine::apply when aborting server
service: strong_consistency: Abort ongoing operations when shutting down
service: client_state: Extend with abort_source
service: strong_consistency: Handle abort when removing Raft group
service: strong_consistency: Abort Raft operations on timeout
service: strong_consistency: Use timeout when mutating
service: strong_consistency: Fix indentation
service: strong_consistency: Enclose coordinator methods with try-catch
service: strong_consistency: Crash at unexpected exception
test: cluster: Extract default config & cmdline in test_strong_consistency.py
This reverts commit 8b4a91982b.
Two commits independently added rolling_max_tracker_test to test/boost/CMakeLists.txt:
8b4a919 cmake: add missing rolling_max_tracker_test and symmetric_key_test
f3a91df test/cmake: add missing tests to boost test suite
The second was merged two days after the first. They didn't conflict on
code-level and applied cleanly resulting in a duplicate add_scylla_test()
entries that breaks the CMake build:
CMake Error: add_executable cannot create target
"test_boost_rolling_max_tracker_test" because another target
with the same name already exists.
Remove the duplicate.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Reported-by: Łukasz Paszkowski <lukasz.paszkowski@scylladb.com>
The PR contains more code cleanups, mostly in gossiper. Dropping more gossiper state leaving only NORMAL and SHUTDOWN. All other states are checked against topology state. Those two are left because SHUTDOWN state is propagated through gossiper only and when the node is not in SHUTDOWN it should be in some other state.
No need to backport. Cleanups.
Closesscylladb/scylladb#29129
* https://github.com/scylladb/scylladb:
storage_service: cleanup unused code
storage_service: simplify get_peer_info_for_update
gossiper: send shutdown notifications in parallel
gms: remove unused code
virtual_tables: no need to call gossiper if we already know that the node is in shutdown
gossiper: print node state from raft topology in the logs
gossiper: use is_shutdown instead of code it manually
gossiper: mark endpoint_state(inet_address ip) constructor as explicit
gossiper: remove unused code
gossiper: drop last use of LEFT state and drop the state
gossiper: drop unused STATUS_BOOTSTRAPPING state
gossiper: rename is_dead_state to is_left since this is all that the function checks now.
gossiper: use raft topology state instead of gossiper one when checking node's state
storage_service: drop check_for_endpoint_collision function
storage_service: drop is_first_node function
gossiper: remove unused REMOVED_TOKEN state
gossiper: remove unused advertise_token_removed function
Cassandra's native vector index type is StorageAttachedIndex (SAI). Libraries such as CassIO, LangChain, and LlamaIndex generate `CREATE CUSTOM INDEX` statements using the SAI class name. Previously, ScyllaDB rejected these with "Non-supported custom class".
This PR adds compatibility so that SAI-style CQL statements work on ScyllaDB without modification.
1. **test: enable SAI_VECTOR_ALLOW_CUSTOM_PARAMETERS for Cassandra tests**
Enables the `SAI_VECTOR_ALLOW_CUSTOM_PARAMETERS` Cassandra system property so that `search_beam_width` tests pass against Cassandra 5.0.7.
2. **test: modernize vector index test comments and fix xfail**
Updates test comments from "Reproduces" to "Validates fix for" for clarity, and converts the `test_ann_query_with_pk_restriction` xfail into a stripped-down CREATE INDEX syntax test (removing unused INSERT/SELECT lines). Removes the redundant `test_ann_query_with_non_pk_restriction` test.
3. **cql: add Cassandra SAI (StorageAttachedIndex) compatibility**
Core implementation: the SAI class name is detected and translated to ScyllaDB's native `vector_index`. The fully-qualified class name (`org.apache.cassandra.index.sai.StorageAttachedIndex`) requires exact case; short names (`StorageAttachedIndex`, `sai`) are matched case-insensitively — matching Cassandra's behavior. Non-vector and multi-column SAI targets are rejected with clear errors. Adds `skip_on_scylla_vnodes` fixture, SAI compatibility docs, and the Cassandra compatibility table entry (split into "SAI general" vs "SAI for vector search").
4. **cql: accept source_model option for Cassandra SAI compatibility**
The `source_model` option is a Cassandra SAI property used by Cassandra libraries (e.g., CassIO) to tag vector indexes with the name of the embedding model. ScyllaDB accepts it for compatibility but does not use it — the validator is a no-op lambda. The option is preserved in index metadata and returned in DESCRIBE INDEX output.
- `cql3/statements/create_index_statement.cc`: SAI class detection and rewriting logic
- `index/secondary_index_manager.cc`: case-insensitive class name lookup (lowercasing restored before `classes.find()`)
- `index/vector_index.cc`: `source_model` accepted as a valid option with no-op validator
- `docs/cql/secondary-indexes.rst`: SAI compatibility documentation with `source_model` table row
- `docs/using-scylla/cassandra-compatibility.rst`: SAI entry split into general (not supported) and vector search (supported)
- `test/cqlpy/conftest.py`: `scylla_with_tablets` renamed to `skip_on_scylla_vnodes`
- `test/cqlpy/test_vector_index.py`: SAI tests inlined (no constants), `check_bad_option()` helper for numeric validation, uppercase class name test, merged `source_model` tests with DESCRIBE check
| Backend | Passed | Skipped | Failed |
|--------------------|--------|---------|--------|
| ScyllaDB (dev) | 42 | 0 | 0 |
| Cassandra 5.0.7 | 16 | 26 | 0 |
None: new feature.
Fixes: SCYLLADB-239
Closesscylladb/scylladb#28645
* github.com:scylladb/scylladb:
cql: accept source_model option and show options in DESCRIBE
cql: add Cassandra SAI (StorageAttachedIndex) compatibility
test: modernize vector index test comments and fix xfail
test: enable SAI_VECTOR_ALLOW_CUSTOM_PARAMETERS for Cassandra tests
Add system.tablets to the set of system resources that can be
accessed with the VECTOR_SEARCH_INDEXING permission.
Fixes: VECTOR-605
Closesscylladb/scylladb#29397
Accept the Cassandra SAI 'source_model' option for vector indexes.
This option is used by Cassandra libraries (e.g., CassIO, LangChain)
to tag vector indexes with the name of the embedding model that
produced the vectors.
ScyllaDB does not use the source_model value but stores it and
includes it in the DESCRIBE INDEX output for Cassandra compatibility.
Additionally, extend vector_index::describe() to emit a
WITH OPTIONS = {...} clause containing all user-provided index options
(filtering out system keys: target, class_name, index_version).
This makes options like similarity_function, source_model, etc.
visible in DESCRIBE output.
Libraries such as CassIO, LangChain, and LlamaIndex create vector
indexes using Cassandra's StorageAttachedIndex (SAI) class name.
This commit lets ScyllaDB accept these statements without modification.
When a CREATE CUSTOM INDEX statement specifies an SAI class name on a
vector column, ScyllaDB automatically rewrites it to the native
vector_index implementation. Accepted class names (case-insensitive):
- org.apache.cassandra.index.sai.StorageAttachedIndex
- StorageAttachedIndex
- sai
SAI on non-vector columns is rejected with a clear error directing
users to a secondary index instead.
The SAI detection and rewriting logic is extracted into a dedicated
static function (maybe_rewrite_sai_to_vector_index) to keep the
already-long validate_while_executing method manageable.
Multi-column (local index) targets and nonexistent columns are
skipped with continue — the former are treated as filtering columns
by vector_index::check_target(), and the latter are caught later by
vector_index::validate().
Tests that exercise features common to both backends (basic creation,
similarity_function, IF NOT EXISTS, bad options, etc.) now use the
SAI class name with the skip_on_scylla_vnodes fixture so they run
against both ScyllaDB and Cassandra. ScyllaDB-specific tests continue
to use USING 'vector_index' with scylla_only.
- Change 'Reproduces' to 'Validates fix for' in test comments to
reflect that the referenced issues are already fixed.
- Condense the VECTOR-179 comment to two lines.
- Replace the xfailed test_ann_query_with_restriction_works_only_on_pk
with a focused test (test_ann_query_with_pk_restriction) that creates
a vector index on a table with a PK column restriction, validating
the VECTOR-374 fix.
Every time someone modifies the build system — adding a source file, changing a compilation flag, or wiring a new test — the change tends to land in only one of our two build systems (configure.py or CMake). Over time this causes three classes of problems:
1. **CMake stops compiling entirely.** Missing defines, wrong sanitizer flags, or misplaced subdirectory ordering cause hard build failures that are only discovered when someone tries to use CMake (e.g. for IDE integration).
2. **Missing build targets.** Tests or binaries present in configure.py are never added to CMake, so `cmake --build` silently skips them. This PR fixes several such cases (e.g. `symmetric_key_test`, `auth_cache_test`, `sstable_tablet_streaming`).
3. **Missing compilation units in targets.** A `.cc` file is added to a test binary in one system but not the other, causing link errors or silently omitted test coverage.
To fix the existing drift and prevent future divergence, this series:
**Adds a build-system comparison script**
(`scripts/compare_build_systems.py`) that configures both systems into a temporary directory, parses their generated `build.ninja` files, and compares per-file compilation flags, link target sets, and per-target libraries. configure.py is treated as the baseline; CMake must match it. The script supports a `--ci` mode suitable for gating PRs that touch
build files.
**Fixes all current mismatches** found by the script:
- Mode flag alignment in `mode.common.cmake` and `mode.Coverage.cmake`
(sanitizer flags, `-fno-lto`, stack-usage warnings, coverage defines).
- Global define alignment (`SEASTAR_NO_EXCEPTION_HACK`, `XXH_PRIVATE_API`,
`BOOST_ALL_DYN_LINK`, `SEASTAR_TESTING_MAIN` placement).
- Seastar build configuration (shared vs static per mode, coverage
sanitizer link options).
- Abseil sanitizer flags (`-fno-sanitize=vptr`).
- Missing test targets in `test/boost/CMakeLists.txt`.
- Redundant per-test flags now covered by global settings.
- Lua library resolution via a custom `cmake/FindLua.cmake` using
pkg-config, matching configure.py's approach.
**Adds documentation** (`docs/dev/compare-build-systems.md`) describing how to run the script and interpret its output.
No backport needed — this is build infrastructure improvement only.
Closesscylladb/scylladb#29273
* github.com:scylladb/scylladb:
scripts: remove lua library rename workaround from comparison script
cmake: add custom FindLua using pkg-config to match configure.py
test/cmake: add missing tests to boost test suite
test/cmake: remove per-test LTO disable
cmake: add BOOST_ALL_DYN_LINK and strip per-component defines
cmake: move SEASTAR_TESTING_MAIN after seastar and abseil subdirs
cmake: add -fno-sanitize=vptr for abseil sanitizer flags
cmake: align Seastar build configuration with configure.py
cmake: align global compile defines and options with configure.py
cmake: fix Coverage mode in mode.Coverage.cmake
cmake: align mode.common.cmake flags with configure.py
configure.py: add sstable_tablet_streaming to combined_tests
docs: add compare-build-systems.md
scripts: add compare_build_systems.py to compare ninja build files
Add .set_skip_when_empty() to all error-path metrics in the tracing
module. Tracing itself is not a commonly used feature, making all of
these metrics almost always zero:
Tier 1 (very rare - corruption/schema issues):
- tracing_keyspace_helper::bad_column_family_errors: tracing schema
missing or incompatible, should never happen post-bootstrap
- tracing::trace_errors: internal error building trace parameters
Tier 2 (overload - tracing backend saturated):
- tracing::dropped_sessions: too many pending sessions
- tracing::dropped_records: too many pending records
Tier 3 (general tracing write errors):
- tracing_keyspace_helper::tracing_errors: errors during writes to
system_traces keyspace
Since tracing is an opt-in feature that most deployments rarely use,
all five metrics are almost always zero and create unnecessary
reporting overhead.
AI-Assisted: yes
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
Closesscylladb/scylladb#29346
Add a documentation of the counters feature implementation in
docs/dev/counters.md.
The documentation is taken from the wiki and updated according to the
current state of the code - legacy details are removed, and a section
about the counter id is added.
For counter updates, use a counter ID that is constructed from the
node's rack instead of the node's host ID.
A rack can have at most two active tablet replicas at a time: a single
normal tablet replica, and during tablet migration there are two active
replicas, the normal and pending replica. Therefore we can have two
unique counter IDs per rack that are reused by all replicas in the rack.
We construct the counter ID from the rack UUID, which is constructed
from the name "dc:rack". The pending replica uses a deterministic
variation of the rack's counter ID by negating it.
This improves the performance and size of counter cells by having less
unique counter IDs and less counter shards in a counter cell.
Previously the number of counter shards was the number of different
host_id's that updated the counter, which can be typically the number of
nodes in the cluster and continue growing indefinitely when nodes are
replaced. with the rack-based counter id the number of counter shards
will be at most twice the number of different racks (including removed
racks, which should not be significant).
Fixes SCYLLADB-356
Add .set_skip_when_empty() to four metrics in replica/database.cc that
are only incremented on very rare error paths and are almost always zero:
- database::dropped_view_updates: view updates dropped due to overload.
NOTE: this metric appears to never be incremented in the current
codebase and may be a candidate for removal.
- database::multishard_query_failed_reader_stops: documented as a 'hard
badness counter' that should always be zero. NOTE: no increment site
was found in the current codebase; may be a candidate for removal.
- database::multishard_query_failed_reader_saves: documented as a 'hard
badness counter' that should always be zero.
- database::total_writes_rejected_due_to_out_of_space_prevention: only
fires when disk utilization is critical and user table writes are
disabled, a very rare operational state.
These metrics create unnecessary reporting overhead when they are
perpetually zero. set_skip_when_empty() suppresses them from metrics
output until they become non-zero.
AI-Assisted: yes
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
Closesscylladb/scylladb#29345
After obtaining the CQL response, check if its actual size exceeds the initially acquired memory permit. If so, acquire additional semaphore units and adopt them into the permit, ensuring accurate memory accounting for large responses.
Additionally, move the permit into a .then() continuation so that the semaphore units are kept alive until write_message finishes, preventing premature release of memory permit. This is especially important with slow networks and big responses when buffers can accumulate and deplete a node's memory.
Fixes: https://scylladb.atlassian.net/browse/SCYLLADB-1306
Related https://scylladb.atlassian.net/browse/SCYLLADB-740
Backport: all supported versions
Closesscylladb/scylladb#29288
* github.com:scylladb/scylladb:
transport: add per-service-level pending response memory metric
transport: hold memory permit until response write completes
transport: account for response size exceeding initial memory estimate
Add .set_skip_when_empty() to four metrics in the db module that are
only incremented on very rare error paths and are almost always zero:
- cache::pinned_dirty_memory_overload: described as 'should sit
constantly at 0, nonzero is indicative of a bug'
- corrupt_data::entries_reported: only fires on actual data corruption
- hints::corrupted_files: only fires on on-disk hint file corruption
- rate_limiter::failed_allocations: only fires when the rate limiter
hash table is completely full and gives up allocating, requiring
extreme cardinality pressure
These metrics create unnecessary reporting overhead when they are
perpetually zero. set_skip_when_empty() suppresses them from metrics
output until they become non-zero.
AI-Assisted: yes
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
Closesscylladb/scylladb#29344
get_live_members function called is_shutdown which inet_address
argument, which caused temporary endpoint_state to be created. Fix
it by prohibiting implicit conversion and calling the correct
is_shutdown function instead.
The decommission sets left gossiper state only to prevent shutdown
notification be issued by the node during shutdown. Since the
notification code now checks the state in raft topology this is no
longer needed.
The state machine used by strongly consistent tablets may block on a
read barrier if the local schema is insufficient to resolve pending
mutations [1]. To deal with that, we perform a read barrier that may
block for a long time.
When a strongly consistent tablet is being removed, we'd like to cancel
all ongoing executions of `state_machine::apply`: the shard is no
longer responsible for the tablet, so it doesn't matter what the outcome
is.
---
In the implementation, we abort the operations by simply throwing
an exception from `state_machine::apply` and not doing anything.
That's a red flag considering that it may lead to the instance
being killed on the spot [2].
Fortunately for us, strongly consistent tables use the default Raft
server implementation, i.e. `raft::server_impl`, which actually
handles one type of an exception thrown by the method: namely,
`abort_requested_exception`, which is the default exception thrown
by `seastar::abort_source` [3]. We leverage this property.
---
Unfortunately, `raft::server_impl::abort` isn't perfectly suited for
us. If we look into its code, we'll see that the relevant portion of
the procedure boils down to three steps:
1. Prevent scheduling adding new entries.
2. Wait for the applier fiber.
3. Abort the state machine.
Since aborting the state machine happens only after the applier fiber
has already finished, there will no longer be anything to abort. Either
all executions of `state_machine::apply` have already finished, or they
are hanging and we cannot do anything.
That's a pre-existing problem that we won't be solving here (even
though it's possible). We hope the problem will be solved, and it seems
likely: the code suggests that the behavior is not intended. For more
details, see e.g. [4].
---
We provide two validation tests. They simulate the abortion of
`state_machine::apply` in two different scenarios:
* when the table is dropped (which should also cover the case of tablet
migration),
* when the node is shutting down.
The value of the tests isn't high since they don't ensure that the
state of the group is still valid (though it should be), nor do they
perform any other check. Instead, we rely on the testing framework to
spot any anomalies or errors. That's probably the best we can do at
the moment.
Unfortunately, both tests are marked as skipped becuause of the current
limitations of `raft::server_impl::abort` described above and in [4].
References:
[1] 4c8dba1
[2] See the description of `raft::state_machine` in `raft/raft.hh`.
[3] See `server_impl::applier_fiber` in `raft/server.cc`.
[4] SCYLLADB-1056
These changes are complementary to those from a recent commit where we
handled aborting ongoing operations during tablet events, such as
tablet migration. In this commit, we consider the case of shutting down
a node.
When a node is shutting down, we eventually close the connections. When
the client can no longer get a response from the server, it makes no
sense to continue with the queries. We'd like to cancel them at that
point.
We leverage the abort source passed down via `client_state` down to
the strongly consistent coordinator. This way, the transport layer can
communicate with it and signal that the queries should be canceled.
The abort source is triggered by the CQL server (cf.
`generic_server::server::{stop,shutdown}`).
---
Note that this is not an optional change. In fact, if we don't abort
those requests, we might hang for an indefinite amount of time when
executing the following code in `main.cc`:
```
// Register at_exit last, so that storage_service::drain_on_shutdown will be called first
auto do_drain = defer_verbose_shutdown("local storage", [&ss] {
ss.local().drain_on_shutdown().get();
});
```
The problem boils down to the fact that `generic_server::server::stop`
will wait for all connections to be closed, but that won't happen until
all ongoing operations (at least those to strongly consistent tables)
are finished.
It's important to highlight that even though we hang on this, the
client can no longer get any response. Thus, it's crucial that at that
point we simply abort ongoing operations to proceed with the rest of
shutdown.
---
Two tests are added to verify that the implementation is correct:
one focusing on local operations, the other -- on a forwarded write.
Difference in time spent on the whole test file
`test_strong_consistency.py` on my local machine, in dev mode:
Before:
```
real 0m31.775s
user 1m4.475s
sys 0m22.615s
```
After:
```
real 0m32.024s
user 1m10.751s
sys 0m23.871s
```
Individual runs of the added tests:
test_queries_when_shutting_down:
```
real 0m12.818s
user 0m36.726s
sys 0m4.577s
```
test_abort_forwarded_write_upon_shutdown:
```
real 0m12.930s
user 0m36.622s
sys 0m4.752s
```
We make `client_state` store a pointer to an `abort_source`. This will
be useful in the following commit that will implement aborting ongoing
requests to strongly consistent tables upon connection shutdowns.
It might also be useful in some other places in the code in the future.
We set the abort source for client states in relevant places.
When a strongly consistent Raft group is being removed, it means one of
the following cases:
(A) The node is shutting down and it's simply part of the the shutdown
procedure.
(B) The tablet is somehow leaving the replica. For example, due to:
- Tablet migration
- Tablet split/merge
- Tablet removal (e.g. because the table is dropped)
In this commit, we focus on case (A). Case (B) will be handled in the
following one.
---
The changes in the code are literally none, and there's a reason to it.
First, let's note that we've already implemented abortion of timed-out
requests. There is a limit to how long a query can run and sooner or
later it will finish, regardless of what we do.
Second, we need to ask ourselves if the cases we're considering in this
commit (i.e. case (B)) is a situation where we'd like to speed up the
process. The answer is no.
Tablet migrations are effectively internal operations that are invisible
to the users. User requests are, quite obviously, the opposite of that.
Because of that, we want to patiently wait for the queries to finish or
time out, even though it's technically possible to lead to an abort
earlier.
Lastly, the changes in the code that actually appear in this commit are
not completely irrelevant either. We consider the important case of
the `leader_info_updater` fiber and argue that it's safe to not pass
any abort source to the Raft methods used by it.
---
Unfortunately, we don't have tablet migrations implemented yet [1],
so our testing capabilities are limited. Still, we provide a new test
that corresponds to case (B) described above. We simulate a tablet
migration by dropping a table and observe how reads and writes behave
in such a situation. There's no extremely careful validation involved
there, but that's what we can have for the time being.
Difference in time spent on the whole test file
`test_strong_consistency.py` on my local machine, in dev mode:
Before:
```
real 0m30.841s
user 1m3.294s
sys 0m21.091s
```
After:
```
real 0m31.775s
user 1m4.475s
sys 0m22.615s
```
The time spent on the new test only:
```
real 0m5.264s
user 0m34.646s
sys 0m3.374s
```
References:
[1] SCYLLADB-868
If a query, either a write, or a read to a strongly consistent table,
times out, we immediately abort the operation and throw an exception.
Unfortunately, due to the inconsistency in exception types thrown
on timeout by the many methods we use in the code, it results in
pretty messy `try-catch` clauses. Perhaps there's a better alternative
to this, but it's beyond the scope of this work, so we leave it as-is.
We provide a validation test that consists of three cases corresponding
to reads, writes, and waiting for the leader. They verify that the code
works as expected in all affected places.
A comparison of time spent on the whole `test_strong_consistency.py` on
my local machine, in dev mode:
Before:
```
real 0m32.185s
user 0m55.391s
sys 0m15.745s
```
After:
```
real 0m30.841s
user 1m3.294s
sys 0m21.091s
```
The time spent on the new test only:
```
real 0m7.077s
user 0m35.359s
sys 0m3.717s
```
- Document Alternator (DynamoDB-compatible API) auditing support in
the operator-facing auditing guide (docs/operating-scylla/security/auditing.rst)
- Cover operation-to-category mapping, operation field format,
keyspace/table filtering, and audit log examples
- Document the audit_tables=alternator.<table> shorthand format
- Minor wording improvements throughout (Scylla -> ScyllaDB,
clarify default audit backend)
Closesscylladb/scylladb#29231
We remove the inconsistency between reads and writes to strongly
consistent tables. Before the commit, only reads used a timeout.
Now, writes do as well.
Although the parameter isn't used yet, that will change in the following
commit. This is a prerequisite for it.
We enclose `coordinator::{mutate,query}` with `try-catch` clauses. They
do nothing at the moment, but we'll use them later. We do this now to
avoid noise in the upcoming commits.
We'll fix the indentation in the following commit.
The loop shouldn't throw any other exception than the ones already
covered by the `catch` claues. Crash, at least when
`abort_on_internal_error` is set, if we catch any other type since
that may be a sign of a bug.
All used configs and cmdlines share the same values. Let's extract them
to avoid repeating them every time a new test is written. Those options
should be enabled for all tests in the file anyway.
Fixes#29043 with the following docs changes:
- docs/dev/system-keyspaces.md: Added a new file that documents all keyspaces created internally
Closesscylladb/scylladb#29044
Spreading db::config around and making all services depend on it is not nice. Most other service that need configuration provide their own config that's populated from db::config in main.cc/cql_test_env.cc and use it, not the global config.
This PR does the same for repair_service.
Enhancing components dependencies, not backporting
Closesscylladb/scylladb#29153
* github.com:scylladb/scylladb:
repair: Remove db/config.hh from repair/*.cc files
repair: Move repair_multishard_reader options onto repair_service::config
repair: Move critical_disk_utilization_level onto repair_service::config
repair: Move repair_partition_count_estimation_ratio onto repair_service::config
repair: Move repair_hints_batchlog_flush_cache_time_in_ms onto repair_service::config
repair: Move enable_small_table_optimization_for_rbno onto repair_service::config
repair: Introduce repair_service::config
While working on benchmarks for strong consistency we noticed that the raft logic attempted to take snapshots during the benchmark. Snapshot transfer is not implemented for strong consistency yet and the methods that take or transfer snapshots throw exceptions. This causes the raft groups to stop working completely.
While implementing snapshot transfers is out of scope, we can implement some mitigations now to stop the tests from breaking:
- The first commit adjusts the configuration options. First, it disables periodic snapshotting (i.e. creating a snapshot every X log entries). Second, it increases the memory threshold for the raft log before which a snapshot is created from 2MB to 10MB.
- The second commit relaxes the take snapshot / drop snapshot methods and makes it possible to actually use them - they are no-ops. It is still forbidden to transfer snapshots.
I am including both commits because applying only the first one didn't completely prevent the issue from occurring when testing locally.
Refs: SCYLLADB-1115
Strong consistency is experimental, no need for backport.
Closesscylladb/scylladb#29189
* github.com:scylladb/scylladb:
strong_consistency: fake taking and dropping snapshots
strong_consistency: adjust limits for snapshots
This commit removes references ScyllaDB versions ("Since x.y")
from the ScyllaDB documentation on Docker Hub, as they are
redundant and confusing (some versions are super ancient).
Fixes SCYLLADB-1212
Closesscylladb/scylladb#29204
The code responds ealry with READY message, but lack some necessary set up, namely:
* update_scheduling_group(): without it, the connection runs under the default scheduling group instead of the one mapped to the user's service level.
* on_connection_ready(): without it, the connection never releases its slot in the uninitialized-connections concurrency semaphore (acquired at connection creation), leaking one unit per cert-authenticated connection for the lifetime of the connection.
* _authenticating = false / _ready = true: without them, system.clients reports connection_stage = AUTHENTICATING forever instead of READY (not critical, but not nice either)
The PR fixes it and adds a regression test, that (for sanity) also covers AllowAll and Password authrticators
Fixes SCYLLADB-1226
Present since 2025.1, probably worth backporting
Closesscylladb/scylladb#29220
* github.com:scylladb/scylladb:
transport: fix process_startup cert-auth path missing connection-ready setup
transport: test that connection_stage is READY after auth via all process_startup paths
During incremental repair, each tablet replica holds three SSTable views:
UNREPAIRED, REPAIRING, and REPAIRED. The repair lifecycle is:
1. Replicas snapshot unrepaired SSTables and mark them REPAIRING.
2. Row-level repair streams missing rows between replicas.
3. mark_sstable_as_repaired() runs on all replicas, rewriting the
SSTables with repaired_at = sstables_repaired_at + 1 (e.g. N+1).
4. The coordinator atomically commits sstables_repaired_at=N+1 and
the end_repair stage to Raft, then broadcasts
repair_update_compaction_ctrl which calls clear_being_repaired().
The bug lives in the window between steps 3 and 4. After step 3, each
replica has on-disk SSTables with repaired_at=N+1, but sstables_repaired_at
in Raft is still N. The classifier therefore sees:
is_repaired(N, sst{repaired_at=N+1}) == false
sst->being_repaired == null (lost on restart, or not yet set)
and puts them in the UNREPAIRED view. If a new write arrives and is
flushed (repaired_at=0), STCS minor compaction can fire immediately and
merge the two SSTables. The output gets repaired_at = max(N+1, 0) = N+1
because compaction preserves the maximum repaired_at of its inputs.
Once step 4 commits sstables_repaired_at=N+1, the compacted output is
classified REPAIRED on the affected replica even though it contains data
that was never part of the repair scan. Other replicas, which did not
experience this compaction, classify the same rows as UNREPAIRED. This
divergence is never healed by future repairs because the repaired set is
considered authoritative. The result is data resurrection: deleted rows
can reappear after the next compaction that merges unrepaired data with the
wrongly-promoted repaired SSTable.
The fix has two layers:
Layer 1 (in-memory, fast path): mark_sstable_as_repaired() now also calls
mark_as_being_repaired(session) on the new SSTables it writes. This keeps
them in the REPAIRING view from the moment they are created until
repair_update_compaction_ctrl clears the flag after step 4, covering the
race window in the normal (no-restart) case.
Layer 2 (durable, restart-safe): a new is_being_repaired() helper on
tablet_storage_group_manager detects the race window even after a node
restart, when being_repaired has been lost from memory. It checks:
sst.repaired_at == sstables_repaired_at + 1
AND tablet transition kind == tablet_transition_kind::repair
Both conditions survive restarts: repaired_at is on-disk in SSTable
metadata, and the tablet transition is persisted in Raft. Once the
coordinator commits sstables_repaired_at=N+1 (step 4), is_repaired()
returns true and the SSTable naturally moves to the REPAIRED view.
The classifier in make_repair_sstable_classifier_func() is updated to call
is_being_repaired(sst, sstables_repaired_at) in place of the previous
sst->being_repaired.uuid().is_null() check.
A new test, test_incremental_repair_race_window_promotes_unrepaired_data,
reproduces the bug by:
- Running repair round 1 to establish sstables_repaired_at=1.
- Injecting delay_end_repair_update to hold the race window open.
- Running repair round 2 so all replicas complete mark_sstable_as_repaired
(repaired_at=2) but the coordinator has not yet committed step 4.
- Writing post-repair keys to all replicas and flushing servers[1] to
create an SSTable with repaired_at=0 on disk.
- Restarting servers[1] so being_repaired is lost from memory.
- Waiting for autocompaction to merge the two SSTables on servers[1].
- Asserting that the merged SSTable contains post-repair keys (the bug)
and that servers[0] and servers[2] do not see those keys as repaired.
NOTE FOR MAINTAINER: Copilot initially only implemented Layer 1 (the
in-memory being_repaired guard), missing the restart scenario entirely.
I pointed out that being_repaired is lost on restart and guided Copilot
to add the durable Layer 2 check. I also polished the implementation:
moving is_being_repaired into tablet_storage_group_manager so it can
reuse the already-held _tablet_map (avoiding an ERM lookup and try/catch),
passing sstables_repaired_at in from the classifier to avoid re-reading it,
and using compaction_group_for_sstable inside the function rather than
threading a tablet_id parameter through the classifier.
Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1239.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29244
This series fixes two related inconsistencies around secondary-index
names.
1. `DESCRIBE INDEX ... WITH INTERNALS` returned the backing
materialized-view name in the `name` column instead of the logical
index name.
2. The snapshot REST API accepted backing table names for MV-backed
secondary indexes, but not the logical index names exposed to users.
The snapshot side now resolves logical secondary-index names to backing
table names where applicable, reports logical index names in snapshot
details, rejects vector index names with HTTP 400, and keeps multi-keyspace
DELETE atomic by resolving all keyspaces before deleting anything.
The tests were also extended accordingly, and the snapshot test helper
was fixed to clean up multi-table snapshots using one DELETE per table.
Fixes: SCYLLADB-1122
Minor bugfix, no need to backport.
Closesscylladb/scylladb#29083
* github.com:scylladb/scylladb:
cql3: fix DESCRIBE INDEX WITH INTERNALS name
test: add snapshot REST API tests for logical index names
test: fix snapshot cleanup helper
api: clarify snapshot REST parameter descriptions
api: surface no_such_column_family as HTTP 400
db: fix clear_snapshot() atomicity and use C++23 lambda form
db: normalize index names in get_snapshot_details()
db: add resolve_table_name() to snapshot_ctl
To create `process_staging` view building tasks, we firstly need to collect informations about them on shard0, create necessary mutations, commit them to group0 and move staging sstables objects to their original shards.
But there is a possible race after committing the group0 command and before moving the staging sstables to their shards. Between those two events, the coordinator may schedule freshly created tasks and dispatch them to the worker but the worker won't have the sstables objects because they weren't moved yet.
This patch fixes the race by holding `_staging_sstables_mutex` locks from all necessary shards when executing `create_staging_sstable_tasks()`. With this, even if the task will be scheduled and dispatched quickly, the worker will wait with executing it until the sstables objects are moved and the locks are released.
Fixes SCYLLADB-816
This PR should be backported to all versions containing view building coordinator (2025.4 and newer).
Closesscylladb/scylladb#29174
* github.com:scylladb/scylladb:
db/view/view_building_worker: fix indentation
db/view/view_building_worker: lock staging sstables mutex for necessary shards when creating tasks
DESCRIBE INDEX ... WITH INTERNALS returned the name of
the backing materialized view in the name column instead
of the logical index name.
Return the logical index name from schema::describe()
for index schemas so all callers observe the
user-facing name consistently.
Fixes: SCYLLADB-1122
Add focused REST coverage for logical secondary-index
names in snapshot creation, deletion, and details
output.
Also cover vector-index rejection and verify
multi-keyspace delete resolves all keyspaces before
deleting anything so mixed index kinds cannot cause
partial removal.
The snapshot REST helper cleaned up multi-table
snapshots with a single DELETE request that passed a
comma-separated cf filter, but the API accepts only one
table name there.
Delete each table snapshot separately so existing tests
that snapshot multiple tables use the API as
documented.
Document the current /storage_service/snapshots behavior
more accurately.
For DELETE, cf is a table filter applied independently
in each keyspace listed in kn. If cf is omitted or
empty, snapshots for all tables are eligible, and
secondary indexes can be addressed by their logical
index name.
Snapshot requests that name a non-existent table or a
non-snapshotable logical index currently surface an
internal server error.
Translate no_such_column_family into a bad request so
callers get a client-facing error that matches the
invalid input.
clear_snapshot() applies a table filter independently in
each keyspace, so logical index names must be resolved
per keyspace on the delete path as well.
Resolve all keyspaces before deleting anything so a later
failure cannot partially remove a snapshot, and use the
explicit-object-parameter coroutine lambda form for the
asynchronous implementation.
Snapshot details exposed backing secondary-index view
names instead of logical index names.
Normalize index entries in get_snapshot_details() so the
REST API reports the user-facing name, and update the
existing REST test to assert that behavior directly.
The snapshot REST API accepted backing secondary-index
table names, but not logical index names.
Introduce resolve_table_name() so snapshot creation can
translate a logical index name to the backing table when
the index is materialized as a view.
Currently we don't support 'local' consistency, which would
imply maintaining separate raft group for each dc. What we
support is actually 'global' consistency -- one raft group
per tablet replica set. We don't plan to support local
consistency for the first GA.
Closesscylladb/scylladb#29221
Hi, thanks for Scylla!
We found a small issue in tracker::set_configuration() during joint consensus and put together a fix.
When a server is demoted from voter to non-voter, set_configuration processes the current config first (can_vote=false), then the previous config. But when it finds the server already in the progress map (tracker.cc:118), it hits `continue` without updating can_vote. So the server's follower_progress::can_vote stays false even though it's still a voter in the previous config.
This causes broadcast_read_quorum (fsm.cc:1055) to skip the demoted server, reducing the pool of responders. Since committed() correctly includes the server in _previous_voters for quorum calculation, read barriers can stall if other servers are slow.
The fix is to use configuration::can_vote() in tracker::set_configuration.
We included a reproduction unit test (test_tracker_voter_demotion_joint_config) that extracts the set_configuration algorithm and demonstrates the mismatch. We weren't able to build the full Scylla test suite to add an in-tree test, so we kept it as a standalone file for reference.
No backport: the bug is non-critical and the change needs some soak time in master.
Closesscylladb/scylladb#29226
* https://github.com/scylladb/scylladb:
fix: use is_voter::yes instead of true in test assertions
test: add tracker voter demotion test to fsm_test.cc
fix: use configuration::can_vote() in tracker::set_configuration
The test_create_index_synchronous_updates test in test_secondary_index_properties.py
was intermittently failing with 'assert found_wanted_trace' because the expected
trace event 'Forcing ... view update to be synchronous' was missing from the
trace events returned by get_query_trace().
Root cause: trace events are written asynchronously to system_traces.events.
The Python driver's populate() method considers a trace complete once the
session row in system_traces.sessions has duration IS NOT NULL, then reads
events exactly once. Since the session row and event rows are written as
separate mutations with no transactional guarantee, the driver can read an
incomplete set of events.
Evidence from the failed CI run logs:
- The entire test (CREATE TABLE through DROP TABLE) completed in ~300ms
(01:38:54,859 - 01:38:55,157)
- The INSERT with tracing happened in a ~50ms window between the second
CREATE INDEX completing (01:38:55,108) and DROP TABLE starting
(01:38:55,157)
- The 'Forcing ... synchronous' trace message is generated during the
INSERT write path (db/view/view.cc:2061), so it was produced, but
not yet flushed to system_traces.events when the driver read them
- This matches the known limitation documented in test/alternator/
test_tracing.py: 'we have no way to know whether the tracing events
returned is the entire trace'
Fix: replace the single-shot trace.events read with a retry loop that
directly queries system_traces.events until the expected event appears
(with a 30s timeout). Use ConsistencyLevel.ONE since system_traces has
RF=2 and cqlpy tests run on a single-node cluster.
The same race condition pattern exists in test_mv_synchronous_updates in
test_materialized_view.py (which this test was modeled after), so the
same fix is proactively applied there as well.
Fixes SCYLLADB-1314
Closesscylladb/scylladb#29374
Previously, the timestamp decoded from a timeuuid was printed using the
local timezone via datetime.fromtimestamp(), which produces different
output depending on the machine's locale settings.
ScyllaDB logs are emitted in UTC by default. Printing the decoded date
in UTC makes it straightforward to correlate SSTable identifiers with
log entries without having to mentally convert timezones.
Also fix the embedded pytest assertion, which was accidentally correct
only on machines in UTC+8 — it now uses an explicit UTC-aware datetime.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29253
Add explicit permissions block (contents: read, pull-requests: write,
statuses: write) matching the requirements of the called reusable
workflow which checks out code, posts PR comments, and sets commit
statuses. Fixes code scanning alert #172.
Closesscylladb/scylladb#29183
Replace strict case-sensitive '== "True"' check with strcasecmp(..., "true")
so that Python's str(True) -> "True" is properly recognized. Accepts any
case variation of "true" ("True", "TRUE", etc.), with empty string
defaulting to false.
Maintains backward compatibility with out-of-tree tests that rely on
Python's bool stringification.
The goal is to reduce the number of distinct ways API handlers use to
convert string http query parameters into bool variables. This place is the
only one that simply compares param to "True".
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29236
Switch to branch lts_2026_01_07, which is exactly equal to
upstream now.
There were no notable changes in the release notes, but the
new versions are more friendly to newer compilers (specifically,
in include hygiene).
configure.py needs a few library updates; cmake works without
change.
scylla-gdb.py updated for new hash table layout (by Claude Opus 4.6).
* abseil d7aaad83...255c84da (1179):
> Abseil LTS branch, Jan 2026, Patch 1 (#2007)
> Cherry-picks for LTS 20260107 (#1990)
> Apply LTS transformations for 20260107 LTS branch (#1989)
> Mark legacy Mutex methods and MutexLock pointer constructors as deprecated
> `cleanup`: specify that it's safe to use the class in a signal handler.
> Suppress bugprone-use-after-move in benign cases
> StrFormat: format scientific notation without heap allocation
> Introduce a legacy copy of GetDebugStackTraceHook API.
> Report 1ns instead of 0ns for probe_benchmarks. Some tools incorrectly assume that benchmark was not run if 0ns reported.
> Add absl::chunked_queue
> `CRC32` version of `CombineContiguous` for length <= 32.
> Add `absl::down_cast`
> Fix FixedArray iterator constructor, which should require input_iterator, not forward_iterator
> Add a latency benchmark for hashing a pair of integers.
> Delete absl::strings_internal::STLStringReserveAmortized()
> As IsAtLeastInputIterator helper
> Use StringAppendAndOverwrite() in CEscapeAndAppendInternal()
> Add support for absl::(u)int128 in FastIntToBuffer()
> absl/strings: Prepare helper for printing objects to string representations.
> Use SimpleAtob() for parsing bool flags
> No-op changes to relative timeout support code.
> Adjust visibility of heterogeneous_lookup_testing.h
> Remove -DUNORDERED_SET_CXX17 since the macro no longer exists
> [log] Prepare helper for streaming container contents to strings.
> Restrict the visibility of some internal testing utilities
> Add absl::linked_hash_set and absl::linked_hash_map
> [meta] Add constexpr testing helper.
> BUILD file reformatting.
> `absl/meta`: Add C++17 port of C++20 `requires` expression for internal use
> Remove the implementation of `absl::string_view`, which was only needed prior to C++17. `absl::string_view` is now an alias for `std::string_view`. It is recommended that clients simply use `std::string_view`.
> No public description
> absl:🎏 Stop echoing file content in flagfile parsing errors Modified ArgsList::ReadFromFlagfile to redact the content of unexpected lines from error messages. \
> Refactor the declaration of `raw_hash_set`/`btree` to omit default template parameters from the subclasses.
> Import of CCTZ from GitHub.
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND to Flag help generator
> Correct `Mix4x16Vectors` comment.
> Special implementation for string hash with sizes greater than 64.
> Reorder function parameters so that hash state is the first argument.
> Search more aggressively for open slots in absl::internal_stacktrace::BorrowedFixupBuffer
> Implement SpinLockHolder in terms of std::lock_guard.
> No public description
> Avoid discarding test matchers.
> Import of CCTZ from GitHub.
> Automated rollback of commit 9f40d6d6f3cfc1fb0325dd8637eb65f8299a4b00.
> Enable clang-specific warnings on the clang-cl build instead of just trying to be MSVC
> Enable clang-specific warnings on the clang-cl build instead of just trying to be MSVC
> Make AnyInvocable remember more information
> Add further diagnostics under clang for string_view(nullptr)
> Import of CCTZ from GitHub.
> Document the differing trimming behavior of absl::Span::subspan() and std::span::subspan()
> Special implementation for string hash with sizes in range [33, 64].
> Add the deleted string_view(std::nullptr_t) constructor from C++23
> CI: Use a cached copy of GoogleTest in CMake builds if possible to minimize the possibility of errors downloading from GitHub
> CI: Enable libc++ hardening in the ASAN build for even more checks https://libcxx.llvm.org/Hardening.html
> Call the common case of AllocateBackingArray directly instead of through the function pointer.
> Change AlignedType to have a void* array member so that swisstable backing arrays end up in the pointer-containing partition for heap partitioning.
> base: Discourage use of ABSL_ATTRIBUTE_PACKED
> Revert: Add an attribute to HashtablezInfo which performs a bitwise XOR on all hashes. The purposes of this attribute is to identify if identical hash tables are being created. If we see a large number of identical tables, it's likely the code can be improved by using a common table as opposed to keep rebuilding the same one.
> Import of CCTZ from GitHub.
> Record insert misses in hashtable profiling.
> Add absl::StatusCodeToStringView.
> Add a missing dependency on str_format that was being pulled in transitively
> Pico-optimize `SkipWhitespace` to use `StripLeadingAsciiWhitespace`.
> absl::string_view: Upgrade the debug assert on the single argument char* constructor to ABSL_HARDENING_ASSERT
> Use non-stack storage for stack trace buffers
> Fixed incorrect include for ABSL_NAMESPACE_BEGIN
> Add ABSL_REFACTOR_INLINE to separate the inliner directive from the deprecated directive so that we can give users a custom deprecation message.
> Reduce stack usage when unwinding without fixups
> Reduce stack usage when unwinding from 170 to 128 on x64
> Rename RecordInsert -> RecordInsertMiss.
> PR #1968: Use std::move_backward within InlinedVector's Storage::Insert
> Use the new absl::StringResizeAndOverwrite() in CUnescape()
> Explicitly instantiate common `raw_hash_set` backing array functions.
> Rollback reduction of maximum load factor. Now it is back to 28/32.
> Export Mutex::Dtor from shared libraries in NDEBUG mode
> Allow `IsOkAndHolds` to rely on duck typing for matching `StatusOr` like types instead of uniquely `absl::StatusOr`, e.g. `google::cloud::StatusOr`.
> Fix typo in macro and add missing static_cast for WASM builds.
> windows(cmake): add abseil_test_dll to target link libraries when required
> Handle empty strings in `SimpleAtof` after stripping whitespace
> Avoid using a thread_local in an inline function since this causes issues on some platforms.
> (Roll forward) Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes.
> Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes.
> Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes.
> Fixes for String{Resize|Append}AndOverwrite - StringAppendAndOverwrite() should always call StringResizeAndOverwrite() with at least capacity() in case the standard library decides to shrink the buffer (Fixes#1965) - Small refactor to make the minimum growth an addition for clarity and to make it easier to test 1.5x growth in the future - Turn an ABSL_HARDENING_ASSERT into a ThrowStdLengthError - Add a missing std::move
> Correct the supported features of Status Matchers
> absl/time: Use "memory order acquire" for loads, which would allow for the safe removal of the data memory barrier.
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> Add an internal-only helper StringAppendAndOverwrite() similar to StringResizeAndOverwrite() but optimized for repeated appends, using exponential growth to ensure amortized complexity of increasing a string size by a small amount is O(1).
> Release `ABSL_EXPECT_OK` and `ABSL_ASSERT_OK`.
> Fix the CHECK_XX family of macros to not print `char*` arguments as C-strings if the comparison happened as pointers. Printing as pointers is more relevant to the result of the comparison.
> Rollback StringAppendAndOverwrite() - the problem is that StringResizeAndOverwrite has MSAN testing of the entire string. This causes quadratic MSAN verification on small appends.
> Add an internal-only helper StringAppendAndOverwrite() similar to StringResizeAndOverwrite() but optimized for repeated appends, using exponential growth to ensure amortized complexity of increasing a string size by a small amount is O(1).
> PR #1961: Fix Clang warnings on powerpc
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> macOS CI: Move the Bazel vendor_dir to ${HOME} to workaround a Bazel issue where it does not work when it is in ${TMP} and also fix the quoting which was causing it to incorrectly receive the argument
> Use __msan_check_mem_is_initialized for detailed MSan report
> Optimize stack unwinding by reducing `AddressIsReadable` calls.
> Add internal API to allow bypassing stack trace fixups when needed
> absl::StrFormat: improve test coverage with scientific exponent test cases
> Add throughput and latency benchmarks for `absl::ToDoubleXYZ` functions.
> CordzInfo: Use absl::NoDestructor to remove a global destructor. Chromium requires no global destructors.
> string_view: Enable std::view and std::borrowed_range
> cleanup: s/logging_internal/log_internal/ig for consistency
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> Use the new absl::StringResizeAndOverwrite() in string escaping utilities
> Use the new absl::StringResizeAndOverwrite() in absl::AsciiStrTo{Lower|Upper}
> Use the new absl::StringResizeAndOverwrite() in absl::StrJoin()
> Use the new absl::StringResizeAndOverwrite() in absl::StrCat()
> string_view: Fix include order
> Don't pass nullptr as the 1st arg of `from_chars`
> absl/types: format code with clang-format.
> Validate absl::StringResizeAndOverwrite op has written bytes as expected.
> Skip the ShortStringCollision test on WASM.
> Rollback `absl/types`: format code with clang-format.
> Remove usage of the WasmOffsetConverter for Wasm / Emscripten stack-traces.
> Use the new absl::StringResizeAndOverwrite() in absl::CordCopyToString()
> Remove an undocumented behavior of --vmodule and absl::SetVLogLevel that could set a module_pattern to defer to the global vlog threshold.
> Update to rules_cc 0.2.9
> Avoid redefine warnings with ntstatus constants
> PR #1944: Use same element-width for non-temporal loads and stores on Arm
> absl::StringResizeAndOverwrite(): Add the requirement that the only value that can be written to buf[size] is the terminator character.
> absl/types: format code with clang-format.
> Minor formatting changes.
> Remove `IntIdentity` and `PtrIdentity` from `raw_hash_set_probe_benchmark`.
> Automated rollback of commit cad60580dba861d36ed813564026d9774d9e4e2b.
> FlagStateInterface implementors need only support being restored once.
> Clarify the post-condition of `reserve()` in Abseil hash containers.
> Clarify the post-condition of `reserve()` in Abseil hash containers.
> Represent dropped samples in hashtable profile.
> Add lifetimebound to absl::implicit_cast and make it work for rvalue references as it already does with lvalue references
> Clean up a doc example where we had `absl_nonnull` and `= nullptr;`
> Change Cordz to synchronize tracked cords with Snapshots / DeleteQueue
> Minor refactor to `num_threads` in deadlock test
> Rename VLOG macro parameter to match other uses of this pseudo type.
> `time`: Fix indentation
> Automated Code Change
> Adds `absl::StringResizeAndOverwrite` as a polyfill for C++23's `std::basic_string<CharT,Traits,Allocator>::resize_and_overwrite`
> Internal-only change
> absl/time: format code with clang-format.
> No public description
> Expose typed releasers of externally appended memory.
> Fix __declspec support for ABSL_DECLARE_FLAG()
> Annotate absl::AnyInvocable as an owner type via [[gsl::Owner]] and absl_internal_is_view = std::false_type
> Annotate absl::FunctionRef as a view type via [[gsl::Pointer]] and absl_internal_is_view
> Remove unnecessary dep on `core_headers` from the `nullability` cc_library
> type_traits: Add type_identity and type_traits_t backfills
> Refactor raw_hash_set range insertion to call private insert_range function.
> Fix bug in absl::FunctionRef conversions from non-const to const
> PR #1937: Simplify ConvertSpecialToEmptyAndFullToDeleted
> Improve absl::FunctionRef compatibility with C++26
> Add a workaround for unused variable warnings inside of not-taken if-constexpr codepaths in older versions of GCC
> Annotate ABSL_DIE_IF_NULL's return type with `absl_nonnull`
> Move insert index computation into `PrepareInsertLarge` in order to reduce inlined part of insert/emplace operations.
> Automated Code Change
> PR #1939: Add missing rules_cc loads
> Expose (internally) a LogMessage constructor taking file as a string_view for (internal, upcoming) FFI integration.
> Fixed up some #includes in mutex.h
> Make absl::FunctionRef support non-const callables, aligning it with std::function_ref from C++26
> Move capacity update in `Grow1To3AndPrepareInsert` after accessing `common.infoz()` to prevent assertion failure in `control()`.
> Fix check_op(s) compilation failures on gcc 8 which eagerly tries to instantiate std::underlying_type for non-num types.
> Use `ABSL_ATTRIBUTE_ALWAYS_INLINE`for lambda in `find_or_prepare_insert_large`.
> Mark the implicit floating operators as constexpr for `absl::int128` and `absl::uint128`
> PR #1931: raw_hash_set: fix instantiation for recursive types on MSVC with /Zc:__cplusplus
> Add std::pair specializations for IsOwner and IsView
> Cast ABSL_MIN_LOG_LEVEL to absl::LogSeverityAtLeast instead of absl::LogSeverity.
> Fix a corner case in the aarch64 unwinder
> Fix inconsistent nullability annotation in ReleasableMutexLock
> Remove support for Native Client
> Rollback f040e96b93dba46e8ed3ca59c0444cbd6c0a0955
> When printing CHECK_XX failures and both types are unprintable, don't bother printing " (UNPRINTABLE vs. UNPRINTABLE)".
> PR #1929: Fix shorten-64-to-32 warning in stacktrace_riscv-inl.inc
> Refactor `find_or_prepare_insert_large` to use a single return statement using a lambda.
> Use possible CPUs to identify NumCPUs() on Linux.
> Fix incorrect nullability annotation of `absl::Cord::InlineRep::set_data()`.
> Move SetCtrl* family of functions to cc file.
> Change absl::InlinedVector::clear() so that it does not deallocate any allocated space. This allows allocations to be reused and matches the behavior specification of std::vector::clear().
> Mark Abseil container algorithms as `constexpr` for C++20.
> Fix `CHECK_<OP>` ambiguous overload for `operator<<` in older versions of GCC when C-style strings are compared
> stacktrace_test: avoid spoiling errno in the test signal handler.
> Optimize `CRC32AcceleratedX86ARMCombinedMultipleStreams::Extend` by interleaving the `CRC32_u64` calls at a lower level.
> stacktrace_test: avoid spoiling errno in the test signal handler.
> stacktrace_test: avoid spoiling errno in the test signal handler.
> std::multimap::find() is not guaranteed to return the first entry with the requested key. Any may be returned if many exist.
> Mark `/`, `%`, and `*` operators as constexpr when intrinsics are available.
> Add the C++20 string_view contructor that uses iterators
> Implement absl::erase_if for absl::InlinedVector
> Adjust software prefetch to fetch 5 cachelines ahead, as benchmarking suggests this should perform better.
> Reduce maximum load factor to 27/32 (from 28/32).
> Remove unused include
> Remove unused include statement
> PR #1921: Fix ABSL_BUILD_DLL mode (absl_make_dll) with mingw
> PR #1922: Enable mmap for WASI if it supports the mman header
> Rollback C++20 string_view constructor that uses iterators due to broken builds
> Add the C++20 string_view contructor that uses iterators
> Bump versions of dependencies in MODULE.bazel
> Automated Code Change
> PR #1918: base: add musl + ppc64le fallback for UnscaledCycleClock::Frequency
> Optimize crc32 Extend by removing obsolete length alignment.
> Fix typo in comment of `ABSL_ATTRIBUTE_UNUSED`.
> Mark AnyInvocable as being nullability compatible.
> Ensure stack usage remains low when unwinding the stack, to prevent stack overflows
> Shrink #if ABSL_HAVE_ATTRIBUTE_WEAK region sizes in stacktrace_test.cc
> <filesystem> is not supported for XTENSA. Disable it in //absl/hash/internal/hash.h.
> Use signal-safe dynamic memory allocation for stack traces when necessary
> PR #1915: Fix SYCL Build Compatibility with Intel LLVM Compiler on Windows for abseil
> Import of CCTZ from GitHub.
> Tag tests that currently fail on ios_sim_arm64 with "no_test_ios_sim_arm64"
> Automated Code Change
> Automated Code Change
> Import of CCTZ from GitHub.
> Move comment specific to pointer-taking MutexLock variant to its definition.
> Add lifetime annotations to MutexLock, SpinLockHolder, etc.
> Add lifetimebound annotations to absl::MakeSpan and absl::MakeConstSpan to detect dangling references
> Remove comment mentioning deferenceability.
> Add referenceful MutexLock with Condition overload.
> Mark SpinLock camel-cased methods as ready for inlining.
> Whitespace change
> In logging tests that write expectations against `ScopedMockLog::Send`, suppress the default behavior that forwards to `ScopedMockLog::Log` so that unexpected logs are printed with full metadata. Many of these tests are poking at those metadata, and a failure message that doesn't include them is unhelpful.
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::ClippedSubstr
> Inline internal usages of Mutex::Lock, etc. in favor of lock.
> Inline internal usages of pointerful SpinLockHolder/MutexLock.
> Remove wrong comment in Cord::Unref
> Update the crc32 dynamic dispatch table with newer platforms.
> PR #1914: absl/base/internal/poison.cc: Minor build fix
> Accept references on SpinLockHolder/MutexLock
> Import of CCTZ from GitHub.
> Fix typos in comments.
> Inline SpinLock Lock->lock, Unlock->unlock internal to Abseil.
> Rename Mutex methods to use the typical C++ lower case names.
> Rename SpinLock methods to use the typical C++ lower case names.
> Add an assert that absl::StrSplit is not called with a null char* argument.
> Fix sign conversion warning
> PR #1911: Fix absl_demangle_test on ppc64
> Disallow using a hash function whose return type is smaller than size_t.
> Optimize CRC-32C extension by zeroes
> Deduplicate stack trace implementations in stacktrace.cc
> Align types of location_table_ and mapping_table_ keys (-Wshorten-64-to-32).
> Move SigSafeArena() out to absl/base/internal/low_level_alloc.h
> Allow CHECK_<OP> variants to be used with unprintable types.
> Import of CCTZ from GitHub.
> Adds required load statements for C++ rules to BUILD and bzl files.
> Disable sanitizer bounds checking in ComputeZeroConstant.
> Roll back NDK weak symbol mode for backtrace() due to internal test breakage
> Add converter for extracting SwissMap profile information into a https://github.com/google/pprof suitable format for inspection.
> Allocate memory for frames and sizes during stack trace fix-up when no memory is provided
> Support NDK weak symbol mode for backtrace() on Android.
> Change skip_empty_or_deleted to not use groups.
> Fix bug of dereferencing invalidated iterator in test case.
> Refactor: split erase_meta_only into large and small versions.
> Fix a TODO to use std::is_nothrow_swappable when it became available.
> Clean up the testing of alternate options that were removed in previous changes
> Only use generic stacktrace when ABSL_HAVE_THREAD_LOCAL.
> Automated Code Change
> Add triviality tests for absl::Span
> Loosen the PointerAlignment test to allow up to 5 stuck bits to avoid flakiness.
> Prevent conversion constructions from absl::Span to itself
> Skip flaky expectations in waiter_test for MSVC.
> Refactor: call AssertIsFull from iterator::assert_is_full to avoid passing the same arguments repeatedly.
> In AssertSameContainer, remove the logic checking for whether the iterators are from SOO tables or not since we don't use it to generate a more informative debug message.
> Remove unused NonIterableBitMask::HighestBitSet function.
> Refactor: move iterator unchecked_* members before data members to comply with Google C++ style guide.
> Mix pointers once instead of twice now that we've improved mixing on 32-bit platforms and improved the kMul constant.
> Remove unused utility functions/constants.
> Revert a change for breaking downstream third party libs
> Remove unneeded include from cord_rep_btree_navigator.h
> Refactor: move find_first_non_full into raw_hash_set.cc.
> Perform stronger mixing on 32-bit platforms and enable the LowEntropyStrings test.
> Include deallocated caller-provided size in delete hooks.
> Roll back one more time: In debug mode, assert that the probe sequence isn't excessively long.
> Allow a `std::move` of `delimiter_` to happen in `ByString::ByString(ByString&&)`. Right now the move ctor is making a copy because the source object is `const`.
> Assume that control bytes don't alias CommonFields.
> Consistently use [[maybe_unused]] in raw_hash_set.h for better compiler warning compatibility.
> Roll forward: In debug mode, assert that the probe sequence isn't excessively long.
> Add a new test for hash collisions for short strings when PrecombineLengthMix has low quality.
> Refactor: define CombineRawImpl for repeated `Mix(state ^ value, kMul)` operations.
> Automated Code Change
> Mark hash_test as large so that the timeout is increased.
> Change the value of kMul to have higher entropy and prevent collisions when keys are aligned integers or pointers.
> Fix LIFETIME annotations for op*/op->/value operators for reference types.
> Update StatusOr to support lvalue reference value types.
> Rollback debug assertion that the probe sequence isn't excessively long.
> AnyInvocable: Fix operator==/!= comments
> In debug mode, assert that the probe sequence isn't excessively long.
> Improve NaN handling in absl::Duration arithmetic.
> Change PrecombineLengthMix to sample data from kStaticRandomData.
> Fix includes and fuse constructors of SpinLock.
> Enable `operator==` for `StatusOr` only if the contained type is equality-comparable
> Enable SIMD memcpy-crc on ARM cores.
> Improve mixing on 32-bit platforms.
> Change DurationFromDouble to return -InfiniteDuration() for all NaNs.
> Change return type of hash internal `Seed` to `size_t` from `uint64_t`
> CMake: Add a fatal error when the compiler defaults to or is set to a C++ language standard prior to C++17.
> Make bool true hash be ~size_t{} instead of 1 so that all bits are different between true/false instead of only one.
> Automated Code Change
> Pass swisstable seed as seed to absl::Hash so we can save an XOR in H1.
> Add support for scoped enumerations in CHECK_XX().
> Revert no-inline on Voidify::operator&&() -- caused unexpected binary size growth
> Mark Voidify::operator&&() as no-inline. This improves stack trace for `LOG(FATAL)` with optimization on.
> Refactor long strings hash computations and move `len <= PiecewiseChunkSize()` out of the line to keep only one function call in the inlined hash code.
> rotr/rotl: Fix undefined behavior when passing INT_MIN as the number of positions to rotate by
> Reorder members of MixingHashState to comply with Google C++ style guide ordering of type declarations, static constants, ctors, non-ctor functions.
> Delete unused function ShouldSampleHashtablezInfoOnResize.
> Remove redundant comments that just name the following symbol without providing additional information.
> Remove unnecessary modification of growth info in small table case.
> Suppress CFI violation on VDSO call.
> Replace WeakMix usage with Mix and change H2 to use the most significant 7 bits - saving 1 cycle in H1.
> Fix -Wundef warning
> Fix conditional constexpr in ToInt64{Nano|Micro|Milli}seconds under GCC7 and GCC8 using an else clause as a workaround
> Enable CompressedTupleTest.NestedEbo test case.
> Lift restriction on using EBCO[1] for nested CompressedTuples. The current implementation of CompressedTuple explicitly disallows EBCO for cases where CompressedTuples are nested. This is because the implentation for a tuple with EBCO-compatible element T inherits from Storage<T, I>, where I is the index of T in the tuple, and
> absl::string_view: assert against (data() == nullptr && size() != 0)
> Fix a false nullability warning in [Q]CHECK_OK by replacing nullptr with an empty char*
> Make `combine_contiguous` to mix length in a weak way by adding `size << 24`, so that we can avoid a separate mixing of size later. The empty range is mixing 0x57 byte.
> Add a test case that -1.0 and 1.0 have different hashes.
> Update CI to a more recent Clang on Linux x86-64
> `absl::string_view`: Add a debug assert to the single-argument constructor that the argument is not `nullptr`.
> Fix CI on macOS Sequoia
> Use Xcode 16.3 for testing
> Use a proper fix instead of a workaround for a parameter annotated absl_nonnull since the latest Clang can see through the workaround
> Assert that SetCtrl isn't called on small tables - there are no control bytes in such cases.
> Use `MaskFullOrSentinel` in `skip_empty_or_deleted`.
> Reduce flakiness in MockDistributions.Examples test case.
> Rename PrepareInsertNonSoo to PrepareInsertLarge now that it's no longer used in all non-SOO cases.
> PR #1895: use c++17 in podspec
> Avoid hashing the key in prefetch() for small tables.
> Remove template alias nullability annotations.
> Add `Group::MaskFullOrSentinel` implementation without usage.
> Move `hashtable_control_bytes` tests into their own file.
> Simplify calls to `EqualElement` by introducing `equal_to` helper function.
> Do `common.increment_size()` directly in SmallNonSooPrepareInsert if inserting to reserved 1 element table.
> Import of CCTZ from GitHub.
> Small cleanup of `infoz` processing to get the logic out of the line or removed.
> Extract the entire PrepareInsert to Small non SOO table out of the line.
> Take `get_hash` implementation out of the SwissTable class to minimize number of instantiations.
> Change kEmptyGroup to kDefaultIterControl now that it's only used for default-constructed iterators.
> [bits] Add tests for return types
> Avoid allocating control bytes in capacity==1 swisstables.
> PR #1888: Adjust Table.GrowExtremelyLargeTable to avoid OOM on i386
> Avoid mixing after `Hash64` calls for long strings by passing `state` instead of `Seed` to low level hash.
> Indent absl container examples consistently
> Revert- Doesn't actually work because SWIG doesn't use the full preprocessor
> Add tags to skip some tests under UBSAN.
> Avoid subtracting `it.control()` and `table.control()` in single element table during erase.
> Remove the `salt` parameter from low level hash and use a global constant. That may potentially remove some loads.
> In SwissTable, don't hash the key when capacity<=1 on insertions.
> Remove the "small" size designation for thread_identity_test, which causes the test to timeout after 60s.
> Add comment explaining math behind expressions.
> Exclude SWIG from ABSL_DEPRECATED and ABSL_DEPRECATE_AND_INLINE
> stacktrace_x86: Handle nested signals on altstack
> Import of CCTZ from GitHub.
> Simplify MixingHashState::Read9To16 to not depend on endianness.
> Delete deprecated `absl::Cord::Get` and its remaining call sites.
> PR #1884: Remove duplicate dependency
> Remove relocatability test that is no longer useful
> Import of CCTZ from GitHub.
> Fix a bug of casting sizeof(slot_type) to uint16_t instead of uint32_t.
> Rewrite `WideToUtf8` for improved readability.
> Avoid requiring default-constructability of iterator type in algorithms that use ContainerIterPairType
> Added test cases for invalid surrogates sequences.
> Use __builtin_is_cpp_trivially_relocatable to implement absl::is_trivially_relocatable in a way that is compatible with PR2786 in the upcoming C++26.
> Remove dependency on `wcsnlen` for string length calculation.
> Stop being strict about validating the "clone" part of mangled names
> Add support for logging wide strings in `absl::log`.
> Deprecate `ABSL_HAVE_STD_STRING_VIEW`.
> Change some nullability annotations in absl::Span to absl_nullability_unknown to workaround a bug that makes nullability checks trigger in foreach loops, while still fixing the -Wnullability-completeness warnings.
> Linux CI update
> Fix new -Wnullability-completeness warnings found after upgrading the Clang version used in the Linux ARM CI to Clang 19.
> Add __restrict for uses of PolicyFunctions.
> Use Bazel vendor mode to cache external dependencies on Windows and macOS
> Move PrepareInsertCommon from header file to cc file.
> Remove the explicit from the constructor to a test allocator in hash_policy_testing.h. This is rejected by Clang when using the libstdc++ that ships with GCC15
> Extract `WideToUtf8` helper to `utf8.h`.
> Updates the documentation for `CHECK` to make it more explicit that it is used to require that a condition is true.
> Add PolicyFunctions::soo_capacity() so that the compiler knows that soo_capacity() is always 0 or 1.
> Expect different representations of pointers from the Windows toolchain.
> Add set_no_seed_for_testing for use in GrowExtremelyLargeTable test.
> Update GoogleTest dependency to 1.17.0 to support GCC15
> Assume that frame pointers inside known stack bounds are readable.
> Remove fallback code in absl/algorithm/container.h
> Fix GCC15 warning that <ciso646> is deprecated in C++17
> Fix misplaced closing brace
> Remove unused include.
> Automated Code Change
> Type erase copy constructor.
> Refactor to use hash_of(key) instead of hash_ref()(key).
> Create Table.Prefetch test to make sure that it works.
> Remove NOINLINE on the constructor with buckets.
> In SwissTable, don't hash the key in find when capacity<=1.
> Use 0x57 instead of Seed() for weakly mixing of size.
> Use absl::InsecureBitGen in place of std::random_device in Abseil tests.
> Remove unused include.
> Use large 64 bits kMul for 32 bits platforms as well.
> Import of CCTZ from GitHub.
> Define `combine_weakly_mixed_integer` in HashSelect::State in order to allow `friend auto AbslHashValue` instead of `friend H AbslHashValue`.
> PR #1878: Fix typos in comments
> Update Abseil dependencies in preparation for release
> Use weaker mixing for absl::Hash for types that mix their sizes.
> Update comments on UnscaledCycleClock::Now.
> Use alignas instead of the manual alignment for the Randen entropy pool.
> Document nullability annotation syntax for array declarations (not many people may know the syntax).
> Import of CCTZ from GitHub.
> Release tests for ABSL_RAW_DCHECK and ABSL_RAW_DLOG.
> Adjust threshold for stuck bits to avoid flaky failures.
> Deprecate template type alias nullability annotations.
> Add more probe benchmarks
> PR #1874: Simplify detection of the powerpc64 ELFv1 ABI
> Make `absl::FunctionRef` copy-assignable. This brings it more in line with `std::function_ref`.
> Remove unused #includes from absl/base/internal/nullability_impl.h
> PR #1870: Retry SymInitialize on STATUS_INFO_LENGTH_MISMATCH
> Prefetch from slots in parallel with reading from control.
> Migrate template alias nullability annotations to macros.
> Improve dependency graph in `TryFindNewIndexWithoutProbing` hot path evaluation.
> Add latency benchmarks for Hash for strings with size 3, 5 and 17.
> Exclude UnwindImpl etc. from thread sanitizer due to false positives.
> Use `GroupFullEmptyOrDeleted` inside of `transfer_unprobed_elements_to_next_capacity_fn`.
> PR #1863: [minor] Avoid variable shadowing for absl btree
> Extend stack-frame walking functionality to allow dynamic fixup
> Fix "unsafe narrowing" in absl for Emscripten
> Roll back change to address breakage
> Extend stack-frame walking functionality to allow dynamic fixup
> Introduce `absl::Cord::Distance()`
> Avoid aliasing issues in growth information initialization.
> Make `GrowSooTableToNextCapacityAndPrepareInsert` in order to initialize control bytes all at once and avoid two function calls on growth right after SOO.
> Simplify `SingleGroupTableH1` since we do not need to mix all bits anymore. Per table seed has a good last bit distribution.
> Use `NextSeed` instead of `NextSeedBaseNumber` and make the result type to be `uint16_t`. That avoids unnecessary bit twiddling and simplify the code.
> Optimize `GrowthToLowerBoundCapacity` in order to avoid division.
> [base] Make :endian internal to absl
> Fully qualify absl names in check macros to avoid invalid name resolution when the user scope has those names defined.
> Fix memory sanitization in `GrowToNextCapacityAndPrepareInsert`.
> Define and use `ABSL_SWISSTABLE_ASSERT` in cc file since a lot of logic moved there.
> Remove `ShouldInsertBackwards` functionality. It was used for additional order randomness in debug mode. It is not necessary anymore with introduction of separate per table `seed`.
> Fast growing to the next capacity based on carbon hash table ideas.
> Automated Code Change
> Refactor CombinePiecewiseBuffer test case to (a) call PiecewiseChunkSize() to get the chunk size and (b) use ASSERT for expectation in a loop.
> PR #1867: Remove global static in stacktrace_win32-inl.inc
> Mark Abseil hardening assert in AssertIsValidForComparison as slow.
> Roll back a problematic change.
> Add absl::FastTypeId<T>()
> Automated Code Change
> Update TestIntrinsicInt128 test to print the indices with the conflicting hashes.
> Code simplification: we don't need XOR and kMul when mixing large string hashes into hash state.
> Refactor absl::CUnescape() to use direct string output instead of pointer/size.
> Rename `policy.transfer` to `policy.transfer_n`.
> Optimize `ResetCtrl` for small tables with `capacity < Group::KWidth * 2` (<32 if SSE enabled and <16 if not).
> Use 16 bits of per-table-seed so that we can save an `and` instruction in H1.
> Fully annotate nullability in headers where it is partially annotated.
> Add note about sparse containers to (flat|node)_hash_(set|map).
> Make low_level_alloc compatible with -Wthread-safety-pointer
> Add missing direct includes to enable the removal of unused includes from absl/base/internal/nullability_impl.h.
> Add tests for macro nullability annotations analogous to existing tests for type alias annotations.
> Adds functionality to return stack frame pointers during stack walking, in addition to code addresses
> Use even faster reduction algorithm in FinalizePclmulStream()
> Add nullability annotations to some very-commonly-used APIs.
> PR #1860: Add `unsigned` to character buffers to ensure they can provide storage (https://eel.is/c++draft/intro.object#3)
> Release benchmarks for absl::Status and absl::StatusOr
> Use more efficient reduction algorithm in FinalizePclmulStream()
> Add a test case to make it clear that `--vmodule=foo/*=1` does match any children and grandchildren and so on under `foo/`.
> Gate use of clang nullability qualifiers through absl nullability macros on `nullability_on_classes`.
> Mark `absl::StatusOr::status()` as ABSL_MUST_USE_RESULT
> Cleanups related to benchmarks * Fix many benchmarks to be cc_binary instead of cc_test * Add a few benchmarks for StrFormat * Add benchmarks for Substitute * Add benchmarks for Damerau-Levenshtein distance used in flags
> Add a log severity alias `DO_NOT_$UBMIT` intended for logging during development
> Avoid relying on true and false tokens in the preprocessor macros used in any_invocable.h
> Avoid relying on true and false tokens in the preprocessor macros used in absl/container
> Refactor to make it clear that H2 computation is not repeated in each iteration of the probe loop.
> Turn on C++23 testing for GCC and Clang on Linux
> Fix overflow of kSeedMask on 32 bits platform in `generate_new_seed`.
> Add a workaround for std::pair not being trivially copyable in C++23 in some standard library versions
> Refactor WeakMix to include the XOR of the state with the input value.
> Migrate ClearPacBits() to a more generic implementation and location
> Annotate more Abseil container methods with [[clang::lifetime_capture_by(...)]] and make them all forward to the non-captured overload
> Make PolicyFunctions always be the second argument (after CommonFields) for type-erased functions.
> Move GrowFullSooTableToNextCapacity implementation with some dependencies to cc file.
> Optimize btree_iterator increment/decrement to avoid aliasing issues by using local variables instead of repeatedly writing to `this`.
> Add constexpr conversions from absl::Duration to int64_t
> PR #1853: Add support for QCC compiler
> Fix documentation for key requirements of flat_hash_set
> Use `extern template` for `GrowFullSooTableToNextCapacity` since we know the most common set of paramenters.
> C++23: Fix log_format_test to match the stream format for volatile pointers
> C++23: Fix compressed_tuple_test.
> Implement `btree::iterator::+=` and `-=`.
> Stop calling `ABSL_ANNOTATE_MEMORY_IS_INITIALIZED` for threadlocal counter.
> Automated Code Change
> Introduce seed stored in the hash table inside of the size.
> Replace ABSL_ATTRIBUTE_UNUSED with [[maybe_unused]]
> Minor consistency cleanups to absl::BitGen mocking.
> Restore the empty CMake targets for bad_any_cast, bad_optional_access, and bad_variant_access to allow clients to migrate.
> bits.h: Add absl::endian and absl::byteswap polyfills
> Use absl::NoDestructor an absl::Mutex instance in the flags library to prevent some exit-time destructor warnings
> Add thread GetEntropyFromRandenPool test
> Update nullability annotation documentation to focus on macro annotations.
> Simplify some random/internal types; expose one function to acquire entropy.
> Remove pre-C++17 workarounds for lack of std::launder
> UBSAN: Use -fno-sanitize-recover
> int128_test: Avoid testing signed integer overflow
> Remove leading commas in `Describe*` methods of `StatusIs` matcher.
> absl::StrFormat: Avoid passing null to memcpy
> str_cat_test: Avoid using invalid enum values
> hash_generator_testing: Avoid using invalid enum values
> absl::Cord: Avoid passing null to memcpy and memset
> graphcycles_test: Avoid applying a non-zero offset to a null pointer
> Make warning about wrapping empty std::function in AnyInvocable stronger.
> absl/random: Convert absl::BitGen / absl::InsecureBitGen to classes from aliases.
> Fix buffer overflow the internal demangling function
> Avoid calling `ShouldRehashForBugDetection` on the first two inserts to the table.
> Remove the polyfill implementations for many type traits and alias them to their std equivalents. It is recommended that clients now simple use the std equivalents.
> ROLLBACK: Limit slot_size to 2^16-1 and maximum table size to 2^43-1.
> Limit `slot_size` to `2^16-1` and maximum table size to `2^43-1`.
> Use C++17 [[nodiscard]] instead of the deprecated ABSL_MUST_USE_RESULT
> Remove the polyfills for absl::apply and absl::make_from_tuple, which were only needed prior to C++17. It is recommended that clients simply use std::apply and std::make_from_tuple.
> PR #1846: Fix build on big endian
> Bazel: Move environment variables to --action_env
> Remove the implementation of `absl::variant`, which was only needed prior to C++17. `absl::variant` is now an alias for `std::variant`. It is recommended that clients simply use `std::variant`.
> MSVC: Fix warnings c4244 and c4267 in the main library code
> Update LowLevelHashLenGt16 to be LowLevelHashLenGt32 now that the input is guaranteed to be >32 in length.
> Xtensa does not support thread_local. Disable it in absl/base/config.h.
> Add support for 8-bit and 16-bit integers to absl::SimpleAtoi
> CI: Update Linux ARM latest container
> Add time hash tests
> `any_invocable`: Update comment that refer to C++17 and C++11
> `check_test_impl.inc`: Use C++17 features unconditionally
> Remove the implementation of `absl::optional`, which was only needed prior to C++17. `absl::optional` is now an alias for `std::optional`. It is recommended that clients simply use `std::optional`.
> Move hashtable control bytes manipulation to a separate file.
> Fix a use-after-free bug in which the string passed to `AtLocation` may be referenced after it is destroyed. While the string does live until the end of the full statement, logging (previously occurred) in the destructor of the `LogMessage` which may be constructed before the temporary string (and thus destroyed after the temporary string's destructor).
> `internal/layout`: Delete pre-C++17 out of line definition of constexpr class member
> Extract slow path for PrepareInsertNonSoo to a separate function `PrepareInsertNonSooSlow`.
> Minor code cleanups
> `internal/log_message`: Use `if constexpr` instead of SFINAE for `operator<<`
> [absl] Use `std::min` in `constexpr` contexts in `absl::string_view`
> Remove the implementation of `absl::any`, which was only needed prior to C++17. `absl::any` is now an alias for `std::any`. It is recommended that clients simply use `std::any`.
> Remove ABSL_INTERNAL_NEED_REDUNDANT_CONSTEXPR_DECL which is longer needed with the C++17 floor
> Make `OptimalMemcpySizeForSooSlotTransfer` ready to work with MaxSooSlotSize upto `3*sizeof(size_t)`.
> `internal/layout`: Replace SFINAE with `if constexpr`
> PR #1830: C++17 improvement: use if constexpr in internal/hash.h
> `absl`: Deprecate `ABSL_HAVE_CLASS_TEMPLATE_ARGUMENT_DEDUCTION`
> Add a verification for access of being destroyed table. Also enabled access after destroy check in ASAN optimized mode.
> Store `CharAlloc` in SwissTable in order to simplify type erasure of functions accepting allocator as `void*`.
> Introduce and use `SetCtrlInLargeTable`, when we know that table is at least one group. Similarly to `SetCtrlInSingleGroupTable`, we can save some operations.
> Make raw_hash_set::slot_type private.
> Delete absl/utility/internal/if_constexpr.h
> `internal/any_invocable`: Use `if constexpr` instead of SFINAE when initializing storage accessor
> Depend on string_view directly
> Optimize and slightly simplify `PrepareInsertNonSoo`.
> PR #1833: Make ABSL_INTERNAL_STEP_n macros consistent in crc code
> `internal/any_invocable`: Use alias `RawT` consistently in `InitializeStorage`
> Move the implementation of absl::ComputeCrc32c to the header file, to facilitate inlining.
> Delete absl/base/internal/inline_variable.h
> Add lifetimebound to absl::StripAsciiWhitespace
> Revert: Random: Use target attribute instead of -march
> Add return for opt mode in AssertNotDebugCapacity to make sure that code is not evaluated in opt mode.
> `internal/any_invocable`: Delete TODO, improve comment and simplify pragma in constructor
> Split resizing routines and type erase similar instructions.
> Random: Use target attribute instead of -march
> `internal/any_invocable`: Use `std::launder` unconditionally
> `internal/any_invocable`: Remove suppresion of false positive -Wmaybe-uninitialized on GCC 12
> Fix feature test for ABSL_HAVE_STD_OPTIONAL
> Support C++20 iterators in raw_hash_map's random-access iterator detection
> Fix mis-located test dependency
> Disable the DestroyedCallsFail test on GCC due to flakiness.
> `internal/any_invocable`: Implement invocation using `if constexpr` instead of SFINAE
> PR #1835: Bump deployment_target version and add visionos to podspec
> PR #1828: Fix spelling of pseudorandom in README.md
> Make raw_hash_map::key_arg private.
> `overload`: Delete obsolete macros for undefining `absl::Overload` when C++ < 17
> `absl/base`: Delete `internal/invoke.h` and `invoke_test.cc`
> Remove `WORKSPACE.bazel`
> `absl`: Replace `base_internal::{invoke,invoke_result_t,is_invocable_r}` with `std` equivalents
> Allow C++20 forward iterators to use fast paths
> Factor out some iterator traits detection code
> Type erase IterateOverFullSlots to decrease code size.
> `any_invocable`: Delete pre-C++17 workarounds for `noexcept` and guaranteed copy elision
> Make raw_hash_set::key_arg private.
> Rename nullability macros to use new lowercase spelling.
> Fix bug where ABSL_REQUIRE_EXPLICIT_INIT did not actually result in a linker error
> Make Randen benchmark program use runtime CPU detection.
> Add CI for the C++20/Clang/libstdc++ combination
> Move Abseil to GoogleTest 1.16.0
> `internal/any_invocable`: Use `if constexpr` instead of SFINAE in `InitializeStorage`
> More type-erasing of InitializeSlots by removing the Alloc and AlignOfSlot template parameters.
> Actually use the hint space instruction to strip PAC bits for return addresses in stack traces as the comment says
> `log/internal`: Replace `..._ATTRIBUTE_UNUSED_IF_STRIP_LOG` with C++17 `[[maybe_unused]]`
> `attributes`: Document `ABSL_ATTRIBUTE_UNUSED` as deprecated
> `internal/any_invocable`: Initialize using `if constexpr` instead of ternary operator, enum, and templates
> Fix flaky tests due to sampling by introducing utility to refresh sampling counters for the current thread.
> Minor reformatting in raw_hash_set: - Add a clear_backing_array member to declutter calls to ClearBackingArray. - Remove some unnecessary `inline` keywords on functions. - Make PoisonSingleGroupEmptySlots static.
> Update CI for linux_gcc-floor to use GCC9, Bazel 7.5, and CMake 3.31.5.
> `internal/any_invocable`: Rewrite `IsStoredLocally` type trait into a simpler constexpr function
> Add ABSL_REQUIRE_EXPLICIT_INIT to Abseil to enable enforcing explicit field initializations
> Require C++17
> Minimize number of `InitializeSlots` with respect to SizeOfSlot.
> Leave the call to `SampleSlow` only in type erased InitializeSlots.
> Update comments for Read4To8 and Read1To3.
> PR #1819: fix compilation with AppleClang
> Move SOO processing inside of InitializeSlots and move it once.
> PR #1816: Random: use getauxval() via <sys/auxv.h>
> Optimize `InitControlBytesAfterSoo` to have less writes and make them with compile time known size.
> Remove stray plus operator in cleanup_internal::Storage
> Include <cerrno> to fix compilation error in chromium build.
> Adjust internal logging namespacing for consistency s/ABSL_LOGGING_INTERNAL_/ABSL_LOG_INTERNAL_/
> Rewrite LOG_EVERY_N (et al) docs to clarify that the first instance is logged. Also, deliberately avoid giving exact numbers or examples since IRL behavior is not so exact.
> ABSL_ASSUME: Use a ternary operator instead of do-while in the implementations that use a branch marked unreachable so that it is usable in more contexts.
> Simplify the comment for raw_hash_set::erase.
> Remove preprocessors for now unsupported compilers.
> `absl::ScopedMockLog`: Explicitly document that it captures logs emitted by all threads
> Fix potential integer overflow in hash container create/resize
> Add lifetimebound to StripPrefix/StripSuffix.
> Random: Rollforward support runtime dispatch on AArch64 macOS
> Crc: Only test non_temporal_store_memcpy_avx on AVX targets
> Provide information about types of all flags.
> Deprecate the precomputed hash find() API in swisstable.
> Import of CCTZ from GitHub.
> Adjust whitespace
> Expand documentation for absl::raw_hash_set::erase to include idiom example of iterator post-increment.
> Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower()
> Crc: Remove the __builtin_cpu_supports path for SupportsArmCRC32PMULL
> Use absl::NoDestructor for some absl::Mutex instances in the flags library to prevent some exit-time destructor warnings
> Update the WORKSPACE dependency of rules_cc to 0.1.0
> Rollback support runtime dispatch on AArch64 macOS for breaking some builds
> Downgrade to rules_cc 0.0.17 because 0.1.0 was yanked
> Use unused set in testing.
> Random: Support runtime dispatch on AArch64 macOS
> crc: Use absl::nullopt when returning absl::optional
> Annotate absl::FixedArray to warn when unused.
> PR #1806: Fix undefined symbol: __android_log_write
> Move ABSL_HAVE_PTHREAD_CPU_NUMBER_NP to the file where it is needed
> Use rbit instruction on ARM rather than rev.
> Debugging: Report the CPU we are running on under Darwin
> Add a microbenchmark for very long int/string tuples.
> Crc: Detect support for pmull and crc instructions on Apple AArch64 With a newer clang, we can use __builtin_cpu_supports which caches all the feature bits.
> Add special handling for hashing integral types so that we can optimize Read1To3 and Read4To8 for the strings case.
> Use unused FixedArray instances.
> Minor reformatting
> Avoid flaky expectation in WaitDurationWoken test case in MSVC.
> Use Bazel rules_cc for many compiler-specific rules instead of our custom ones from before the Bazel rules existed.
> Mix pointers twice in absl::Hash.
> New internal-use-only classes `AsStructuredLiteralImpl` and `AsStructuredValueImpl`
> Annotate some Abseil container methods with [[clang::lifetime_capture_by(...)]]
> Faster copy from inline Cords to inline Strings
> Add new benchmark cases for hashing string lengths 1,2,4,8.
> Move the Arm implementation of UnscaledCycleClock::Now() into the header file, like the x86 implementation, so it can be more easily inlined.
> Minor include cleanup in absl/random/internal
> Import of CCTZ from GitHub.
> Use Bazel Platforms to support AES-NI compile options for Randen
> In HashState::Create, require that T is a subclass of HashStateBase in order to discourage users from defining their own HashState types.
> PR #1801: Remove unncessary <iostream> includes
> New class StructuredProtoField
> Mix pointers twice in TSan and MSVC to avoid flakes in the PointerAlignment test.
> Add a test case that type-erased absl::HashState is consistent with absl::HashOf.
> Mix pointers twice in build modes in which the PointerAlignment test is flaky if we mix once.
> Increase threshold for stuck bits in PointerAlignment test on android.
> Use hashing ideas from Carbon's hashtable in absl hashing: - Use byte swap instead of mixing pointers twice. - Change order of branches to check for len<=8 first. - In len<=16 case, do one multiply to mix the data instead of using the logic from go/absl-hash-rl (reinforcement learning was used to optimize the instruction sequence). - Add special handling for len<=32 cases in 64-bit architectures.
> Test that using a table that was moved-to from a moved-from table fails in sanitizer mode.
> Remove a trailing comma causing an issue for an OSS user
> Add missing includes in hash.h.
> Use the public implementation rule for "@bazel_tools//tools/cpp:clang-cl"
> Import of CCTZ from GitHub.
> Change the definition of is_trivially_relocatable to be a bit less conservative.
> Updates to CI to support newer versions of tools
> Check if ABSL_HAVE_INTRINSIC_INT128 is defined
> Print hash expansions in the hash_testing error messages.
> Avoid flakiness in notification_test on MSVC.
> Roll back: Add more debug capacity validation checks on moves.
> Add more debug capacity validation checks on moves.
> Add macro versions of nullability annotations.
> Improve fork-safety by opening files with `O_CLOEXEC`.
> Move ABSL_HARDENING_ASSERTs in constexpr methods to their own lines.
> Add test cases for absl::Hash: - That hashes are consistent for the same int value across different int types. - That hashes of vectors of strings are unequal even when their concatenations are equal. - That FragmentedCord hashes works as intended for small Cords.
> Skip the IterationOrderChangesOnRehash test case in ASan mode because it's flaky.
> Add missing includes in absl hash.
> Try to use file descriptors in the 2000+ range to avoid mis-behaving client interference.
> Add weak implementation of the __lsan_is_turned_off in Leak Checker
> Fix a bug where EOF resulted in infinite loop.
> static_assert that absl::Time and absl::Duration are trivially destructible.
> Move Duration ToInt64<unit> functions to be inline.
> string_view: Add defaulted copy constructor and assignment
> Use `#ifdef` to avoid errors when `-Wundef` is used.
> Strip PAC bits for return addresses in stack traces
> PR #1794: Update cpu_detect.cc fix hw crc32 and AES capability check, fix undefined
> PR #1790: Respect the allocator's .destroy method in ~InlinedVector
> Cast away nullability in the guts of CHECK_EQ (et al) where Clang doesn't see that the nullable string returned by Check_EQImpl is statically nonnull inside the loop.
> string_view: Correct string_view(const char*, size_type) docs
> Add support for std::string_view in StrCat even when absl::string_view != std::string_view.
> Misc. adjustments to unit tests for logging.
> Use local_config_cc from rules_cc and make it a dev dependency
> Add additional iteration order tests with reservation. Reserved tables have a different way of iteration randomization compared to gradually resized tables (at least for small tables).
> Use all the bits (`popcount`) in `FindFirstNonFullAfterResize` and `PrepareInsertAfterSoo`.
> Mark ConsumePrefix, ConsumeSuffix, StripPrefix, and StripSuffix as constexpr since they are all pure functions.
> PR #1789: Add missing #ifdef pp directive to the TypeName() function in the layout.h
> PR #1788: Fix warning for sign-conversion on riscv
> Make StartsWith and EndsWith constexpr.
> Simplify logic for growing single group table.
> Document that absl::Time and absl::Duration are trivially destructible.
> Change some C-arrays to std::array as this enables bounds checking in some hardened standard library builds
> Replace outdated select() on --cpu with platform API equivalent.
> Take failure_message as const char* instead of string_view in LogMessageFatal and friends.
> Mention `c_any_of` in the function comment of `absl::c_linear_search`.
> Import of CCTZ from GitHub.
> Rewrite some string_view methods to avoid a -Wunreachable-code warning
> IWYU: Update includes and fix minor spelling mistakes.
> Add comment on how to get next element after using erase.
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND and a doc note about absl::LogAsLiteral to clarify its intended use.
> Import of CCTZ from GitHub.
> Reduce memory consumption of structured logging proto encoding by passing tag value
> Remove usage of _LIBCPP_HAS_NO_FILESYSTEM_LIBRARY.
> Make Span's relational operators constexpr since C++20.
> distributions: support a zero max value in Zipf.
> PR #1786: Fix typo in test case.
> absl/random: run clang-format.
> Add some nullability annotations in logging and tidy up some NOLINTs and comments.
> CMake: Change the default for ABSL_PROPAGATE_CXX_STD to ON
> Delete UnvalidatedMockingBitGen
> PR #1783: [riscv][debugging] Fix a few warnings in RISC-V inlines
> Add conversion operator to std::array for StrSplit.
> Add a comment explaining the extra comparison in raw_hash_set::operator==. Also add a small optimization to avoid the extra comparison in sets that use hash_default_eq as the key_equal functor.
> Add benchmark for absl::HexStringToBytes
> Avoid installing options.h with the other headers
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::Span constructors.
> Annotate absl::InlinedVector to warn when unused.
> Make `c_find_first_of`'s `options` parameter a const reference to allow temporaries.
> Disable Elf symbols for Xtensa
> PR #1775: Support symbolize only on WINAPI_PARTITION_DESKTOP
> Require through an internal presubmit that .h|.cc|.inc files contain either the string ABSL_NAMESPACE_BEGIN or SKIP_ABSL_INLINE_NAMESPACE_CHECK
> Xtensa supports mmap, enable it in absl/base/config.h
> PR #1777: Avoid std::ldexp in `operator double(int128)`.
> Marks absl::Span as view and borrowed_range, like std::span.
> Mark inline functions with only a simple comparison in strings/ascii.h as constexpr.
> Add missing Abseil inline namespace and fix includes
> Fix bug where the high bits of `__int128_t`/`__uint128_t` might go unused in the hash function. This fix increases the hash quality of these types.
> Add a test to verify bit casting between signed and unsigned int128 works as expected
> Add suggestions to enable sanitizers for asserts when doing so may be helpful.
> Add nullability attributes to nullability type aliases.
> Refactor swisstable moves.
> Improve ABSL_ASSERT performance by guaranteeing it is optimized away under NDEBUG in C++20
> Mark Abseil hardening assert in AssertSameContainer as slow.
> Add workaround for q++ 8.3.0 (QNX 7.1) compiler by making sure MaskedPointer is trivially copyable and copy constructible.
> Small Mutex::Unlock optimization
> Optimize `CEscape` and `CEscapeAndAppend` by up to 40%.
> Fix the conditional compilation of non_temporal_store_memcpy_avx to verify that AVX can be forced via `gnu::target`.
> Delete TODOs to move functors when moving hashtables and add a test that fails when we do so.
> Fix benchmarks in `escaping_benchmark.cc` by properly calling `benchmark::DoNotOptimize` on both inputs and outputs and by removing the unnecessary and wrong `ABSL_RAW_CHECK` condition (`check != 0`) of `BM_ByteStringFromAscii_Fail` benchmark.
> It seems like commit abc9b916a94ebbf251f0934048295a07ecdbf32a did not work as intended.
> Fix a bug in `absl::SetVLogLevel` where a less generic pattern incorrectly removed a more generic one.
> Remove the side effects between tests in vlog_is_on_test.cc
> Attempt to fix flaky Abseil waiter/sleep tests
> Add an explicit tag for non-SOO CommonFields (removing default ctor) and add a small optimization for early return in AssertNotDebugCapacity.
> Make moved-from swisstables behave the same as empty tables. Note that we may change this in the future.
> Tag tests that currently fail on darwin_arm64 with "no_test_darwin_arm64"
> add gmock to cmake defs for no_destructor_test
> Optimize raw_hash_set moves by allowing some members of CommonFields to be uninitialized when moved-from.
> Add more debug capacity validation checks on iteration/size.
> Add more debug capacity validation checks on copies.
> constinit -> constexpr for DisplayUnits
> LSC: Fix null safety issues diagnosed by Clang’s `-Wnonnull` and `-Wnullability`.
> Remove the extraneous variable creation in Match().
> Import of CCTZ from GitHub.
> Add more debug capacity validation checks on merge/swap.
> Add `absl::` namespace to c_linear_search implementation in order to avoid ADL
> Distinguish the debug message for the case of self-move-assigned swiss tables.
> Update LowLevelHash comment regarding number of hash state variables.
> Add an example for the `--vmodule` flag.
> Remove first prefetch.
> Add moved-from validation for the case of self-move-assignment.
> Allow slow and fast abseil hardening checks to be enabled independently.
> Update `ABSL_RETIRED_FLAG` comment to reflect `default_value` is no longer used.
> Add validation against use of moved-from hash tables.
> Provide file-scoped pragma behind macro ABSL_POINTERS_DEFAULT_NONNULL to indicate the default nullability. This is a no-op for now (not understood by checkers), but does communicate intention to human readers.
> Add stacktrace config for android using the generic implementation
> Fix nullability annotations in ABSL code.
> Replace CHECKs with ASSERTs and EXPECTs -- no reason to crash on failure.
> Remove ABSL_INTERNAL_ATTRIBUTE_OWNER and ABSL_INTERNAL_ATTRIBUTE_VIEW
> Migrate ABSL_INTERNAL_ATTRIBUTE_OWNER and ABSL_INTERNAL_ATTRIBUTE_VIEW to ABSL_ATTRIBUTE_OWNER and ABSL_ATTRIBUTE_VIEW
> Disable ABSL_ATTRIBUTE_OWNER and ABSL_ATTRIBUTE_VIEW prior to Clang-13 due to false positives.
> Make ABSL_ATTRIBUTE_VIEW and ABSL_ATTRIBUTE_OWNER public
> Optimize raw_hash_set::AssertHashEqConsistent a bit to avoid having as much runtime overhead.
> PR #1728: Workaround broken compilation against NDK r25
> Add validation against use of destroyed hash tables.
> Do not truncate `ABSL_RAW_LOG` output at null bytes
> Use several unused cord instances in tests and benchmarks.
> Add comments about ThreadIdentity struct allocation behavior.
> Refactoring followup for reentrancy validation in swisstable.
> Add debug mode checks that element constructors/destructors don't make reentrant calls to raw_hash_set member functions.
> Add tagging for cc_tests that are incompatible with Fuchsia
> Add GetTID() implementation for Fuchsia
> PR #1738: Fix shell option group handling in pkgconfig files
> Disable weak attribute when absl compiled as windows DLL
> Remove `CharIterator::operator->`.
> Mark non-modifying container algorithms as constexpr for C++20.
> PR #1739: container/internal: Explicitly include <cstdint>
> Don't match -Wnon-virtual-dtor in the "flags are needed to suppress warnings in headers". It should fall through to the "don't impose our warnings on others" case. Do this by matching on "-Wno-*" instead of "-Wno*".
> PR #1732: Fix build on NVIDIA Jetson board. Fix#1665
> Update GoogleTest dependency to 1.15.2
> Enable AsciiStrToLower and AsciiStrToUpper overloads for rvalue references.
> PR #1735: Avoid `int` to `bool` conversion warning
> Add `absl::swap` functions for `*_hash_*` to avoid calling `std::swap`
> Change internal visibility
> Remove resolved issue.
> Increase test timeouts to support running on Fuchsia emulators
> Add tracing annotations to absl::Notification
> Suppress compiler optimizations which may break container poisoning.
> Disable ABSL_INTERNAL_HAVE_DEBUGGING_STACK_CONSUMPTION for Fuchsia
> Add tracing annotations to absl::BlockingCounter
> Add absl_vlog_is_on and vlog_is_on to ABSL_INTERNAL_DLL_TARGETS
> Update swisstable swap API comments to no longer guarantee that we don't move/swap individual elements.
> PR #1726: cmake: Fix RUNPATH when using BUILD_WITH_INSTALL_RPATH=True
> Avoid unnecessary copying when upper-casing or lower-casing ASCII string_view
> Add weak internal tracing API
> Fix LINT.IfChange syntax
> PR #1720: Fix spelling mistake: occurrance -> occurrence
> Add missing include for Windows ASAN configuration in poison.cc
> Delete absl/strings/internal/has_absl_stringify.h now that the GoogleTest version we depend on uses the public file
> Update versions of dependencies in preparation for release
> PR #1699: Add option to build with MSVC static runtime
> Remove unneeded 'be' from comment.
> PR #1715: Generate options.h using CMake only once
> Small type fix in absl/log/internal/log_impl.h
> PR #1709: Handle RPATH CMake configuration
> PR #1710: fixup! PR #1707: Fixup absl_random compile breakage in Apple ARM64 targets
> PR #1695: Fix time library build for Apple platforms
> Remove cyclic cmake dependency that breaks in cmake 3.30.0
> Roll forward poisoned pointer API and fix portability issues.
> Use GetStatus in IsOkAndHoldsMatcher
> PR #1707: Fixup absl_random compile breakage in Apple ARM64 targets
> PR #1706: Require CMake version 3.16
> Add an MSVC implementation of ABSL_ATTRIBUTE_LIFETIME_BOUND
> Mark c_min_element, c_max_element, and c_minmax_element as constexpr in C++17.
> Optimize the absl::GetFlag cost for most non built-in flag types (including string).
> Encode some additional metadata when writing protobuf-encoded logs.
> Replace signed integer overflow, since that's undefined behavior, with unsigned integer overflow.
> Make mutable CompressedTuple::get() constexpr.
> vdso_support: support DT_GNU_HASH
> Make c_begin, c_end, and c_distance conditionally constexpr.
> Add operator<=> comparison to absl::Time and absl::Duration.
> Deprecate `ABSL_ATTRIBUTE_NORETURN` in favor of the `[[noreturn]]` standardized in C++11
> Rollback new poisoned pointer API
> Static cast instead of reinterpret cast raw hash set slots as casting from void* to T* is well defined
> Fix absl::NoDestructor documentation about its use as a global
> Declare Rust demangling feature-complete.
> Split demangle_internal into a tree of smaller libraries.
> Decode Rust Punycode when it's not too long.
> Add assertions to detect reentrance in `IterateOverFullSlots` and `absl::erase_if`.
> Decoder for Rust-style Punycode encodings of bounded length.
> Add `c_contains()` and `c_contains_subrange()` to `absl/algorithm/container.h`.
> Three-way comparison spaceship <=> operators for Cord.
> internal-only change
> Remove erroneous preprocessor branch on SGX_SIM.
> Add an internal API to get a poisoned pointer.
> optimization.h: Add missing <utility> header for C++
> Add a compile test for headers that require C compatibility
> Fix comment typo
> Expand documentation for SetGlobalVLogLevel and SetVLogLevel.
> Roll back 6f972e239f668fa29cab43d7968692cd285997a9
> PR #1692: Add missing `<utility>` include
> Remove NOLINT for `#include <new>` for __cpp_lib_launder
> Remove not used after all kAllowRemoveReentrance parameter from IterateOverFullSlots.
> Create `absl::container_internal::c_for_each_fast` for SwissTable.
> Disable flaky test cases in kernel_timeout_internal_test.
> Document that swisstable and b-tree containers are not exception-safe.
> Add `ABSL_NULLABILITY_COMPATIBLE` attribute.
> LSC: Move expensive variables on their last use to avoid copies.
> Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to more types in Abseil
> Drop std:: qualification from integer types like uint64_t.
> Increase slop time on MSVC in PerThreadSemTest.Timeouts again due to continued flakiness.
> Turn on validation for out of bounds MockUniform in MockingBitGen
> Use ABSL_UNREACHABLE() instead of equivalent
> If so configured, report which part of a C++ mangled name didn't parse.
> Sequence of 1-to-4 values with prefix sum to support Punycode decoding.
> Add the missing inline namespace to the nullability files
> Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to types in Abseil
> Disallow reentrance removal in `absl::erase_if`.
> Fix implicit conversion of temporary bitgen to BitGenRef
> Use `IterateOverFullSlots` in `absl::erase_if` for hash table.
> UTF-8 encoding library to support Rust Punycode decoding.
> Disable negative NaN float ostream format checking on RISC-V
> PR #1689: Minor: Add missing quotes in CMake string view library definition
> Demangle template parameter object names, TA <template-arg>.
> Demangle sr St <simple-id> <simple-id>, a dubious encoding found in the wild.
> Try not to lose easy type combinators in S::operator const int*() and the like.
> Demangle fixed-width floating-point types, DF....
> Demangle _BitInt types DB..., DU....
> Demangle complex floating-point literals.
> Demangle <extended-qualifier> in types, e.g., U5AS128 for address_space(128).
> Demangle operator co_await (aw).
> Demangle fully general vendor extended types (any <template-args>).
> Demangle transaction-safety notations GTt and Dx.
> Demangle C++11 user-defined literal operator functions.
> Demangle C++20 constrained friend names, F (<source-name> | <operator-name>).
> Demangle dependent GNU vector extension types, Dv <expression> _ <type>.
> Demangle elaborated type names, (Ts | Tu | Te) <name>.
> Add validation that hash/eq functors are consistent, meaning that `eq(k1, k2) -> hash(k1) == hash(k2)`.
> Demangle delete-expressions with the global-scope operator, gs (dl | da) ....
> Demangle new-expressions with braced-init-lists.
> Demangle array new-expressions, [gs] na ....
> Demangle object new-expressions, [gs] nw ....
> Demangle preincrement and predecrement, pp_... and mm_....
> Demangle throw and rethrow (tw... and tr).
> Remove redundant check of is_soo() while prefetching heap blocks.
> Demangle ti... and te... expressions (typeid).
> Demangle nx... syntax for noexcept(e) as an expression in a dependent signature.
> Demangle alignof expressions, at... and az....
> Demangle C++17 structured bindings, DC...E.
> Demangle modern _ZGR..._ symbols.
> Remove redundant check of is_soo() while prefetching heap blocks.
> Demangle sizeof...(pack captured from an alias template), sP ... E.
> Demangle types nested under vendor extended types.
> Demangle il ... E syntax (braced list other than direct-list-initialization).
> Avoid signed overflow for Ed <number> _ manglings with large <number>s.
> Remove redundant check of is_soo() while prefetching heap blocks.
> Remove obsolete TODO
> Clarify function comment for `erase` by stating that this idiom only works for "some" standard containers.
> Move SOVERSION to global CMakeLists, apply SOVERSION to DLL
> Set ABSL_HAVE_THREAD_LOCAL to 1 on all platforms
> Demangle constrained auto types (Dk <type-constraint>).
> Parse <discriminator> more accurately.
> Demangle lambdas in class member functions' default arguments.
> Demangle unofficial <unresolved-qualifier-level> encodings like S0_IT_E.
> Do not make std::filesystem::path hash available for macOS <10.15
> Include flags in DLL build (non-Windows only)
> Enable building monolithic shared library on macOS and Linux.
> Demangle Clang's last-resort notation _SUBSTPACK_.
> Demangle C++ requires-expressions with parameters (rQ ... E).
> Demangle Clang's encoding of __attribute__((enable_if(condition, "message"))).
> Demangle static_cast and friends.
> Demangle decltype(expr)::nested_type (NDT...E).
> Optimize GrowIntoSingleGroupShuffleControlBytes.
> Demangle C++17 fold-expressions.
> Demangle thread_local helper functions.
> Demangle lambdas with explicit template arguments (UlTy and similar forms).
> Demangle &-qualified function types.
> Demangle valueless literals LDnE (nullptr) and LA<number>_<type>E ("foo").
> Correctly demangle the <unresolved-name> at the end of dt and pt (x.y, x->y).
> Add missing targets to ABSL_INTERNAL_DLL_TARGETS
> Build abseil_test_dll with ABSL_BUILD_TESTING
> Demangle C++ requires-expressions without parameters (rq ... E).
> overload: make the constructor constexpr
> Update Abseil CI Docker image to use Clang 19, GCC 14, and CMake 3.29.3
> Workaround symbol resolution bug in Clang 19
> Workaround bogus GCC14 -Wmaybe-uninitialized warning
> Silence a bogus GCC14 -Warray-bounds warning
> Forbid absl::Uniform<absl::int128>(gen)
> Use IN_LIST to replace list(FIND) + > -1
> Recognize C++ vendor extended expressions (e.g., u9__is_same...E).
> `overload_test`: Remove a few unnecessary trailing return types
> Demangle the C++ this pointer (fpT).
> Stop eating an extra E in ParseTemplateArg for some L<type><value>E literals.
> Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to Abseil.
> Demangle C++ direct-list-initialization (T{1, 2, 3}, tl ... E).
> Demangle the C++ spaceship operator (ss, operator<=>).
> Demangle C++ sZ encodings (sizeof...(pack)).
> Demangle C++ so ... E encodings (typically array-to-pointer decay).
> Recognize dyn-trait-type in Rust demangling.
> Rework casting in raw_hash_set's IsFull().
> Remove test references to absl::SharedBitGen, which was never part of the open source release. This was only used in tests that never ran as part in the open source release.
> Recognize fn-type and lifetimes in Rust demangling.
> Support int128/uint128 in validated MockingBitGen
> Recognize inherent-impl and trait-impl in Rust demangling.
> Recognize const and array-type in Rust mangled names.
> Remove Asylo from absl.
> Recognize generic arguments containing only types in Rust mangled names.
> Fix missing #include <random> for std::uniform_int_distribution
> Move `prepare_insert` out of the line as type erased `PrepareInsertNonSoo`.
> Revert: Add -Wdead-code-aggressive to ABSL_LLVM_FLAGS
> Add (unused) validation to absl::MockingBitGen
> Support `AbslStringify` with `DCHECK_EQ`.
> PR #1672: Optimize StrJoin with tuple without user defined formatter
> Give ReturnAddresses and N<uppercase> namespaces separate stacks for clarity.
> Demangle Rust backrefs.
> Use Nt for struct and trait names in Rust demangler test inputs.
> Allow __cxa_demangle on MIPS
> Add a `string_view` overload to `absl::StrJoin`
> Demangle Rust's Y<type><path> production for passably simple <type>s.
> `convert_test`: Delete obsolete condition around ASSERT_EQ in TestWithMultipleFormatsHelper
> `any_invocable`: Clean up #includes
> Resynchronize absl/functional/CMakeLists.txt with BUILD.bazel
> `any_invocable`: Add public documentation for undefined behavior when invoking an empty AnyInvocable
> `any_invocable`: Delete obsolete reference to proposed standard type
> PR #1662: Replace shift with addition in crc multiply
> Doc fix.
> `convert_test`: Extract loop over tested floats from helper function
> Recognize some simple Rust mangled names in Demangle.
> Use __builtin_ctzg and __builtin_clzg in the implementations of CountTrailingZeroesNonzero16 and CountLeadingZeroes16 when they are available.
> Remove the forked absl::Status matchers implementation in statusor_test
> Add comment hack to fix copybara reversibility
> Add GoogleTest matchers for absl::Status
> [random] LogUniform: Document as a discrete distribution
> Enable Cord tests with Crc.
> Fix order of qualifiers in `absl::AnyInvocable` documentation.
> Guard against null pointer dereference in DumpNode.
> Apply ABSL_MUST_USE_RESULT to try lock functions.
> Add public aliases for default hash/eq types in hash-based containers
> Import of CCTZ from GitHub.
> Remove the hand-rolled CordLeaker and replace with absl::NoDestructor to test the after-exit behavior
> `convert_test`: Delete obsolete `skip_verify` parameter in test helper
> overload: allow using the underlying type with CTAD directly.
> PR #1653: Remove unnecessary casts when calling CRC32_u64
> PR #1652: Avoid C++23 deprecation warnings from float_denorm_style
> Minor cleanup for `absl::Cord`
> PR #1651: Implement ABSL_INTERNAL_DISABLE_DEPRECATED_DECLARATION_WARNING for MSVC compiler
> Add `operator<=>` support to `absl::int128` and `absl::uint128`
> [absl] Re-use the existing `std::type_identity` backfill instead of redefining it again
> Add `absl::AppendCordToString`
> `str_format/convert_test`: Delete workaround for [glibc bug](https://sourceware.org/bugzilla/show_bug.cgi?id=22142)
> `absl/log/internal`: Document conditional ABSL_ATTRIBUTE_UNUSED, add C++17 TODO
> `log/internal/check_op`: Add ABSL_ATTRIBUTE_UNUSED to CHECK macros when STRIP_LOG is enabled
> log_benchmark: Add VLOG_IS_ON benchmark
> Restore string_view detection check
> Remove an unnecessary ABSL_ATTRIBUTE_UNUSED from a logging macro
< Abseil LTS Branch, Jan 2024, Patch 2 (#1650)
> In example code, add missing template parameter.
> Optimize crc32 V128_From2x64 on Arm
> Annotate that Mutex should warn when unused.
> Add ABSL_ATTRIBUTE_LIFETIME_BOUND to Cord::Flatten/TryFlat
> Deprecate `absl::exchange`, `absl::forward` and `absl::move`, which were only useful before C++14.
> Temporarily revert dangling std::string_view detection until dependent is fixed
> Use _decimal_ literals for the CivilDay example.
> Fix bug in BM_EraseIf.
> Add internal traits to absl::string_view for lifetimebound detection
> Add internal traits to absl::StatusOr for lifetimebound detection
> Add internal traits to absl::Span for lifetimebound detection
> Add missing dependency for log test build target
> Add internal traits for lifetimebound detection
> Use local decoding buffer in HexStringToBytes
> Only check if the frame pointer is inside a signal stack with known bounds
> Roll forward: enable small object optimization in swisstable.
> Optimize LowLevelHash by breaking dependency between final loads and previous len/ptr updates.
> Fix the wrong link.
> Optimize InsertMiss for tables without kDeleted slots.
> Use GrowthInfo without applying any optimizations based on it.
> Disable small object optimization while debugging some failing tests.
> Adjust conditonal compilation in non_temporal_memcpy.h
> Reformat log/internal/BUILD
> Remove deprecated errno constants from the absl::Status mapping
> Introduce GrowthInfo with tests, but without usage.
> Enable small object optimization in swisstable.
> Refactor the GCC unintialized memory warning suppression in raw_hash_set.h.
> Respect `NDEBUG_SANITIZER`
> Revert integer-to-string conversion optimizations pending more thorough analysis
> Fix a bug in `Cord::{Append,Prepend}(CordBuffer)`: call `MaybeRemoveEmptyCrcNode()`. Otherwise appending a `CordBuffer` an empty Cord with a CRC node crashes (`RemoveCrcNode()` which increases the refcount of a nullptr child).
> Add `BM_EraseIf` benchmark.
> Record sizeof(key_type), sizeof(value_type) in hashtable profiles.
> Fix ClangTidy warnings in btree.h.
> LSC: Move expensive variables on their last use to avoid copies.
> PR #1644: unscaledcycleclock: remove RISC-V support
> Reland: Make DLOG(FATAL) not understood as [[noreturn]]
> Separate out absl::StatusOr constraints into statusor_internal.h
> Use Layout::WithStaticSizes in btree.
> `layout`: Delete outdated comments about ElementType alias not being used because of MSVC
> Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower()
> `layout_benchmark`: Replace leftover comment with intended call to MyAlign
> Remove absl::aligned_storage_t
> Delete ABSL_ANNOTATE_MEMORY_IS_INITIALIZED under Thread Sanitizer
> Remove vestigial variables in the DumpNode() helper in absl::Cord
> Do hashtablez sampling on the first insertion into an empty SOO hashtable.
> Add explicit #include directives for <tuple>, "absl/base/config.h", and "absl/strings/string_view.h".
> Add a note about the cost of `VLOG` in non-debug builds.
> Fix flaky test failures on MSVC.
> Add template keyword to example comment for Layout::WithStaticSizes.
> PR #1643: add xcprivacy to all subspecs
> Record sampling stride in cord profiling to facilitate unsampling.
> Fix a typo in a comment.
> [log] Correct SetVLOGLevel to SetVLogLevel in comments
> Add a feature to container_internal::Layout that lets you specify some array sizes at compile-time as template parameters. This can make offset and size calculations faster.
> `layout`: Mark parameter of Slices with ABSL_ATTRIBUTE_UNUSED, remove old workaround
> `layout`: Use auto return type for functions that explicitly instantiate std::tuple in return statements
> Remove redundant semicolons introduced by macros
> [log] Make :vlog_is_on/:absl_vlog_is_on public in BUILD.bazel
> Add additional checks for size_t overflows
> Replace //visibility:private with :__pkg__ for certain targets
> PR #1603: Disable -Wnon-virtual-dtor warning for CommandLineFlag implementations
> Add several missing includes in crc/internal
> Roll back extern template instatiations in swisstable due to binary size increases in shared libraries.
> Add nodiscard to SpinLockHolder.
> Test that rehash(0) reduces capacity to minimum.
> Add extern templates for common swisstable types.
> Disable ubsan for benign unaligned access in crc_memcpy
> Make swisstable SOO support GDB pretty printing and still compile in OSS.
> Fix OSX support with CocoaPods and Xcode 15
> Fix GCC7 C++17 build
> Use UnixEpoch and ZeroDuration
> Make flaky failures much less likely in BasicMocking.MocksNotTriggeredForIncorrectTypes test.
> Delete a stray comment
> Move GCC uninitialized memory warning suppression into MaybeInitializedPtr.
> Replace usages of absl::move, absl::forward, and absl::exchange with their std:: equivalents
> Fix the move to itself
> Work around an implicit conversion signedness compiler warning
> Avoid MSan: use-of-uninitialized-value error in find_non_soo.
> Fix flaky MSVC test failures by using longer slop time.
> Add ABSL_ATTRIBUTE_UNUSED to variables used in an ABSL_ASSUME.
> Implement small object optimization in swisstable - disabled for now.
> Document and test ability to use absl::Overload with generic lambdas.
> Extract `InsertPosition` function to be able to reuse it.
> Increase GraphCycles::PointerMap size
> PR #1632: inlined_vector: Use trivial relocation for `erase`
> Create `BM_GroupPortable_Match`.
> [absl] Mark `absl::NoDestructor` methods with `absl::Nonnull` as appropriate
> Automated Code Change
> Rework casting in raw_hash_set's `IsFull()`.
> Adds ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::BitGenRef
> Workaround for NVIDIA C++ compiler being unable to parse variadic expansions in range of range-based for loop
> Rollback: Make DLOG(FATAL) not understood as [[noreturn]]
> Make DLOG(FATAL) not understood as [[noreturn]]
> Optimize `absl::Duration` division and modulo: Avoid repeated redundant comparisons in `IDivFastPath`.
> Optimize `absl::Duration` division and modulo: Allow the compiler to inline `time_internal::IDivDuration`, by splitting the slow path to a separate function.
> Fix typo in example code snippet.
> Automated Code Change
> Add braces for conditional statements in raw_hash_map functions.
> Optimize `prepare_insert`, when resize happens. It removes single unnecessary probing before resize that is beneficial for small tables the most.
> Add noexcept to move assignment operator and swap function
> Import of CCTZ from GitHub.
> Minor documentation updates.
> Change find_or_prepare_insert to return std::pair<iterator, bool> to match return type of insert.
> PR #1618: inlined_vector: Use trivial relocation for `SwapInlinedElements`
> Improve raw_hash_set tests.
> Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower()
> Use const_cast to avoid duplicating the implementation of raw_hash_set::find(key).
> Import of CCTZ from GitHub.
> Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower()
> Annotate that SpinLock should warn when unused.
> PR #1625: absl::is_trivially_relocatable now respects assignment operators
> Introduce `Group::MaskNonFull` without usage.
> `demangle`: Parse template template and C++20 lambda template param substitutions
> PR #1617: fix MSVC 32-bit build with -arch:AVX
> Minor documentation fix for `absl::StrSplit()`
> Prevent overflow in `absl::CEscape()`
> `demangle`: Parse optional single template argument for built-in types
> PR #1412: Filter out `-Xarch_` flags from pkg-config files
> `demangle`: Add complexity guard to `ParseQRequiresExpr`
< Prepare 20240116.1 patch for Apple Privacy Manifest (#1623)
> Remove deprecated symbol absl::kuint128max
> Add ABSL_ATTRIBUTE_WARN_UNUSED.
> `demangle`: Parse `requires` clauses on template params, before function return type
> On Apple, implement absl::is_trivially_relocatable with the fallback.
> `demangle`: Parse `requires` clauses on functions
> Make `begin()` to return `end()` on empty tables.
> `demangle`: Parse C++20-compatible template param declarations, except those with `requires` expressions
> Add the ABSL_DEPRECATE_AND_INLINE() macro
> Span: Fixed comment referencing std::span as_writable_bytes() as as_mutable_bytes().
> Switch rank structs to be consistent with written guidance in go/ranked-overloads
> Avoid hash computation and `Group::Match` in small tables copy and use `IterateOverFullSlots` for iterating for all tables.
> Optimize `absl::Hash` by making `LowLevelHash` faster.
> Add -Wdead-code-aggressive to ABSL_LLVM_FLAGS
< Backport Apple Privacy Manifest (#1613)
> Stop using `std::basic_string<uint8_t>` which relies on a non-standard generic `char_traits<>` implementation, recently removed from `libc++`.
> Add absl_container_hash-based HashEq specialization
> `demangle`: Implement parsing for simplest constrained template arguments
> Roll forward 9d8588bfc4566531c4053b5001e2952308255f44 (which was rolled back in 146169f9ad357635b9cd988f976b38bcf83476e3) with fix.
> Add a version of absl::HexStringToBytes() that returns a bool to validate that the input was actually valid hexadecimal data.
> Enable StringLikeTest in hash_function_defaults_test
> Fix a typo.
> Minor changes to the BUILD file for absl/synchronization
> Avoid static initializers in case of ABSL_FLAGS_STRIP_NAMES=1
> Rollback 9d8588bfc4566531c4053b5001e2952308255f44 for breaking the build
> No public description
> Decrease the precision of absl::Now in x86-64 debug builds
> Optimize raw_hash_set destructor.
> Add ABSL_ATTRIBUTE_UNINITIALIZED macros for use with clang and GCC's `uninitialized`
> Optimize `Cord::Swap()` for missed compiler optimization in clang.
> Type erased hash_slot_fn that depends only on key types (and hash function).
> Replace `testonly = 1` with `testonly = True` in abseil BUILD files.
> Avoid extra `& msbs` on every iteration over the mask for GroupPortableImpl.
> Missing parenthesis.
> Early return from destroy_slots for trivially destructible types in flat_hash_{*}.
> Avoid export of testonly target absl::test_allocator in CMake builds
> Use absl::NoDestructor for cordz global queue.
> Add empty WORKSPACE.bzlmod
> Introduce `RawHashSetLayout` helper class.
> Fix a corner case in SpyHashState for exact boundaries.
> Add nullability annotations
> Use absl::NoDestructor for global HashtablezSampler.
> Always check if the new frame pointer is readable.
> PR #1604: Add privacy manifest
< Disable ABSL_ATTRIBUTE_TRIVIAL_ABI in open-source builds (#1606)
> Remove code pieces for no longer supported GCC versions.
> Disable ABSL_ATTRIBUTE_TRIVIAL_ABI in open-source builds
> Prevent brace initialization of AlphaNum
> Remove code pieces for no longer supported MSVC versions.
> Added benchmarks for smaller size copy constructors.
> Migrate empty CrcCordState to absl::NoDestructor.
> Add protected copy ctor+assign to absl::LogSink, and clarify thread-safety requirements to apply to the interface methods.
< Apply LTS transformations for 20240116 LTS branch (#1599)
Closesscylladb/scylladb#28756
The rack option was fully implemented in the code but omitted from
both docs/operating-scylla/admin.rst and conf/scylla.yaml comments.
Closesscylladb/scylladb#29239
In a multi-declarator declaration, the & ref-qualifier is part of each
individual declarator, not the shared type specifier. So:
const auto& a = x(), b = y();
declares 'a' as a reference but 'b' as a value, silently copying y().
The same applies to:
const T& a = v[i], b = v[j];
Both operator== lines had this pattern, causing an unnecessary copy of
the column vector and an unnecessary copy of each entry on every call.
Fix by repeating & on the second declarator in both lines.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29213
clear_gently() (introduced in 322aa2f8b5) clears all token_metadata_impl
members using co_await to avoid reactor stalls on large data structures.
_topology_change_info (introduced in 10bf8c7901) was added later and not
included in clear_gently().
update_topology_change_info() already uses utils::clear_gently() when
replacing the value, so it looks reasonable to apply the same pattern
in clear_gently().
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closesscylladb/scylladb#29210
Some on_internal_error() calls have the selector argument to a format
string with no placeholder for it in the format string.
"While at it", disambiguate selector type in the message text.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Closesscylladb/scylladb#29208
There's only one test left that uses it, and it can be patched to use standard ks/cf creation helpers from pylib. This patch does so and drops the lengthy create_dataset() helper
Tests improvements, no need to backport
Closesscylladb/scylladb#29176
* github.com:scylladb/scylladb:
test/backup: drop create_dataset helper
test/backup: use new_test_keyspace in test_restore_primary_replica
The enable_tablets(false) was added when LWT wasn't supported for tablets, now it's, so no need in this attribute are more.
The test covers behavior which should work in similar way for both vnodes and tablets -> it doesn't seem it would benefit much from running it in both enable_tablets(true) and enable_tablets(false) modes.
Closesscylladb/scylladb#29167
In order to apply fsult-injected delay, there's the inject(duration)
overload. Results in shorter code
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Closesscylladb/scylladb#29168
The endpoint in question has some places worth fixing, in particular
- the keyspace parameter is not validated
- the validated table name is resolved into table_id, but the id is unused
- two ugly static helpers to stream obtained token ranges into json
Improving the API code flow, not backporting
Closesscylladb/scylladb#29154
* github.com:scylladb/scylladb:
api: Inline describe_ring JSON handling
storage_service: Make describe_ring_for_table() take table_id
Add skip_reason_plugin.py — a framework-agnostic pytest plugin that
provides typed skip markers (skip_bug, skip_not_implemented, skip_slow,
skip_env) so that the reason a test is skipped is machine-readable in
JUnit XML and Allure reports. Bare untyped pytest.mark.skip now
triggers a warning (to become an error after full migration). Runtime
skips via skip() are also enriched by parsing the [type] prefix from
the skip message.
The plugin is a class (SkipReasonPlugin) that receives the concrete
SkipType enum and an optional report_callback from conftest.py, keeping
it decoupled from allure and project-specific types.
Extract SkipType enum and convenience runtime skip wrappers (skip_bug,
skip_env, etc.) into test/pylib/skip_types.py so callers only need a
single import instead of importing both SkipType and skip() separately.
conftest.py imports SkipType from the new module and registers the
plugin instance unconditionally (for all test runners).
New files:
- test/pylib/skip_reason_plugin.py: core plugin — typed marker
processing, bare-skip warnings, JUnit/Allure report enrichment
(including runtime skip() parsing via _parse_skip_type helper)
- test/pylib/skip_types.py: SkipType enum and convenience wrappers
(skip_bug, skip_not_implemented, skip_slow, skip_env)
- test/pylib_test/test_skip_reason_plugin.py: 17 pytester-based
test functions (51 cases across 3 build modes) covering markers,
warnings, reports, callbacks, and skip_mode interaction
Infrastructure changes:
- test/conftest.py: import SkipType from skip_types, register
SkipReasonPlugin with allure report callback
- test/pylib/runner.py: set SKIP_TYPE_KEY/SKIP_REASON_KEY stash keys
for skip_mode so the report hook can enrich JUnit/Allure with
skip_type=mode without longrepr parsing
- test/pytest.ini: register typed marker definitions (required for
--strict-markers even when plugin is not loaded)
Migrated test files (representative samples):
- test/cluster/test_tablet_repair_scheduler.py:
skip -> skip_bug (#26844), skip -> skip_not_implemented
- test/cqlpy/.../timestamp_test.py: skip -> skip_slow
- test/cluster/dtest/schema_management_test.py: skip -> skip_not_implemented
- test/cluster/test_change_replication_factor_1_to_0.py: skip -> skip_bug (#20282)
- test/alternator/conftest.py: skip -> skip_env
- test/alternator/test_https.py: use skip_env() wrapper
Fixes SCYLLADB-79
Closesscylladb/scylladb#29235
This patch series implements `object_storage_base::clone`, which was previously a stub that aborted at runtime. Clone creates a copy of an sstable under a new generation and is used during compaction.
The implementation uses server-side object copies (S3 CopyObject / GCS Objects: rewrite) and mirrors the filesystem clone semantics: TemporaryTOC is written first to mark the operation as in-progress, component objects are copied, and TemporaryTOC is removed to commit (unless the caller requested the destination be left unsealed).
The first two patches fix pre-existing bugs in the underlying storage clients that were exposed by the new clone code path:
- GCS `copy_object` used the wrong HTTP method (PUT instead of POST) and sent an invalid empty request body.
- S3 `copy_object` silently ignored the abort_source parameter.
1. **gcp_client: fix copy_object request method and body** — Fix two bugs in the GCS rewrite API call.
2. **s3_client: pass through abort_source in copy_object** — Stop ignoring the abort_source parameter.
3. **object_storage: add copy_object to object_storage_client** — New interface method with S3 and GCS implementations.
4. **storage: add make_object_name overload with generation** — Helper for building destination object names with a different generation.
5. **storage: make delete_object const** — Needed by the const clone method.
6. **storage: implement object_storage_base::clone** — The actual clone implementation plus a copy_object wrapper.
7. **test/boost: enable sstable clone tests for S3 and GCS** — Re-enable the previously skipped tests.
A test similar to `sstable_clone_leaving_unsealed_dest_sstable` was added to properly test the sealed/unsealed states for object storage. Works for both S3 and GCS.
Fixes: https://scylladb.atlassian.net/browse/SCYLLADB-1045
Prerequisite: https://github.com/scylladb/scylladb/pull/28790
No need to backport since this code targets future feature
Closesscylladb/scylladb#29166
* github.com:scylladb/scylladb:
compaction_test: enable sstable clone tests for S3 and GCS
storage: implement object_storage_base::clone
storage: make delete_object const in object_storage_base
storage: add make_object_name overload with generation
sstables: add get_format() accessor to sstable
object_storage: add copy_object to object_storage_client
s3_client: pass through abort_source in copy_object
gcp_client: fix copy_object request method and body
The vector-search feature introduced the somewhat confusing feature of
enabling CDC without explicitly enabling CDC: When a vector index is
enabled on a table, CDC is "enabled" for it even if the user didn't
ask to enable CDC.
For this, write-path code began to use a new cdc_enabled() function
instead of checking schema.cdc_options.enabled() directly. This
cdc_enabled() function checks if either this enabled() is true, or
has_vector_index() is true.
Unfortunately, LWT writes continued to use cdc_options.enabled() instead
of the new cdc_enabled(). This means that if a vector index is used and
a vector is written using an LWT write, the new value is not indexed.
This patch fixes this bug. It also adds a regression test that fails
before this patch and passes afterwards - the new test verifies that
when a table has a vector index (but no explicit CDC enabled), the CDC
log is updated both after regular writes and after successful LWT writes.
This patch was also tested in the context of the upcoming vector-search-
for-Alternator pull request, which has a test reproducing this bug
(Alternator uses LWT frequently, so this is very important there).
It will also be tested by the vector-store test suite ("validator").
Fixes SCYLLADB-1342
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Closesscylladb/scylladb#29300
In the unregistered-ID branch, ldap_msgfree() was called on a result
already owned by an RAII ldap_msg_ptr, causing a double-free on scope
exit. Remove the redundant manual free.
Fixes: SCYLLADB-1344
Backport: 2026.1, 2025.4, 2025.1 - it's a memory corruption, with a one-line fix, so better backport it everywhere.
Closesscylladb/scylladb#29302
* github.com:scylladb/scylladb:
test: ldap: add regression test for double-free on unregistered message ID
ldap: fix double-free of LDAPMessage in poll_results()
Now that object_storage_base::clone is implemented,
remove the early-return skips and re-enable the
sstable_clone_leaving_unsealed_dest_sstable tests for
both S3 and GCS storage backends.
Implement the clone method for object_storage_base, which creates
a copy of an sstable with a new generation using server-side object
copies. Also add a const copy_object convenience wrapper, similar
to the existing put_object and delete_object wrappers.
A dedicated test for the new object storage clone path will be
added in the following commit. The preexisting local-filesystem
clone is already covered by the sstable_clone_leaving_unsealed_dest_sstable
test.
Add a make_object_name overload that accepts a target
generation parameter for constructing object names with
a generation different from the source sstable's own.
Refactor the original make_object_name to delegate to
the new overload, eliminating code duplication.
This is needed by clone to build destination object
names for the new generation.
Add a public get_format() accessor for the _format member, following
the same pattern as the existing get_version(). This allows storage
implementations to access the sstable format without reaching into
private members, and is needed by the upcoming object_storage_base::clone
to construct entry_descriptor for the sstables registry.
Add a copy_object method to the object_storage_client
interface for server-side object copies, with
implementations for both S3 and GCS wrappers.
The S3 wrapper delegates to s3::client::copy_object.
The GCS wrapper delegates to gcp::storage::client's
cross-bucket copy_object overload.
This is a prerequisite for implementing sstable clone
on object storage.
The abort_source parameter in s3::client::copy_object
was ignored — the function accepted it but always passed
nullptr to the underlying copy_s3_object. Forward it
properly so callers can cancel in-progress copies.
The GCP copy_object (rewrite API) had two bugs:
1. The request body was an empty string, but the GCP
rewrite endpoint always parses it as JSON metadata.
An empty string is not valid JSON, resulting in
400 "Metadata in the request couldn't decode".
Fix: send "{}" (empty JSON object) as the body.
2. The HTTP method was PUT, but the GCP Objects: rewrite
API requires POST per the documentation.
Fix: use POST.
Test coverage in a follow-up patch
To create `process_staging` view building tasks, we firstly need to
collect informations about them on shard0, create necessary mutations,
commit them to group0 and move staging sstables objects to their
original shards.
But there is a possible race after committing the group0 command
and before moving the staging sstables to their shards.
Between those two events, the coordinator may schedule freshly created
tasks and dispatch them to the worker but the worker won't have the
sstables objects because they weren't moved yet.
This patch fixes the race by holding `_staging_sstables_mutex` locks
from necessary shards when executing `create_staging_sstable_tasks()`.
With this, even if the task will be scheduled and dispatched quickly,
the worker will wait with executing it until the sstables objects are
moved and the locks are released.
Fixes SCYLLADB-816
Verify that upgrading from 2025.1 to master does not silently drop DDL
auditing for table-scoped audit configurations (SCYLLADB-1155).
Test time in dev: 4s
Refs: SCYLLADB-1155
Fixes: SCYLLADB-1305
The old execute_and_validate_audit_entry required every caller to
pass audit_settings so it could decide internally whether to expect
an entry. A test added later in this series needs to simply assert
an entry was produced, without specifying audit_settings at all.
Split into two methods:
- execute_and_validate_new_audit_entry: unconditionally expects an
audit entry.
- execute_and_validate_if_category_enabled: checks audit_settings
to decide whether to expect an entry or assert absence.
Local wrapper functions and **kwargs forwarding are removed in favor
of explicit arguments at each call site, and expected-error cases are
handled inline with assert_invalid + assert_entries_were_added.
AuditTester uses self.manager throughout but never declares it.
The attribute is only assigned in the CQLAuditTester subclass
__init__, so the type checker reports 'Attribute "manager" is
unknown' on every self.manager reference in the base class.
Add an __init__ to AuditTester that accepts and stores the manager
instance, and update CQLAuditTester to forward it via super().__init__
instead of assigning self.manager directly.
Track the total memory consumed by responses waiting to be
written to the socket, exposed as a per-scheduling-group gauge
(cql_pending_response_memory). This complements the response
memory accounting added in the previous commits by giving
visibility into how much memory each service level is holding
in unsent response buffers.
Sends a search via the raw LDAP handle (bypassing _msgid_to_promise
registration), then triggers poll_results() through the public API
to exercise the unregistered-ID branch.
Refs: SCYLLADB-1344
In the unregistered-ID branch, ldap_msgfree() was called on a result
already owned by an RAII ldap_msg_ptr, causing a double-free on scope
exit. Remove the redundant manual free.
Fixes: SCYLLADB-1344
Capture the memory permit in the leave lambda's .finally()
continuation so that the semaphore units are kept alive until
write_response finishes, preventing premature release of
memory accounting.
This is especially important with slow network and big responses
when buffers can accumulate and deplete node's memory.
After obtaining the CQL response, check if its actual size exceeds
the initially acquired memory permit. If so, take semaphore units
and adopt them into the permit (non blocking).
This doesn't fully prevent from allocating too much memory as
size is known when buffer is already allocated but improves
memory accounting for big responses.
Now that cmake/FindLua.cmake uses pkg-config (matching configure.py),
both build systems resolve to the same 'lua' library name. Remove the
lua/lua-5.4 entries from _KNOWN_LIB_ASYMMETRIES and add 'm' (math
library) as a known transitive dependency that configure.py gets via
pkg-config for lua.
CMake's built-in FindLua resolves to the versioned library file
(e.g. liblua-5.4.so) instead of the unversioned symlink (liblua.so),
causing a library name mismatch between the two build systems.
Add a custom cmake/FindLua.cmake that uses pkg-config — matching
configure.py's approach — and find_library(NAMES lua) to find the
unversioned symlink. This also mirrors the pattern used by other
Find modules in cmake/ (FindxxHash, Findlz4, etc.).
Add symmetric_key_test (standalone, links encryption library) and
auth_cache_test to the combined_tests binary. These tests already
exist in configure.py; this aligns the CMake build.
The per-test -fno-lto link option is now redundant since -fno-lto
was added globally in mode.common.cmake. LTO-enabled targets
(the scylla binary in RelWithDebInfo) override it via enable_lto().
Match configure.py's Boost handling:
- Add BOOST_ALL_DYN_LINK when using shared Boost libraries.
- Strip per-component defines (BOOST_UNIT_TEST_FRAMEWORK_DYN_LINK,
BOOST_REGEX_DYN_LINK, etc.) that CMake's Boost package config
adds on imported targets. configure.py only uses the umbrella
BOOST_ALL_DYN_LINK define.
Place add_compile_definitions(SEASTAR_TESTING_MAIN) after both
add_subdirectory(seastar) and add_subdirectory(abseil) are processed.
This matches configure.py's global define without leaking into
seastar's subdirectory build (which would cause a duplicate main
symbol in seastar_testing).
Remove the now-redundant per-test SEASTAR_TESTING_MAIN compile
definition from test/CMakeLists.txt.
Match configure.py line 2192: abseil gets sanitizer flags with
-fno-sanitize=vptr to exclude vptr checks which are incompatible
with abseil's usage of type-punning patterns.
- Set BUILD_SHARED_LIBS based on build type to match configure.py's
build_seastar_shared_libs: Debug and Dev build Seastar as a shared
library, all other modes build it static.
- Add sanitizer link options on the seastar target for Coverage
mode. Seastar's CMake only activates sanitizer targets for
Debug/Sanitize configs, but Coverage mode needs them too since
configure.py's seastar_libs_coverage carries -fsanitize flags.
- Disable CMake's automatic -fcolor-diagnostics injection for
Clang+Ninja (CMake 3.24+), matching configure.py which does not
add any color diagnostics flags.
- Add SEASTAR_NO_EXCEPTION_HACK and XXH_PRIVATE_API as global
defines (previously SEASTAR_NO_EXCEPTION_HACK was only on the
seastar target as PRIVATE; it needs to be project-wide).
- Add -fpch-validate-input-files-content to check precompiled
header content when timestamps don't match.
Fix multiple deviations from configure.py's coverage mode:
- Remove -fprofile-list from CMAKE_CXX_FLAGS_COVERAGE. That flag
belongs in COVERAGE_INST_FLAGS applied to other modes, not to
coverage mode itself.
- Replace incorrect defines (DEBUG, SANITIZE, DEBUG_LSA_SANITIZER,
SCYLLA_ENABLE_ERROR_INJECTION) with the correct Seastar debug
defines (SEASTAR_DEBUG, SEASTAR_DEFAULT_ALLOCATOR, etc.) that
configure.py's pkg-config query produces for coverage mode.
- Add sanitizer and stack-clash-protection compile flags for
Coverage config, matching the flags that Seastar's pkg-config
--cflags output includes for debug builds.
- Change CMAKE_STATIC_LINKER_FLAGS_COVERAGE to
CMAKE_EXE_LINKER_FLAGS_COVERAGE. Coverage flags need to reach
the executable linker, not the static archiver.
Add three flag-alignment changes:
- -Wno-error=stack-usage= alongside the stack-usage threshold flag,
preventing hard errors from stack-usage warnings (matching
configure.py behavior).
- -fno-lto global link option. configure.py adds -fno-lto to all
binaries; LTO-enabled targets override it via enable_lto().
- Sanitizer link flags (-fsanitize=address, -fsanitize=undefined) for
Debug/Sanitize configs, matching configure.py's cxx_ld_flags.
Document the purpose, usage, and examples for
scripts/compare_build_systems.py which compares the configure.py
and CMake build systems by parsing their ninja build files.
Add a script that compares configure.py and CMake build systems by
parsing their generated build.ninja files. The script checks:
- Per-file compilation flags (defines, warnings, optimization)
- Link target sets (detect missing/extra targets)
- Per-target linker flags and libraries
configure.py is treated as the baseline. CMake should match it.
Both systems are always configured into a temporary directory so the
user's build tree is never touched.
Usage:
scripts/compare_build_systems.py -m dev # single mode
scripts/compare_build_systems.py # all modes
scripts/compare_build_systems.py --ci # CI mode (strict)
When authenticate() returns a user directly (certificate-based auth,
introduced in 20e9619bb1), process_startup was missing the same
post-authentication bookkeeping that the no-auth and SASL paths perform:
- update_scheduling_group(): without it, the connection runs under the
default scheduling group instead of the one mapped to the user's
service level.
- _authenticating = false / _ready = true: without them,
system.clients reports connection_stage = AUTHENTICATING forever
instead of READY.
- on_connection_ready(): without it, the connection never releases its
slot in the uninitialized-connections concurrency semaphore (acquired
at connection creation), leaking one unit per cert-authenticated
connection for the lifetime of the connection.
The omission was introduced when on_connection_ready() was added to the
else and SASL branches in 474e84199c but the cert-auth branch was missed.
Fixes: 20e9619bb1 ("auth: support certificate-based authentication")
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The cert-auth path in process_startup (introduced in 20e9619bb1) was
missing _ready = true, _authenticating = false, update_scheduling_group()
and on_connection_ready(). The result is that connections authenticated
via certificate show connection_stage = AUTHENTICATING in system.clients
forever, run under the wrong service-level scheduling group, and hold
the uninitialized-connections semaphore slot for the lifetime of the
connection.
Add a parametrized cluster test that verifies all three process_startup
branches result in connection_stage = READY:
- allow_all: AllowAllAuthenticator (no-auth path)
- password: PasswordAuthenticator (SASL/process_auth_response path)
- cert_bypass: CertificateAuthenticator with transport_early_auth_bypass
error injection (cert-auth path -- the buggy one)
The injection is added to certificate_authenticator::authenticate() so
tests can bypass actual TLS certificate parsing while still exercising
the cert-auth code path in process_startup.
The cert_bypass case is marked xfail until the bug is fixed.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Snapshots are not implemented yet for strong consistency - attempting to
take, transfer or drop a snapshot results in an exception. However, the
logic of our state machine forces snapshot transfer even if there are no
lagging replicas - every raft::server::configuration::snapshot_threshold
log entries. We have actually encountered an issue in our benchmarks
where snapshots were being taken even though the cluster was not under
any disruption, and this is one of the possible causes.
It turns out that we can safely allow for taking snapshots right now -
we can just implement it as a no-op and return a random UUID.
Conversely, dropping a snapshot can also be a no-op. This is safe
because snapshot transfer still throws an exception - as long as the
taken/recovered snapshots are never attempted to be transferred.
Raft snapshots are not implemented yet for strong consistency. Adjust
the current raft group config to make them much less likely to occur:
- snapshot_threshold config option decides how many log entries need to
be applied after the last snapshot. Set it to the maximum value for
size_t in order to effectively disable it.
- snapshot_threshold_log_size defines a threshold for the log memory
usage over which a snapshot is created. Increase it from the default
2MB to 10MB.
- max_log_size defines the threshold for the log memory usage over which
requests are stopped to be admitted until the log is shrunk back by a
snapshot. Set it to 20MB, as this option is recommended to be at least
twice as much as snapshot_threshold_log_size.
Refs: SCYLLADB-1115
Replace create_dataset + manual DROP/CREATE KEYSPACE with two sequential
new_test_keyspace context manager blocks, matching the pattern used by
do_test_streaming_scopes. The first block covers backup, the second
covers restore. Keyspace lifecycle is now automatic.
The streaming directions validation loop is moved outside of the second
context block, since it only parses logs and has no dependency on the
keyspace being alive.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There are two helpers for describe_ring endpoint. Both can be squashed
together for code brevity.
Also, while at it, the "keyspace" parameter is not properly validated by
the endpoint.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
All callers already have it. It makes no difference for the method
itself with which table identifier to work, but will help to simplify
the flow in API handler (next patch)
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This actually uses two interconnected options:
repair_multishard_reader_buffer_hint_size and
repair_multishard_reader_enable_read_ahead.
Both are propagated through repair_service::config and pass their
values to repair_reader/make_reader at construction time.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Most other services have their configs, rpair still uses global
db::config.
Add an empty config struct to repair_service to carry db::config options
the repair service needs.
Subsequent patches will populate the struct with options.
The config is created in main.cc as sharded_parameter because all future
options are live-updateable and should capture theirs source from
db::config on correct shard.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
2026-03-20 19:23:47 +03:00
136 changed files with 5588 additions and 1072 deletions
"description":"The snapshot tag to delete. If omitted, all snapshots are removed.",
"required":false,
"allowMultiple":false,
"type":"string",
@@ -751,7 +751,7 @@
},
{
"name":"kn",
"description":"Comma-separated keyspaces name that their snapshot will be deleted",
"description":"Comma-separated list of keyspace names to delete snapshots from. If omitted, snapshots are deleted from all keyspaces.",
"required":false,
"allowMultiple":false,
"type":"string",
@@ -759,7 +759,7 @@
},
{
"name":"cf",
"description":"an optional table name that its snapshot will be deleted",
"description":"A table name used to filter which table's snapshots are deleted. If omitted or empty, snapshots for all tables are eligible. When provided together with 'kn', the table is looked up in each listed keyspace independently. For secondary indexes, the logical index name (e.g. 'myindex') can be used and is resolved automatically.",
sm::make_counter("sstable_reader_recreations",sm::description("number of times sstable reader was recreated due to memtable flush"),_stats.underlying_recreations),
sm::make_counter("sstable_partition_skips",sm::description("number of times sstable reader was fast forwarded across partitions"),_stats.underlying_partition_skips),
sm::make_counter("sstable_row_skips",sm::description("number of times sstable reader was fast forwarded within a partition"),_stats.underlying_row_skips),
sm::make_counter("pinned_dirty_memory_overload",sm::description("amount of pinned bytes that we tried to unpin over the limit. This should sit constantly at 0, and any number different than 0 is indicative of a bug"),_stats.pinned_dirty_memory_overload),
sm::make_counter("pinned_dirty_memory_overload",sm::description("amount of pinned bytes that we tried to unpin over the limit. This should sit constantly at 0, and any number different than 0 is indicative of a bug"),_stats.pinned_dirty_memory_overload).set_skip_when_empty(),
Counters are special kinds of cells which value can only be incremented, decremented, read and (with some limitations) deleted. In particular, once deleted, that counter cannot be used again. For example:
```cql
>UPDATEcfSETmy_counter=my_counter+6WHEREpk=0
>SELECT*FROMcf;
pk|my_counter
----+------------
0|6
(1rows)
>UPDATEcfSETmy_counter=my_counter-1WHEREpk=0
>SELECT*FROMcf;
pk|my_counter
----+------------
0|5
(1rows)
>DELETEmy_counterFROMcfWHEREpk=0;
>SELECT*FROMcf;
pk|my_counter
----+------------
(0rows)
>UPDATEcfSETmy_counter=my_counter+3WHEREpk=0
>SELECT*FROMcf;
pk|my_counter
----+------------
(0rows)
```
## Counters representation
Counters are represented as sets of, so called, shards which are triples containing:
* counter id – uuid identifying the writer owning that shard (see below)
* logical clock – incremented each time the owning writer modifies the shard value
* current value – sum of increments and decrements done by the owning writer
During each write operation one of the replicas is chosen as a leader. The leader reads its shard, increments logical clock, updates current value and then sends the new version of its shard to the other replicas.
Shards owned by the same writer are merged (see below) so that each counter cell contains only one shard per counter id. Reading the actual counter value requires summing values of all shards.
### Counter id
The counter id is a 128-bit UUID that identifies which writer owns a shard. How it is assigned depends on whether the table uses vnodes or tablets.
**Vnodes:** the counter id is the host id of the node that owns the shard. Each node in the cluster gets a unique counter id, so the number of shards in a counter cell grows with the number of distinct nodes that have ever written to it.
**Tablets:** the counter id is rack-based rather than node-based. It is a deterministic type-3 (name-based) UUID derived from the string `"<datacenter>:<rack>"`. All nodes in the same rack share the same counter id.
During tablet migration, since there are two active replicas in a rack and in order to avoid conflicts, the node that is a *pending replica* uses the **negated** rack UUID as its counter id.
This bounds the number of shards in a counter cell to at most `2 × (number of racks)` regardless of node replacements.
### Merging and reconciliation
Reconciliation of two counters requires merging all shards belonging to the same counter id. The rule is: the shard with the highest logical clock wins.
Since support of deleting counters is limited so that once deleted they cannot be used again, during reconciliation tombstones win with live counter cell regardless of their timestamps.
### Digest
Computing a digest of counter cells needs to be done based solely on the shard contents (counter id, value, logical clock) rather than any structural metadata.
## Writes
1. Counter update starts with a client sending counter delta as a long (CQL3 `bigint`) to the coordinator.
2. CQL3 creates a `CounterMutation` containing a `counter_update` cell which is just a delta.
3. Coordinator chooses the leader of the counter update and sends it the mutation. The leader is always one of the replicas owning the partition the modified counter belongs to.
4. Now, the leader needs to transform counter deltas into shards. To do that it reads the current value of the shard it owns, and produces a new shard with the value modified by the delta and the logical clock incremented.
5. The mutation with the newly created shard is both used to update the memtable on the leader as well as sent to the other nodes for replication.
### Choosing leader
Choosing a replica which becomes a leader for a counter update is completely at the coordinator discretion. It is not a static role in any way and any concurrent update could be forwarded to a different leader. This means that all problems related to leader election are avoided.
The coordinator chooses the leader using the following algorithm:
1. If the coordinator can be a leader it chooses itself.
2. Otherwise, a random replica from the local DC is chosen.
3. If there is no eligible node available in the local DC the replica closest to the coordinator (according to the snitch) is chosen.
## Reads
Querying counter values is much simpler than updating it. First part of the read operation is performed as for all other cell types. When counter cells from different sources are being reconciled their shards are merged. Once the final counter cell value is known and the `CounterCell` is serialised, current values of all shards are summed up and the output of serialisation is a long integer.
@@ -192,14 +192,10 @@ For example, to configure ScyllaDB to use listen address `10.0.0.5`:
$ docker run --name some-scylla -d scylladb/scylla --listen-address 10.0.0.5
```
**Since: 1.4**
#### `--alternator-address ADDR`
The `--alternator-address` command line option configures the Alternator API listen address. The default value is the same as `--listen-address`.
**Since: 3.2**
#### `--alternator-port PORT`
The `--alternator-port` command line option configures the Alternator API listen port. The Alternator API is disabled by default. You need to specify the port to enable it.
@@ -210,22 +206,16 @@ For example, to configure ScyllaDB to listen to Alternator API at port `8000`:
$ docker run --name some-scylla -d scylladb/scylla --alternator-port 8000
```
**Since: 3.2**
#### `--alternator-https-port PORT`
The `--alternator-https-port` option is similar to `--alternator-port`, just enables an encrypted (HTTPS) port. Either the `--alternator-https-port` or `--alternator-http-port`, or both, can be used to enable Alternator.
Note that the `--alternator-https-port` option also requires that files `/etc/scylla/scylla.crt` and `/etc/scylla/scylla.key` be inserted into the image. These files contain an SSL certificate and key, respectively.
**Since: 4.2**
#### `--alternator-write-isolation policy`
The `--alternator-write-isolation` command line option chooses between four allowed write isolation policies described in docs/alternator/alternator.md. This option must be specified if Alternator is enabled - it does not have a default.
**Since: 4.1**
#### `--broadcast-address ADDR`
The `--broadcast-address` command line option configures the IP address the ScyllaDB instance tells other ScyllaDB nodes in the cluster to connect to.
@@ -304,8 +294,6 @@ For example, to skip running I/O setup:
$ docker run --name some-scylla -d scylladb/scylla --io-setup 0
```
**Since: 4.3**
#### `--cpuset CPUSET`
The `--cpuset` command line option restricts ScyllaDB to run on only on CPUs specified by `CPUSET`.
@@ -341,26 +329,18 @@ For example, to enable the User Defined Functions (UDF) feature:
$ docker run --name some-scylla -d scylladb/scylla --experimental-feature=udf
```
**Since: 2.0**
#### `--disable-version-check`
The `--disable-version-check` disable the version validation check.
**Since: 2.2**
#### `--authenticator AUTHENTICATOR`
The `--authenticator` command lines option allows to provide the authenticator class ScyllaDB will use. By default ScyllaDB uses the `AllowAllAuthenticator` which performs no credentials checks. The second option is using the `PasswordAuthenticator` parameter, which relies on username/password pairs to authenticate users.
**Since: 2.3**
#### `--authorizer AUTHORIZER`
The `--authorizer` command lines option allows to provide the authorizer class ScyllaDB will use. By default ScyllaDB uses the `AllowAllAuthorizer` which allows any action to any user. The second option is using the `CassandraAuthorizer` parameter, which stores permissions in `system.permissions` table.
**Since: 2025.4**
#### `--dc NAME`
The `--dc` command line option sets the datacenter name for the ScyllaDB node.
Internal keyspace for encryption-at-rest key material used by the replicated key provider. It stores encrypted data keys so nodes can retrieve the correct key IDs when reading encrypted data.
This keyspace is created as an internal system keyspace and uses `EverywhereStrategy` so key metadata is available on every node. It is not intended for user data.
## `system_distributed`
Internal distributed metadata keyspace used for cluster-wide coordination data that is shared across nodes.
In practice, it is used for metadata such as:
- materialized view build coordination state
- CDC stream/timestamp metadata exposed to clients
- service level definitions used by workload prioritization
This keyspace is managed by Scylla and is not intended for application tables.
It is created as an internal keyspace (historically with `SimpleStrategy` and RF=3 by default).
## `system_distributed_everywhere`
Legacy keyspace. It is no longer used.
## `system_auth`
Legacy auth keyspace name kept primarily for compatibility.
Auth tables have moved to the `system` keyspace (`roles`, `role_members`, `role_permissions`, and related auth state). `system_auth` may still exist for compatibility with legacy tooling/queries, but it is no longer where current auth state is primarily stored.
## `system`
This keyspace is local one, so each node has its own, independent content for tables in this keyspace. For some tables, the content is coordinated at a higher level (RAFT), but not via the traditional replication systems (storage proxy).
See the detailed table-level documentation here: [system_keyspace](system_keyspace.md)
## `system_schema`
This keyspace is local one, so each node has its own, independent content for tables in this keyspace. All tables in this keyspace are coordinated via the schema replication system.
See the detailed table-level documentation here: [system_schema_keyspace](system_schema_keyspace.md)
## `system_traces`
Internal tracing keyspace used for query tracing and slow-query logging records (`sessions`, `events`, and related index/log tables).
This keyspace is written by Scylla's tracing subsystem for diagnostics and observability. It is operational metadata, not user application data (historically created with `SimpleStrategy` and RF=2).
## `system_audit`/`audit`
Internal audit-logging keyspace used to persist audit events when table-backed auditing is enabled.
Scylla's audit table storage is implemented as an internal audit keyspace for audit records (for example, auth/admin/DCL activity depending on audit configuration). In current code this keyspace is named `audit`, while operational material may refer to it as its historical name (`system_audit`). It is intended for security/compliance observability, not for application data.
@@ -289,7 +289,7 @@ Yes, but it will require running a full repair (or cleanup) to change the replic
- If you're reducing the replication factor, run ``nodetool cleanup <updated Keyspace>`` on the keyspace you modified to remove surplus replicated data.
Cleanup runs on a per-node basis.
- If you're increasing the replication factor, refer to :doc:`How to Safely Increase the RF </kb/rf-increase>`
- Note that you need to provide the keyspace namr. If you do not, the cleanup or repair operation runs on all keyspaces for the specific node.
- Note that you need to provide the keyspace name. If you do not, the cleanup or repair operation runs on all keyspaces for the specific node.
Why can't I set ``listen_address`` to listen to 0.0.0.0 (all my addresses)?
Auditing allows the administrator to monitor activities on a Scylla cluster, including queries and data changes.
The information is stored in a Syslog or a Scylla table.
Auditing allows the administrator to monitor activities on a ScyllaDB cluster, including CQL queries and data changes, as well as Alternator (DynamoDB-compatible API) requests.
The information is stored in a Syslog or a ScyllaDB table.
By default, table auditing is **enabled**. Enabling auditing is controlled by the ``audit:`` parameter in the ``scylla.yaml`` file.
By default, auditing is **enabled** with the ``table`` backend. Enabling auditing is controlled by the ``audit:`` parameter in the ``scylla.yaml`` file.
You can set the following options:
*``none`` - Audit is disabled.
*``table`` - Audit is enabled, and messages are stored in a Scylla table (default).
*``table`` - Audit is enabled, and messages are stored in a ScyllaDB table (default).
*``syslog`` - Audit is enabled, and messages are sent to Syslog.
*``syslog,table`` - Audit is enabled, and messages are stored in a Scylla table and sent to Syslog.
*``syslog,table`` - Audit is enabled, and messages are stored in a ScyllaDB table and sent to Syslog.
Configuring any other value results in an error at Scylla startup.
Configuring any other value results in an error at ScyllaDB startup.
For details on auditing Alternator operations, see :ref:`alternator-auditing`.
Note that enabling audit may negatively impact performance and audit-to-table may consume extra storage. That's especially true when auditing DML and QUERY categories, which generate a high volume of audit messages.
.._alternator-auditing:
Auditing Alternator Requests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When auditing is enabled, Alternator (DynamoDB-compatible API) requests are audited using the same
backends and the same filtering configuration (``audit_categories``, ``audit_keyspaces``,
``audit_tables``) as CQL operations. No additional configuration is needed.
Both successful and failed Alternator requests are audited.
Alternator Operation Categories
""""""""""""""""""""""""""""""""
Each Alternator API operation is assigned to one of the standard audit categories:
By default, audit messages are written to the same destination as Scylla :doc:`logging </getting-started/logging>`, with ``scylla-audit`` as the process name.
By default, audit messages are written to the same destination as ScyllaDB:doc:`logging </getting-started/logging>`, with ``scylla-audit`` as the process name.
Logging output example (drop table):
Logging output example (CQL drop table):
..code-block::shell
@@ -123,7 +232,7 @@ To redirect the Syslog output to a file, follow the steps below (available only
Storing Audit Messages in a Table
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Messages are stored in a Scylla table named ``audit.audit_log``.
Messages are stored in a ScyllaDB table named ``audit.audit_log``.
@@ -196,7 +305,7 @@ Storing Audit Messages in a Table and Syslog Simultaneously
**Procedure**
#. Follow both procedures from above, and set the ``audit`` parameter in the ``scylla.yaml`` file to both ``syslog`` and ``table``. You need to restart scylla only once.
#. Follow both procedures from above, and set the ``audit`` parameter in the ``scylla.yaml`` file to both ``syslog`` and ``table``. You need to restart ScyllaDB only once.
To have both syslog and table you need to specify both backends separated by a comma:
logger.debug("Ignore SUPPORTED_FEATURES of local node: features={}",features);
continue;
}
if(features.empty()){
autoit=loaded_peer_features.find(host_id);
if(it!=loaded_peer_features.end()){
logger.info("Node {} does not contain SUPPORTED_FEATURES in gossip, using features saved in system table, features={}",host_id,feature_service::to_feature_set(it->second));
}else{
logger.warn("Node {} does not contain SUPPORTED_FEATURES in gossip or system table",host_id);
sm::description("The total number of bytes that were extracted from the shard reader but were unconsumed by the query and moved back into the reader.")),
// NOTE: multishard_query_failed_reader_stops appears to have no increment site in the
// current codebase. Consider removing it entirely if it is confirmed dead.
e.execute_cql("INSERT INTO my_table (key, val) VALUES ('foo', null);").get();
e.execute_cql("CREATE FUNCTION my_func1(val int) CALLED ON NULL INPUT RETURNS int LANGUAGE Lua AS 'return val + 1';").get();
e.execute_cql("CREATE FUNCTION my_func2(val int) CALLED ON NULL INPUT RETURNS int LANGUAGE Lua AS 'return val';").get();
BOOST_REQUIRE_EXCEPTION(e.execute_cql("SELECT my_func1(val) FROM my_table;").get(),ire,message_equals("lua execution failed: ?:-1: attempt to perform arithmetic on a nil value"));
BOOST_REQUIRE_EXCEPTION(e.execute_cql("SELECT my_func1(val) FROM my_table;").get(),ire,message_contains("attempt to perform arithmetic on a nil value"));
autores=e.execute_cql("SELECT my_func2(val) FROM my_table;").get();
e.execute_cql("CREATE TABLE my_table (key text PRIMARY KEY, val int);").get();
e.execute_cql("INSERT INTO my_table (key, val) VALUES ('foo', 42);").get();
e.execute_cql("CREATE FUNCTION my_func(val int) RETURNS NULL ON NULL INPUT RETURNS int LANGUAGE Lua AS 'return 2 * bar';").get();
BOOST_REQUIRE_EXCEPTION(e.execute_cql("SELECT my_func(val) FROM my_table;").get(),ire,message_equals("lua execution failed: ?:-1: attempt to perform arithmetic on a nil value (field 'bar')"));
BOOST_REQUIRE_EXCEPTION(e.execute_cql("SELECT my_func(val) FROM my_table;").get(),ire,message_contains("attempt to perform arithmetic on a nil value (field 'bar')"));
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
