mirror of
https://github.com/scylladb/scylladb.git
synced 2026-05-22 07:42:16 +00:00
b7fc661fa983f754ed25cd94de5dd8dd1bc9a98c
2243 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Avi Kivity
|
6db152afbb |
Update seastar submodule
Drop local formatter for seastar::http::reply, which should have been added to Seastar in the first place, and now conflicts. Also drop local formatters for types that are aliases for Seastar types which have gained formatters. Disable recently-gained TLS use of OpenSSL instead of gnutls. We don't need it, and it causes link errors with LTO. Fix incorrect skipping in encrypted_file_test, which computed the remaining stream length but did not account for already consumed size_to_compare. Change utils::gcp::storage::client::object_data_source::skip() to match new Seastar behavior (rejecting skip-past-eof with an exception). This is needed since |
||
Michael Litvak
|
5a5c7c6241 |
strong_consistency: wait for raft servers to start in create table
When creating a strongly consistent table, wait for the table's raft servers to start and be ready to serve queries before completing the operation. We want the create table operation to absorb the delay of starting the raft groups instead of the first queries. The create table coordinator commits and applies the schema statement, then it waits for all hosts that have a tablet replica to create and start the raft groups for the table's tablets. It does this by sending an RPC to all the relevant hosts that executes a group0 barrier, in order to ensure the table and raft groups are created, then waits for all raft groups on the host to finish starting and be ready. Fixes SCYLLADB-807 |
||
Pavel Emelyanov
|
39ae59da9c |
messaging: Add RESTORE_TABLET RPC verb
The topology coordinator will need to call this verb against existing tablet replicas to ask them restore tablet sstables. Here's the RPC verb to do it. It now returns an empty restore_result to make it "synchronous" -- the co_await send_restore_tablets() won't resolve until client call finishes. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com> |
||
Ernest Zaslavsky
|
aa00048753 |
sstables_loader: move download_sstable and get_sstables_for_tablet
Move the download_sstable and get_sstables_for_tablet static functions from sstables_loader into a new file to make them reusable by the tablet-aware restore code. |
||
Robert Bindar
|
2f19d84ad7 |
Add system_distributed_keyspace.snapshot_sstables
This patch adds the snapshot_sstables table with the following
schema:
```cql
CREATE TABLE system_distributed.snapshot_sstables (
snapshot_name text,
keyspace text, table text,
datacenter text, rack text,
id uuid,
first_token bigint, last_token bigint,
toc_name text, prefix text)
PRIMARY KEY ((snapshot_name, keyspace, table, datacenter, rack), first_token, id);
```
The table will be populated by the coordinator node during the restore
phase (and later on during the backup phase to accomodate live-restore).
The content of this table is meant to be consumed by the restore worker nodes
which will use this data to filter and file-based download sstables.
Fixes SCYLLADB-263
Signed-off-by: Robert Bindar <robert.bindar@scylladb.com>
|
||
Nadav Har'El
|
f1b2b9bd52 |
Merge 'Register fulltext_index custom index type' from Dawid Pawlik
This PR adds the `fulltext_index` custom index class, laying the groundwork for full-text search in ScyllaDB. It focuses on the CQL-facing layer - schema validation, option parsing, and metadata - without implementing the search backend itself.
Users can now write:
```cql
CREATE CUSTOM INDEX ON t(content) USING 'fulltext_index'
WITH OPTIONS = {'analyzer': 'english', 'positions': 'false'};
```
The implementation follows the same custom index pattern established by vector search: a `custom_index` subclass registered in the factory map, with no backing materialized view. This keeps the door open for a CDC-based indexing pipeline similar to the one vector search uses.
As part of this work, the option validation helpers (`validate_enumerated_option`, `validate_positive_option`, `validate_factor_option`) were extracted from `vector_index.cc` into a shared header so both index types can reuse them. The `custom_index` base class also gained a virtual `index_type_name()` method, giving each subclass a self-describing name for error messages without hardcoding strings in shared code.
The PR is split into three commits:
1. Extract shared validation utilities and add `index_type_name()` to `custom_index`
2. Implement `fulltext_index` with column type and option validation
3. Integration tests covering creation, validation, describe, and metadata
Fixes: SCYLLADB-1517
Fixes: SCYLLADB-1510
References: SCYLLADB-1516
Closes scylladb/scylladb#29658
* github.com:scylladb/scylladb:
test/cqlpy: add integration tests for `fulltext_index`
index: unify custom index description
index: add `fulltext_index` custom index implementation
index: extract option validation helpers
|
||
Yaniv Michael Kaul
|
b8a150e22c |
build: add -ftime-trace support for compilation profiling
Add a --time-trace flag to configure.py and a Scylla_TIME_TRACE CMake option that enable Clang's -ftime-trace on all C++ compilations. When enabled, each .o file produces a companion .json trace that can be analyzed with ClangBuildAnalyzer or loaded in chrome://tracing to identify slow headers and costly template instantiations. This is the first step toward data-driven build speed improvements. Refs #1 Usage: configure.py: ./configure.py --time-trace --mode dev CMake: cmake -DScylla_TIME_TRACE=ON -DCMAKE_BUILD_TYPE=Dev .. Closes scylladb/scylladb#29462 |
||
Yaron Kaikov
|
5694c93c12 |
build: add collect-dist target to organize build artifacts
Build artifacts are currently scattered across build/dist/$mode/redhat/, tools/python3/build/, tools/cqlsh/build/, etc. with unpredictable names. Add a new 'collect-dist' ninja target that gathers all distributable artifacts into a well-known structure: build/$mode/dist/rpm/ -- all binary RPMs (no SRPMs) build/$mode/dist/deb/ -- all .deb packages build/$mode/dist/tar/ -- relocatable tarballs (already here) The collection is done via a reusable 'collect_pkgs' ninja rule defined directly in configure.py, which knows all the source paths. No external script is needed. Fixes: SCYLLADB-75 Closes scylladb/scylladb#29475 |
||
|
Michael Litvak
|
274024a76b |
configure.py: update compile_commands.json if stale
configure.py creates compile_commands.json in the root directory as a symbolic link to the file in one of the build directories. If the file already exists it does nothing. However it may happen that the file exists but the target file does not exist. For example, if the build directory is removed and then building with a different mode. Then the file will remain as a stale symbolic link. To address this, when the file exists check also if it's a valid symbolic link. If not, then recreate it with a valid target. Closes scylladb/scylladb#29680 |
||
Dawid Pawlik
|
fcd15b5cd4 |
index: add fulltext_index custom index implementation
Introduce `fulltext_index`, a new `custom_index` subclass for full-text search (FTS). The index validates that the target column is a text type (text, varchar, or ascii) and supports two WITH OPTIONS keys: - 'analyzer': one of standard, english, german, french, spanish, italian, portuguese, russian, chinese, japanese, korean, simple, whitespace - 'positions': boolean controlling whether term positions are stored `view_should_exist()` returns false — no backing materialized view is created, matching the CDC-backed pattern used by `vector_index`. Fixes: SCYLLADB-1517 |
||
|
Dawid Pawlik
|
a396129e5c |
index: extract option validation helpers
Move `validate_enumerated_option`, `validate_positive_option`, and `validate_factor_option` into shared index option utilities under the `secondary_index::util` namespace. These functions were previously defined as file-local statics in `vector_index.cc` with hardcoded index names in error messages. The shared versions take `index_type_name` as a parameter, allowing each `custom_index` subclass to pass its own name via the virtual `index_type_name()` method at the call site. The options maps use `std::bind_front` to bind config params (supported values, limits), leaving `index_name` as the first unbound argument passed by `check_index_options()`. Add `index_type_name()` as a pure virtual method on `custom_index`. Move the shared utility implementations into `index_option_utils.cc` and update `vector_index.cc` to use them. |
||
Piotr Dulikowski
|
efcc0b6376 |
Merge 'table_helper: fix use-after-free on prepared-statement invalidation' from Marcin Maliszkiewicz
insert() held no local strong ref to the prepared modification_statement across the suspension in execute(). On a single shard: 1. Fiber A suspends inside _insert_stmt->execute(). 2. DROP TABLE / DROP KEYSPACE on the target, or LRU eviction, removes the prepared_statements_cache entry, releasing its strong ref. 3. Fiber B re-enters cache_table_info(), sees _prepared_stmt (checked_weak_ptr) invalidated, and runs _insert_stmt = nullptr, releasing the last strong ref. The modification_statement is freed. 4. Fiber A resumes inside execute() and touches freed *this. Pin strong ref to _insert_stmt locally before the suspension. Fixes https://scylladb.atlassian.net/browse/SCYLLADB-1667 Backport: all supported branches, it's memory corruption bug, long present Closes scylladb/scylladb#29588 * github.com:scylladb/scylladb: test/boost: add dummy case to table_helper_test for non-injection modes test/boost: add regression test for table_helper insert() UAF utils/error_injection: add waiters() API table_helper: fix use-after-free on prepared-statement invalidation |
||
Marcin Maliszkiewicz
|
515b5722fd |
test/boost: add regression test for table_helper insert() UAF
Deterministic reproducer using an error injection point placed in table_helper::insert() between cache_table_info() and execute(). The test parks fiber A at the injection, drops the target table (evicting the prepared_statements_cache entry), runs fiber B which nulls _insert_stmt, then releases fiber A. Without the fix this crashes in execute(); with the fix fiber A holds a local strong ref and proceeds. Uses the new waiters() API to synchronize with fiber A's entry into the injection. |
||
|
Ernest Zaslavsky
|
1febfbd9b5 |
test: rename sstable_tablet_streaming.cc to match the naming convention
apparently, boost test MUST end with "_test" to be executed by the test.py Closes scylladb/scylladb#29693 |
||
Wojciech Mitros
|
ebaf536449 |
replica/database: fix cross-shard deadlock in lock_tables_metadata()
lock_tables_metadata() acquires a write lock on tables_metadata._cf_lock on every shard. It used invoke_on_all(), which dispatches lock acquisitions to all shards in parallel via parallel_for_each + smp::submit_to. When two fibers call lock_tables_metadata() concurrently, this can deadlock. parallel_for_each starts all iterations unconditionally: even when the local shard's lock attempt blocks (because the other fiber already holds it), SMP messages are still sent to remote shards. Both fibers' lock-acquisition messages land in the per-shard SMP queues. The SMP queue itself is FIFO, but process_incoming() drains it and schedules each item as a reactor task via add_task(), which — in debug and sanitize builds with SEASTAR_SHUFFLE_TASK_QUEUE — shuffles each newly added task against all pending tasks in the same scheduling group's reactor task queue. This means fiber A's lock acquisition can be reordered past fiber B's (and past unrelated tasks) on a given shard. If fiber A wins the lock on shard X while fiber B wins on shard Y, this creates a classic cross-shard lock-ordering deadlock (circular wait). In production builds without SEASTAR_SHUFFLE_TASK_QUEUE, the reactor task queue is FIFO. Still, even in release builds, the SMP queues can reorder messages even, so the deadlock is still possible, even if it's much less likely. In debug and sanitize builds, the task-queue shuffle makes the deadlock very likely whenever both fibers' lock-acquisition tasks are pending simultaneously in the reactor task queue on any shard. This deadlock was exposed by |
||
|
Piotr Dulikowski
|
37fc1507f0 |
Merge 'Alternator: Add vector search support' from Nadav Har'El
This series adds support for vector search in Alternator based on the existing implementation in CQL. The series adds APIs for `CreateTable` and `UpdateTable` to add or remove vector indexes to Alternator tables, `DescribeTable` to list them and check the indexing status, and `Query` to perform a vector search - which contacts the vector store for the actual ANN (approximate nearest neighbor) search. Correct functionality of these features depend on some features of the the vector store, that were already done (see https://github.com/scylladb/vector-store/pull/394). This initial implementation is fully functional, and can already be useful, but we do not yet support all the features we hope to eventually support. Here are things that we have **not** done yet, and plan to do later in follow-up pull requests: 1. Support a new optimized vector type ("V") - in addition to the "list of numbers" type supported in this version. 2. Allow choosing a different similarity function when creating an index, by SimilarityFunction in VectorIndex definition. 3. Allow choosing quantization (f32/f16/bf16/i8/b1) to ask the vector index to compress stored vectors. 4. Support oversampling and rescoring, defined per-index and per-query. 5. Support HNSW tuning parameters — maximum_node_connections, construction_beam_width, search_beam_width. 6. Support pre-filtering over key columns, which are available at the vector store, by sending the filter to the vector store (translated from DynamoDB filter syntax to the vector's store's filter syntax). A decision still need to be made if this will use KeyConditionExpression or FilterExpression. This version supports only post-filtering (with `FilterExpression`). 7. Support projecting non-key attributes into the index (Projection=INCLUDE and Projection=ALL), and then 1. pre-filtering using these attributes, and 2. efficiently return these attributes (using Select=ALL_PROJECTED_ATTRIBUTES, which today returns just the key columns). 8. Optimize the performance of `Query`, which today is inefficient for Select=ALL_ATTRIBUTES because it serially retrieves the matching items one at a time. 9. Returning the similarity scores with the items (the design proposes ReturnVectorSearchSimilarity). 10. Add more vector-search-specific metrics, beyond the metric we already have counting Query requests. For example separate latency and request-count metrics for vector-search Queries (distinct from GSI/LSI queries), and a metric accumulating the total Limit (K) across all vector search queries. 11. Consider how (and if at all) we want to run the tests in test/alternator/test_vector.py that need the vector store in the CI. Currently they are skipped in CI and only run manually (with `test/alternator/run --vs test_vector`). 12. UpdateTable 'Update' operation to modify index parameters. Only some can be modified, e.g., Oversampling. 13. Support for "local index" (separate index for each partition). 14. Make sure that vector search and Streams can be enabled concurrently on the same table - both need CDC but we need to verify that one doesn't confuse the other or disables options that the other needs. We can only do this after we have Alternator Streams running on tablets (since vector store requires tablets). Testing the new Alternator vector search end-to-end requires running both Scylla and the vector store together. We will have such end-to-end tests in the vector store repository (see https://github.com/scylladb/vector-store/pull/392), but we also add in this pull request many end-to-end tests written in Python, that can be run with the command "test/alternator/run --vs test_vector.py". The "--vs" option tells the run script to run both Scylla and the vector store (currently assumed to be in `.../vector-store/target/release/vector-store`). About 65% of the tests in this pull request check supported syntax and error paths so can run without the vector store, while about 35% of the tests do perform actual Query operations and require the vector store to be running. Currently, the tests that do require the vector store will not get run by CI, but can be easily re-run manually with `test/alternator/run --vs test_vector.py`. In total, this series includes 78 functional tests in 2200 lines of Python code. This series also includes documentation for the new Alternator feature and the new APIs introduced. You can see a more detailed design document here: https://docs.google.com/document/d/1cxLI7n-AgV5hhH1DTyU_Es8_f-t8Acql-1f58eQjZLY/edit Two patches in this series split the huge alternator/executor.cc, after this series continued to grow it and it reached a whoppng 7,000 lines. These patches are just reorganization of code, no functional changes. But it's time that we finally do this (Refs #5783), we can't just continue to grow executor.cc with no end... Closes scylladb/scylladb#29046 * github.com:scylladb/scylladb: test/alternator: add option to "run" script to run with vector search alternator: document vector search test/alternator: fix retries in new_dynamodb_session test/alternator: test for allowed characters in attribute names test/alternator: tests for vector index support alternator, vector: add validation of non-finite numbers in Query alternator: Query: improve error message when VectorSearch is missing alternator: add per-table metrics for vector query alternator: clean up duplicated code alternator: fix default Select of Query alternator: split executor.cc even more alternator: split alternator/executor.cc alternator: validate vector index attribute values on write alternator: DescribeTable for vector index: add IndexStatus and Backfilling alternator: implement Query with a vector index alternator: fix bug in describe_multi_item() alternator: prevent adding GSI conflicting with a vector index alternator: implement UpdateTable with a vector index alternator: implement DescribeTable with a vector index alternator: implement CreateTable with a vector index alternator: reject empty attribute names cdc: fix on_pre_create_column_families to create CDC log for vector search |
||
|
Nadav Har'El
|
2e274bbdba |
alternator: split executor.cc even more
This patch continues the effort to split the huge executor.cc (5000 lines before this patch) even more. In this patch we introduce a new source file, executor_util.cc, for various utility functions that are used for many different operations and therefore are useful to have in a header file. These utility functions will now be in executor_util.cc and executor_util.hh - instead of executor.cc and executor.hh. Various source files, including executor.cc, the executor_read.cc introduced in the previous patch, as well as older source files like as streams.cc, ttl.cc and serialization.cc, use the new header file. This patch removes over 700 lines of code from executor.cc, and also removes a large amount of utility functions declerations from executor.hh. Originally, executor.hh was meant to be about the interface that the Alternator server needs to *execute* the different DynamoDB API operations - and after this patch it returns closer to this original goal. Signed-off-by: Nadav Har'El <nyh@scylladb.com> |
||
|
Nadav Har'El
|
751da00692 |
alternator: split alternator/executor.cc
Already six years ago, in #5783, we noticed that alternator/executor.cc has grown too large. The previous patches added hundreds of more lines to it to implement vector search, and it reached a whopping 7,000 lines of code. This is too much. This patch splits from executor.cc two major chunks: 1. The implementation of **read** requests - GetItem, BatchGetItem, Query (base table, GSI/LSI, and vector-search), and Scan - was moved to a new source file alternator/executor_read.cc. The new file has 2,000 lines. 2. Moved 250 lines of template functions dealing with attribute paths and maps of them to a new header file, attribute_path.hh. These utilities are used for many different operations - various read operations use them for ProjectionExpression, and UpdateItem uses them for modifications to nested attributes, so we need the new header file from both executor.cc and executor_read.cc The remaining executor.cc is still pretty big, 5,000 lines, and contains write operations (PutItem, UpdateItem, DeleteItem, BatchWriteItem) as well as various table and other operations, and also many utility functions used by many types of operations, so we can later continue this refactoring effort. Refs #5783 Signed-off-by: Nadav Har'El <nyh@scylladb.com> |
||
Nikos Dragazis
|
26f0c038af |
test: boost: Add test for wrap-around vnodes
Add a Boost test to verify that `prepare_for_tablets_migration()` produces the correct tablet boundaries when a wrap-around vnode exists. Tablets cannot wrap around the token ring as vnodes do; the last token of the last tablet must always be MAX_TOKEN. When the last vnode token does not coincide with MAX_TOKEN, the wrap-around vnode must be split into two tablets. The test is parameterized over both cases: unaligned (split expected) and aligned (no split expected). Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com> |
||
|
Avi Kivity
|
0ae22a09d4 |
LICENSE: Update to version 1.1
Updated terms of non-commercial use (must be a never-customer). |
||
|
Avi Kivity
|
ca80ee8586 |
Merge 'Introduce maintenance scheduling supergroup and do initial population' from Pavel Emelyanov
The supergroup replaces streaming (a.k.a. maintenance as well) group, inherits 200 shares from it and consists of four sub-groups (all have equal shares of 200 withing the new supergroup) * maintenance_compaction. This group configures `compaction_manager::maintenance_sg()` group. User-triggered compaction runs in it * backup. This group configures `snapshot_ctl::config::backup_sched_group`. Native backup activity runs there * maintenance. It's a new "visible" name, everything that was called "maintenance" in the code ran in "streaming" group. Now it will run in "maintenance". The activities include those that don't communicate over RPC (see below why) * `tablet_allocator::balance_tablets()` * `sstables_manager::components_reclaim_reload_fiber()` * `tablet_storage_group_manager::merge_completion_fiber()` * metrics exporting http server altogether * streaming. This is purely existing streaming group that just moves under the new supergroup. Everything else that was run there, continues doing so, including * hints sender * all view building related components (update generator, builder, workers) * repair * stream_manager * messaging service (except for verb handlers that switch groups) * join_cluster() activity * REST API * ... something else I forgot The `--maintenance_io_throughput_mb_per_sec` option is introduced. It controls the IO throughput limit applied to the maintenance supergroup. If not set, the `--stream_io_throughput_mb_per_sec` option is used to preserve backward compatibility. All new sched groups inherit `request_class::maintenance` (however, "backup" seem not to make any requests yet). Moving more activities from "streaming" into "maintenance" (or its own group) is possible, but one will need to take care of RPC group switching. The thing is that when a client makes an RPC call, the server may switch to one of pre-negotiated scheduling groups. Verbs for existing activities that run in "streaming" group are routed through RPC index that negotiates "streaming" group on the server side. If any of that client code moves to some other group, server will still run the handlers in "streaming" which is not quite expected. That's one of the main reasons why only the selected fibers were moved to their own "maintenance" group. Similar for backup -- this code doesn't use RPC, so it can be moved. Restoring code uses load-and-stream and corresponding RPCs, so it cannot be just moved into its own new group. Fixes SCYLLADB-351 New feature, not backporting Closes scylladb/scylladb#28542 * github.com:scylladb/scylladb: code: Add maintenance/maintenance group backup: Add maintenance/backup group compaction: Add maintenance/maintenance_compaction group main: Introduce maintenance supergroup main: Move all maintenance sched group into streaming one database: Use local variable for current_scheduling_group code: Live-update IO throughputs from main |
||
Botond Dénes
|
5886d1841a |
Merge 'cmake: align CMake build system with configure.py and add comparison script' from Ernest Zaslavsky
Every time someone modifies the build system — adding a source file, changing a compilation flag, or wiring a new test — the change tends to land in only one of our two build systems (configure.py or CMake). Over time this causes three classes of problems: 1. **CMake stops compiling entirely.** Missing defines, wrong sanitizer flags, or misplaced subdirectory ordering cause hard build failures that are only discovered when someone tries to use CMake (e.g. for IDE integration). 2. **Missing build targets.** Tests or binaries present in configure.py are never added to CMake, so `cmake --build` silently skips them. This PR fixes several such cases (e.g. `symmetric_key_test`, `auth_cache_test`, `sstable_tablet_streaming`). 3. **Missing compilation units in targets.** A `.cc` file is added to a test binary in one system but not the other, causing link errors or silently omitted test coverage. To fix the existing drift and prevent future divergence, this series: **Adds a build-system comparison script** (`scripts/compare_build_systems.py`) that configures both systems into a temporary directory, parses their generated `build.ninja` files, and compares per-file compilation flags, link target sets, and per-target libraries. configure.py is treated as the baseline; CMake must match it. The script supports a `--ci` mode suitable for gating PRs that touch build files. **Fixes all current mismatches** found by the script: - Mode flag alignment in `mode.common.cmake` and `mode.Coverage.cmake` (sanitizer flags, `-fno-lto`, stack-usage warnings, coverage defines). - Global define alignment (`SEASTAR_NO_EXCEPTION_HACK`, `XXH_PRIVATE_API`, `BOOST_ALL_DYN_LINK`, `SEASTAR_TESTING_MAIN` placement). - Seastar build configuration (shared vs static per mode, coverage sanitizer link options). - Abseil sanitizer flags (`-fno-sanitize=vptr`). - Missing test targets in `test/boost/CMakeLists.txt`. - Redundant per-test flags now covered by global settings. - Lua library resolution via a custom `cmake/FindLua.cmake` using pkg-config, matching configure.py's approach. **Adds documentation** (`docs/dev/compare-build-systems.md`) describing how to run the script and interpret its output. No backport needed — this is build infrastructure improvement only. Closes scylladb/scylladb#29273 * github.com:scylladb/scylladb: scripts: remove lua library rename workaround from comparison script cmake: add custom FindLua using pkg-config to match configure.py test/cmake: add missing tests to boost test suite test/cmake: remove per-test LTO disable cmake: add BOOST_ALL_DYN_LINK and strip per-component defines cmake: move SEASTAR_TESTING_MAIN after seastar and abseil subdirs cmake: add -fno-sanitize=vptr for abseil sanitizer flags cmake: align Seastar build configuration with configure.py cmake: align global compile defines and options with configure.py cmake: fix Coverage mode in mode.Coverage.cmake cmake: align mode.common.cmake flags with configure.py configure.py: add sstable_tablet_streaming to combined_tests docs: add compare-build-systems.md scripts: add compare_build_systems.py to compare ninja build files |
||
|
Avi Kivity
|
0fd9ea9701 |
abseil: update to lts_2026_01_07
Switch to branch lts_2026_01_07, which is exactly equal to upstream now. There were no notable changes in the release notes, but the new versions are more friendly to newer compilers (specifically, in include hygiene). configure.py needs a few library updates; cmake works without change. scylla-gdb.py updated for new hash table layout (by Claude Opus 4.6). * abseil d7aaad83...255c84da (1179): > Abseil LTS branch, Jan 2026, Patch 1 (#2007) > Cherry-picks for LTS 20260107 (#1990) > Apply LTS transformations for 20260107 LTS branch (#1989) > Mark legacy Mutex methods and MutexLock pointer constructors as deprecated > `cleanup`: specify that it's safe to use the class in a signal handler. > Suppress bugprone-use-after-move in benign cases > StrFormat: format scientific notation without heap allocation > Introduce a legacy copy of GetDebugStackTraceHook API. > Report 1ns instead of 0ns for probe_benchmarks. Some tools incorrectly assume that benchmark was not run if 0ns reported. > Add absl::chunked_queue > `CRC32` version of `CombineContiguous` for length <= 32. > Add `absl::down_cast` > Fix FixedArray iterator constructor, which should require input_iterator, not forward_iterator > Add a latency benchmark for hashing a pair of integers. > Delete absl::strings_internal::STLStringReserveAmortized() > As IsAtLeastInputIterator helper > Use StringAppendAndOverwrite() in CEscapeAndAppendInternal() > Add support for absl::(u)int128 in FastIntToBuffer() > absl/strings: Prepare helper for printing objects to string representations. > Use SimpleAtob() for parsing bool flags > No-op changes to relative timeout support code. > Adjust visibility of heterogeneous_lookup_testing.h > Remove -DUNORDERED_SET_CXX17 since the macro no longer exists > [log] Prepare helper for streaming container contents to strings. > Restrict the visibility of some internal testing utilities > Add absl::linked_hash_set and absl::linked_hash_map > [meta] Add constexpr testing helper. > BUILD file reformatting. > `absl/meta`: Add C++17 port of C++20 `requires` expression for internal use > Remove the implementation of `absl::string_view`, which was only needed prior to C++17. `absl::string_view` is now an alias for `std::string_view`. It is recommended that clients simply use `std::string_view`. > No public description > absl:🎏 Stop echoing file content in flagfile parsing errors Modified ArgsList::ReadFromFlagfile to redact the content of unexpected lines from error messages. \ > Refactor the declaration of `raw_hash_set`/`btree` to omit default template parameters from the subclasses. > Import of CCTZ from GitHub. > Add ABSL_ATTRIBUTE_LIFETIME_BOUND to Flag help generator > Correct `Mix4x16Vectors` comment. > Special implementation for string hash with sizes greater than 64. > Reorder function parameters so that hash state is the first argument. > Search more aggressively for open slots in absl::internal_stacktrace::BorrowedFixupBuffer > Implement SpinLockHolder in terms of std::lock_guard. > No public description > Avoid discarding test matchers. > Import of CCTZ from GitHub. > Automated rollback of commit 9f40d6d6f3cfc1fb0325dd8637eb65f8299a4b00. > Enable clang-specific warnings on the clang-cl build instead of just trying to be MSVC > Enable clang-specific warnings on the clang-cl build instead of just trying to be MSVC > Make AnyInvocable remember more information > Add further diagnostics under clang for string_view(nullptr) > Import of CCTZ from GitHub. > Document the differing trimming behavior of absl::Span::subspan() and std::span::subspan() > Special implementation for string hash with sizes in range [33, 64]. > Add the deleted string_view(std::nullptr_t) constructor from C++23 > CI: Use a cached copy of GoogleTest in CMake builds if possible to minimize the possibility of errors downloading from GitHub > CI: Enable libc++ hardening in the ASAN build for even more checks https://libcxx.llvm.org/Hardening.html > Call the common case of AllocateBackingArray directly instead of through the function pointer. > Change AlignedType to have a void* array member so that swisstable backing arrays end up in the pointer-containing partition for heap partitioning. > base: Discourage use of ABSL_ATTRIBUTE_PACKED > Revert: Add an attribute to HashtablezInfo which performs a bitwise XOR on all hashes. The purposes of this attribute is to identify if identical hash tables are being created. If we see a large number of identical tables, it's likely the code can be improved by using a common table as opposed to keep rebuilding the same one. > Import of CCTZ from GitHub. > Record insert misses in hashtable profiling. > Add absl::StatusCodeToStringView. > Add a missing dependency on str_format that was being pulled in transitively > Pico-optimize `SkipWhitespace` to use `StripLeadingAsciiWhitespace`. > absl::string_view: Upgrade the debug assert on the single argument char* constructor to ABSL_HARDENING_ASSERT > Use non-stack storage for stack trace buffers > Fixed incorrect include for ABSL_NAMESPACE_BEGIN > Add ABSL_REFACTOR_INLINE to separate the inliner directive from the deprecated directive so that we can give users a custom deprecation message. > Reduce stack usage when unwinding without fixups > Reduce stack usage when unwinding from 170 to 128 on x64 > Rename RecordInsert -> RecordInsertMiss. > PR #1968: Use std::move_backward within InlinedVector's Storage::Insert > Use the new absl::StringResizeAndOverwrite() in CUnescape() > Explicitly instantiate common `raw_hash_set` backing array functions. > Rollback reduction of maximum load factor. Now it is back to 28/32. > Export Mutex::Dtor from shared libraries in NDEBUG mode > Allow `IsOkAndHolds` to rely on duck typing for matching `StatusOr` like types instead of uniquely `absl::StatusOr`, e.g. `google::cloud::StatusOr`. > Fix typo in macro and add missing static_cast for WASM builds. > windows(cmake): add abseil_test_dll to target link libraries when required > Handle empty strings in `SimpleAtof` after stripping whitespace > Avoid using a thread_local in an inline function since this causes issues on some platforms. > (Roll forward) Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes. > Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes. > Change Abseil's SpinLock adaptive_spin_count to a class static variable that can be set by tcmalloc friend classes. > Fixes for String{Resize|Append}AndOverwrite - StringAppendAndOverwrite() should always call StringResizeAndOverwrite() with at least capacity() in case the standard library decides to shrink the buffer (Fixes #1965) - Small refactor to make the minimum growth an addition for clarity and to make it easier to test 1.5x growth in the future - Turn an ABSL_HARDENING_ASSERT into a ThrowStdLengthError - Add a missing std::move > Correct the supported features of Status Matchers > absl/time: Use "memory order acquire" for loads, which would allow for the safe removal of the data memory barrier. > Use the new absl::StringResizeAndOverwrite() in string escaping utilities > Add an internal-only helper StringAppendAndOverwrite() similar to StringResizeAndOverwrite() but optimized for repeated appends, using exponential growth to ensure amortized complexity of increasing a string size by a small amount is O(1). > Release `ABSL_EXPECT_OK` and `ABSL_ASSERT_OK`. > Fix the CHECK_XX family of macros to not print `char*` arguments as C-strings if the comparison happened as pointers. Printing as pointers is more relevant to the result of the comparison. > Rollback StringAppendAndOverwrite() - the problem is that StringResizeAndOverwrite has MSAN testing of the entire string. This causes quadratic MSAN verification on small appends. > Add an internal-only helper StringAppendAndOverwrite() similar to StringResizeAndOverwrite() but optimized for repeated appends, using exponential growth to ensure amortized complexity of increasing a string size by a small amount is O(1). > PR #1961: Fix Clang warnings on powerpc > Use the new absl::StringResizeAndOverwrite() in string escaping utilities > Use the new absl::StringResizeAndOverwrite() in string escaping utilities > macOS CI: Move the Bazel vendor_dir to ${HOME} to workaround a Bazel issue where it does not work when it is in ${TMP} and also fix the quoting which was causing it to incorrectly receive the argument > Use __msan_check_mem_is_initialized for detailed MSan report > Optimize stack unwinding by reducing `AddressIsReadable` calls. > Add internal API to allow bypassing stack trace fixups when needed > absl::StrFormat: improve test coverage with scientific exponent test cases > Add throughput and latency benchmarks for `absl::ToDoubleXYZ` functions. > CordzInfo: Use absl::NoDestructor to remove a global destructor. Chromium requires no global destructors. > string_view: Enable std::view and std::borrowed_range > cleanup: s/logging_internal/log_internal/ig for consistency > Use the new absl::StringResizeAndOverwrite() in string escaping utilities > Use the new absl::StringResizeAndOverwrite() in string escaping utilities > Use the new absl::StringResizeAndOverwrite() in absl::AsciiStrTo{Lower|Upper} > Use the new absl::StringResizeAndOverwrite() in absl::StrJoin() > Use the new absl::StringResizeAndOverwrite() in absl::StrCat() > string_view: Fix include order > Don't pass nullptr as the 1st arg of `from_chars` > absl/types: format code with clang-format. > Validate absl::StringResizeAndOverwrite op has written bytes as expected. > Skip the ShortStringCollision test on WASM. > Rollback `absl/types`: format code with clang-format. > Remove usage of the WasmOffsetConverter for Wasm / Emscripten stack-traces. > Use the new absl::StringResizeAndOverwrite() in absl::CordCopyToString() > Remove an undocumented behavior of --vmodule and absl::SetVLogLevel that could set a module_pattern to defer to the global vlog threshold. > Update to rules_cc 0.2.9 > Avoid redefine warnings with ntstatus constants > PR #1944: Use same element-width for non-temporal loads and stores on Arm > absl::StringResizeAndOverwrite(): Add the requirement that the only value that can be written to buf[size] is the terminator character. > absl/types: format code with clang-format. > Minor formatting changes. > Remove `IntIdentity` and `PtrIdentity` from `raw_hash_set_probe_benchmark`. > Automated rollback of commit cad60580dba861d36ed813564026d9774d9e4e2b. > FlagStateInterface implementors need only support being restored once. > Clarify the post-condition of `reserve()` in Abseil hash containers. > Clarify the post-condition of `reserve()` in Abseil hash containers. > Represent dropped samples in hashtable profile. > Add lifetimebound to absl::implicit_cast and make it work for rvalue references as it already does with lvalue references > Clean up a doc example where we had `absl_nonnull` and `= nullptr;` > Change Cordz to synchronize tracked cords with Snapshots / DeleteQueue > Minor refactor to `num_threads` in deadlock test > Rename VLOG macro parameter to match other uses of this pseudo type. > `time`: Fix indentation > Automated Code Change > Adds `absl::StringResizeAndOverwrite` as a polyfill for C++23's `std::basic_string<CharT,Traits,Allocator>::resize_and_overwrite` > Internal-only change > absl/time: format code with clang-format. > No public description > Expose typed releasers of externally appended memory. > Fix __declspec support for ABSL_DECLARE_FLAG() > Annotate absl::AnyInvocable as an owner type via [[gsl::Owner]] and absl_internal_is_view = std::false_type > Annotate absl::FunctionRef as a view type via [[gsl::Pointer]] and absl_internal_is_view > Remove unnecessary dep on `core_headers` from the `nullability` cc_library > type_traits: Add type_identity and type_traits_t backfills > Refactor raw_hash_set range insertion to call private insert_range function. > Fix bug in absl::FunctionRef conversions from non-const to const > PR #1937: Simplify ConvertSpecialToEmptyAndFullToDeleted > Improve absl::FunctionRef compatibility with C++26 > Add a workaround for unused variable warnings inside of not-taken if-constexpr codepaths in older versions of GCC > Annotate ABSL_DIE_IF_NULL's return type with `absl_nonnull` > Move insert index computation into `PrepareInsertLarge` in order to reduce inlined part of insert/emplace operations. > Automated Code Change > PR #1939: Add missing rules_cc loads > Expose (internally) a LogMessage constructor taking file as a string_view for (internal, upcoming) FFI integration. > Fixed up some #includes in mutex.h > Make absl::FunctionRef support non-const callables, aligning it with std::function_ref from C++26 > Move capacity update in `Grow1To3AndPrepareInsert` after accessing `common.infoz()` to prevent assertion failure in `control()`. > Fix check_op(s) compilation failures on gcc 8 which eagerly tries to instantiate std::underlying_type for non-num types. > Use `ABSL_ATTRIBUTE_ALWAYS_INLINE`for lambda in `find_or_prepare_insert_large`. > Mark the implicit floating operators as constexpr for `absl::int128` and `absl::uint128` > PR #1931: raw_hash_set: fix instantiation for recursive types on MSVC with /Zc:__cplusplus > Add std::pair specializations for IsOwner and IsView > Cast ABSL_MIN_LOG_LEVEL to absl::LogSeverityAtLeast instead of absl::LogSeverity. > Fix a corner case in the aarch64 unwinder > Fix inconsistent nullability annotation in ReleasableMutexLock > Remove support for Native Client > Rollback f040e96b93dba46e8ed3ca59c0444cbd6c0a0955 > When printing CHECK_XX failures and both types are unprintable, don't bother printing " (UNPRINTABLE vs. UNPRINTABLE)". > PR #1929: Fix shorten-64-to-32 warning in stacktrace_riscv-inl.inc > Refactor `find_or_prepare_insert_large` to use a single return statement using a lambda. > Use possible CPUs to identify NumCPUs() on Linux. > Fix incorrect nullability annotation of `absl::Cord::InlineRep::set_data()`. > Move SetCtrl* family of functions to cc file. > Change absl::InlinedVector::clear() so that it does not deallocate any allocated space. This allows allocations to be reused and matches the behavior specification of std::vector::clear(). > Mark Abseil container algorithms as `constexpr` for C++20. > Fix `CHECK_<OP>` ambiguous overload for `operator<<` in older versions of GCC when C-style strings are compared > stacktrace_test: avoid spoiling errno in the test signal handler. > Optimize `CRC32AcceleratedX86ARMCombinedMultipleStreams::Extend` by interleaving the `CRC32_u64` calls at a lower level. > stacktrace_test: avoid spoiling errno in the test signal handler. > stacktrace_test: avoid spoiling errno in the test signal handler. > std::multimap::find() is not guaranteed to return the first entry with the requested key. Any may be returned if many exist. > Mark `/`, `%`, and `*` operators as constexpr when intrinsics are available. > Add the C++20 string_view contructor that uses iterators > Implement absl::erase_if for absl::InlinedVector > Adjust software prefetch to fetch 5 cachelines ahead, as benchmarking suggests this should perform better. > Reduce maximum load factor to 27/32 (from 28/32). > Remove unused include > Remove unused include statement > PR #1921: Fix ABSL_BUILD_DLL mode (absl_make_dll) with mingw > PR #1922: Enable mmap for WASI if it supports the mman header > Rollback C++20 string_view constructor that uses iterators due to broken builds > Add the C++20 string_view contructor that uses iterators > Bump versions of dependencies in MODULE.bazel > Automated Code Change > PR #1918: base: add musl + ppc64le fallback for UnscaledCycleClock::Frequency > Optimize crc32 Extend by removing obsolete length alignment. > Fix typo in comment of `ABSL_ATTRIBUTE_UNUSED`. > Mark AnyInvocable as being nullability compatible. > Ensure stack usage remains low when unwinding the stack, to prevent stack overflows > Shrink #if ABSL_HAVE_ATTRIBUTE_WEAK region sizes in stacktrace_test.cc > <filesystem> is not supported for XTENSA. Disable it in //absl/hash/internal/hash.h. > Use signal-safe dynamic memory allocation for stack traces when necessary > PR #1915: Fix SYCL Build Compatibility with Intel LLVM Compiler on Windows for abseil > Import of CCTZ from GitHub. > Tag tests that currently fail on ios_sim_arm64 with "no_test_ios_sim_arm64" > Automated Code Change > Automated Code Change > Import of CCTZ from GitHub. > Move comment specific to pointer-taking MutexLock variant to its definition. > Add lifetime annotations to MutexLock, SpinLockHolder, etc. > Add lifetimebound annotations to absl::MakeSpan and absl::MakeConstSpan to detect dangling references > Remove comment mentioning deferenceability. > Add referenceful MutexLock with Condition overload. > Mark SpinLock camel-cased methods as ready for inlining. > Whitespace change > In logging tests that write expectations against `ScopedMockLog::Send`, suppress the default behavior that forwards to `ScopedMockLog::Log` so that unexpected logs are printed with full metadata. Many of these tests are poking at those metadata, and a failure message that doesn't include them is unhelpful. > Add ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::ClippedSubstr > Inline internal usages of Mutex::Lock, etc. in favor of lock. > Inline internal usages of pointerful SpinLockHolder/MutexLock. > Remove wrong comment in Cord::Unref > Update the crc32 dynamic dispatch table with newer platforms. > PR #1914: absl/base/internal/poison.cc: Minor build fix > Accept references on SpinLockHolder/MutexLock > Import of CCTZ from GitHub. > Fix typos in comments. > Inline SpinLock Lock->lock, Unlock->unlock internal to Abseil. > Rename Mutex methods to use the typical C++ lower case names. > Rename SpinLock methods to use the typical C++ lower case names. > Add an assert that absl::StrSplit is not called with a null char* argument. > Fix sign conversion warning > PR #1911: Fix absl_demangle_test on ppc64 > Disallow using a hash function whose return type is smaller than size_t. > Optimize CRC-32C extension by zeroes > Deduplicate stack trace implementations in stacktrace.cc > Align types of location_table_ and mapping_table_ keys (-Wshorten-64-to-32). > Move SigSafeArena() out to absl/base/internal/low_level_alloc.h > Allow CHECK_<OP> variants to be used with unprintable types. > Import of CCTZ from GitHub. > Adds required load statements for C++ rules to BUILD and bzl files. > Disable sanitizer bounds checking in ComputeZeroConstant. > Roll back NDK weak symbol mode for backtrace() due to internal test breakage > Add converter for extracting SwissMap profile information into a https://github.com/google/pprof suitable format for inspection. > Allocate memory for frames and sizes during stack trace fix-up when no memory is provided > Support NDK weak symbol mode for backtrace() on Android. > Change skip_empty_or_deleted to not use groups. > Fix bug of dereferencing invalidated iterator in test case. > Refactor: split erase_meta_only into large and small versions. > Fix a TODO to use std::is_nothrow_swappable when it became available. > Clean up the testing of alternate options that were removed in previous changes > Only use generic stacktrace when ABSL_HAVE_THREAD_LOCAL. > Automated Code Change > Add triviality tests for absl::Span > Loosen the PointerAlignment test to allow up to 5 stuck bits to avoid flakiness. > Prevent conversion constructions from absl::Span to itself > Skip flaky expectations in waiter_test for MSVC. > Refactor: call AssertIsFull from iterator::assert_is_full to avoid passing the same arguments repeatedly. > In AssertSameContainer, remove the logic checking for whether the iterators are from SOO tables or not since we don't use it to generate a more informative debug message. > Remove unused NonIterableBitMask::HighestBitSet function. > Refactor: move iterator unchecked_* members before data members to comply with Google C++ style guide. > Mix pointers once instead of twice now that we've improved mixing on 32-bit platforms and improved the kMul constant. > Remove unused utility functions/constants. > Revert a change for breaking downstream third party libs > Remove unneeded include from cord_rep_btree_navigator.h > Refactor: move find_first_non_full into raw_hash_set.cc. > Perform stronger mixing on 32-bit platforms and enable the LowEntropyStrings test. > Include deallocated caller-provided size in delete hooks. > Roll back one more time: In debug mode, assert that the probe sequence isn't excessively long. > Allow a `std::move` of `delimiter_` to happen in `ByString::ByString(ByString&&)`. Right now the move ctor is making a copy because the source object is `const`. > Assume that control bytes don't alias CommonFields. > Consistently use [[maybe_unused]] in raw_hash_set.h for better compiler warning compatibility. > Roll forward: In debug mode, assert that the probe sequence isn't excessively long. > Add a new test for hash collisions for short strings when PrecombineLengthMix has low quality. > Refactor: define CombineRawImpl for repeated `Mix(state ^ value, kMul)` operations. > Automated Code Change > Mark hash_test as large so that the timeout is increased. > Change the value of kMul to have higher entropy and prevent collisions when keys are aligned integers or pointers. > Fix LIFETIME annotations for op*/op->/value operators for reference types. > Update StatusOr to support lvalue reference value types. > Rollback debug assertion that the probe sequence isn't excessively long. > AnyInvocable: Fix operator==/!= comments > In debug mode, assert that the probe sequence isn't excessively long. > Improve NaN handling in absl::Duration arithmetic. > Change PrecombineLengthMix to sample data from kStaticRandomData. > Fix includes and fuse constructors of SpinLock. > Enable `operator==` for `StatusOr` only if the contained type is equality-comparable > Enable SIMD memcpy-crc on ARM cores. > Improve mixing on 32-bit platforms. > Change DurationFromDouble to return -InfiniteDuration() for all NaNs. > Change return type of hash internal `Seed` to `size_t` from `uint64_t` > CMake: Add a fatal error when the compiler defaults to or is set to a C++ language standard prior to C++17. > Make bool true hash be ~size_t{} instead of 1 so that all bits are different between true/false instead of only one. > Automated Code Change > Pass swisstable seed as seed to absl::Hash so we can save an XOR in H1. > Add support for scoped enumerations in CHECK_XX(). > Revert no-inline on Voidify::operator&&() -- caused unexpected binary size growth > Mark Voidify::operator&&() as no-inline. This improves stack trace for `LOG(FATAL)` with optimization on. > Refactor long strings hash computations and move `len <= PiecewiseChunkSize()` out of the line to keep only one function call in the inlined hash code. > rotr/rotl: Fix undefined behavior when passing INT_MIN as the number of positions to rotate by > Reorder members of MixingHashState to comply with Google C++ style guide ordering of type declarations, static constants, ctors, non-ctor functions. > Delete unused function ShouldSampleHashtablezInfoOnResize. > Remove redundant comments that just name the following symbol without providing additional information. > Remove unnecessary modification of growth info in small table case. > Suppress CFI violation on VDSO call. > Replace WeakMix usage with Mix and change H2 to use the most significant 7 bits - saving 1 cycle in H1. > Fix -Wundef warning > Fix conditional constexpr in ToInt64{Nano|Micro|Milli}seconds under GCC7 and GCC8 using an else clause as a workaround > Enable CompressedTupleTest.NestedEbo test case. > Lift restriction on using EBCO[1] for nested CompressedTuples. The current implementation of CompressedTuple explicitly disallows EBCO for cases where CompressedTuples are nested. This is because the implentation for a tuple with EBCO-compatible element T inherits from Storage<T, I>, where I is the index of T in the tuple, and > absl::string_view: assert against (data() == nullptr && size() != 0) > Fix a false nullability warning in [Q]CHECK_OK by replacing nullptr with an empty char* > Make `combine_contiguous` to mix length in a weak way by adding `size << 24`, so that we can avoid a separate mixing of size later. The empty range is mixing 0x57 byte. > Add a test case that -1.0 and 1.0 have different hashes. > Update CI to a more recent Clang on Linux x86-64 > `absl::string_view`: Add a debug assert to the single-argument constructor that the argument is not `nullptr`. > Fix CI on macOS Sequoia > Use Xcode 16.3 for testing > Use a proper fix instead of a workaround for a parameter annotated absl_nonnull since the latest Clang can see through the workaround > Assert that SetCtrl isn't called on small tables - there are no control bytes in such cases. > Use `MaskFullOrSentinel` in `skip_empty_or_deleted`. > Reduce flakiness in MockDistributions.Examples test case. > Rename PrepareInsertNonSoo to PrepareInsertLarge now that it's no longer used in all non-SOO cases. > PR #1895: use c++17 in podspec > Avoid hashing the key in prefetch() for small tables. > Remove template alias nullability annotations. > Add `Group::MaskFullOrSentinel` implementation without usage. > Move `hashtable_control_bytes` tests into their own file. > Simplify calls to `EqualElement` by introducing `equal_to` helper function. > Do `common.increment_size()` directly in SmallNonSooPrepareInsert if inserting to reserved 1 element table. > Import of CCTZ from GitHub. > Small cleanup of `infoz` processing to get the logic out of the line or removed. > Extract the entire PrepareInsert to Small non SOO table out of the line. > Take `get_hash` implementation out of the SwissTable class to minimize number of instantiations. > Change kEmptyGroup to kDefaultIterControl now that it's only used for default-constructed iterators. > [bits] Add tests for return types > Avoid allocating control bytes in capacity==1 swisstables. > PR #1888: Adjust Table.GrowExtremelyLargeTable to avoid OOM on i386 > Avoid mixing after `Hash64` calls for long strings by passing `state` instead of `Seed` to low level hash. > Indent absl container examples consistently > Revert- Doesn't actually work because SWIG doesn't use the full preprocessor > Add tags to skip some tests under UBSAN. > Avoid subtracting `it.control()` and `table.control()` in single element table during erase. > Remove the `salt` parameter from low level hash and use a global constant. That may potentially remove some loads. > In SwissTable, don't hash the key when capacity<=1 on insertions. > Remove the "small" size designation for thread_identity_test, which causes the test to timeout after 60s. > Add comment explaining math behind expressions. > Exclude SWIG from ABSL_DEPRECATED and ABSL_DEPRECATE_AND_INLINE > stacktrace_x86: Handle nested signals on altstack > Import of CCTZ from GitHub. > Simplify MixingHashState::Read9To16 to not depend on endianness. > Delete deprecated `absl::Cord::Get` and its remaining call sites. > PR #1884: Remove duplicate dependency > Remove relocatability test that is no longer useful > Import of CCTZ from GitHub. > Fix a bug of casting sizeof(slot_type) to uint16_t instead of uint32_t. > Rewrite `WideToUtf8` for improved readability. > Avoid requiring default-constructability of iterator type in algorithms that use ContainerIterPairType > Added test cases for invalid surrogates sequences. > Use __builtin_is_cpp_trivially_relocatable to implement absl::is_trivially_relocatable in a way that is compatible with PR2786 in the upcoming C++26. > Remove dependency on `wcsnlen` for string length calculation. > Stop being strict about validating the "clone" part of mangled names > Add support for logging wide strings in `absl::log`. > Deprecate `ABSL_HAVE_STD_STRING_VIEW`. > Change some nullability annotations in absl::Span to absl_nullability_unknown to workaround a bug that makes nullability checks trigger in foreach loops, while still fixing the -Wnullability-completeness warnings. > Linux CI update > Fix new -Wnullability-completeness warnings found after upgrading the Clang version used in the Linux ARM CI to Clang 19. > Add __restrict for uses of PolicyFunctions. > Use Bazel vendor mode to cache external dependencies on Windows and macOS > Move PrepareInsertCommon from header file to cc file. > Remove the explicit from the constructor to a test allocator in hash_policy_testing.h. This is rejected by Clang when using the libstdc++ that ships with GCC15 > Extract `WideToUtf8` helper to `utf8.h`. > Updates the documentation for `CHECK` to make it more explicit that it is used to require that a condition is true. > Add PolicyFunctions::soo_capacity() so that the compiler knows that soo_capacity() is always 0 or 1. > Expect different representations of pointers from the Windows toolchain. > Add set_no_seed_for_testing for use in GrowExtremelyLargeTable test. > Update GoogleTest dependency to 1.17.0 to support GCC15 > Assume that frame pointers inside known stack bounds are readable. > Remove fallback code in absl/algorithm/container.h > Fix GCC15 warning that <ciso646> is deprecated in C++17 > Fix misplaced closing brace > Remove unused include. > Automated Code Change > Type erase copy constructor. > Refactor to use hash_of(key) instead of hash_ref()(key). > Create Table.Prefetch test to make sure that it works. > Remove NOINLINE on the constructor with buckets. > In SwissTable, don't hash the key in find when capacity<=1. > Use 0x57 instead of Seed() for weakly mixing of size. > Use absl::InsecureBitGen in place of std::random_device in Abseil tests. > Remove unused include. > Use large 64 bits kMul for 32 bits platforms as well. > Import of CCTZ from GitHub. > Define `combine_weakly_mixed_integer` in HashSelect::State in order to allow `friend auto AbslHashValue` instead of `friend H AbslHashValue`. > PR #1878: Fix typos in comments > Update Abseil dependencies in preparation for release > Use weaker mixing for absl::Hash for types that mix their sizes. > Update comments on UnscaledCycleClock::Now. > Use alignas instead of the manual alignment for the Randen entropy pool. > Document nullability annotation syntax for array declarations (not many people may know the syntax). > Import of CCTZ from GitHub. > Release tests for ABSL_RAW_DCHECK and ABSL_RAW_DLOG. > Adjust threshold for stuck bits to avoid flaky failures. > Deprecate template type alias nullability annotations. > Add more probe benchmarks > PR #1874: Simplify detection of the powerpc64 ELFv1 ABI > Make `absl::FunctionRef` copy-assignable. This brings it more in line with `std::function_ref`. > Remove unused #includes from absl/base/internal/nullability_impl.h > PR #1870: Retry SymInitialize on STATUS_INFO_LENGTH_MISMATCH > Prefetch from slots in parallel with reading from control. > Migrate template alias nullability annotations to macros. > Improve dependency graph in `TryFindNewIndexWithoutProbing` hot path evaluation. > Add latency benchmarks for Hash for strings with size 3, 5 and 17. > Exclude UnwindImpl etc. from thread sanitizer due to false positives. > Use `GroupFullEmptyOrDeleted` inside of `transfer_unprobed_elements_to_next_capacity_fn`. > PR #1863: [minor] Avoid variable shadowing for absl btree > Extend stack-frame walking functionality to allow dynamic fixup > Fix "unsafe narrowing" in absl for Emscripten > Roll back change to address breakage > Extend stack-frame walking functionality to allow dynamic fixup > Introduce `absl::Cord::Distance()` > Avoid aliasing issues in growth information initialization. > Make `GrowSooTableToNextCapacityAndPrepareInsert` in order to initialize control bytes all at once and avoid two function calls on growth right after SOO. > Simplify `SingleGroupTableH1` since we do not need to mix all bits anymore. Per table seed has a good last bit distribution. > Use `NextSeed` instead of `NextSeedBaseNumber` and make the result type to be `uint16_t`. That avoids unnecessary bit twiddling and simplify the code. > Optimize `GrowthToLowerBoundCapacity` in order to avoid division. > [base] Make :endian internal to absl > Fully qualify absl names in check macros to avoid invalid name resolution when the user scope has those names defined. > Fix memory sanitization in `GrowToNextCapacityAndPrepareInsert`. > Define and use `ABSL_SWISSTABLE_ASSERT` in cc file since a lot of logic moved there. > Remove `ShouldInsertBackwards` functionality. It was used for additional order randomness in debug mode. It is not necessary anymore with introduction of separate per table `seed`. > Fast growing to the next capacity based on carbon hash table ideas. > Automated Code Change > Refactor CombinePiecewiseBuffer test case to (a) call PiecewiseChunkSize() to get the chunk size and (b) use ASSERT for expectation in a loop. > PR #1867: Remove global static in stacktrace_win32-inl.inc > Mark Abseil hardening assert in AssertIsValidForComparison as slow. > Roll back a problematic change. > Add absl::FastTypeId<T>() > Automated Code Change > Update TestIntrinsicInt128 test to print the indices with the conflicting hashes. > Code simplification: we don't need XOR and kMul when mixing large string hashes into hash state. > Refactor absl::CUnescape() to use direct string output instead of pointer/size. > Rename `policy.transfer` to `policy.transfer_n`. > Optimize `ResetCtrl` for small tables with `capacity < Group::KWidth * 2` (<32 if SSE enabled and <16 if not). > Use 16 bits of per-table-seed so that we can save an `and` instruction in H1. > Fully annotate nullability in headers where it is partially annotated. > Add note about sparse containers to (flat|node)_hash_(set|map). > Make low_level_alloc compatible with -Wthread-safety-pointer > Add missing direct includes to enable the removal of unused includes from absl/base/internal/nullability_impl.h. > Add tests for macro nullability annotations analogous to existing tests for type alias annotations. > Adds functionality to return stack frame pointers during stack walking, in addition to code addresses > Use even faster reduction algorithm in FinalizePclmulStream() > Add nullability annotations to some very-commonly-used APIs. > PR #1860: Add `unsigned` to character buffers to ensure they can provide storage (https://eel.is/c++draft/intro.object#3) > Release benchmarks for absl::Status and absl::StatusOr > Use more efficient reduction algorithm in FinalizePclmulStream() > Add a test case to make it clear that `--vmodule=foo/*=1` does match any children and grandchildren and so on under `foo/`. > Gate use of clang nullability qualifiers through absl nullability macros on `nullability_on_classes`. > Mark `absl::StatusOr::status()` as ABSL_MUST_USE_RESULT > Cleanups related to benchmarks * Fix many benchmarks to be cc_binary instead of cc_test * Add a few benchmarks for StrFormat * Add benchmarks for Substitute * Add benchmarks for Damerau-Levenshtein distance used in flags > Add a log severity alias `DO_NOT_$UBMIT` intended for logging during development > Avoid relying on true and false tokens in the preprocessor macros used in any_invocable.h > Avoid relying on true and false tokens in the preprocessor macros used in absl/container > Refactor to make it clear that H2 computation is not repeated in each iteration of the probe loop. > Turn on C++23 testing for GCC and Clang on Linux > Fix overflow of kSeedMask on 32 bits platform in `generate_new_seed`. > Add a workaround for std::pair not being trivially copyable in C++23 in some standard library versions > Refactor WeakMix to include the XOR of the state with the input value. > Migrate ClearPacBits() to a more generic implementation and location > Annotate more Abseil container methods with [[clang::lifetime_capture_by(...)]] and make them all forward to the non-captured overload > Make PolicyFunctions always be the second argument (after CommonFields) for type-erased functions. > Move GrowFullSooTableToNextCapacity implementation with some dependencies to cc file. > Optimize btree_iterator increment/decrement to avoid aliasing issues by using local variables instead of repeatedly writing to `this`. > Add constexpr conversions from absl::Duration to int64_t > PR #1853: Add support for QCC compiler > Fix documentation for key requirements of flat_hash_set > Use `extern template` for `GrowFullSooTableToNextCapacity` since we know the most common set of paramenters. > C++23: Fix log_format_test to match the stream format for volatile pointers > C++23: Fix compressed_tuple_test. > Implement `btree::iterator::+=` and `-=`. > Stop calling `ABSL_ANNOTATE_MEMORY_IS_INITIALIZED` for threadlocal counter. > Automated Code Change > Introduce seed stored in the hash table inside of the size. > Replace ABSL_ATTRIBUTE_UNUSED with [[maybe_unused]] > Minor consistency cleanups to absl::BitGen mocking. > Restore the empty CMake targets for bad_any_cast, bad_optional_access, and bad_variant_access to allow clients to migrate. > bits.h: Add absl::endian and absl::byteswap polyfills > Use absl::NoDestructor an absl::Mutex instance in the flags library to prevent some exit-time destructor warnings > Add thread GetEntropyFromRandenPool test > Update nullability annotation documentation to focus on macro annotations. > Simplify some random/internal types; expose one function to acquire entropy. > Remove pre-C++17 workarounds for lack of std::launder > UBSAN: Use -fno-sanitize-recover > int128_test: Avoid testing signed integer overflow > Remove leading commas in `Describe*` methods of `StatusIs` matcher. > absl::StrFormat: Avoid passing null to memcpy > str_cat_test: Avoid using invalid enum values > hash_generator_testing: Avoid using invalid enum values > absl::Cord: Avoid passing null to memcpy and memset > graphcycles_test: Avoid applying a non-zero offset to a null pointer > Make warning about wrapping empty std::function in AnyInvocable stronger. > absl/random: Convert absl::BitGen / absl::InsecureBitGen to classes from aliases. > Fix buffer overflow the internal demangling function > Avoid calling `ShouldRehashForBugDetection` on the first two inserts to the table. > Remove the polyfill implementations for many type traits and alias them to their std equivalents. It is recommended that clients now simple use the std equivalents. > ROLLBACK: Limit slot_size to 2^16-1 and maximum table size to 2^43-1. > Limit `slot_size` to `2^16-1` and maximum table size to `2^43-1`. > Use C++17 [[nodiscard]] instead of the deprecated ABSL_MUST_USE_RESULT > Remove the polyfills for absl::apply and absl::make_from_tuple, which were only needed prior to C++17. It is recommended that clients simply use std::apply and std::make_from_tuple. > PR #1846: Fix build on big endian > Bazel: Move environment variables to --action_env > Remove the implementation of `absl::variant`, which was only needed prior to C++17. `absl::variant` is now an alias for `std::variant`. It is recommended that clients simply use `std::variant`. > MSVC: Fix warnings c4244 and c4267 in the main library code > Update LowLevelHashLenGt16 to be LowLevelHashLenGt32 now that the input is guaranteed to be >32 in length. > Xtensa does not support thread_local. Disable it in absl/base/config.h. > Add support for 8-bit and 16-bit integers to absl::SimpleAtoi > CI: Update Linux ARM latest container > Add time hash tests > `any_invocable`: Update comment that refer to C++17 and C++11 > `check_test_impl.inc`: Use C++17 features unconditionally > Remove the implementation of `absl::optional`, which was only needed prior to C++17. `absl::optional` is now an alias for `std::optional`. It is recommended that clients simply use `std::optional`. > Move hashtable control bytes manipulation to a separate file. > Fix a use-after-free bug in which the string passed to `AtLocation` may be referenced after it is destroyed. While the string does live until the end of the full statement, logging (previously occurred) in the destructor of the `LogMessage` which may be constructed before the temporary string (and thus destroyed after the temporary string's destructor). > `internal/layout`: Delete pre-C++17 out of line definition of constexpr class member > Extract slow path for PrepareInsertNonSoo to a separate function `PrepareInsertNonSooSlow`. > Minor code cleanups > `internal/log_message`: Use `if constexpr` instead of SFINAE for `operator<<` > [absl] Use `std::min` in `constexpr` contexts in `absl::string_view` > Remove the implementation of `absl::any`, which was only needed prior to C++17. `absl::any` is now an alias for `std::any`. It is recommended that clients simply use `std::any`. > Remove ABSL_INTERNAL_NEED_REDUNDANT_CONSTEXPR_DECL which is longer needed with the C++17 floor > Make `OptimalMemcpySizeForSooSlotTransfer` ready to work with MaxSooSlotSize upto `3*sizeof(size_t)`. > `internal/layout`: Replace SFINAE with `if constexpr` > PR #1830: C++17 improvement: use if constexpr in internal/hash.h > `absl`: Deprecate `ABSL_HAVE_CLASS_TEMPLATE_ARGUMENT_DEDUCTION` > Add a verification for access of being destroyed table. Also enabled access after destroy check in ASAN optimized mode. > Store `CharAlloc` in SwissTable in order to simplify type erasure of functions accepting allocator as `void*`. > Introduce and use `SetCtrlInLargeTable`, when we know that table is at least one group. Similarly to `SetCtrlInSingleGroupTable`, we can save some operations. > Make raw_hash_set::slot_type private. > Delete absl/utility/internal/if_constexpr.h > `internal/any_invocable`: Use `if constexpr` instead of SFINAE when initializing storage accessor > Depend on string_view directly > Optimize and slightly simplify `PrepareInsertNonSoo`. > PR #1833: Make ABSL_INTERNAL_STEP_n macros consistent in crc code > `internal/any_invocable`: Use alias `RawT` consistently in `InitializeStorage` > Move the implementation of absl::ComputeCrc32c to the header file, to facilitate inlining. > Delete absl/base/internal/inline_variable.h > Add lifetimebound to absl::StripAsciiWhitespace > Revert: Random: Use target attribute instead of -march > Add return for opt mode in AssertNotDebugCapacity to make sure that code is not evaluated in opt mode. > `internal/any_invocable`: Delete TODO, improve comment and simplify pragma in constructor > Split resizing routines and type erase similar instructions. > Random: Use target attribute instead of -march > `internal/any_invocable`: Use `std::launder` unconditionally > `internal/any_invocable`: Remove suppresion of false positive -Wmaybe-uninitialized on GCC 12 > Fix feature test for ABSL_HAVE_STD_OPTIONAL > Support C++20 iterators in raw_hash_map's random-access iterator detection > Fix mis-located test dependency > Disable the DestroyedCallsFail test on GCC due to flakiness. > `internal/any_invocable`: Implement invocation using `if constexpr` instead of SFINAE > PR #1835: Bump deployment_target version and add visionos to podspec > PR #1828: Fix spelling of pseudorandom in README.md > Make raw_hash_map::key_arg private. > `overload`: Delete obsolete macros for undefining `absl::Overload` when C++ < 17 > `absl/base`: Delete `internal/invoke.h` and `invoke_test.cc` > Remove `WORKSPACE.bazel` > `absl`: Replace `base_internal::{invoke,invoke_result_t,is_invocable_r}` with `std` equivalents > Allow C++20 forward iterators to use fast paths > Factor out some iterator traits detection code > Type erase IterateOverFullSlots to decrease code size. > `any_invocable`: Delete pre-C++17 workarounds for `noexcept` and guaranteed copy elision > Make raw_hash_set::key_arg private. > Rename nullability macros to use new lowercase spelling. > Fix bug where ABSL_REQUIRE_EXPLICIT_INIT did not actually result in a linker error > Make Randen benchmark program use runtime CPU detection. > Add CI for the C++20/Clang/libstdc++ combination > Move Abseil to GoogleTest 1.16.0 > `internal/any_invocable`: Use `if constexpr` instead of SFINAE in `InitializeStorage` > More type-erasing of InitializeSlots by removing the Alloc and AlignOfSlot template parameters. > Actually use the hint space instruction to strip PAC bits for return addresses in stack traces as the comment says > `log/internal`: Replace `..._ATTRIBUTE_UNUSED_IF_STRIP_LOG` with C++17 `[[maybe_unused]]` > `attributes`: Document `ABSL_ATTRIBUTE_UNUSED` as deprecated > `internal/any_invocable`: Initialize using `if constexpr` instead of ternary operator, enum, and templates > Fix flaky tests due to sampling by introducing utility to refresh sampling counters for the current thread. > Minor reformatting in raw_hash_set: - Add a clear_backing_array member to declutter calls to ClearBackingArray. - Remove some unnecessary `inline` keywords on functions. - Make PoisonSingleGroupEmptySlots static. > Update CI for linux_gcc-floor to use GCC9, Bazel 7.5, and CMake 3.31.5. > `internal/any_invocable`: Rewrite `IsStoredLocally` type trait into a simpler constexpr function > Add ABSL_REQUIRE_EXPLICIT_INIT to Abseil to enable enforcing explicit field initializations > Require C++17 > Minimize number of `InitializeSlots` with respect to SizeOfSlot. > Leave the call to `SampleSlow` only in type erased InitializeSlots. > Update comments for Read4To8 and Read1To3. > PR #1819: fix compilation with AppleClang > Move SOO processing inside of InitializeSlots and move it once. > PR #1816: Random: use getauxval() via <sys/auxv.h> > Optimize `InitControlBytesAfterSoo` to have less writes and make them with compile time known size. > Remove stray plus operator in cleanup_internal::Storage > Include <cerrno> to fix compilation error in chromium build. > Adjust internal logging namespacing for consistency s/ABSL_LOGGING_INTERNAL_/ABSL_LOG_INTERNAL_/ > Rewrite LOG_EVERY_N (et al) docs to clarify that the first instance is logged. Also, deliberately avoid giving exact numbers or examples since IRL behavior is not so exact. > ABSL_ASSUME: Use a ternary operator instead of do-while in the implementations that use a branch marked unreachable so that it is usable in more contexts. > Simplify the comment for raw_hash_set::erase. > Remove preprocessors for now unsupported compilers. > `absl::ScopedMockLog`: Explicitly document that it captures logs emitted by all threads > Fix potential integer overflow in hash container create/resize > Add lifetimebound to StripPrefix/StripSuffix. > Random: Rollforward support runtime dispatch on AArch64 macOS > Crc: Only test non_temporal_store_memcpy_avx on AVX targets > Provide information about types of all flags. > Deprecate the precomputed hash find() API in swisstable. > Import of CCTZ from GitHub. > Adjust whitespace > Expand documentation for absl::raw_hash_set::erase to include idiom example of iterator post-increment. > Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower() > Crc: Remove the __builtin_cpu_supports path for SupportsArmCRC32PMULL > Use absl::NoDestructor for some absl::Mutex instances in the flags library to prevent some exit-time destructor warnings > Update the WORKSPACE dependency of rules_cc to 0.1.0 > Rollback support runtime dispatch on AArch64 macOS for breaking some builds > Downgrade to rules_cc 0.0.17 because 0.1.0 was yanked > Use unused set in testing. > Random: Support runtime dispatch on AArch64 macOS > crc: Use absl::nullopt when returning absl::optional > Annotate absl::FixedArray to warn when unused. > PR #1806: Fix undefined symbol: __android_log_write > Move ABSL_HAVE_PTHREAD_CPU_NUMBER_NP to the file where it is needed > Use rbit instruction on ARM rather than rev. > Debugging: Report the CPU we are running on under Darwin > Add a microbenchmark for very long int/string tuples. > Crc: Detect support for pmull and crc instructions on Apple AArch64 With a newer clang, we can use __builtin_cpu_supports which caches all the feature bits. > Add special handling for hashing integral types so that we can optimize Read1To3 and Read4To8 for the strings case. > Use unused FixedArray instances. > Minor reformatting > Avoid flaky expectation in WaitDurationWoken test case in MSVC. > Use Bazel rules_cc for many compiler-specific rules instead of our custom ones from before the Bazel rules existed. > Mix pointers twice in absl::Hash. > New internal-use-only classes `AsStructuredLiteralImpl` and `AsStructuredValueImpl` > Annotate some Abseil container methods with [[clang::lifetime_capture_by(...)]] > Faster copy from inline Cords to inline Strings > Add new benchmark cases for hashing string lengths 1,2,4,8. > Move the Arm implementation of UnscaledCycleClock::Now() into the header file, like the x86 implementation, so it can be more easily inlined. > Minor include cleanup in absl/random/internal > Import of CCTZ from GitHub. > Use Bazel Platforms to support AES-NI compile options for Randen > In HashState::Create, require that T is a subclass of HashStateBase in order to discourage users from defining their own HashState types. > PR #1801: Remove unncessary <iostream> includes > New class StructuredProtoField > Mix pointers twice in TSan and MSVC to avoid flakes in the PointerAlignment test. > Add a test case that type-erased absl::HashState is consistent with absl::HashOf. > Mix pointers twice in build modes in which the PointerAlignment test is flaky if we mix once. > Increase threshold for stuck bits in PointerAlignment test on android. > Use hashing ideas from Carbon's hashtable in absl hashing: - Use byte swap instead of mixing pointers twice. - Change order of branches to check for len<=8 first. - In len<=16 case, do one multiply to mix the data instead of using the logic from go/absl-hash-rl (reinforcement learning was used to optimize the instruction sequence). - Add special handling for len<=32 cases in 64-bit architectures. > Test that using a table that was moved-to from a moved-from table fails in sanitizer mode. > Remove a trailing comma causing an issue for an OSS user > Add missing includes in hash.h. > Use the public implementation rule for "@bazel_tools//tools/cpp:clang-cl" > Import of CCTZ from GitHub. > Change the definition of is_trivially_relocatable to be a bit less conservative. > Updates to CI to support newer versions of tools > Check if ABSL_HAVE_INTRINSIC_INT128 is defined > Print hash expansions in the hash_testing error messages. > Avoid flakiness in notification_test on MSVC. > Roll back: Add more debug capacity validation checks on moves. > Add more debug capacity validation checks on moves. > Add macro versions of nullability annotations. > Improve fork-safety by opening files with `O_CLOEXEC`. > Move ABSL_HARDENING_ASSERTs in constexpr methods to their own lines. > Add test cases for absl::Hash: - That hashes are consistent for the same int value across different int types. - That hashes of vectors of strings are unequal even when their concatenations are equal. - That FragmentedCord hashes works as intended for small Cords. > Skip the IterationOrderChangesOnRehash test case in ASan mode because it's flaky. > Add missing includes in absl hash. > Try to use file descriptors in the 2000+ range to avoid mis-behaving client interference. > Add weak implementation of the __lsan_is_turned_off in Leak Checker > Fix a bug where EOF resulted in infinite loop. > static_assert that absl::Time and absl::Duration are trivially destructible. > Move Duration ToInt64<unit> functions to be inline. > string_view: Add defaulted copy constructor and assignment > Use `#ifdef` to avoid errors when `-Wundef` is used. > Strip PAC bits for return addresses in stack traces > PR #1794: Update cpu_detect.cc fix hw crc32 and AES capability check, fix undefined > PR #1790: Respect the allocator's .destroy method in ~InlinedVector > Cast away nullability in the guts of CHECK_EQ (et al) where Clang doesn't see that the nullable string returned by Check_EQImpl is statically nonnull inside the loop. > string_view: Correct string_view(const char*, size_type) docs > Add support for std::string_view in StrCat even when absl::string_view != std::string_view. > Misc. adjustments to unit tests for logging. > Use local_config_cc from rules_cc and make it a dev dependency > Add additional iteration order tests with reservation. Reserved tables have a different way of iteration randomization compared to gradually resized tables (at least for small tables). > Use all the bits (`popcount`) in `FindFirstNonFullAfterResize` and `PrepareInsertAfterSoo`. > Mark ConsumePrefix, ConsumeSuffix, StripPrefix, and StripSuffix as constexpr since they are all pure functions. > PR #1789: Add missing #ifdef pp directive to the TypeName() function in the layout.h > PR #1788: Fix warning for sign-conversion on riscv > Make StartsWith and EndsWith constexpr. > Simplify logic for growing single group table. > Document that absl::Time and absl::Duration are trivially destructible. > Change some C-arrays to std::array as this enables bounds checking in some hardened standard library builds > Replace outdated select() on --cpu with platform API equivalent. > Take failure_message as const char* instead of string_view in LogMessageFatal and friends. > Mention `c_any_of` in the function comment of `absl::c_linear_search`. > Import of CCTZ from GitHub. > Rewrite some string_view methods to avoid a -Wunreachable-code warning > IWYU: Update includes and fix minor spelling mistakes. > Add comment on how to get next element after using erase. > Add ABSL_ATTRIBUTE_LIFETIME_BOUND and a doc note about absl::LogAsLiteral to clarify its intended use. > Import of CCTZ from GitHub. > Reduce memory consumption of structured logging proto encoding by passing tag value > Remove usage of _LIBCPP_HAS_NO_FILESYSTEM_LIBRARY. > Make Span's relational operators constexpr since C++20. > distributions: support a zero max value in Zipf. > PR #1786: Fix typo in test case. > absl/random: run clang-format. > Add some nullability annotations in logging and tidy up some NOLINTs and comments. > CMake: Change the default for ABSL_PROPAGATE_CXX_STD to ON > Delete UnvalidatedMockingBitGen > PR #1783: [riscv][debugging] Fix a few warnings in RISC-V inlines > Add conversion operator to std::array for StrSplit. > Add a comment explaining the extra comparison in raw_hash_set::operator==. Also add a small optimization to avoid the extra comparison in sets that use hash_default_eq as the key_equal functor. > Add benchmark for absl::HexStringToBytes > Avoid installing options.h with the other headers > Add ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::Span constructors. > Annotate absl::InlinedVector to warn when unused. > Make `c_find_first_of`'s `options` parameter a const reference to allow temporaries. > Disable Elf symbols for Xtensa > PR #1775: Support symbolize only on WINAPI_PARTITION_DESKTOP > Require through an internal presubmit that .h|.cc|.inc files contain either the string ABSL_NAMESPACE_BEGIN or SKIP_ABSL_INLINE_NAMESPACE_CHECK > Xtensa supports mmap, enable it in absl/base/config.h > PR #1777: Avoid std::ldexp in `operator double(int128)`. > Marks absl::Span as view and borrowed_range, like std::span. > Mark inline functions with only a simple comparison in strings/ascii.h as constexpr. > Add missing Abseil inline namespace and fix includes > Fix bug where the high bits of `__int128_t`/`__uint128_t` might go unused in the hash function. This fix increases the hash quality of these types. > Add a test to verify bit casting between signed and unsigned int128 works as expected > Add suggestions to enable sanitizers for asserts when doing so may be helpful. > Add nullability attributes to nullability type aliases. > Refactor swisstable moves. > Improve ABSL_ASSERT performance by guaranteeing it is optimized away under NDEBUG in C++20 > Mark Abseil hardening assert in AssertSameContainer as slow. > Add workaround for q++ 8.3.0 (QNX 7.1) compiler by making sure MaskedPointer is trivially copyable and copy constructible. > Small Mutex::Unlock optimization > Optimize `CEscape` and `CEscapeAndAppend` by up to 40%. > Fix the conditional compilation of non_temporal_store_memcpy_avx to verify that AVX can be forced via `gnu::target`. > Delete TODOs to move functors when moving hashtables and add a test that fails when we do so. > Fix benchmarks in `escaping_benchmark.cc` by properly calling `benchmark::DoNotOptimize` on both inputs and outputs and by removing the unnecessary and wrong `ABSL_RAW_CHECK` condition (`check != 0`) of `BM_ByteStringFromAscii_Fail` benchmark. > It seems like commit abc9b916a94ebbf251f0934048295a07ecdbf32a did not work as intended. > Fix a bug in `absl::SetVLogLevel` where a less generic pattern incorrectly removed a more generic one. > Remove the side effects between tests in vlog_is_on_test.cc > Attempt to fix flaky Abseil waiter/sleep tests > Add an explicit tag for non-SOO CommonFields (removing default ctor) and add a small optimization for early return in AssertNotDebugCapacity. > Make moved-from swisstables behave the same as empty tables. Note that we may change this in the future. > Tag tests that currently fail on darwin_arm64 with "no_test_darwin_arm64" > add gmock to cmake defs for no_destructor_test > Optimize raw_hash_set moves by allowing some members of CommonFields to be uninitialized when moved-from. > Add more debug capacity validation checks on iteration/size. > Add more debug capacity validation checks on copies. > constinit -> constexpr for DisplayUnits > LSC: Fix null safety issues diagnosed by Clang’s `-Wnonnull` and `-Wnullability`. > Remove the extraneous variable creation in Match(). > Import of CCTZ from GitHub. > Add more debug capacity validation checks on merge/swap. > Add `absl::` namespace to c_linear_search implementation in order to avoid ADL > Distinguish the debug message for the case of self-move-assigned swiss tables. > Update LowLevelHash comment regarding number of hash state variables. > Add an example for the `--vmodule` flag. > Remove first prefetch. > Add moved-from validation for the case of self-move-assignment. > Allow slow and fast abseil hardening checks to be enabled independently. > Update `ABSL_RETIRED_FLAG` comment to reflect `default_value` is no longer used. > Add validation against use of moved-from hash tables. > Provide file-scoped pragma behind macro ABSL_POINTERS_DEFAULT_NONNULL to indicate the default nullability. This is a no-op for now (not understood by checkers), but does communicate intention to human readers. > Add stacktrace config for android using the generic implementation > Fix nullability annotations in ABSL code. > Replace CHECKs with ASSERTs and EXPECTs -- no reason to crash on failure. > Remove ABSL_INTERNAL_ATTRIBUTE_OWNER and ABSL_INTERNAL_ATTRIBUTE_VIEW > Migrate ABSL_INTERNAL_ATTRIBUTE_OWNER and ABSL_INTERNAL_ATTRIBUTE_VIEW to ABSL_ATTRIBUTE_OWNER and ABSL_ATTRIBUTE_VIEW > Disable ABSL_ATTRIBUTE_OWNER and ABSL_ATTRIBUTE_VIEW prior to Clang-13 due to false positives. > Make ABSL_ATTRIBUTE_VIEW and ABSL_ATTRIBUTE_OWNER public > Optimize raw_hash_set::AssertHashEqConsistent a bit to avoid having as much runtime overhead. > PR #1728: Workaround broken compilation against NDK r25 > Add validation against use of destroyed hash tables. > Do not truncate `ABSL_RAW_LOG` output at null bytes > Use several unused cord instances in tests and benchmarks. > Add comments about ThreadIdentity struct allocation behavior. > Refactoring followup for reentrancy validation in swisstable. > Add debug mode checks that element constructors/destructors don't make reentrant calls to raw_hash_set member functions. > Add tagging for cc_tests that are incompatible with Fuchsia > Add GetTID() implementation for Fuchsia > PR #1738: Fix shell option group handling in pkgconfig files > Disable weak attribute when absl compiled as windows DLL > Remove `CharIterator::operator->`. > Mark non-modifying container algorithms as constexpr for C++20. > PR #1739: container/internal: Explicitly include <cstdint> > Don't match -Wnon-virtual-dtor in the "flags are needed to suppress warnings in headers". It should fall through to the "don't impose our warnings on others" case. Do this by matching on "-Wno-*" instead of "-Wno*". > PR #1732: Fix build on NVIDIA Jetson board. Fix #1665 > Update GoogleTest dependency to 1.15.2 > Enable AsciiStrToLower and AsciiStrToUpper overloads for rvalue references. > PR #1735: Avoid `int` to `bool` conversion warning > Add `absl::swap` functions for `*_hash_*` to avoid calling `std::swap` > Change internal visibility > Remove resolved issue. > Increase test timeouts to support running on Fuchsia emulators > Add tracing annotations to absl::Notification > Suppress compiler optimizations which may break container poisoning. > Disable ABSL_INTERNAL_HAVE_DEBUGGING_STACK_CONSUMPTION for Fuchsia > Add tracing annotations to absl::BlockingCounter > Add absl_vlog_is_on and vlog_is_on to ABSL_INTERNAL_DLL_TARGETS > Update swisstable swap API comments to no longer guarantee that we don't move/swap individual elements. > PR #1726: cmake: Fix RUNPATH when using BUILD_WITH_INSTALL_RPATH=True > Avoid unnecessary copying when upper-casing or lower-casing ASCII string_view > Add weak internal tracing API > Fix LINT.IfChange syntax > PR #1720: Fix spelling mistake: occurrance -> occurrence > Add missing include for Windows ASAN configuration in poison.cc > Delete absl/strings/internal/has_absl_stringify.h now that the GoogleTest version we depend on uses the public file > Update versions of dependencies in preparation for release > PR #1699: Add option to build with MSVC static runtime > Remove unneeded 'be' from comment. > PR #1715: Generate options.h using CMake only once > Small type fix in absl/log/internal/log_impl.h > PR #1709: Handle RPATH CMake configuration > PR #1710: fixup! PR #1707: Fixup absl_random compile breakage in Apple ARM64 targets > PR #1695: Fix time library build for Apple platforms > Remove cyclic cmake dependency that breaks in cmake 3.30.0 > Roll forward poisoned pointer API and fix portability issues. > Use GetStatus in IsOkAndHoldsMatcher > PR #1707: Fixup absl_random compile breakage in Apple ARM64 targets > PR #1706: Require CMake version 3.16 > Add an MSVC implementation of ABSL_ATTRIBUTE_LIFETIME_BOUND > Mark c_min_element, c_max_element, and c_minmax_element as constexpr in C++17. > Optimize the absl::GetFlag cost for most non built-in flag types (including string). > Encode some additional metadata when writing protobuf-encoded logs. > Replace signed integer overflow, since that's undefined behavior, with unsigned integer overflow. > Make mutable CompressedTuple::get() constexpr. > vdso_support: support DT_GNU_HASH > Make c_begin, c_end, and c_distance conditionally constexpr. > Add operator<=> comparison to absl::Time and absl::Duration. > Deprecate `ABSL_ATTRIBUTE_NORETURN` in favor of the `[[noreturn]]` standardized in C++11 > Rollback new poisoned pointer API > Static cast instead of reinterpret cast raw hash set slots as casting from void* to T* is well defined > Fix absl::NoDestructor documentation about its use as a global > Declare Rust demangling feature-complete. > Split demangle_internal into a tree of smaller libraries. > Decode Rust Punycode when it's not too long. > Add assertions to detect reentrance in `IterateOverFullSlots` and `absl::erase_if`. > Decoder for Rust-style Punycode encodings of bounded length. > Add `c_contains()` and `c_contains_subrange()` to `absl/algorithm/container.h`. > Three-way comparison spaceship <=> operators for Cord. > internal-only change > Remove erroneous preprocessor branch on SGX_SIM. > Add an internal API to get a poisoned pointer. > optimization.h: Add missing <utility> header for C++ > Add a compile test for headers that require C compatibility > Fix comment typo > Expand documentation for SetGlobalVLogLevel and SetVLogLevel. > Roll back 6f972e239f668fa29cab43d7968692cd285997a9 > PR #1692: Add missing `<utility>` include > Remove NOLINT for `#include <new>` for __cpp_lib_launder > Remove not used after all kAllowRemoveReentrance parameter from IterateOverFullSlots. > Create `absl::container_internal::c_for_each_fast` for SwissTable. > Disable flaky test cases in kernel_timeout_internal_test. > Document that swisstable and b-tree containers are not exception-safe. > Add `ABSL_NULLABILITY_COMPATIBLE` attribute. > LSC: Move expensive variables on their last use to avoid copies. > Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to more types in Abseil > Drop std:: qualification from integer types like uint64_t. > Increase slop time on MSVC in PerThreadSemTest.Timeouts again due to continued flakiness. > Turn on validation for out of bounds MockUniform in MockingBitGen > Use ABSL_UNREACHABLE() instead of equivalent > If so configured, report which part of a C++ mangled name didn't parse. > Sequence of 1-to-4 values with prefix sum to support Punycode decoding. > Add the missing inline namespace to the nullability files > Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to types in Abseil > Disallow reentrance removal in `absl::erase_if`. > Fix implicit conversion of temporary bitgen to BitGenRef > Use `IterateOverFullSlots` in `absl::erase_if` for hash table. > UTF-8 encoding library to support Rust Punycode decoding. > Disable negative NaN float ostream format checking on RISC-V > PR #1689: Minor: Add missing quotes in CMake string view library definition > Demangle template parameter object names, TA <template-arg>. > Demangle sr St <simple-id> <simple-id>, a dubious encoding found in the wild. > Try not to lose easy type combinators in S::operator const int*() and the like. > Demangle fixed-width floating-point types, DF.... > Demangle _BitInt types DB..., DU.... > Demangle complex floating-point literals. > Demangle <extended-qualifier> in types, e.g., U5AS128 for address_space(128). > Demangle operator co_await (aw). > Demangle fully general vendor extended types (any <template-args>). > Demangle transaction-safety notations GTt and Dx. > Demangle C++11 user-defined literal operator functions. > Demangle C++20 constrained friend names, F (<source-name> | <operator-name>). > Demangle dependent GNU vector extension types, Dv <expression> _ <type>. > Demangle elaborated type names, (Ts | Tu | Te) <name>. > Add validation that hash/eq functors are consistent, meaning that `eq(k1, k2) -> hash(k1) == hash(k2)`. > Demangle delete-expressions with the global-scope operator, gs (dl | da) .... > Demangle new-expressions with braced-init-lists. > Demangle array new-expressions, [gs] na .... > Demangle object new-expressions, [gs] nw .... > Demangle preincrement and predecrement, pp_... and mm_.... > Demangle throw and rethrow (tw... and tr). > Remove redundant check of is_soo() while prefetching heap blocks. > Demangle ti... and te... expressions (typeid). > Demangle nx... syntax for noexcept(e) as an expression in a dependent signature. > Demangle alignof expressions, at... and az.... > Demangle C++17 structured bindings, DC...E. > Demangle modern _ZGR..._ symbols. > Remove redundant check of is_soo() while prefetching heap blocks. > Demangle sizeof...(pack captured from an alias template), sP ... E. > Demangle types nested under vendor extended types. > Demangle il ... E syntax (braced list other than direct-list-initialization). > Avoid signed overflow for Ed <number> _ manglings with large <number>s. > Remove redundant check of is_soo() while prefetching heap blocks. > Remove obsolete TODO > Clarify function comment for `erase` by stating that this idiom only works for "some" standard containers. > Move SOVERSION to global CMakeLists, apply SOVERSION to DLL > Set ABSL_HAVE_THREAD_LOCAL to 1 on all platforms > Demangle constrained auto types (Dk <type-constraint>). > Parse <discriminator> more accurately. > Demangle lambdas in class member functions' default arguments. > Demangle unofficial <unresolved-qualifier-level> encodings like S0_IT_E. > Do not make std::filesystem::path hash available for macOS <10.15 > Include flags in DLL build (non-Windows only) > Enable building monolithic shared library on macOS and Linux. > Demangle Clang's last-resort notation _SUBSTPACK_. > Demangle C++ requires-expressions with parameters (rQ ... E). > Demangle Clang's encoding of __attribute__((enable_if(condition, "message"))). > Demangle static_cast and friends. > Demangle decltype(expr)::nested_type (NDT...E). > Optimize GrowIntoSingleGroupShuffleControlBytes. > Demangle C++17 fold-expressions. > Demangle thread_local helper functions. > Demangle lambdas with explicit template arguments (UlTy and similar forms). > Demangle &-qualified function types. > Demangle valueless literals LDnE (nullptr) and LA<number>_<type>E ("foo"). > Correctly demangle the <unresolved-name> at the end of dt and pt (x.y, x->y). > Add missing targets to ABSL_INTERNAL_DLL_TARGETS > Build abseil_test_dll with ABSL_BUILD_TESTING > Demangle C++ requires-expressions without parameters (rq ... E). > overload: make the constructor constexpr > Update Abseil CI Docker image to use Clang 19, GCC 14, and CMake 3.29.3 > Workaround symbol resolution bug in Clang 19 > Workaround bogus GCC14 -Wmaybe-uninitialized warning > Silence a bogus GCC14 -Warray-bounds warning > Forbid absl::Uniform<absl::int128>(gen) > Use IN_LIST to replace list(FIND) + > -1 > Recognize C++ vendor extended expressions (e.g., u9__is_same...E). > `overload_test`: Remove a few unnecessary trailing return types > Demangle the C++ this pointer (fpT). > Stop eating an extra E in ParseTemplateArg for some L<type><value>E literals. > Add ABSL_INTERNAL_ATTRIBUTE_VIEW and ABSL_INTERNAL_ATTRIBUTE_OWNER attributes to Abseil. > Demangle C++ direct-list-initialization (T{1, 2, 3}, tl ... E). > Demangle the C++ spaceship operator (ss, operator<=>). > Demangle C++ sZ encodings (sizeof...(pack)). > Demangle C++ so ... E encodings (typically array-to-pointer decay). > Recognize dyn-trait-type in Rust demangling. > Rework casting in raw_hash_set's IsFull(). > Remove test references to absl::SharedBitGen, which was never part of the open source release. This was only used in tests that never ran as part in the open source release. > Recognize fn-type and lifetimes in Rust demangling. > Support int128/uint128 in validated MockingBitGen > Recognize inherent-impl and trait-impl in Rust demangling. > Recognize const and array-type in Rust mangled names. > Remove Asylo from absl. > Recognize generic arguments containing only types in Rust mangled names. > Fix missing #include <random> for std::uniform_int_distribution > Move `prepare_insert` out of the line as type erased `PrepareInsertNonSoo`. > Revert: Add -Wdead-code-aggressive to ABSL_LLVM_FLAGS > Add (unused) validation to absl::MockingBitGen > Support `AbslStringify` with `DCHECK_EQ`. > PR #1672: Optimize StrJoin with tuple without user defined formatter > Give ReturnAddresses and N<uppercase> namespaces separate stacks for clarity. > Demangle Rust backrefs. > Use Nt for struct and trait names in Rust demangler test inputs. > Allow __cxa_demangle on MIPS > Add a `string_view` overload to `absl::StrJoin` > Demangle Rust's Y<type><path> production for passably simple <type>s. > `convert_test`: Delete obsolete condition around ASSERT_EQ in TestWithMultipleFormatsHelper > `any_invocable`: Clean up #includes > Resynchronize absl/functional/CMakeLists.txt with BUILD.bazel > `any_invocable`: Add public documentation for undefined behavior when invoking an empty AnyInvocable > `any_invocable`: Delete obsolete reference to proposed standard type > PR #1662: Replace shift with addition in crc multiply > Doc fix. > `convert_test`: Extract loop over tested floats from helper function > Recognize some simple Rust mangled names in Demangle. > Use __builtin_ctzg and __builtin_clzg in the implementations of CountTrailingZeroesNonzero16 and CountLeadingZeroes16 when they are available. > Remove the forked absl::Status matchers implementation in statusor_test > Add comment hack to fix copybara reversibility > Add GoogleTest matchers for absl::Status > [random] LogUniform: Document as a discrete distribution > Enable Cord tests with Crc. > Fix order of qualifiers in `absl::AnyInvocable` documentation. > Guard against null pointer dereference in DumpNode. > Apply ABSL_MUST_USE_RESULT to try lock functions. > Add public aliases for default hash/eq types in hash-based containers > Import of CCTZ from GitHub. > Remove the hand-rolled CordLeaker and replace with absl::NoDestructor to test the after-exit behavior > `convert_test`: Delete obsolete `skip_verify` parameter in test helper > overload: allow using the underlying type with CTAD directly. > PR #1653: Remove unnecessary casts when calling CRC32_u64 > PR #1652: Avoid C++23 deprecation warnings from float_denorm_style > Minor cleanup for `absl::Cord` > PR #1651: Implement ABSL_INTERNAL_DISABLE_DEPRECATED_DECLARATION_WARNING for MSVC compiler > Add `operator<=>` support to `absl::int128` and `absl::uint128` > [absl] Re-use the existing `std::type_identity` backfill instead of redefining it again > Add `absl::AppendCordToString` > `str_format/convert_test`: Delete workaround for [glibc bug](https://sourceware.org/bugzilla/show_bug.cgi?id=22142) > `absl/log/internal`: Document conditional ABSL_ATTRIBUTE_UNUSED, add C++17 TODO > `log/internal/check_op`: Add ABSL_ATTRIBUTE_UNUSED to CHECK macros when STRIP_LOG is enabled > log_benchmark: Add VLOG_IS_ON benchmark > Restore string_view detection check > Remove an unnecessary ABSL_ATTRIBUTE_UNUSED from a logging macro < Abseil LTS Branch, Jan 2024, Patch 2 (#1650) > In example code, add missing template parameter. > Optimize crc32 V128_From2x64 on Arm > Annotate that Mutex should warn when unused. > Add ABSL_ATTRIBUTE_LIFETIME_BOUND to Cord::Flatten/TryFlat > Deprecate `absl::exchange`, `absl::forward` and `absl::move`, which were only useful before C++14. > Temporarily revert dangling std::string_view detection until dependent is fixed > Use _decimal_ literals for the CivilDay example. > Fix bug in BM_EraseIf. > Add internal traits to absl::string_view for lifetimebound detection > Add internal traits to absl::StatusOr for lifetimebound detection > Add internal traits to absl::Span for lifetimebound detection > Add missing dependency for log test build target > Add internal traits for lifetimebound detection > Use local decoding buffer in HexStringToBytes > Only check if the frame pointer is inside a signal stack with known bounds > Roll forward: enable small object optimization in swisstable. > Optimize LowLevelHash by breaking dependency between final loads and previous len/ptr updates. > Fix the wrong link. > Optimize InsertMiss for tables without kDeleted slots. > Use GrowthInfo without applying any optimizations based on it. > Disable small object optimization while debugging some failing tests. > Adjust conditonal compilation in non_temporal_memcpy.h > Reformat log/internal/BUILD > Remove deprecated errno constants from the absl::Status mapping > Introduce GrowthInfo with tests, but without usage. > Enable small object optimization in swisstable. > Refactor the GCC unintialized memory warning suppression in raw_hash_set.h. > Respect `NDEBUG_SANITIZER` > Revert integer-to-string conversion optimizations pending more thorough analysis > Fix a bug in `Cord::{Append,Prepend}(CordBuffer)`: call `MaybeRemoveEmptyCrcNode()`. Otherwise appending a `CordBuffer` an empty Cord with a CRC node crashes (`RemoveCrcNode()` which increases the refcount of a nullptr child). > Add `BM_EraseIf` benchmark. > Record sizeof(key_type), sizeof(value_type) in hashtable profiles. > Fix ClangTidy warnings in btree.h. > LSC: Move expensive variables on their last use to avoid copies. > PR #1644: unscaledcycleclock: remove RISC-V support > Reland: Make DLOG(FATAL) not understood as [[noreturn]] > Separate out absl::StatusOr constraints into statusor_internal.h > Use Layout::WithStaticSizes in btree. > `layout`: Delete outdated comments about ElementType alias not being used because of MSVC > Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower() > `layout_benchmark`: Replace leftover comment with intended call to MyAlign > Remove absl::aligned_storage_t > Delete ABSL_ANNOTATE_MEMORY_IS_INITIALIZED under Thread Sanitizer > Remove vestigial variables in the DumpNode() helper in absl::Cord > Do hashtablez sampling on the first insertion into an empty SOO hashtable. > Add explicit #include directives for <tuple>, "absl/base/config.h", and "absl/strings/string_view.h". > Add a note about the cost of `VLOG` in non-debug builds. > Fix flaky test failures on MSVC. > Add template keyword to example comment for Layout::WithStaticSizes. > PR #1643: add xcprivacy to all subspecs > Record sampling stride in cord profiling to facilitate unsampling. > Fix a typo in a comment. > [log] Correct SetVLOGLevel to SetVLogLevel in comments > Add a feature to container_internal::Layout that lets you specify some array sizes at compile-time as template parameters. This can make offset and size calculations faster. > `layout`: Mark parameter of Slices with ABSL_ATTRIBUTE_UNUSED, remove old workaround > `layout`: Use auto return type for functions that explicitly instantiate std::tuple in return statements > Remove redundant semicolons introduced by macros > [log] Make :vlog_is_on/:absl_vlog_is_on public in BUILD.bazel > Add additional checks for size_t overflows > Replace //visibility:private with :__pkg__ for certain targets > PR #1603: Disable -Wnon-virtual-dtor warning for CommandLineFlag implementations > Add several missing includes in crc/internal > Roll back extern template instatiations in swisstable due to binary size increases in shared libraries. > Add nodiscard to SpinLockHolder. > Test that rehash(0) reduces capacity to minimum. > Add extern templates for common swisstable types. > Disable ubsan for benign unaligned access in crc_memcpy > Make swisstable SOO support GDB pretty printing and still compile in OSS. > Fix OSX support with CocoaPods and Xcode 15 > Fix GCC7 C++17 build > Use UnixEpoch and ZeroDuration > Make flaky failures much less likely in BasicMocking.MocksNotTriggeredForIncorrectTypes test. > Delete a stray comment > Move GCC uninitialized memory warning suppression into MaybeInitializedPtr. > Replace usages of absl::move, absl::forward, and absl::exchange with their std:: equivalents > Fix the move to itself > Work around an implicit conversion signedness compiler warning > Avoid MSan: use-of-uninitialized-value error in find_non_soo. > Fix flaky MSVC test failures by using longer slop time. > Add ABSL_ATTRIBUTE_UNUSED to variables used in an ABSL_ASSUME. > Implement small object optimization in swisstable - disabled for now. > Document and test ability to use absl::Overload with generic lambdas. > Extract `InsertPosition` function to be able to reuse it. > Increase GraphCycles::PointerMap size > PR #1632: inlined_vector: Use trivial relocation for `erase` > Create `BM_GroupPortable_Match`. > [absl] Mark `absl::NoDestructor` methods with `absl::Nonnull` as appropriate > Automated Code Change > Rework casting in raw_hash_set's `IsFull()`. > Adds ABSL_ATTRIBUTE_LIFETIME_BOUND to absl::BitGenRef > Workaround for NVIDIA C++ compiler being unable to parse variadic expansions in range of range-based for loop > Rollback: Make DLOG(FATAL) not understood as [[noreturn]] > Make DLOG(FATAL) not understood as [[noreturn]] > Optimize `absl::Duration` division and modulo: Avoid repeated redundant comparisons in `IDivFastPath`. > Optimize `absl::Duration` division and modulo: Allow the compiler to inline `time_internal::IDivDuration`, by splitting the slow path to a separate function. > Fix typo in example code snippet. > Automated Code Change > Add braces for conditional statements in raw_hash_map functions. > Optimize `prepare_insert`, when resize happens. It removes single unnecessary probing before resize that is beneficial for small tables the most. > Add noexcept to move assignment operator and swap function > Import of CCTZ from GitHub. > Minor documentation updates. > Change find_or_prepare_insert to return std::pair<iterator, bool> to match return type of insert. > PR #1618: inlined_vector: Use trivial relocation for `SwapInlinedElements` > Improve raw_hash_set tests. > Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower() > Use const_cast to avoid duplicating the implementation of raw_hash_set::find(key). > Import of CCTZ from GitHub. > Performance improvement for absl::AsciiStrToUpper() and absl::AsciiStrToLower() > Annotate that SpinLock should warn when unused. > PR #1625: absl::is_trivially_relocatable now respects assignment operators > Introduce `Group::MaskNonFull` without usage. > `demangle`: Parse template template and C++20 lambda template param substitutions > PR #1617: fix MSVC 32-bit build with -arch:AVX > Minor documentation fix for `absl::StrSplit()` > Prevent overflow in `absl::CEscape()` > `demangle`: Parse optional single template argument for built-in types > PR #1412: Filter out `-Xarch_` flags from pkg-config files > `demangle`: Add complexity guard to `ParseQRequiresExpr` < Prepare 20240116.1 patch for Apple Privacy Manifest (#1623) > Remove deprecated symbol absl::kuint128max > Add ABSL_ATTRIBUTE_WARN_UNUSED. > `demangle`: Parse `requires` clauses on template params, before function return type > On Apple, implement absl::is_trivially_relocatable with the fallback. > `demangle`: Parse `requires` clauses on functions > Make `begin()` to return `end()` on empty tables. > `demangle`: Parse C++20-compatible template param declarations, except those with `requires` expressions > Add the ABSL_DEPRECATE_AND_INLINE() macro > Span: Fixed comment referencing std::span as_writable_bytes() as as_mutable_bytes(). > Switch rank structs to be consistent with written guidance in go/ranked-overloads > Avoid hash computation and `Group::Match` in small tables copy and use `IterateOverFullSlots` for iterating for all tables. > Optimize `absl::Hash` by making `LowLevelHash` faster. > Add -Wdead-code-aggressive to ABSL_LLVM_FLAGS < Backport Apple Privacy Manifest (#1613) > Stop using `std::basic_string<uint8_t>` which relies on a non-standard generic `char_traits<>` implementation, recently removed from `libc++`. > Add absl_container_hash-based HashEq specialization > `demangle`: Implement parsing for simplest constrained template arguments > Roll forward 9d8588bfc4566531c4053b5001e2952308255f44 (which was rolled back in 146169f9ad357635b9cd988f976b38bcf83476e3) with fix. > Add a version of absl::HexStringToBytes() that returns a bool to validate that the input was actually valid hexadecimal data. > Enable StringLikeTest in hash_function_defaults_test > Fix a typo. > Minor changes to the BUILD file for absl/synchronization > Avoid static initializers in case of ABSL_FLAGS_STRIP_NAMES=1 > Rollback 9d8588bfc4566531c4053b5001e2952308255f44 for breaking the build > No public description > Decrease the precision of absl::Now in x86-64 debug builds > Optimize raw_hash_set destructor. > Add ABSL_ATTRIBUTE_UNINITIALIZED macros for use with clang and GCC's `uninitialized` > Optimize `Cord::Swap()` for missed compiler optimization in clang. > Type erased hash_slot_fn that depends only on key types (and hash function). > Replace `testonly = 1` with `testonly = True` in abseil BUILD files. > Avoid extra `& msbs` on every iteration over the mask for GroupPortableImpl. > Missing parenthesis. > Early return from destroy_slots for trivially destructible types in flat_hash_{*}. > Avoid export of testonly target absl::test_allocator in CMake builds > Use absl::NoDestructor for cordz global queue. > Add empty WORKSPACE.bzlmod > Introduce `RawHashSetLayout` helper class. > Fix a corner case in SpyHashState for exact boundaries. > Add nullability annotations > Use absl::NoDestructor for global HashtablezSampler. > Always check if the new frame pointer is readable. > PR #1604: Add privacy manifest < Disable ABSL_ATTRIBUTE_TRIVIAL_ABI in open-source builds (#1606) > Remove code pieces for no longer supported GCC versions. > Disable ABSL_ATTRIBUTE_TRIVIAL_ABI in open-source builds > Prevent brace initialization of AlphaNum > Remove code pieces for no longer supported MSVC versions. > Added benchmarks for smaller size copy constructors. > Migrate empty CrcCordState to absl::NoDestructor. > Add protected copy ctor+assign to absl::LogSink, and clarify thread-safety requirements to apply to the interface methods. < Apply LTS transformations for 20240116 LTS branch (#1599) Closes scylladb/scylladb#28756 |
||
Karol Nowacki
|
6bc88e817f |
vector_search: fix SELECT on local vector index
Queries against local vector indexes were failing with the error:
"ANN ordering by vector requires the column to be indexed using 'vector_index'"
This was a regression introduced by
|
||
|
Ernest Zaslavsky
|
38088a8a94 | configure.py: add sstable_tablet_streaming to combined_tests | ||
|
Pavel Emelyanov
|
cb329b10bf |
code: Add maintenance/maintenance group
And move some activities from streaming group into it, namely - tablet_allocator background group - sstables_manager-s components reclaimer - tablet storage group manager merge completion fiber - prometheus All other activity that was in streaming group remains there, but can be moved to this group (or to new maintenance subgroup) later. All but prometheus are patched here, prometheus still uses the maintenance_sched_group variable in main.cc, so it transparently moves into new group Signed-off-by: Pavel Emelyanov <xemul@scylladb.com> |
||
|
Pavel Emelyanov
|
de9bfe0f1d |
backup: Add maintenance/backup group
The snapshot_ctl::backup_task_impl runs in configured scheduling group. Now it's streaming one. This patch introduces the maintenance/backup group and re-configures backup task with it. The group gets its --backup_io_throughput_mb_per_sec option that controls bandwidth limit for this sub-group only. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com> |
||
|
Pavel Emelyanov
|
6f43e8562e |
compaction: Add maintenance/maintenance_compaction group
Compaction manager tells compaction_sched_group from maintenance_compaction_sched_group. The latter, however, is set to be "streaming" group. This patch adds real maintenance_compaction group under the maintenance supergroup and makes compaction manager use it. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com> |
||
|
Michael Litvak
|
e7c3942d43 |
logstor: move segments to replica::compaction_group
Add a segment_set member to replica::compaction_group that manages the logstor segments that belong to the compaction group, similarly to how it manages sstables. Add also a separator buffer in each compaction group. When writing a mutation to a compaction group, the mutation is written to the active segment and to the separator buffer of the compaction group, and when the separator buffer is flushed the segment is added to the compaction_group's segment set. |
||
|
Michael Litvak
|
0b1343747f |
logstor: initial commit
initial implementation of the logstor storage engine for key-value tables that supports writes, reads and basic compaction. main components: * logstor: this is the main interface to users that supports writing and reading back mutations, and manages the internal components. * index: the primary index in-memory that maps a key to a location on disk. * write buffer: writes go initially to a write buffer. it accumulates multiple records in a buffer and writes them to the segment manager in 4k sized blocks. * segment manager: manages the storage - files, segments, compaction. it manages file and segment allocation, and writes 4k aligned buffers to the active segment sequentially. it tracks the used space in each segment. the compaction finds segment with low space usage and writes them to new segments, and frees the old segments. |
||
|
Piotr Dulikowski
|
d8b283e1fb |
Merge 'Add CQL forwarding for strongly consistent tables' from Wojciech Mitros
In this series we add support for forwarding strongly consistent CQL requests to suitable replicas, so that clients can issue reads/writes to any node and have the request executed on an appropriate tablet replica (and, for writes, on the Raft leader). We return the same CQL response as what the user would get while sending the request to the correct replica and we perform the same logging/stats updates on the request coordinator as if the coordinator was the appropriate replica. The core mechanism of forwarding a strongly consistent request is sending an RPC containing the user's cql request frame to the appropriate replica and returning back a ready, serialized `cql_transport::response`. We do this in the CQL server - it is most prepared for handling these types and forwarding a request containing a CQL frame allows us to reuse near-top-level methods for CQL request handling in the new RPC handler (such as the general `process`) For sending the RPC, the CQL server needs to obtain the information about who should it forward the request to. This requires knowledge about the tablet raft group members and leader. We obtain this information during the execution of a `cql3/strong_consistency` statement, and we return this information back to the CQL server using the generalized `bounce_to_shard` `response_message`, where we now store the information about either a shard, or a specific replica to which we should forward to. Similarly to `bounce_to_shard`, we need to handle this `result_message` in a loop - a replica may move during statement execution, or the Raft leader can change. We also use it for forwarding strongly consistent writes when we're not a member of the affected tablet raft group - in that case we need to forward the statement twice - once to any replica of the affected tablet, then that replica can find the leader and return this information to the coordinator, which allows the second request to be directed to the leader. This feature also allows passing through exception messages which happened on the target replica while executing the statement. For that, many methods of the `cql_transport::cql_server::connection` for creating error responses needed to be moved to `cql_transport::cql_server`. And for final exception handling on the coordinator, we added additional error info to the RPC response, so that the handling can be performed without having the `result_message::exception` or `exception_ptr` itself. Fixes [SCYLLADB-71](https://scylladb.atlassian.net/browse/SCYLLADB-71) [SCYLLADB-71]: https://scylladb.atlassian.net/browse/SCYLLADB-71?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Closes scylladb/scylladb#27517 * github.com:scylladb/scylladb: test: add tests for CQL forwarding transport: enable CQL forwarding for strong consistency statements transport: add remote statement preparation for CQL forwarding transport: handle redirect responses in CQL forwarding transport: add exception handling for forwarded CQL requests transport: add basic CQL request forwarding idl: add a representation of client_state for forwarding cql_server: handle query, execute, batch in one case transport: inline process_on_shard in cql_server::process transport: extract process() to cql_server transport: add messaging_service to cql_server transport: add response reconstruction helpers for forwarding transport: generalize the bounce result message for bouncing to other nodes strong consistency: redirect requests to live replicas from the same rack transport: pass foreign_ptr into sleep_until_timeout_passes and move it to cql_server transport: extract the error handling from process_request_one transport: move error response helpers from connection to cql_server |
||
|
Wojciech Mitros
|
23bff5dfef |
transport: add basic CQL request forwarding
Add the infrastructure for forwarding CQL requests to other nodes. When a process() call results in a node bounce (as opposed to a shard bounce), the coordinator serializes the request and sends it via the FORWARD_CQL_EXECUTE RPC verb to the target node. In this patch we omit several features that allow handling more scenarios that can happen when trying to forward a CQL request, but the RPC request and response are already prepared for them. They will be handled in the following commits. |
||
|
Wojciech Mitros
|
170b82ddca |
idl: add a representation of client_state for forwarding
In the following patches, when we start allowing to forward CQL requests to other nodes, we'll need to use the same client state for executing the request on the destination node as we had on the source. client_state contains many fields and we need to create a new instance of it when we start handling the forwarded request, so to prepare for the forwarding RPC, we add a serializable format of the client_state as an IDL struct. The new class is missing some fields that are not used while executing requests, and some whose value is determined by the fact that the client state is used for a forwarded request. These include: - driver name, driver version, client options - not used for executing requests. Instead, we use these as data sources for the virtual "clients" system table. - auth_state - must be READY - we reached a bounce message, so we were able to try executing the request locally - _control_connection - used for altering a cql_server::connection, which we don't have on the target node - _default_timeout_config - used when updating service levels, also only per-connection - workload_type - used for deciding whether to allow shedding at the start of processing the request, and for getting per-connection service level params (for an API) |
||
|
Marcin Maliszkiewicz
|
5b2a07b408 |
utils: add rolling max tracker
We will use it later to track parser memory usage via per query samples. Tests runtime in dev: 1.6s |
||
Patryk Jędrzejczak
|
37aeba9c8c |
Merge 'raft: add global read barrier to group0_batch::commit and switch auth and service levels' from Marcin Maliszkiewicz
This series adds a global read barrier to raft_group0_client, ensuring that Raft group0 mutations are applied on all live nodes before returning to the caller. Currently, after a group0_batch::commit, the mutations are only guaranteed to be applied on the leader. Other nodes may still be catching up, leading to stale reads. This patch introduces a broadcast read barrier mechanism. Calling send_group0_read_barrier_to_live_members after committing will cause the coordinator to send a read barrier RPC to all live nodes (discovered via gossiper) and waits for them to complete. This is best effort attempt to get cluster-wide visibility of the committed state before the response is returned to the user. Auth and service levels write paths are switched to use this new mechanism. Fixes https://scylladb.atlassian.net/browse/SCYLLADB-650 Backport: no, new feature Closes scylladb/scylladb#28731 * https://github.com/scylladb/scylladb: test: add tests for global group0_batch barrier feature qos: switch service levels write paths to use global group0_batch barrier auth: switch write paths to use global group0_batch barrier raft: add function to broadcast read barrier request raft: add gossiper dependency to raft_group0_client raft: add read barrier RPC |
||
Gleb Natapov
|
b59b3d4f8a | service level: remove version 1 service level code | ||
|
Gleb Natapov
|
6a7e850161 |
cdc: remove legacy code
The patch removes test/boost/cdc_generation_test.cc since it unit tests cdc::limit_number_of_streams_if_needed function which is remove here. |
||
|
Gleb Natapov
|
1d188f0394 |
auth: remove legacy auth mode and upgrade code
A system needs to be upgraded to use v2 auth before moving to this ScyllaDB version otherwise the boot will fail. |
||
|
Marcin Maliszkiewicz
|
8422fbca9f |
raft: add read barrier RPC
The RPC does read barrier on a destination node. It will be issued in following commits to live nodes to assure that command was applied everywhere. |
||
|
Marcin Maliszkiewicz
|
c7d3f80863 |
Merge 'auth: do not create default 'cassandra:cassandra' superuser' from Dario Mirovic
This patch series removes creation of default 'cassandra:cassandra' superuser on system start. Disable creation of a superuser with default 'cassandra:cassandra' credentials to improve security. The current flow requires clients to create another superuser and then drop the default `cassandra:cassandra' role. For those who do, there is a time window where the default credentials exist. For those who do not, that role stays. We want to improve security by forcing the client to either use config to specify default values for default superuser name and password or use cqlsh over maintenance socket connection to explicitly create/alter a superuser role. The patch series: - Enable role modification over the maintenance socket - Stop using default 'cassandra' value for default superuser, skipping creation instead Design document: https://scylladb.atlassian.net/wiki/spaces/RND/pages/165773327/Drop+default+cassandra+superuser Fixes scylladb/scylla-enterprise#5657 This is an improvement. It does not need a backport. Closes scylladb/scylladb#27215 * github.com:scylladb/scylladb: config: enable maintenance socket in workdir by default docs: auth: do not specify password with -p option docs: update documentation related to default superuser test: maintenance socket role management test: cluster: add logs to test_maintenance_socket.py test: pylib: fix connect_driver handling when adding and starting server auth: do not create default 'cassandra:cassandra' superuser auth: remove redundant DEFAULT_USER_NAME from password authenticator auth: enable role management operations via maintenance socket client_state: add has_superuser method client_state: add _bypass_auth_checks flag auth: let maintenance_socket_role_manager know if node is in maintenance mode auth: remove class registrator usage auth: instantiate auth service with factory functors auth: add service constructor with factory functors auth: add transitional.hh file service: qos: handle special scheduling group case for maintenance socket service: qos: use _auth_integration as condition for using _auth_integration |
||
|
Avi Kivity
|
85bd6d0114 |
Merge 'Add multiple-shard persistent metadata storage for strongly consistent tables' from Wojciech Mitros
In this series we introduce new system tables and use them for storing the raft metadata for strongly consistent tables. In contrast to the previously used raft group0 tables, the new tables can store data on any shard. The tables also allow specifying the shard where each partition should reside, which enables the tablets of strongly consistent tables to have their raft group metadata co-located on the same shard as the tablet replica. The new tables have almost the same schemas as the raft group0 tables. However, they have an additional column in their partition keys. The additional column is the shard that specifies where the data should be located. While a tablet and its corresponding raft group server resides on some shard, it now writes and reads all requests to the metadata tables using its shard in addition to the group_id. The extra partition key column is used by the new partitioner and sharder which allow this special shard routing. The partitioner encodes the shard in the token and the sharder decodes the shard from the token. This approach for routing avoids any additional lookups (for the tablet mapping) during operations on the new tables and it also doesn't require keeping any state. It also doesn't interact negatively with resharding - as long as tablets (and their corresponding raft metadata) occupy some shard, we do not allow starting the node with a shard count lower than the id of this shard. When increasing the shard count, the routing does not change, similarly to how tablet allocation doesn't change. To use the new tables, a new implementation of `raft::persistence` is added. Currently, it's almost an exact copy of the `raft_sys_table_storage` which just uses the new tables, but in the future we can modify it with changes specific to metadata (or mutation) storage for strongly consistent tables. The new storage is used in the `groups_manager`, which combined with the removal of some `this_shard_id() == 0` checks, allows strongly consistent tables to be used on all shards. This approach for making sure that the reads/writes to the new tables end up on the correct shards won in the balance of complexity/usability/performance against a few other approaches we've considered. They include: 1. Making the Raft server read/write directly to the database, skipping the sharder, on its shard, while using the default partitioner/sharder. This approach could let us avoid changing the schema and there should be no problems for reads and writes performed by the Raft server. However, in this approach we would input data in tables conflicting with the placement determined by the sharder. As a result, any read going through the sharder could miss the rows it was supposed to read. Even when reading all shards to find a specific value, there is a risk of polluting the cache - the rows loaded on incorrect shards may persist in the cache for an unknown amount of time. The cache may also mistakenly remember that a row is missing, even though it's actually present, just on an incorrect shard. Some of the issues with this approach could be worked around using another sharder which always returns this_shard_id() when asked about a shard. It's not clear how such a sharder would implement a method like `token_for_next_shard`, and how much simpler it would be compared to the current "identity" sharder. 2. Using a sharder depending on the current allocation of tablets on the node. This approach relies on the knowledge of group_id -> shard mapping at any point in time in the cluster. For this approach we'd also need to either add a custom partitioner which encodes the group_id in the token, or we'd need to track the token(group_id) -> shard mapping. This approach has the benefit over the one used in the series of keeping the partition key as just group_id. However, it requires more logic, and the access to the live state of the node in the sharder, and it's not static - the same token may be sharded differently depending on the state of the node - it shouldn't occur in practice, but if we changed the state of the node before adjusting the table data, we would be unable to access/fix the stale data without artificially also changing the state of the node. 3. Using metadata tables co-located to the strongly consistent tables. This approach could simplify the metadata migrations in the future, however it would require additional schema management of all co-located metadata tables, and it's not even obvious what could be used as the partition key in these tables - some metadata is per-raft-group, so we couldn't reuse the partition key of the strongly consistent table for it. And finding and remembering a partition key that is routed to a specific shard is not a simple task. Finally, splits and merges will most likely need special handling for metadata anyway, so we wouldn't even make use of co-located table's splits and merges. Fixes [SCYLLADB-361](https://scylladb.atlassian.net/browse/SCYLLADB-361) [SCYLLADB-361]: https://scylladb.atlassian.net/browse/SCYLLADB-361?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Closes scylladb/scylladb#28509 * github.com:scylladb/scylladb: docs: add strong consistency doc test/cluster: add tests for strongly-consistent tables' metadata persistence raft: enable multi-shard raft groups for strongly consistent tablets test/raft: add unit tests for raft_groups_storage raft: add raft_groups_storage persistence class db: add system tables for strongly consistent tables' raft groups dht: add fixed_shard_partitioner and fixed_shard_sharder raft: add group_id -> shard mapping to raft_group_registry schema: add with_sharder overload accepting static_sharder reference |
||
Dario Mirovic
|
45628cf041 |
auth: enable role management operations via maintenance socket
Introduce maintenance_socket_authenticator and rework maintenance_socket_role_manager to support role management operations. Maintenance auth service uses allow_all_authenticator. To allow role modification statements over the maintenance socket connections, we need to treat the maintenance socket connections as superusers and give them proper access rights. Possible approaches are: 1. Modify allow_all_authenticator with conditional logic that password_authenticator already does 2. Modify password_authenticator with conditional logic specific for the maintenance socket connections 3. Extend password_authenticator, overriding the methods that differ Option 3 is chosen: maintenance_socket_authenticator extends password_authenticator with authentication disabled. The maintenance_socket_role_manager is reworked to lazily create a standard_role_manager once the node joins the cluster, delegating role operations to it. In maintenance mode role operations remain disabled. Refs SCYLLADB-409 |
||
|
Patryk Jędrzejczak
|
9a9202c909 |
Merge 'Remove gossiper topology code' from Gleb Natapov
The PR removes most of the code that assumes that group0 and raft topology is not enabled. It also makes sure that joining a cluster in no raft mode or upgrading a node in a cluster that not yet uses raft topology to this version will fail. Refs #15422 No backport needed since this removes functionality. Closes scylladb/scylladb#28514 * https://github.com/scylladb/scylladb: group0: fix indentation after previous patch raft_group0: simplify get_group0_upgrade_state function since no upgrade can happen any more raft_group0: move service::group0_upgrade_state to use fmt::formatter instead of iostream raft_group0: remove unused code from raft_group0 node_ops: remove topology over node ops code topology: fix indentation after the previous patch topology: drop topology_change_enabled parameter from raft_group0 code storage_service: remove unused handle_state_* functions gossiper: drop wait_for_gossip_to_settle and deprecate correspondent option storage_service: fix indentation after the last patch storage_service: remove gossiper bootstrapping code storage_service: drop get_group_server_if_raft_topolgy_enabled storage_service: drop is_topology_coordinator_enabled and its uses storage_service: drop run_with_api_lock_in_gossiper_mode_only topology: remove code that assumes raft_topology_change_enabled() may return false test: schema_change_test: make test_schema_digest_does_not_change_with_disabled_features tests run in raft mode test: schema_change_test: drop schema tests relevant for no raft mode only topology: remove upgrade to raft topology code group0: remove upgrade to group0 code group0: refuse to boot if a cluster is still is not in a raft topology mode storage_service: refuse to join a cluster in legacy mode |
||
|
Wojciech Mitros
|
16977d7aa0 |
raft: add raft_groups_storage persistence class
Add raft_groups_storage, a raft::persistence implementation for strongly consistent tablet groups. Currently, it's almost an exact copy of the raft_sys_table_storage that uses the new raft tables for strongly consistent tables (raft_groups, raft_groups_snapshots, raft_groups_snapshot_config) which have a (shard, group_id) partition key. In the future, the mutation, term and commit_idx data will be stored differently for for strongly consistent tables than for group0, which will differentiate this class from the original raft_sys_table_storage. The storage is created for each raft group server and it takes a shard parameter at construction time to ensure all queries target the correct partition (and thus shard). |
||
|
Wojciech Mitros
|
cb0caea8bf |
dht: add fixed_shard_partitioner and fixed_shard_sharder
Add a custom partitioner and sharder that will be used for Raft tables for strongly consistent tables. These tables will have partition keys of the form (shard, group_id) and the partitioner creates tokens that encode the target shard in the high 16 bits. Token layout: [shard: 16 bits][partition key hash: 48 bits] This encoding guarantees that raft group data will be located on the same shard as the tablet replica corresponding to that raft group as long we use the tablet replica's shard as the value in the partition key. Storing the shard directly in the partition key avoids additional lookups for request routing to the incoming new raft tables. For even more simplicity, we avoid biasing between uint64_t and int64_t by limiting the acceptable shard ids up to 32767 (leaving the top bit 0), which results in the same value of the token when interpreting either as uint64_t or int64_t. The sharder decodes the shard by extracting the high bits, which is shard-count independent. This allows the partition key:shard mapping to remain the same even during smp changes (only increases are allowed, the same limitation as for tablets). |
||
|
Gleb Natapov
|
6173ea476b |
node_ops: remove topology over node ops code
The code is no longer called. |
||
|
Ernest Zaslavsky
|
321d4caf0c |
object_storage: add retryable machinery to object storage
remove hand rolled error handling from object storage client and replace with common machinery that supports exception handling and retrying when appropriate |
||
|
Piotr Dulikowski
|
b9db3c9c75 |
Merge 'Add consistent permissions cache' from Marcin Maliszkiewicz
This patchset replaces permissions cache based on loading_cache with a new unified (permissions and roles), full, coherent auth cache. Reason for the change is that we want to improve scenarios under stress and simplify operation manuals. New cache doesn't require any tweaking. And it behaves particularly better in scenarios with lots of schema entities (e.g. tables) combined with unprepared queries. Old cache can generate few thousands of extra internal tps due to cache refresh. Benchmark of unprepared statements (just to populate the cache) with 1000 tables shows 3k tps of internal reads reduction and 9.1% reduction of median instructions per op. So many tables were used to show resource impact, cache could be filled with other resource types to show the same improvement. Backport: no, it's a new feature. Fixes https://github.com/scylladb/scylladb/issues/7397 Fixes https://github.com/scylladb/scylladb/issues/3693 Fixes https://github.com/scylladb/scylladb/issues/2589 Fixes https://scylladb.atlassian.net/browse/SCYLLADB-147 Closes scylladb/scylladb#28078 * github.com:scylladb/scylladb: test: boost: add auth cache tests auth: add cache size metrics docs: conf: update permissions cache documentation auth: remove old permissions cache auth: use unified cache for permissions auth: ldap: add permissions reload to unified cache auth: add permissions cache to auth/cache auth: add service::revoke_all as main entry point auth: explicitly life-extend resource in auth_migration_listener |
||
|
Marcin Maliszkiewicz
|
741969cf4c |
test: boost: add auth cache tests
The cache is covered already with general auth dtests but some cases are more tricky and easier to express directly as calls to cache class. For such tests boost test file was added. |
||
|
Marcin Maliszkiewicz
|
a23e503e7b | auth: remove old permissions cache |