Compare commits

..

13 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
91dd15d3bf Fix batch size check to only apply to atomic batches with tests
The verify_batch_size() check was applied to all batches before determining mutate_atomic, causing unnecessary failures for batches that never touch system.batchlog. Only logged multi-partition batches (mutate_atomic=true) are stored as blobs in system.batchlog and require size validation.

Changed:

* Moved verify_batch_size() call in execute_without_conditions() directly
  into the logged multi-partition branch for clearer intent
* The check is now placed exactly where mutate_atomic=true, making the
  code more intuitive

Result:
Batch Type  Single-Partition  Size Check
LOGGED      false             YES
LOGGED      true              NO
UNLOGGED    N/A               NO

Tests:

* test_unlogged_batch_size_not_checked: Unlogged batch >1024KB succeeds
* test_logged_multi_partition_batch_size_checked: Logged multi-partition batch >1024KB fails
* test_logged_single_partition_batch_size_not_checked: Logged single-partition batch >1024KB succeeds

Fixes #27605

Co-authored-by: mykaul <yaniv.kaul@scylladb.com>
2025-12-15 10:45:29 +02:00
Nadav Har'El
c06e63daed Merge 'auth: start using SHA 512 hashing originated from musl with added yielding' from Andrzej Jackowski
This patch series contains the following changes:
 - Incorporation of `crypt_sha512.c` from musl to out codebase
 - Conversion of `crypt_sha512.c` to C++ and coroutinization
 - Coroutinization of `auth::passwords::check`
 - Enabling use of `__crypt_sha512` orignated from `crypt_sha512.c` for
   computing SHA 512 passwords of length <=255
 - Addition of yielding in the aforementioned hashing implementation.

The alien thread was a solution for reactor stalls caused by indivisible
password‑hashing tasks (https://github.com/scylladb/scylladb/issues/24524).
However, because there is only one alien thread, overall hashing throughput was reduced
(see, e.g., https://github.com/scylladb/scylla-enterprise/issues/5711). To address this,
the alien‑thread solution is reverted, and a hashing implementation
with yielding is introduced in this patch series.

Before this patch series, ScyllaDB used SHA-512 hashing provided
by the `crypt_r` function, which in our case meant using the implementation
from the `libxcrypt` library. Adding yielding to this `libxcrypt`
implementation is problematic, both due to licensing (LGPL) and because the
implementation is split into many functions across multiple files. In
contrast, the SHA-512 implementation from `musl libc` has a more
permissive license and is concise, which makes it easier to incorporate
into the ScyllaDB codebase.

The performance of this solution was compared with the previous
implementation that used one alien thread and the implementation
after the alien thread was reverted. The results (median) of
`perf-cql-raw` with `--connection-per-request 1 --smp 10` parameters
are as follows:
 - Alien thread: 41.5 new connections/s per shard
 - Reverted alien thread: 244.1 new connections/s per shard
 - This commit (yielding in hashing): 198.4 new connections/s per shard

The roughly 20% performance deterioration compared to
the old implementation without the alien thread comes from the fact
that the new hashing algorithm implemented in `utils/crypt_sha512.cc`
performs an expensive self-verification and stack cleanup.

On the other hand, with smp=10 the current implementation achieves
roughly 5x higher throughput than the alien thread. In addition,
due to yielding added in this commit, the algorithm is expected
to provide similar protection from stalls as the alien thread did.
In a test that in parallel started a cassandra-stress workload and
created thousands of new connections using python-driver, the values of
`scylla_reactor_stalls_count` metric were as follows:
 - Alien thread: 109 stalls/shard total
 - Reverted alien thread: 13186 stalls/shard total
 - This commit (yielding in hashing): 149 stalls/shard total

Similarly, the `scylla_scheduler_time_spent_on_task_quota_violations_ms`
values were:
 - Alien thread: 1087 ms/shard total
 - Reverted alien thread: 72839 ms/shard total
 - This commit (yielding in hashing): 1623 ms/shard total

To summarize, yielding during hashing computations achieves similar
throughput to the old solution without the alien thread but also
prevents stalls similarly to the alien thread.

Fixes: scylladb/scylladb#26859
Refs: scylladb/scylla-enterprise#5711

No automatic backport. After this PR is completed, the alien thread should be rather reverted from older branches (2025.2-2025.4 because on 2025.1 it's already removed). Backporting of the other commits needs further discussion.

Closes scylladb/scylladb#26860

* github.com:scylladb/scylladb:
  test/boost: add too_long_password to auth_passwords_test
  test/boost: add same_hashes_as_crypt_r to auth_passwords_test
  auth: utils: add yielding to crypt_sha512
  auth: change return type of passwords::check to future
  auth: remove code duplication in verify_scheme
  test/boost: coroutinize auth_passwords_test
  utils: coroutinize crypt_sha512
  utils: make crypt_sha512.cc to compile
  utils: license: import crypt_sha512.c from musl to the project
  Revert "auth: move passwords::check call to alien thread"
2025-12-14 14:01:01 +02:00
David Garcia
c1c3b2c5bb docs: fix local build
prevents early exits in metrics docs generation to break the local build.

Fixes #27497

Closes scylladb/scylladb#27615
2025-12-14 11:48:48 +02:00
Andrzej Jackowski
11ad32c85e test/boost: add too_long_password to auth_passwords_test
The test documents the current behavior of hashing algorithms that
fail if the passphrase has 512 bytes or more.

Moreover, it documents the behavior of the current bcrypt
implementation that compares only the first 72 bytes of the password.
Although we don't typically use bcrypt for password hashing, it is
possible to insert such a hash using
`CREATE ROLE ... WITH HASHED PASSWORD ...`.

Refs: scylladb/scylladb#26842
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
4c8c9cd548 test/boost: add same_hashes_as_crypt_r to auth_passwords_test
The test verifies that the old and new implementation of SHA-512
hashing returns exactly the same values.

Refs: scylladb/scylladb#26859
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
98f431dd81 auth: utils: add yielding to crypt_sha512
This change allows yielding during hashing computations to prevent
stalls.

The performance of this solution was compared with the previous
implementation that used one alien thread and the implementation
after the alien thread was reverted. The results (median) of
`perf-cql-raw` with `--connection-per-request 1 --smp 10` parameters
are as follows:
 - Alien thread: 41.5 new connections/s per shard
 - Reverted alien thread: 244.1 new connections/s per shard
 - This commit (yielding in hashing): 198.4 new connections/s per shard

The alien thread is limited by a single-core hashing throughput,
which is roughly 400-500 hashes/s in the test environment. Therefore,
with smp=10, the throughput is below 50 hashes/s, and the difference
between the alien thread and other solutions further increases with
higer smp.

The roughly 20% performance deterioration compared to
the old implementation without the alien thread comes from the fact
that the new hashing algorithm implemented in `utils/crypt_sha512.cc`
performs an expensive self-verification and stack cleanup.

On the other hand, with smp=10 the current implementation achieves
roughly 5x higher throughput than the alien thread. In addition,
due to yielding added in this commit, the algorithm is expected
to provide similar protection from stalls as the alien thread did.
In a test that in parallel started a cassandra-stress workload and
created thousands of new connections using python-driver, the values of
`scylla_reactor_stalls_count` metric were as follows:
 - Alien thread: 109 stalls/shard total
 - Reverted alien thread: 13186 stalls/shard total
 - This commit (yielding in hashing): 149 stalls/shard total

Similarly, the `scylla_scheduler_time_spent_on_task_quota_violations_ms`
values were:
 - Alien thread: 1087 ms/shard total
 - Reverted alien thread: 72839 ms/shard total
 - This commit (yielding in hashing): 1623 ms/shard total

To summarize, yielding during hashing computations achieves similar
throughput to the old solution without the alien thread but also
prevents stalls similarly to the alien thread.

Fixes: scylladb/scylladb#26859
Refs: scylladb/scylla-enterprise#5711
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
4ffdb0721f auth: change return type of passwords::check to future
Introduce a new `passwords::hash_with_salt_async` and change the return
type of `passwords::check` to `future<bool>`. This enables yielding
during password computations later in this patch series.

The old method, `hash_with_salt`, is marked as deprecated because
new code should use the new `hash_with_salt_async` function.
We are not removing `hash_with_salt` now to reduce the regression risk
of changing the hashing implementation—at least the methods that change
persistent hashes (CREATE, ALTER) will continue to use the old hashing
method. However, in the future, `hash_with_salt` should be entirely
removed.

Refs: scylladb/scylladb#26859
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
775906d749 auth: remove code duplication in verify_scheme
Refactoring: create a new function `verify_hashing_output` to reuse
code in `hash_with_salt` and `verify_scheme`. The change is introduced
to facilitate verification of hashing output when the implementation
is extended later in this patch series.

Refs: scylladb/scylladb#26859
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
11eca621b0 test/boost: coroutinize auth_passwords_test
This commit prepares `auth_passwords_test` for using coroutines,
because later in this patch series `auth::passwords::check` and other
similar functions will return Seastar futures.

Refs: scylladb/scylladb#26859
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
d7818b56df utils: coroutinize crypt_sha512
Change `sha512crypt` and `__crypt_sha512` to coroutines to allow
yielding during hash computations later in this patch series.

Refs: scylladb/scylladb#26859
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
033fed5734 utils: make crypt_sha512.cc to compile
The purpose of this change is to allow the usage of Seastar futures in
crypt_sha512 later in this patch series.

Refs: scylladb/scylladb#26859
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
c6c30b7d0a utils: license: import crypt_sha512.c from musl to the project
This patch imports the `crypt_sha512.c` file from the musl library.
We need it to incorporate yielding in the `crypt_r` function to avoid
reactor stalls during long hashing computations.

Before this patch series, ScyllaDB used SHA-512 hashing provided
by the `crypt_r` function, which in our case meant using the implementation
from the `libxcrypt` library. Adding yielding to this `libxcrypt`
implementation is problematic, both due to licensing (LGPL) and because the
implementation is split into many functions across multiple files. In
contrast, the SHA-512 implementation from `musl libc` has a more
permissive license and is concise, which makes it easier to incorporate
into the ScyllaDB codebase.

Both `crypt_sha512.c` and musl license are obtained from
git.musl-libc.org:
 - https://git.musl-libc.org/cgit/musl/tree/src/crypt/crypt_sha512.c
 - https://git.musl-libc.org/cgit/musl/tree/COPYRIGHT

Import commit:
  commit 1b76ff0767d01df72f692806ee5adee13c67ef88
  Author: Alex Rønne Petersen <alex@alexrp.com>
  Date:   Sun Oct 12 05:35:19 2025 +0200

  s390x: shuffle register usage in __tls_get_offset to avoid r0 as address

Refs: scylladb/scylladb#26859
2025-12-10 15:36:18 +01:00
Andrzej Jackowski
5afcec4a3d Revert "auth: move passwords::check call to alien thread"
The alien thread was a solution for reactor stalls caused by indivisible
password‑hashing tasks (scylladb/scylladb#24524). However, because
there is only one alien thread, overall hashing throughput was reduced
(see, e.g., scylladb/scylla-enterprise#5711). To address this,
the alien‑thread solution is reverted, and a hashing implementation
with yielding will be introduced later in this patch series.

This reverts commit 9574513ec1.
2025-12-10 15:36:09 +01:00
30 changed files with 840 additions and 88 deletions

View File

@@ -7,7 +7,7 @@ on:
- enterprise
paths:
- '**/*.cc'
- 'scripts/metrics-config.yml'
- 'scripts/metrics-config.yml'
- 'scripts/get_description.py'
- 'docs/_ext/scylladb_metrics.py'
@@ -15,20 +15,20 @@ jobs:
validate-metrics:
runs-on: ubuntu-latest
name: Check metrics documentation coverage
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: true
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.10'
- name: Install dependencies
run: pip install PyYAML
- name: Validate metrics
run: python3 scripts/get_description.py --validate -c scripts/metrics-config.yml

View File

@@ -9,7 +9,6 @@
#include "auth/allow_all_authenticator.hh"
#include "service/migration_manager.hh"
#include "utils/alien_worker.hh"
#include "utils/class_registrator.hh"
namespace auth {
@@ -23,7 +22,6 @@ static const class_registrator<
cql3::query_processor&,
::service::raft_group0_client&,
::service::migration_manager&,
cache&,
utils::alien_worker&> registration("org.apache.cassandra.auth.AllowAllAuthenticator");
cache&> registration("org.apache.cassandra.auth.AllowAllAuthenticator");
}

View File

@@ -14,7 +14,6 @@
#include "auth/authenticator.hh"
#include "auth/cache.hh"
#include "auth/common.hh"
#include "utils/alien_worker.hh"
namespace cql3 {
class query_processor;
@@ -30,7 +29,7 @@ extern const std::string_view allow_all_authenticator_name;
class allow_all_authenticator final : public authenticator {
public:
allow_all_authenticator(cql3::query_processor&, ::service::raft_group0_client&, ::service::migration_manager&, cache&, utils::alien_worker&) {
allow_all_authenticator(cql3::query_processor&, ::service::raft_group0_client&, ::service::migration_manager&, cache&) {
}
virtual future<> start() override {

View File

@@ -35,14 +35,13 @@ static const class_registrator<auth::authenticator
, cql3::query_processor&
, ::service::raft_group0_client&
, ::service::migration_manager&
, auth::cache&
, utils::alien_worker&> cert_auth_reg(CERT_AUTH_NAME);
, auth::cache&> cert_auth_reg(CERT_AUTH_NAME);
enum class auth::certificate_authenticator::query_source {
subject, altname
};
auth::certificate_authenticator::certificate_authenticator(cql3::query_processor& qp, ::service::raft_group0_client&, ::service::migration_manager&, auth::cache&, utils::alien_worker&)
auth::certificate_authenticator::certificate_authenticator(cql3::query_processor& qp, ::service::raft_group0_client&, ::service::migration_manager&, auth::cache&)
: _queries([&] {
auto& conf = qp.db().get_config();
auto queries = conf.auth_certificate_role_queries();

View File

@@ -10,7 +10,6 @@
#pragma once
#include "auth/authenticator.hh"
#include "utils/alien_worker.hh"
#include <boost/regex_fwd.hpp> // IWYU pragma: keep
namespace cql3 {
@@ -34,7 +33,7 @@ class certificate_authenticator : public authenticator {
enum class query_source;
std::vector<std::pair<query_source, boost::regex>> _queries;
public:
certificate_authenticator(cql3::query_processor&, ::service::raft_group0_client&, ::service::migration_manager&, cache&, utils::alien_worker&);
certificate_authenticator(cql3::query_processor&, ::service::raft_group0_client&, ::service::migration_manager&, cache&);
~certificate_authenticator();
future<> start() override;

View File

@@ -49,8 +49,7 @@ static const class_registrator<
cql3::query_processor&,
::service::raft_group0_client&,
::service::migration_manager&,
cache&,
utils::alien_worker&> password_auth_reg("org.apache.cassandra.auth.PasswordAuthenticator");
cache&> password_auth_reg("org.apache.cassandra.auth.PasswordAuthenticator");
static thread_local auto rng_for_salt = std::default_random_engine(std::random_device{}());
@@ -64,14 +63,13 @@ std::string password_authenticator::default_superuser(const db::config& cfg) {
password_authenticator::~password_authenticator() {
}
password_authenticator::password_authenticator(cql3::query_processor& qp, ::service::raft_group0_client& g0, ::service::migration_manager& mm, cache& cache, utils::alien_worker& hashing_worker)
password_authenticator::password_authenticator(cql3::query_processor& qp, ::service::raft_group0_client& g0, ::service::migration_manager& mm, cache& cache)
: _qp(qp)
, _group0_client(g0)
, _migration_manager(mm)
, _cache(cache)
, _stopped(make_ready_future<>())
, _superuser(default_superuser(qp.db().get_config()))
, _hashing_worker(hashing_worker)
{}
static bool has_salted_hash(const cql3::untyped_result_set_row& row) {
@@ -330,9 +328,7 @@ future<authenticated_user> password_authenticator::authenticate(
}
salted_hash = role->salted_hash;
}
const bool password_match = co_await _hashing_worker.submit<bool>([password = std::move(password), salted_hash] {
return passwords::check(password, *salted_hash);
});
const bool password_match = co_await passwords::check(password, *salted_hash);
if (!password_match) {
throw exceptions::authentication_exception("Username and/or password are incorrect");
}

View File

@@ -18,7 +18,6 @@
#include "auth/passwords.hh"
#include "auth/cache.hh"
#include "service/raft/raft_group0_client.hh"
#include "utils/alien_worker.hh"
namespace db {
class config;
@@ -49,13 +48,12 @@ class password_authenticator : public authenticator {
shared_promise<> _superuser_created_promise;
// We used to also support bcrypt, SHA-256, and MD5 (ref. scylladb#24524).
constexpr static auth::passwords::scheme _scheme = passwords::scheme::sha_512;
utils::alien_worker& _hashing_worker;
public:
static db::consistency_level consistency_for_user(std::string_view role_name);
static std::string default_superuser(const db::config&);
password_authenticator(cql3::query_processor&, ::service::raft_group0_client&, ::service::migration_manager&, cache&, utils::alien_worker&);
password_authenticator(cql3::query_processor&, ::service::raft_group0_client&, ::service::migration_manager&, cache&);
~password_authenticator();

View File

@@ -7,6 +7,8 @@
*/
#include "auth/passwords.hh"
#include "utils/crypt_sha512.hh"
#include <seastar/core/coroutine.hh>
#include <cerrno>
@@ -21,27 +23,48 @@ static thread_local crypt_data tlcrypt = {};
namespace detail {
void verify_hashing_output(const char * res) {
if (!res || (res[0] == '*')) {
throw std::system_error(errno, std::system_category());
}
}
void verify_scheme(scheme scheme) {
const sstring random_part_of_salt = "aaaabbbbccccdddd";
const sstring salt = sstring(prefix_for_scheme(scheme)) + random_part_of_salt;
const char* e = crypt_r("fisk", salt.c_str(), &tlcrypt);
if (e && (e[0] != '*')) {
return;
try {
verify_hashing_output(e);
} catch (const std::system_error& ex) {
throw no_supported_schemes();
}
throw no_supported_schemes();
}
sstring hash_with_salt(const sstring& pass, const sstring& salt) {
auto res = crypt_r(pass.c_str(), salt.c_str(), &tlcrypt);
if (!res || (res[0] == '*')) {
throw std::system_error(errno, std::system_category());
}
verify_hashing_output(res);
return res;
}
seastar::future<sstring> hash_with_salt_async(const sstring& pass, const sstring& salt) {
sstring res;
// Only SHA-512 hashes for passphrases shorter than 256 bytes can be computed using
// the __crypt_sha512 method. For other computations, we fall back to the
// crypt_r implementation from `<crypt.h>`, which can stall.
if (salt.starts_with(prefix_for_scheme(scheme::sha_512)) && pass.size() <= 255) {
char buf[128];
const char * output_ptr = co_await __crypt_sha512(pass.c_str(), salt.c_str(), buf);
verify_hashing_output(output_ptr);
res = output_ptr;
} else {
const char * output_ptr = crypt_r(pass.c_str(), salt.c_str(), &tlcrypt);
verify_hashing_output(output_ptr);
res = output_ptr;
}
co_return res;
}
std::string_view prefix_for_scheme(scheme c) noexcept {
switch (c) {
case scheme::bcrypt_y: return "$2y$";
@@ -58,8 +81,9 @@ no_supported_schemes::no_supported_schemes()
: std::runtime_error("No allowed hashing schemes are supported on this system") {
}
bool check(const sstring& pass, const sstring& salted_hash) {
return detail::hash_with_salt(pass, salted_hash) == salted_hash;
seastar::future<bool> check(const sstring& pass, const sstring& salted_hash) {
const auto pwd_hash = co_await detail::hash_with_salt_async(pass, salted_hash);
co_return pwd_hash == salted_hash;
}
} // namespace auth::passwords

View File

@@ -11,6 +11,7 @@
#include <random>
#include <stdexcept>
#include <seastar/core/future.hh>
#include <seastar/core/sstring.hh>
#include "seastarx.hh"
@@ -75,10 +76,19 @@ sstring generate_salt(RandomNumberEngine& g, scheme scheme) {
///
/// Hash a password combined with an implementation-specific salt string.
/// Deprecated in favor of `hash_with_salt_async`.
///
/// \throws \ref std::system_error when an unexpected implementation-specific error occurs.
///
sstring hash_with_salt(const sstring& pass, const sstring& salt);
[[deprecated("Use hash_with_salt_async instead")]] sstring hash_with_salt(const sstring& pass, const sstring& salt);
///
/// Async version of `hash_with_salt` that returns a future.
/// If possible, hashing uses `coroutine::maybe_yield` to prevent reactor stalls.
///
/// \throws \ref std::system_error when an unexpected implementation-specific error occurs.
///
seastar::future<sstring> hash_with_salt_async(const sstring& pass, const sstring& salt);
} // namespace detail
@@ -107,6 +117,6 @@ sstring hash(const sstring& pass, RandomNumberEngine& g, scheme scheme) {
///
/// \throws \ref std::system_error when an unexpected implementation-specific error occurs.
///
bool check(const sstring& pass, const sstring& salted_hash);
seastar::future<bool> check(const sstring& pass, const sstring& salted_hash);
} // namespace auth::passwords

View File

@@ -35,10 +35,9 @@ static const class_registrator<
cql3::query_processor&,
::service::raft_group0_client&,
::service::migration_manager&,
cache&,
utils::alien_worker&> saslauthd_auth_reg("com.scylladb.auth.SaslauthdAuthenticator");
cache&> saslauthd_auth_reg("com.scylladb.auth.SaslauthdAuthenticator");
saslauthd_authenticator::saslauthd_authenticator(cql3::query_processor& qp, ::service::raft_group0_client&, ::service::migration_manager&, cache&, utils::alien_worker&)
saslauthd_authenticator::saslauthd_authenticator(cql3::query_processor& qp, ::service::raft_group0_client&, ::service::migration_manager&, cache&)
: _socket_path(qp.db().get_config().saslauthd_socket_path())
{}

View File

@@ -12,7 +12,6 @@
#include "auth/authenticator.hh"
#include "auth/cache.hh"
#include "utils/alien_worker.hh"
namespace cql3 {
class query_processor;
@@ -30,7 +29,7 @@ namespace auth {
class saslauthd_authenticator : public authenticator {
sstring _socket_path; ///< Path to the domain socket on which saslauthd is listening.
public:
saslauthd_authenticator(cql3::query_processor&, ::service::raft_group0_client&, ::service::migration_manager&, cache&,utils::alien_worker&);
saslauthd_authenticator(cql3::query_processor&, ::service::raft_group0_client&, ::service::migration_manager&, cache&);
future<> start() override;

View File

@@ -191,8 +191,7 @@ service::service(
::service::migration_manager& mm,
const service_config& sc,
maintenance_socket_enabled used_by_maintenance_socket,
cache& cache,
utils::alien_worker& hashing_worker)
cache& cache)
: service(
std::move(c),
cache,
@@ -200,7 +199,7 @@ service::service(
g0,
mn,
create_object<authorizer>(sc.authorizer_java_name, qp, g0, mm),
create_object<authenticator>(sc.authenticator_java_name, qp, g0, mm, cache, hashing_worker),
create_object<authenticator>(sc.authenticator_java_name, qp, g0, mm, cache),
create_object<role_manager>(sc.role_manager_java_name, qp, g0, mm, cache),
used_by_maintenance_socket) {
}

View File

@@ -27,7 +27,6 @@
#include "cql3/description.hh"
#include "seastarx.hh"
#include "service/raft/raft_group0_client.hh"
#include "utils/alien_worker.hh"
#include "utils/observable.hh"
#include "utils/serialized_action.hh"
#include "service/maintenance_mode.hh"
@@ -131,8 +130,7 @@ public:
::service::migration_manager&,
const service_config&,
maintenance_socket_enabled,
cache&,
utils::alien_worker&);
cache&);
future<> start(::service::migration_manager&, db::system_keyspace&);

View File

@@ -38,8 +38,8 @@ class transitional_authenticator : public authenticator {
public:
static const sstring PASSWORD_AUTHENTICATOR_NAME;
transitional_authenticator(cql3::query_processor& qp, ::service::raft_group0_client& g0, ::service::migration_manager& mm, cache& cache, utils::alien_worker& hashing_worker)
: transitional_authenticator(std::make_unique<password_authenticator>(qp, g0, mm, cache, hashing_worker)) {
transitional_authenticator(cql3::query_processor& qp, ::service::raft_group0_client& g0, ::service::migration_manager& mm, cache& cache)
: transitional_authenticator(std::make_unique<password_authenticator>(qp, g0, mm, cache)) {
}
transitional_authenticator(std::unique_ptr<authenticator> a)
: _authenticator(std::move(a)) {
@@ -241,8 +241,7 @@ static const class_registrator<
cql3::query_processor&,
::service::raft_group0_client&,
::service::migration_manager&,
auth::cache&,
utils::alien_worker&> transitional_authenticator_reg(auth::PACKAGE_NAME + "TransitionalAuthenticator");
auth::cache&> transitional_authenticator_reg(auth::PACKAGE_NAME + "TransitionalAuthenticator");
static const class_registrator<
auth::authorizer,

View File

@@ -859,6 +859,7 @@ scylla_core = (['message/messaging_service.cc',
'utils/alien_worker.cc',
'utils/array-search.cc',
'utils/base64.cc',
'utils/crypt_sha512.cc',
'utils/logalloc.cc',
'utils/large_bitset.cc',
'utils/buffer_input_stream.cc',
@@ -1479,7 +1480,6 @@ deps = {
pure_boost_tests = set([
'test/boost/anchorless_list_test',
'test/boost/auth_passwords_test',
'test/boost/auth_resource_test',
'test/boost/big_decimal_test',
'test/boost/caching_options_test',

View File

@@ -304,8 +304,6 @@ future<coordinator_result<>> batch_statement::execute_without_conditions(
}
}));
#endif
verify_batch_size(qp, mutations);
bool mutate_atomic = true;
if (_type != type::LOGGED) {
_stats.batches_pure_unlogged += 1;
@@ -313,6 +311,7 @@ future<coordinator_result<>> batch_statement::execute_without_conditions(
} else {
if (mutations.size() > 1) {
_stats.batches_pure_logged += 1;
verify_batch_size(qp, mutations);
} else {
_stats.batches_unlogged_from_logged += 1;
mutate_atomic = false;

View File

@@ -41,6 +41,8 @@ class MetricsProcessor:
# Get metrics from the file
try:
metrics_file = metrics.get_metrics_from_file(relative_path, "scylla_", metrics_info, strict=strict)
except SystemExit:
pass
finally:
os.chdir(old_cwd)
if metrics_file:

View File

@@ -15,3 +15,22 @@ with the Apache License (version 2) and ScyllaDB-Source-Available-1.0.
They contain the following tag:
SPDX-License-Identifier: (LicenseRef-ScyllaDB-Source-Available-1.0 and Apache-2.0)
### `musl libc` files
`licenses/musl-license.txt` is obtained from:
https://git.musl-libc.org/cgit/musl/tree/COPYRIGHT
`utils/crypt_sha512.cc` is obtained from:
https://git.musl-libc.org/cgit/musl/tree/src/crypt/crypt_sha512.c
Both files are obtained from git.musl-libc.org.
Import commit:
commit 1b76ff0767d01df72f692806ee5adee13c67ef88
Author: Alex Rønne Petersen <alex@alexrp.com>
Date: Sun Oct 12 05:35:19 2025 +0200
s390x: shuffle register usage in __tls_get_offset to avoid r0 as address
musl as a whole is licensed under the standard MIT license included in
`licenses/musl-license.txt`.

193
licenses/musl-license.txt Normal file
View File

@@ -0,0 +1,193 @@
musl as a whole is licensed under the following standard MIT license:
----------------------------------------------------------------------
Copyright © 2005-2020 Rich Felker, et al.
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
----------------------------------------------------------------------
Authors/contributors include:
A. Wilcox
Ada Worcester
Alex Dowad
Alex Suykov
Alexander Monakov
Andre McCurdy
Andrew Kelley
Anthony G. Basile
Aric Belsito
Arvid Picciani
Bartosz Brachaczek
Benjamin Peterson
Bobby Bingham
Boris Brezillon
Brent Cook
Chris Spiegel
Clément Vasseur
Daniel Micay
Daniel Sabogal
Daurnimator
David Carlier
David Edelsohn
Denys Vlasenko
Dmitry Ivanov
Dmitry V. Levin
Drew DeVault
Emil Renner Berthing
Fangrui Song
Felix Fietkau
Felix Janda
Gianluca Anzolin
Hauke Mehrtens
He X
Hiltjo Posthuma
Isaac Dunham
Jaydeep Patil
Jens Gustedt
Jeremy Huntwork
Jo-Philipp Wich
Joakim Sindholt
John Spencer
Julien Ramseier
Justin Cormack
Kaarle Ritvanen
Khem Raj
Kylie McClain
Leah Neukirchen
Luca Barbato
Luka Perkov
Lynn Ochs
M Farkas-Dyck (Strake)
Mahesh Bodapati
Markus Wichmann
Masanori Ogino
Michael Clark
Michael Forney
Mikhail Kremnyov
Natanael Copa
Nicholas J. Kain
orc
Pascal Cuoq
Patrick Oppenlander
Petr Hosek
Petr Skocik
Pierre Carrier
Reini Urban
Rich Felker
Richard Pennington
Ryan Fairfax
Samuel Holland
Segev Finer
Shiz
sin
Solar Designer
Stefan Kristiansson
Stefan O'Rear
Szabolcs Nagy
Timo Teräs
Trutz Behn
Will Dietz
William Haddon
William Pitcock
Portions of this software are derived from third-party works licensed
under terms compatible with the above MIT license:
The TRE regular expression implementation (src/regex/reg* and
src/regex/tre*) is Copyright © 2001-2008 Ville Laurikari and licensed
under a 2-clause BSD license (license text in the source files). The
included version has been heavily modified by Rich Felker in 2012, in
the interests of size, simplicity, and namespace cleanliness.
Much of the math library code (src/math/* and src/complex/*) is
Copyright © 1993,2004 Sun Microsystems or
Copyright © 2003-2011 David Schultz or
Copyright © 2003-2009 Steven G. Kargl or
Copyright © 2003-2009 Bruce D. Evans or
Copyright © 2008 Stephen L. Moshier or
Copyright © 2017-2018 Arm Limited
and labelled as such in comments in the individual source files. All
have been licensed under extremely permissive terms.
The ARM memcpy code (src/string/arm/memcpy.S) is Copyright © 2008
The Android Open Source Project and is licensed under a two-clause BSD
license. It was taken from Bionic libc, used on Android.
The AArch64 memcpy and memset code (src/string/aarch64/*) are
Copyright © 1999-2019, Arm Limited.
The implementation of DES for crypt (src/crypt/crypt_des.c) is
Copyright © 1994 David Burren. It is licensed under a BSD license.
The implementation of blowfish crypt (src/crypt/crypt_blowfish.c) was
originally written by Solar Designer and placed into the public
domain. The code also comes with a fallback permissive license for use
in jurisdictions that may not recognize the public domain.
The smoothsort implementation (src/stdlib/qsort.c) is Copyright © 2011
Lynn Ochs and is licensed under an MIT-style license.
The x86_64 port was written by Nicholas J. Kain and is licensed under
the standard MIT terms.
The mips and microblaze ports were originally written by Richard
Pennington for use in the ellcc project. The original code was adapted
by Rich Felker for build system and code conventions during upstream
integration. It is licensed under the standard MIT terms.
The mips64 port was contributed by Imagination Technologies and is
licensed under the standard MIT terms.
The powerpc port was also originally written by Richard Pennington,
and later supplemented and integrated by John Spencer. It is licensed
under the standard MIT terms.
All other files which have no copyright comments are original works
produced specifically for use as part of this library, written either
by Rich Felker, the main author of the library, or by one or more
contibutors listed above. Details on authorship of individual files
can be found in the git version control history of the project. The
omission of copyright and license comments in each file is in the
interest of source tree size.
In addition, permission is hereby granted for all public header files
(include/* and arch/*/bits/*) and crt files intended to be linked into
applications (crt/*, ldso/dlstart.c, and arch/*/crt_arch.h) to omit
the copyright notice and permission notice otherwise required by the
license, and to use these files without any requirement of
attribution. These files include substantial contributions from:
Bobby Bingham
John Spencer
Nicholas J. Kain
Rich Felker
Richard Pennington
Stefan Kristiansson
Szabolcs Nagy
all of whom have explicitly granted such permission.
This file previously contained text expressing a belief that most of
the files covered by the above exception were sufficiently trivial not
to be subject to copyright, resulting in confusion over whether it
negated the permissions granted in the license. In the spirit of
permissive licensing, and of not having licensing issues being an
obstacle to adoption, that text has been removed.

View File

@@ -748,8 +748,6 @@ To start the scylla server proper, simply invoke as: scylla server (or just scyl
// inherit Seastar's CPU affinity masks. We want this thread to be free
// to migrate between CPUs; we think that's what makes the most sense.
auto rpc_dict_training_worker = utils::alien_worker(startlog, 19, "rpc-dict");
// niceness=10 is ~10% of normal process time
auto hashing_worker = utils::alien_worker(startlog, 10, "pwd-hash");
return app.run(ac, av, [&] () -> future<int> {
@@ -779,8 +777,7 @@ To start the scylla server proper, simply invoke as: scylla server (or just scyl
return seastar::async([&app, cfg, ext, &disk_space_monitor_shard0, &cm, &sstm, &db, &qp, &bm, &proxy, &mapreduce_service, &mm, &mm_notifier, &ctx, &opts, &dirs,
&prometheus_server, &cf_cache_hitrate_calculator, &load_meter, &feature_service, &gossiper, &snitch,
&token_metadata, &erm_factory, &snapshot_ctl, &messaging, &sst_dir_semaphore, &raft_gr, &service_memory_limiter,
&repair, &sst_loader, &auth_cache, &ss, &lifecycle_notifier, &stream_manager, &task_manager, &rpc_dict_training_worker,
&hashing_worker, &vector_store_client] {
&repair, &sst_loader, &auth_cache, &ss, &lifecycle_notifier, &stream_manager, &task_manager, &rpc_dict_training_worker, &vector_store_client] {
try {
if (opts.contains("relabel-config-file") && !opts["relabel-config-file"].as<sstring>().empty()) {
// calling update_relabel_config_from_file can cause an exception that would stop startup
@@ -2060,7 +2057,7 @@ To start the scylla server proper, simply invoke as: scylla server (or just scyl
maintenance_auth_config.authenticator_java_name = sstring{auth::allow_all_authenticator_name};
maintenance_auth_config.role_manager_java_name = sstring{auth::maintenance_socket_role_manager_name};
maintenance_auth_service.start(perm_cache_config, std::ref(qp), std::ref(group0_client), std::ref(mm_notifier), std::ref(mm), maintenance_auth_config, maintenance_socket_enabled::yes, std::ref(auth_cache), std::ref(hashing_worker)).get();
maintenance_auth_service.start(perm_cache_config, std::ref(qp), std::ref(group0_client), std::ref(mm_notifier), std::ref(mm), maintenance_auth_config, maintenance_socket_enabled::yes, std::ref(auth_cache)).get();
cql_maintenance_server_ctl.emplace(maintenance_auth_service, mm_notifier, gossiper, qp, service_memory_limiter, sl_controller, lifecycle_notifier, *cfg, maintenance_cql_sg_stats_key, maintenance_socket_enabled::yes, dbcfg.statement_scheduling_group);
@@ -2336,7 +2333,7 @@ To start the scylla server proper, simply invoke as: scylla server (or just scyl
auth_config.authenticator_java_name = qualified_authenticator_name;
auth_config.role_manager_java_name = qualified_role_manager_name;
auth_service.start(std::move(perm_cache_config), std::ref(qp), std::ref(group0_client), std::ref(mm_notifier), std::ref(mm), auth_config, maintenance_socket_enabled::no, std::ref(auth_cache), std::ref(hashing_worker)).get();
auth_service.start(std::move(perm_cache_config), std::ref(qp), std::ref(group0_client), std::ref(mm_notifier), std::ref(mm), auth_config, maintenance_socket_enabled::no, std::ref(auth_cache)).get();
std::any stop_auth_service;
// Has to be called after node joined the cluster (join_cluster())

View File

@@ -22,7 +22,7 @@ format_match = re.compile(r'\s*(?:seastar::)?format\(\s*"([^"]+)"\s*,\s*(.*)\s*'
def handle_error(message, strict=True, verbose_mode=False):
if strict:
print(f"[ERROR] {message}")
exit(-1)
exit(1)
elif verbose_mode:
print(f"[WARNING] {message}")
@@ -180,12 +180,11 @@ def get_metrics_from_file(file_name, prefix, metrics_information, verb=None, str
groups = {}
if clean_name in metrics_information:
if (isinstance(metrics_information[clean_name], str) and metrics_information[clean_name] == "skip") or "skip" in metrics_information[clean_name]:
exit(0)
return {}
param_mapping = metrics_information[clean_name]["params"] if clean_name in metrics_information and "params" in metrics_information[clean_name] else {}
groups = metrics_information[clean_name]["groups"] if clean_name in metrics_information and "groups" in metrics_information[clean_name] else {}
metrics = {}
multi_line = False
names = undefined
typ = undefined
line_number = 0;

View File

@@ -1,6 +1,6 @@
"cdc/log.cc":
params:
cdc_group_name: cdc
cdc_group_name: "cdc"
part_name;suffix: [["static_row", "total"],["clustering_row", "total"], ["map", "total"], ["set", "total"], ["list", "total"], ["udt", "total"], ["range_tombstone", "total"],["partition_delete", "total"],["row_delete", "total"], ["static_row", "failed"],["clustering_row", "failed"], ["map", "failed"], ["set", "failed"], ["list", "failed"], ["udt", "failed"], ["range_tombstone", "failed"],["partition_delete", "failed"],["row_delete", "failed"]]
kind: ["total", "failed"]
"db/commitlog/commitlog.cc":
@@ -9,7 +9,7 @@
"cfg.max_active_flushes": "cfg.max_active_flushes"
"cql3/query_processor.cc":
groups:
"80": query_processor
"80": "query_processor"
"replica/dirty_memory_manager.cc":
params:
namestr: ["regular", "system"]
@@ -19,10 +19,11 @@
"replica/database.cc":
params:
"_dirty_memory_manager.throttle_threshold()": "throttle threshold"
"seastar/apps/metrics_tester/metrics_tester.cc": skip
"seastar/tests/unit/metrics_test.cc": skip
"seastar/tests/unit/metrics_tester.cc": skip
"seastar/tests/unit/prometheus_http_test.cc": skip
"seastar/apps/metrics_tester/metrics_tester.cc": "skip"
"seastar/tests/unit/metrics_test.cc": "skip"
"seastar/tests/unit/metrics_tester.cc": "skip"
"seastar/tests/unit/prometheus_http_test.cc": "skip"
"seastar/tests/unit/prometheus_text_test.cc": "skip"
"service/storage_proxy.cc":
params:
COORDINATOR_STATS_CATEGORY: "storage_proxy_coordinator"
@@ -32,25 +33,25 @@
_short_description_prefix: ["total_write_attempts", "write_errors", "background_replica_writes_failed", "read_repair_write_attempts"]
_long_description_prefix: ["total number of write requests", "number of write requests that failed", "background_replica_writes_failed", "number of write operations in a read repair context"]
_category: "storage_proxy_coordinator"
"thrift/server.cc": skip
"thrift/server.cc": "skip"
"tracing/tracing.cc":
params:
"max_pending_trace_records + write_event_records_threshold": "max_pending_trace_records + write_event_records_threshold"
"transport/server.cc":
groups:
"200": transport
"200": "transport"
params:
"_config.max_request_size": "max_request_size"
"seastar/src/net/dpdk.cc": skip
"seastar/src/net/dpdk.cc": "skip"
"db/hints/manager.cc":
params:
"group_name": ["hints_for_views_manager", "hints_manager"]
"seastar/src/core/execution_stage.cc":
groups:
"100": execution_stages
"100": "execution_stages"
"seastar/src/core/fair_queue.cc":
groups:
"300": io_queue
"300": "io_queue"
"seastar/src/net/net.cc":
params:
_stats_plugin_name: ["stats_plugin_name"]

View File

@@ -12,7 +12,7 @@ add_scylla_test(alternator_unit_test
add_scylla_test(anchorless_list_test
KIND BOOST)
add_scylla_test(auth_passwords_test
KIND BOOST
KIND SEASTAR
LIBRARIES auth)
add_scylla_test(auth_resource_test
KIND BOOST)

View File

@@ -6,7 +6,7 @@
* SPDX-License-Identifier: LicenseRef-ScyllaDB-Source-Available-1.0
*/
#define BOOST_TEST_MODULE core
#include <seastar/testing/test_case.hh>
#include <array>
#include <random>
@@ -16,15 +16,21 @@
#include <boost/test/unit_test.hpp>
#include <seastar/core/sstring.hh>
#include <seastar/core/coroutine.hh>
#include "seastarx.hh"
extern "C" {
#include <crypt.h>
#include <unistd.h>
}
static auto rng_for_salt = std::default_random_engine(std::random_device{}());
//
// The same password hashed multiple times will result in different strings because the salt will be different.
//
BOOST_AUTO_TEST_CASE(passwords_are_salted) {
SEASTAR_TEST_CASE(passwords_are_salted) {
const char* const cleartext = "my_excellent_password";
std::unordered_set<sstring> observed_passwords{};
@@ -33,12 +39,13 @@ BOOST_AUTO_TEST_CASE(passwords_are_salted) {
BOOST_REQUIRE(!observed_passwords.contains(e));
observed_passwords.insert(e);
}
co_return;
}
//
// A hashed password will authenticate against the same password in cleartext.
//
BOOST_AUTO_TEST_CASE(correct_passwords_authenticate) {
SEASTAR_TEST_CASE(correct_passwords_authenticate) {
// Common passwords.
std::array<const char*, 3> passwords{
"12345",
@@ -47,14 +54,85 @@ BOOST_AUTO_TEST_CASE(correct_passwords_authenticate) {
};
for (const char* p : passwords) {
BOOST_REQUIRE(auth::passwords::check(p, auth::passwords::hash(p, rng_for_salt, auth::passwords::scheme::sha_512)));
BOOST_REQUIRE(co_await auth::passwords::check(p, auth::passwords::hash(p, rng_for_salt, auth::passwords::scheme::sha_512)));
}
}
std::string long_password(uint32_t len) {
std::string out;
auto pattern = "0123456789";
for (uint32_t i = 0; i < len; ++i) {
out.push_back(pattern[i % strlen(pattern)]);
}
return out;
}
SEASTAR_TEST_CASE(same_hashes_as_crypt_h) {
std::string long_pwd_254 = long_password(254);
std::string long_pwd_255 = long_password(255);
std::string long_pwd_511 = long_password(511);
std::array<const char*, 8> passwords{
"12345",
"1_am_the_greatest!",
"password1",
// Some special characters
"!@#$%^&*()_+-=[]{}|\n;:'\",.<>/?",
// UTF-8 characters
"こんにちは、世界!",
// Passwords close to __crypt_sha512 length limit
long_pwd_254.c_str(),
long_pwd_255.c_str(),
// Password of maximal accepted length
long_pwd_511.c_str(),
};
auto salt = "$6$aaaabbbbccccdddd";
for (const char* p : passwords) {
auto res = co_await auth::passwords::detail::hash_with_salt_async(p, salt);
BOOST_REQUIRE(res == auth::passwords::detail::hash_with_salt(p, salt));
}
}
SEASTAR_TEST_CASE(too_long_password) {
auto p1 = long_password(71);
auto p2 = long_password(72);
auto p3 = long_password(73);
auto too_long_password = long_password(512);
auto salt_bcrypt = "$2a$05$mAyzaIeJu41dWUkxEbn8hO";
auto h1_bcrypt = co_await auth::passwords::detail::hash_with_salt_async(p1, salt_bcrypt);
auto h2_bcrypt = co_await auth::passwords::detail::hash_with_salt_async(p2, salt_bcrypt);
auto h3_bcrypt = co_await auth::passwords::detail::hash_with_salt_async(p3, salt_bcrypt);
BOOST_REQUIRE(h1_bcrypt != h2_bcrypt);
// The check below documents the behavior of the current bcrypt
// implementation that compares only the first 72 bytes of the password.
// Although we don't typically use bcrypt for password hashing, it is
// possible to insert such a hash using`CREATE ROLE ... WITH HASHED PASSWORD ...`.
// Refs: scylladb/scylladb#26842
BOOST_REQUIRE(h2_bcrypt == h3_bcrypt);
// The current implementation of bcrypt password hasing fails with passwords of length 512 and above
BOOST_CHECK_THROW(co_await auth::passwords::detail::hash_with_salt_async(too_long_password, salt_bcrypt), std::system_error);
auto salt_sha512 = "$6$aaaabbbbccccdddd";
auto h1_sha512 = co_await auth::passwords::detail::hash_with_salt_async(p1, salt_sha512);
auto h2_sha512 = co_await auth::passwords::detail::hash_with_salt_async(p2, salt_sha512);
auto h3_sha512 = co_await auth::passwords::detail::hash_with_salt_async(p3, salt_sha512);
BOOST_REQUIRE(h1_sha512 != h2_sha512);
BOOST_REQUIRE(h2_sha512 != h3_sha512);
// The current implementation of SHA-512 password hasing fails with passwords of length 512 and above
BOOST_CHECK_THROW(co_await auth::passwords::detail::hash_with_salt_async(too_long_password, salt_sha512), std::system_error);
}
//
// A hashed password that does not match the password in cleartext does not authenticate.
//
BOOST_AUTO_TEST_CASE(incorrect_passwords_do_not_authenticate) {
SEASTAR_TEST_CASE(incorrect_passwords_do_not_authenticate) {
const sstring hashed_password = auth::passwords::hash("actual_password", rng_for_salt,auth::passwords::scheme::sha_512);
BOOST_REQUIRE(!auth::passwords::check("password_guess", hashed_password));
BOOST_REQUIRE(!co_await auth::passwords::check("password_guess", hashed_password));
}

View File

@@ -74,3 +74,58 @@ def test_batch_with_error(cql, table1):
# exceptions::exception_code::SERVER_ERROR, it gets converted to NoHostAvailable by the driver
with pytest.raises(NoHostAvailable, match="Value too large"):
cql.execute(generate_big_batch(table1, 100) + injection_key)
def test_unlogged_batch_size_not_checked(cql, test_keyspace):
"""Verifies that UNLOGGED batches are NOT subject to batch size limits.
Unlogged batches are applied as independent mutations and don't go through
the system.batchlog table, so their collective size is irrelevant.
This test should succeed even with a batch larger than the fail threshold.
"""
with new_test_table(cql, test_keyspace, "k int primary key, t text") as table:
# Create a batch larger than the fail threshold (1024 KB)
# This would fail for a logged batch, but should succeed for unlogged
statements = [f"INSERT INTO {table} (k, t) VALUES ({idx}, '{'x' * 743}')" for idx in range(1100)]
unlogged_batch = "BEGIN UNLOGGED BATCH\n" + "\n".join(statements) + "\n APPLY BATCH\n"
# This should not raise an exception
cql.execute(unlogged_batch)
# Verify the data was inserted
result = cql.execute(f"SELECT COUNT(*) FROM {table}")
assert result.one()[0] == 1100
def test_logged_multi_partition_batch_size_checked(cql, test_keyspace):
"""Verifies that LOGGED batches targeting multiple partitions ARE subject to batch size limits.
Logged multi-partition batches go through system.batchlog and must be size-checked.
"""
with new_test_table(cql, test_keyspace, "k int primary key, t text") as table:
# Create a batch larger than the fail threshold (1024 KB) with multiple partitions
statements = [f"INSERT INTO {table} (k, t) VALUES ({idx}, '{'x' * 743}')" for idx in range(1100)]
logged_batch = "BEGIN BATCH\n" + "\n".join(statements) + "\n APPLY BATCH\n"
# This should raise "Batch too large" exception
with pytest.raises(InvalidRequest, match="Batch too large"):
cql.execute(logged_batch)
def test_logged_single_partition_batch_size_not_checked(cql, test_keyspace):
"""Verifies that LOGGED batches targeting a single partition are NOT subject to batch size limits.
Logged single-partition batches don't go through system.batchlog,
so their collective size is not relevant.
"""
with new_test_table(cql, test_keyspace, "k int, c int, t text, primary key (k, c)") as table:
# Create a batch larger than the fail threshold (1024 KB) but all targeting the same partition
statements = [f"INSERT INTO {table} (k, c, t) VALUES (1, {idx}, '{'x' * 743}')" for idx in range(1100)]
logged_batch = "BEGIN BATCH\n" + "\n".join(statements) + "\n APPLY BATCH\n"
# This should not raise an exception since it's a single-partition batch
cql.execute(logged_batch)
# Verify the data was inserted
result = cql.execute(f"SELECT COUNT(*) FROM {table} WHERE k = 1")
assert result.one()[0] == 1100

View File

@@ -1128,11 +1128,8 @@ private:
auth_config.authenticator_java_name = qualified_authenticator_name;
auth_config.role_manager_java_name = qualified_role_manager_name;
_auth_service.start(perm_cache_config, std::ref(_qp), std::ref(group0_client), std::ref(_mnotifier), std::ref(_mm), auth_config, maintenance_socket_enabled::no, std::ref(_auth_cache)).get();
const uint64_t niceness = 19;
auto hashing_worker = utils::alien_worker(startlog, niceness, "pwd-hash");
_auth_service.start(perm_cache_config, std::ref(_qp), std::ref(group0_client), std::ref(_mnotifier), std::ref(_mm), auth_config, maintenance_socket_enabled::no, std::ref(_auth_cache), std::ref(hashing_worker)).get();
_auth_service.invoke_on_all([this] (auth::service& auth) {
return auth.start(_mm.local(), _sys_ks.local());
}).get();

View File

@@ -123,7 +123,7 @@ _get_distribution_components() {
continue
;;
esac
echo "$target"
echo $target
done
}
@@ -164,7 +164,7 @@ if [[ "${CLANG_BUILD}" = "INSTALL" ]]; then
echo "[clang-stage3] build the compiler applied CSPGO profile"
cd "${CLANG_BUILD_DIR}"
llvm-profdata merge build/profiles/csir-*.profraw -output=csir.prof
llvm-profdata merge build/csprofiles/default_*.profraw -output=csir.prof
llvm-profdata merge ir.prof csir.prof -output=combined.prof
rm -rf build
# linker flags are needed for BOLT

View File

@@ -15,6 +15,7 @@ target_sources(utils
buffer_input_stream.cc
build_id.cc
config_file.cc
crypt_sha512.cc
directories.cc
disk-error-handler.cc
disk_space_monitor.cc

381
utils/crypt_sha512.cc Normal file
View File

@@ -0,0 +1,381 @@
/*
* This file originates from musl libc (git.musl-libc.org).
* Modifications have been made and are licensed under the following terms:
* Copyright (C) 2025-present ScyllaDB
* SPDX-License-Identifier: LicenseRef-ScyllaDB-Source-Available-1.0
*
* public domain sha512 crypt implementation
*
* original sha crypt design: http://people.redhat.com/drepper/SHA-crypt.txt
* in this implementation at least 32bit int is assumed,
* key length is limited, the $6$ prefix is mandatory, '\n' and ':' is rejected
* in the salt and rounds= setting must contain a valid iteration count,
* on error "*" is returned.
*/
#include <ctype.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include "crypt_sha512.hh"
#include <seastar/core/coroutine.hh>
#include <seastar/coroutine/maybe_yield.hh>
/* public domain sha512 implementation based on fips180-3 */
/* >=2^64 bits messages are not supported (about 2000 peta bytes) */
struct sha512 {
uint64_t len; /* processed message length */
uint64_t h[8]; /* hash state */
uint8_t buf[128]; /* message block buffer */
};
static uint64_t ror(uint64_t n, int k) { return (n >> k) | (n << (64-k)); }
#define Ch(x,y,z) (z ^ (x & (y ^ z)))
#define Maj(x,y,z) ((x & y) | (z & (x | y)))
#define S0(x) (ror(x,28) ^ ror(x,34) ^ ror(x,39))
#define S1(x) (ror(x,14) ^ ror(x,18) ^ ror(x,41))
#define R0(x) (ror(x,1) ^ ror(x,8) ^ (x>>7))
#define R1(x) (ror(x,19) ^ ror(x,61) ^ (x>>6))
static const uint64_t K[80] = {
0x428a2f98d728ae22ULL, 0x7137449123ef65cdULL, 0xb5c0fbcfec4d3b2fULL, 0xe9b5dba58189dbbcULL,
0x3956c25bf348b538ULL, 0x59f111f1b605d019ULL, 0x923f82a4af194f9bULL, 0xab1c5ed5da6d8118ULL,
0xd807aa98a3030242ULL, 0x12835b0145706fbeULL, 0x243185be4ee4b28cULL, 0x550c7dc3d5ffb4e2ULL,
0x72be5d74f27b896fULL, 0x80deb1fe3b1696b1ULL, 0x9bdc06a725c71235ULL, 0xc19bf174cf692694ULL,
0xe49b69c19ef14ad2ULL, 0xefbe4786384f25e3ULL, 0x0fc19dc68b8cd5b5ULL, 0x240ca1cc77ac9c65ULL,
0x2de92c6f592b0275ULL, 0x4a7484aa6ea6e483ULL, 0x5cb0a9dcbd41fbd4ULL, 0x76f988da831153b5ULL,
0x983e5152ee66dfabULL, 0xa831c66d2db43210ULL, 0xb00327c898fb213fULL, 0xbf597fc7beef0ee4ULL,
0xc6e00bf33da88fc2ULL, 0xd5a79147930aa725ULL, 0x06ca6351e003826fULL, 0x142929670a0e6e70ULL,
0x27b70a8546d22ffcULL, 0x2e1b21385c26c926ULL, 0x4d2c6dfc5ac42aedULL, 0x53380d139d95b3dfULL,
0x650a73548baf63deULL, 0x766a0abb3c77b2a8ULL, 0x81c2c92e47edaee6ULL, 0x92722c851482353bULL,
0xa2bfe8a14cf10364ULL, 0xa81a664bbc423001ULL, 0xc24b8b70d0f89791ULL, 0xc76c51a30654be30ULL,
0xd192e819d6ef5218ULL, 0xd69906245565a910ULL, 0xf40e35855771202aULL, 0x106aa07032bbd1b8ULL,
0x19a4c116b8d2d0c8ULL, 0x1e376c085141ab53ULL, 0x2748774cdf8eeb99ULL, 0x34b0bcb5e19b48a8ULL,
0x391c0cb3c5c95a63ULL, 0x4ed8aa4ae3418acbULL, 0x5b9cca4f7763e373ULL, 0x682e6ff3d6b2b8a3ULL,
0x748f82ee5defb2fcULL, 0x78a5636f43172f60ULL, 0x84c87814a1f0ab72ULL, 0x8cc702081a6439ecULL,
0x90befffa23631e28ULL, 0xa4506cebde82bde9ULL, 0xbef9a3f7b2c67915ULL, 0xc67178f2e372532bULL,
0xca273eceea26619cULL, 0xd186b8c721c0c207ULL, 0xeada7dd6cde0eb1eULL, 0xf57d4f7fee6ed178ULL,
0x06f067aa72176fbaULL, 0x0a637dc5a2c898a6ULL, 0x113f9804bef90daeULL, 0x1b710b35131c471bULL,
0x28db77f523047d84ULL, 0x32caab7b40c72493ULL, 0x3c9ebe0a15c9bebcULL, 0x431d67c49c100d4cULL,
0x4cc5d4becb3e42b6ULL, 0x597f299cfc657e2aULL, 0x5fcb6fab3ad6faecULL, 0x6c44198c4a475817ULL
};
static void processblock(struct sha512 *s, const uint8_t *buf)
{
uint64_t W[80], t1, t2, a, b, c, d, e, f, g, h;
int i;
for (i = 0; i < 16; i++) {
W[i] = (uint64_t)buf[8*i]<<56;
W[i] |= (uint64_t)buf[8*i+1]<<48;
W[i] |= (uint64_t)buf[8*i+2]<<40;
W[i] |= (uint64_t)buf[8*i+3]<<32;
W[i] |= (uint64_t)buf[8*i+4]<<24;
W[i] |= (uint64_t)buf[8*i+5]<<16;
W[i] |= (uint64_t)buf[8*i+6]<<8;
W[i] |= buf[8*i+7];
}
for (; i < 80; i++)
W[i] = R1(W[i-2]) + W[i-7] + R0(W[i-15]) + W[i-16];
a = s->h[0];
b = s->h[1];
c = s->h[2];
d = s->h[3];
e = s->h[4];
f = s->h[5];
g = s->h[6];
h = s->h[7];
for (i = 0; i < 80; i++) {
t1 = h + S1(e) + Ch(e,f,g) + K[i] + W[i];
t2 = S0(a) + Maj(a,b,c);
h = g;
g = f;
f = e;
e = d + t1;
d = c;
c = b;
b = a;
a = t1 + t2;
}
s->h[0] += a;
s->h[1] += b;
s->h[2] += c;
s->h[3] += d;
s->h[4] += e;
s->h[5] += f;
s->h[6] += g;
s->h[7] += h;
}
static void pad(struct sha512 *s)
{
unsigned r = s->len % 128;
s->buf[r++] = 0x80;
if (r > 112) {
memset(s->buf + r, 0, 128 - r);
r = 0;
processblock(s, s->buf);
}
memset(s->buf + r, 0, 120 - r);
s->len *= 8;
s->buf[120] = s->len >> 56;
s->buf[121] = s->len >> 48;
s->buf[122] = s->len >> 40;
s->buf[123] = s->len >> 32;
s->buf[124] = s->len >> 24;
s->buf[125] = s->len >> 16;
s->buf[126] = s->len >> 8;
s->buf[127] = s->len;
processblock(s, s->buf);
}
static void sha512_init(struct sha512 *s)
{
s->len = 0;
s->h[0] = 0x6a09e667f3bcc908ULL;
s->h[1] = 0xbb67ae8584caa73bULL;
s->h[2] = 0x3c6ef372fe94f82bULL;
s->h[3] = 0xa54ff53a5f1d36f1ULL;
s->h[4] = 0x510e527fade682d1ULL;
s->h[5] = 0x9b05688c2b3e6c1fULL;
s->h[6] = 0x1f83d9abfb41bd6bULL;
s->h[7] = 0x5be0cd19137e2179ULL;
}
static void sha512_sum(struct sha512 *s, uint8_t *md)
{
int i;
pad(s);
for (i = 0; i < 8; i++) {
md[8*i] = s->h[i] >> 56;
md[8*i+1] = s->h[i] >> 48;
md[8*i+2] = s->h[i] >> 40;
md[8*i+3] = s->h[i] >> 32;
md[8*i+4] = s->h[i] >> 24;
md[8*i+5] = s->h[i] >> 16;
md[8*i+6] = s->h[i] >> 8;
md[8*i+7] = s->h[i];
}
}
static void sha512_update(struct sha512 *s, const void *m, unsigned long len)
{
const uint8_t *p = (const uint8_t *)m;
unsigned r = s->len % 128;
s->len += len;
if (r) {
if (len < 128 - r) {
memcpy(s->buf + r, p, len);
return;
}
memcpy(s->buf + r, p, 128 - r);
len -= 128 - r;
p += 128 - r;
processblock(s, s->buf);
}
for (; len >= 128; len -= 128, p += 128)
processblock(s, p);
memcpy(s->buf, p, len);
}
static const unsigned char b64[] =
"./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
static char *to64(char *s, unsigned int u, int n)
{
while (--n >= 0) {
*s++ = b64[u % 64];
u /= 64;
}
return s;
}
/* key limit is not part of the original design, added for DoS protection.
* rounds limit has been lowered (versus the reference/spec), also for DoS
* protection. runtime is O(klen^2 + klen*rounds) */
#define KEY_MAX 256
#define SALT_MAX 16
#define ROUNDS_DEFAULT 5000
#define ROUNDS_MIN 1000
#define ROUNDS_MAX 9999999
/* hash n bytes of the repeated md message digest */
static void hashmd(struct sha512 *s, unsigned int n, const void *md)
{
unsigned int i;
for (i = n; i > 64; i -= 64)
sha512_update(s, md, 64);
sha512_update(s, md, i);
}
static seastar::future<char *> sha512crypt(const char *key, const char *setting, char *output)
{
struct sha512 ctx;
unsigned char md[64], kmd[64], smd[64];
unsigned int i, r, klen, slen;
char rounds[20] = "";
const char *salt;
char *p;
/* reject large keys */
for (i = 0; i <= KEY_MAX && key[i]; i++);
if (i > KEY_MAX)
co_return nullptr;
klen = i;
/* setting: $6$rounds=n$salt$ (rounds=n$ and closing $ are optional) */
if (strncmp(setting, "$6$", 3) != 0)
co_return nullptr;
salt = setting + 3;
r = ROUNDS_DEFAULT;
if (strncmp(salt, "rounds=", sizeof "rounds=" - 1) == 0) {
unsigned long u;
char *end;
/*
* this is a deviation from the reference:
* bad rounds setting is rejected if it is
* - empty
* - unterminated (missing '$')
* - begins with anything but a decimal digit
* the reference implementation treats these bad
* rounds as part of the salt or parse them with
* strtoul semantics which may cause problems
* including non-portable hashes that depend on
* the host's value of ULONG_MAX.
*/
salt += sizeof "rounds=" - 1;
if (!isdigit(*salt))
co_return nullptr;
u = strtoul(salt, &end, 10);
if (*end != '$')
co_return nullptr;
salt = end+1;
if (u < ROUNDS_MIN)
r = ROUNDS_MIN;
else if (u > ROUNDS_MAX)
co_return nullptr;
else
r = u;
/* needed when rounds is zero prefixed or out of bounds */
sprintf(rounds, "rounds=%u$", r);
}
for (i = 0; i < SALT_MAX && salt[i] && salt[i] != '$'; i++)
/* reject characters that interfere with /etc/shadow parsing */
if (salt[i] == '\n' || salt[i] == ':')
co_return nullptr;
slen = i;
/* B = sha(key salt key) */
sha512_init(&ctx);
sha512_update(&ctx, key, klen);
sha512_update(&ctx, salt, slen);
sha512_update(&ctx, key, klen);
sha512_sum(&ctx, md);
/* A = sha(key salt repeat-B alternate-B-key) */
sha512_init(&ctx);
sha512_update(&ctx, key, klen);
sha512_update(&ctx, salt, slen);
hashmd(&ctx, klen, md);
for (i = klen; i > 0; i >>= 1)
if (i & 1)
sha512_update(&ctx, md, sizeof md);
else
sha512_update(&ctx, key, klen);
sha512_sum(&ctx, md);
/* DP = sha(repeat-key), this step takes O(klen^2) time */
sha512_init(&ctx);
for (i = 0; i < klen; i++)
sha512_update(&ctx, key, klen);
sha512_sum(&ctx, kmd);
/* DS = sha(repeat-salt) */
sha512_init(&ctx);
for (i = 0; i < 16 + md[0]; i++)
sha512_update(&ctx, salt, slen);
sha512_sum(&ctx, smd);
/* iterate A = f(A,DP,DS), this step takes O(rounds*klen) time */
for (i = 0; i < r; i++) {
sha512_init(&ctx);
if (i % 2)
hashmd(&ctx, klen, kmd);
else
sha512_update(&ctx, md, sizeof md);
if (i % 3)
sha512_update(&ctx, smd, slen);
if (i % 7)
hashmd(&ctx, klen, kmd);
if (i % 2)
sha512_update(&ctx, md, sizeof md);
else
hashmd(&ctx, klen, kmd);
sha512_sum(&ctx, md);
co_await seastar::coroutine::maybe_yield();
}
/* output is $6$rounds=n$salt$hash */
p = output;
p += sprintf(p, "$6$%s%.*s$", rounds, slen, salt);
#if 1
static const unsigned char perm[][3] = {
{0,21,42},{22,43,1},{44,2,23},{3,24,45},{25,46,4},
{47,5,26},{6,27,48},{28,49,7},{50,8,29},{9,30,51},
{31,52,10},{53,11,32},{12,33,54},{34,55,13},{56,14,35},
{15,36,57},{37,58,16},{59,17,38},{18,39,60},{40,61,19},
{62,20,41} };
for (i=0; i<21; i++) p = to64(p,
(md[perm[i][0]]<<16)|(md[perm[i][1]]<<8)|md[perm[i][2]], 4);
#else
p = to64(p, (md[0]<<16)|(md[21]<<8)|md[42], 4);
p = to64(p, (md[22]<<16)|(md[43]<<8)|md[1], 4);
p = to64(p, (md[44]<<16)|(md[2]<<8)|md[23], 4);
p = to64(p, (md[3]<<16)|(md[24]<<8)|md[45], 4);
p = to64(p, (md[25]<<16)|(md[46]<<8)|md[4], 4);
p = to64(p, (md[47]<<16)|(md[5]<<8)|md[26], 4);
p = to64(p, (md[6]<<16)|(md[27]<<8)|md[48], 4);
p = to64(p, (md[28]<<16)|(md[49]<<8)|md[7], 4);
p = to64(p, (md[50]<<16)|(md[8]<<8)|md[29], 4);
p = to64(p, (md[9]<<16)|(md[30]<<8)|md[51], 4);
p = to64(p, (md[31]<<16)|(md[52]<<8)|md[10], 4);
p = to64(p, (md[53]<<16)|(md[11]<<8)|md[32], 4);
p = to64(p, (md[12]<<16)|(md[33]<<8)|md[54], 4);
p = to64(p, (md[34]<<16)|(md[55]<<8)|md[13], 4);
p = to64(p, (md[56]<<16)|(md[14]<<8)|md[35], 4);
p = to64(p, (md[15]<<16)|(md[36]<<8)|md[57], 4);
p = to64(p, (md[37]<<16)|(md[58]<<8)|md[16], 4);
p = to64(p, (md[59]<<16)|(md[17]<<8)|md[38], 4);
p = to64(p, (md[18]<<16)|(md[39]<<8)|md[60], 4);
p = to64(p, (md[40]<<16)|(md[61]<<8)|md[19], 4);
p = to64(p, (md[62]<<16)|(md[20]<<8)|md[41], 4);
#endif
p = to64(p, md[63], 2);
*p = 0;
co_return output;
}
seastar::future<const char *> __crypt_sha512(const char *key, const char *setting, char *output)
{
static const char testkey[] = "Xy01@#\x01\x02\x80\x7f\xff\r\n\x81\t !";
static const char testsetting[] = "$6$rounds=1234$abc0123456789$";
static const char testhash[] = "$6$rounds=1234$abc0123456789$BCpt8zLrc/RcyuXmCDOE1ALqMXB2MH6n1g891HhFj8.w7LxGv.FTkqq6Vxc/km3Y0jE0j24jY5PIv/oOu6reg1";
char testbuf[128];
char *p, *q;
p = co_await sha512crypt(key, setting, output);
/* self test and stack cleanup */
q = co_await sha512crypt(testkey, testsetting, testbuf);
if (!p || q != testbuf || memcmp(testbuf, testhash, sizeof testhash))
co_return "*";
co_return p;
}

13
utils/crypt_sha512.hh Normal file
View File

@@ -0,0 +1,13 @@
/*
* Copyright (C) 2025-present ScyllaDB
*/
/*
* SPDX-License-Identifier: LicenseRef-ScyllaDB-Source-Available-1.0
*/
#pragma once
#include <seastar/core/future.hh>
seastar::future<const char *> __crypt_sha512(const char *key, const char *setting, char *output);