0fd1050784
Adds glue needed to pass lz4 and zstd with streaming and/or dictionaries as the network traffic compressors for Seastar's RPC servers. The main jobs of this glue are: 1. Implementing the API expected by Seastar from RPC compressors. 2. Expose metrics about the effectiveness of the compression. 3. Allow dynamically switching algorithms and dictionaries on a running connection, without any extra waits. The biggest design decision here is that the choice of algorithm and dictionary is negotiated by both sides of the connection, not dictated unilaterally by the sender. The negotiation algorithm is fairly complicated (a TLA+ model validating it is included in the commit). Unilateral compression choice would be much simpler. However, negotiation avoids re-sending the same dictionary over every connection in the cluster after dictionary updates (with one-way communication, it's the only reliable way to ensure that our receiver possesses the dictionary we are about to start using), lets receivers ask for a cheaper compression mode if they want, and lets them refuse to update a dictionary if they don't think they have enough free memory for that. In hindsight, those properties probably weren't worth the extra complexity and extra development effort. Zstd can be quite expensive, so this patch also includes a mechanism which temporarily downgrades the compressor from zstd to lz4 if zstd has been using too much CPU in a given slice of time. But it should be noted that this can't be treated as a reliable "protection" from negative performance effects of zstd, since a downgrade can happen on the sender side, and receivers are at the mercy of senders.
48 lines
1.6 KiB
C++
48 lines
1.6 KiB
C++
/*
|
|
* Copyright (C) 2023-present ScyllaDB
|
|
*/
|
|
|
|
/*
|
|
* SPDX-License-Identifier: LicenseRef-ScyllaDB-Source-Available-1.0
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
#include "utils/shared_dict.hh"
|
|
#include "utils/advanced_rpc_compressor.hh"
|
|
|
|
namespace utils {
|
|
|
|
struct control_protocol_frame {
|
|
enum header_enum : uint8_t {
|
|
NONE, // This means that the field isn't filled.
|
|
UPDATE,
|
|
COMMIT,
|
|
};
|
|
|
|
struct one_side {
|
|
header_enum header = NONE;
|
|
uint64_t epoch = 0;
|
|
compression_algorithm_set algo = compression_algorithm_set::singleton(compression_algorithm::type::RAW);
|
|
shared_dict::dict_id dict;
|
|
constexpr static size_t serialized_size = 1+8+1+16+8+32;
|
|
void serialize(std::span<std::byte, serialized_size>);
|
|
static one_side deserialize(std::span<const std::byte, serialized_size>);
|
|
};
|
|
|
|
// The negotiation algorithm is run for each of the two directions of the connection separately.
|
|
// The `receiver` field below is a message for the algorithm instance in which we are the receiver.
|
|
// `sender` is for the instance in which we are the sender.
|
|
//
|
|
// Even though usually only one of these will be filled with something meaningful (not `NONE`),
|
|
// we always send both just to keep the layout of the frame fixed. It simplifies the serialization.
|
|
one_side receiver;
|
|
one_side sender;
|
|
|
|
constexpr static size_t serialized_size = 2 * one_side::serialized_size;
|
|
void serialize(std::span<std::byte, serialized_size>);
|
|
static control_protocol_frame deserialize(std::span<const std::byte, serialized_size>);
|
|
};
|
|
|
|
} // namespace utils
|