mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-23 10:00:35 +00:00
4f3b8f38e2
" The current api design of abstract_replication_strategy provides a can_yield parameter to calls that may stall when traversing the token metadata in O(n^2) and even in O(n) for a large number of token ranges. But, to use this option the caller must run in a seastar thread. It can't be used if the caller runs a coroutine or plain async tasks. Rather than keep adding threads (e.g. in storage_service::load_and_stream or storage_service::describe_ring), the series offers an infrastructure change: precalculating the token->endpoints map once, using an async task, and keeping the results in a `effective_replication_map` object. The latter can be used for efficient and stall-free calls, like get_natural_endpoints, or get_ranges/get_primary_range, replacing their equivalents in abstract_replication_strategy, and dropping the public abstract_replication_strategy::calculate_natural_endpoints and its internal cached_endpoints map. Other than the performance benefits of: 1. The current calls require running a thread to yield. Precalculating the map (using async task) allows us to use synchronous calls without stalling the rector. 2. The replication maps can and should be shared between keyspaces that use the same replication strategy. (Will be sent as a follow-up to the series) The bigger benefits (courtesy of Avi Kivity) are laying the groundwork for: 1. atomic replication metadata - an operation can capture a replication map once, and then use consistent information from the map without worrying that it changes under its feet. We may even be able to s/inet_address/replica_ptr/ later. 2. establish boundaries on the use of replication information - by making a replication map not visible, and observing when its reference count drops to zero, we can tell when the new replication map is fully in use. When we start writing to a new node we'll be able to locate a point in time where all writes that were not aware of the new node were completed (this is the point where we should start streaming). Notes: * The get_natural_endpoints method that uses the effective_replication_map is still provided as a abstract_replication_strategy virtual method so that local_strategy can override it and privide natural endpoints for any search token, even in the absence of token_metadata, when\ called early-on, before token_metadata has been established. The effective_replication_map materializes the replication strategy over a given replication strategy options and token_metadata. Whenever either of those change for a keyspace, we make a new effective_replication_map and keep it in the keyspace for latter use. Methods that depend on an ad-hoc token_metadata (e.g. during node operations like bootstrap or replace) are still provided by abstract_replication_strategy. TODO: - effective_replication_map registry - Move pending ranges from token_metadata to replication map - get rid of abstract_replication_strategy::get_range_addresses(token_metadata&) - calculate replication map and use it instead. Test: unit(dev, debug) Dtest: next-gating, bootstrap_test.py update_cluster_layout_tests.py alternator_tests.py -a 'dtest-full,!dtest-heavy' (release) " * tag 'effective_replication_strategy-v6' of github.com:bhalevy/scylla: (44 commits) effective_replication_map: add get_range_addresses abstract_replication_strategy: get rid of shared_token_metadata member and ctor param abstract_replication_strategy: recognized_options: pass const topology& abstract_replication_strategy: precacluate get_replication_factor for effective_replication_map token_metadata: get rid of now-unused sync methods abstract_replication_strategy: get rid of do_calculate_natural_endpoints abstract_replication_strategy: futurize get_*address_ranges abstract_replication_strategy: futurize get_range_addresses abstract_replication_strategy: futurize get_ranges(inet_address ep, token_metadata_ptr) abstract_replication_strategy: move get_ranges and get_primary_ranges* to effective_replication_map compaction_manager: pass owned_ranges via cleanup/upgrade options abstract_replication_strategy: get rid of cached_endpoints all replication strategies: get rid of do_get_natural_endpoints storage_proxy: use effective_replication_map token_metadata_ptr along with endpoints abstract_replication_strategy: move get_natural_endpoints_without_node_being_replaced to effective_replication_map storage_service: bootstrap: add log messages storage_service: get_mutable_token_metadata_ptr: always invalidate_cached_rings shared_token_metadata: set: check version monotonicity token_metadata: use static ring version token_metadata: get rid of copy constructor and assignment operator ...