Wrapping ranges are a pain, so we are moving wrap handling to the edges.
Since cql can't generate wrapping ranges, this means thrift and the ring
maintenance code; also range->ring transformations need to merge the first
and last ranges.
Message-Id: <1478105905-31613-1-git-send-email-avi@scylladb.com>
pending_endpoints_for is called frequently by
storage_proxy::create_write_response_handler when doing cql query.
Before this patch, each call to pending_endpoints_for involves
converting a multimap (std::unordered_multimap<range<token>,
inet_address>>) to map (std::unordered_map<range<token>,
std::unordered_set<inet_address>>).
To speed up the token to pending endpoint mapping search, a interval map
is introduced. It is faster than searching the map linearly and can
avoid caching the token/pending endpoint mapping.
With this patch, the operations per second drop during adding node
period gets much better.
Before:
45K to 10K
After:
45k to 38K
(The number is measured with the streaming code skipping to send data to
rule out the streaming factor.)
Refs: #1223
When replacing a node, we might ignore the tokens so that the tokens is
empty. In this case, we will have
std::unordered_map<inet_address, std::unordered_set<token>> = {ip, {}}
passed to token_metadata::update_normal_tokens(std::unordered_map<inet_address,
std::unordered_set<token>>& endpoint_tokens)
and hit the assert
assert(!tokens.empty());
The get_token_endpoint API should return a map of tokens to endpoints,
including the bootstrapping ones.
Use get_local_storage_service().get_token_to_endpoint_map() for it.
$ nodetool -p 7100 status
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 127.0.0.1 12645 256 ? eac5b6cf-5fda-4447-8104-a7bf3b773aba rack1
UN 127.0.0.2 12635 256 ? 2ad1b7df-c8ad-4cbc-b1f1-059121d2f0c7 rack1
UN 127.0.0.3 12624 256 ? 61f82ea7-637d-4083-acc9-567e0c01b490 rack1
UJ 127.0.0.4 ? 256 ? ced2725e-a5a4-4ac3-86de-e1c66cecfb8d rack1
Fixes#617
One version returns only the ranges
std::vector<range<token>>
Another version returns a map
std::unordered_map<range<token>, std::unordered_set<inet_address>>
which is converted from
std::unordered_multimap<range<token>, inet_address>
They are needed by token_metadata::pending_endpoints_for,
storage_service::get_all_ranges_with_strict_sources_for and
storage_service::decommission.
This patch adds a const version for get_datacenter_endpoints and
get_topology.
It modified the token iterator to use a const version of token_metadata
and it make first_token, first_token_index, tokens_end and ring_range to
be a const method.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
If we had a range (x; ...] then x is excluded, but token iterator was
initialized with x. The splitting loop would exit prematurely because
it would detect that the token is outside the range.
The fix is to teach ring_range() to recognize this and always give
tokens which are not smaller than the range's lower bound.
"This series aims to add the network_topology_strategy class.
It adds a few things that were missing in the token_metadata and
abstract_replication_strategy classes on the way:
- Full topology (a.k.a. Topology) class implementation.
- ring_range (a.k.a. TokenMetadata.ringIterator). The implementation
is partial since we don't implement the "inclideMin" parameter.
- endpoints caching in the abstract_replication_strategy.
- Add-only Java's LinkedHashSet implementation - sequenced_set.
Then we add the network_topology_strategy class itself and the corresponding
unit test that validates the functionality of both the strategy class and of
the caching mechanism."
Conflicts:
locator/token_metadata.hh
This expose the _leaving_endpoints, _moving_endpoints and
_bootstrap_tokens.
It also changes the signature of get_endpoint_to_host_id_map_for_reading
from auto so the compiler will be able to resolve the method signature
outside of the cc file.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Use std::unordered_map instead of boost:bimap. std::unordered_map is
much much easier to use. It is a premature optimization to user bimap.
We can iterate the map to check if host_id is unique. Modification of
host_id is not a frequent or performance sensitive operation anyway.
This adds an API that expose the token to endpoint mapping and the token
that are associate with an address.
Both method will be used by the API to expose the tokens.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch converts (for very small value of 'converts') some
replication related classes. Only static topology is supported (it is
created in keyspace::create_replication_strategy()). During mutation
no replication is done, since messaging service is not ready yet,
only endpoints are calculated.
