Before this change, if a read executor had just enough targets to achieve query's CL, and there was a connection drop (e.g. node failure), the read executor waited for the entire request timeout to give drivers time to execute a speculative read in a meantime. Such behavior don't work well when a very long query timeout (e.g. 1800s) is set, because the unfinished request blocks topology changes. This change implements a mechanism to thrown a new read_failure_exception_with_timeout in the aforementioned scenario. The exception is caught by CQL server which conducts the waiting, after ERM is released. The new exception inherits from read_failure_exception, because layers that don't catch the exception (such as mapreduce service) should handle the exception just a regular read_failure. However, when CQL server catch the exception, it returns read_timeout_exception to the client because after additional waiting such an error message is more appropriate (read_timeout_exception was also returned before this change was introduced). This change: - Add new read_failure_exception_with_timeout exception - Add throw of read_failure_exception_with_timeout in storage_proxy - Add abort_source to CQL server, as well as to_stop() method for the correct abort handling - Add sleep in CQL server when the new exception is caught Refs #21831
47 lines
1.4 KiB
C++
47 lines
1.4 KiB
C++
/*
|
|
* Copyright (C) 2022-present ScyllaDB
|
|
*/
|
|
|
|
/*
|
|
* SPDX-License-Identifier: (LicenseRef-ScyllaDB-Source-Available-1.0 and Apache-2.0)
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
#include "exceptions.hh"
|
|
#include <boost/outcome/result.hpp>
|
|
#include "utils/exception_container.hh"
|
|
#include "utils/result.hh"
|
|
|
|
namespace exceptions {
|
|
|
|
// Allows to pass a coordinator exception as a value. With coordinator_result,
|
|
// it is possible to handle exceptions and inspect their type/value without
|
|
// resorting to costly rethrows. On the other hand, using them is more
|
|
// cumbersome than just using exceptions and exception futures.
|
|
//
|
|
// Not all exceptions are passed in this way, therefore the container
|
|
// does not allow all types of coordinator exceptions. On the other hand,
|
|
// an exception being listed here does not mean it is _always_ passed
|
|
// in an exception_container - it can be thrown in a regular fashion
|
|
// as well.
|
|
//
|
|
// It is advised to use this mechanism mainly for exceptions which can
|
|
// happen frequently, e.g. signalling timeouts, overloads or rate limits.
|
|
using coordinator_exception_container = utils::exception_container<
|
|
mutation_write_timeout_exception,
|
|
read_timeout_exception,
|
|
read_failure_exception,
|
|
rate_limit_exception,
|
|
overloaded_exception,
|
|
read_failure_exception_with_timeout
|
|
>;
|
|
|
|
template<typename T = void>
|
|
using coordinator_result = bo::result<T,
|
|
coordinator_exception_container,
|
|
utils::exception_container_throw_policy
|
|
>;
|
|
|
|
}
|