This change generalizes the implementation of a `resource` to many
different kinds of resources, though there is still only one
kind (`data`). In the future, we also expect resource kinds for roles,
user-defined functions (UDFs), and possibly on particular REST
end-points.
I considered several approaches to generalizing to different kinds of
resources.
One approach is to have a base class that is inherited from by different
resource kinds. The common functionality would be accessed through
virtual member functions and kind-specific functions would exist in
sub-classes. I rejected this approach because dealing with different
kinds of resources uniformly requires storage and life-time management
through something like `std::unique_ptr<auth::resource>`, which means
that we lose value semantics (including comparison) and must deal with
complications around ownership.
Another option was to use `boost::variant` (or, in future,
`std::variant`). This is closer to what we want, since there a static
set of resource kinds that we support. I rejected this approach for two
reasons. The first is that all resource kinds share the same data (a
list of segments and a root identifier), which would be duplicated in
each type that composed the variant. The second is that the complexity
and source-code overhead of `boost::variant` didn't seem warranted.
The solution I ended up with is home-grown variant. All resources are
described in the same `final` class: `auth::resource`. This class has
value semantics, supports equality comparison, and has a strict
ordering. All resources have in common a tag ("kind") and a list of
parts. Most operations on resources don't care about the kind of
resource (like getting its name, parsing a name, querying for the
parent, etc). These are just member functions of the class.
When we care about a kind-specific interpretation of a resource, we can
produce a "view" of the resource. For example, `data_resource_view`
allows for accessing the (optional) keyspace and table names.
I anticipate in the future to add functions for creating role
resources (`auth::resource::role`) and also `role_resource_view`.
The functional behaviour of the system should be unchanged with this
patch.
I've added new unit tests in `auth_resource_test.cc` and removed the old
test from `auth_test.cc`.
Fixes#3027.
"This is CASSANDRA-7886 and CASSANDRA-8592. The patch series detects
that CL of a request can no longer be reached due to errors and fails
the request earlier. New type of errors are reported: read/write failure
which were introduced in cql v4 protocol. For compatibility if older
protocol is used the error is translated to timeout error."
* 'gleb/request-failure_v2' of github.com:scylladb/seastar-dev:
storage_proxy: fail read/write requests early if it cannot be completed due to errors
storage_service: add WRITE_FAILURE_REPLY_FEATURE feature
gossiper: add node_has_feature() function
cql: add read/write failure exceptions
storage_proxy: fix data presence reporting in read timeout error during
storage_proxy: remove inheritance from enable_shared_from_this for abstract_write_response_handler
storage_proxy: remove unneeded field in abstract_write_response_handler
storage_proxy: fix pending endpoint accounting for EACH_QUORUM
consistency_level: constify quorum_for() and local_quorum_for()
When fast forwarding is enabled and all readers positioned inside the
current partition return EOS, return EOS from the combined-reader
too. Instead of skipping to the next partition if there are idle readers
(positioned at some later partition) available. This will cause rows to
be skipped in some cases.
The fix is to distinguish EOS'd readers that are only halted (waiting
for a fast-forward) from thoose really out of data. To achieve this we
track the last fragment-kind the reader emitted. If that was a
partition-end then the reader is out of data, otherwise it might emit
more fragments after a fast-forward. Without this additional information
it is impossible to determine why a reader reached EOS and the code
later may make the wrong decision about whether the combined-reader as
a whole is at EOS or not.
Also when fast-forwarding between partition-ranges or calling
next_partition() we set the last fragment-kind of forwarded readers
because they should emit a partition-start, otherwise they are out of
data.
Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <6f0b21b1ec62e1197de6b46510d5508cdb4a6977.1512569218.git.bdenes@scylladb.com>
Since pbuilder chroot environment does not install CA certificates by default,
accessing https://download.opensuse.org will cause certificate verification
error.
So we need to install it before installing 3rdparty repo GPG key.
Also, checking existance of gpgkeys_curl is not needed, since it's always
not installed since we are running the script in clean chroot environment.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512517001-27524-1-git-send-email-syuu@scylladb.com>
"This fix for the issue #2989 first adds unit tests for caching_options which
is the only class that uses the helpers from json.hh. This is done to
have regression tests in place for the main change.
The second commit adds conditional use of new recommended JsonCpp API
where available. For older versions of the library, it uses the old
code."
* 'issues/2989/v1' of https://github.com/argenet/scylla:
Use CharReaderBuilder/CharReader and StreamWriterBuilder from JsonCpp.
tests: Add unit tests for caching_options.
"This series makes sstable tests use flat stream interface. The main
motivation is to allow eventual removal of mutation_reader and
streamed_mutation and ensuring that the conversion between the
interfaces doesn't hide any bugs that would be otherwise found."
* tag 'flat_mutation_reader-sstable-tests/v1' of https://github.com/pdziepak/scylla:
sstables: drop read_range_rows()
tests/mutation_reader: stop using read_range_rows()
incremental_reader_selector: do not use read_range_rows()
tests/sstable: stop using read_range_rows()
sstables: drop read_row()
tests/sstables: use read_row_flat() instead of read_row()
database: use read_row_flat() instead of read_row()
tests/sstable_mutation_test: get flat_mutation_readers from mutation sources
tests/sstables: make sstable_reader return flat_mutation_reader
sstable: drop read_row() overload accepting sstable::key
tests/sstable: stop using read_row() with sstable::key
tests/flat_mutation_reader_assertions: add has_monotonic_positions()
tests/flat_mutation_reader_assertions: add produces(Range)
tests/flat_mutation_reader_assertions: add produces(mutation)
tests/flat_mutation_reader_assertions: add produces(dht::decorated_key)
tests/flat_mutation_reader_assertions: add produces(mutation_fragment::kind)
tests/flat_mutation_reader_assertions: fix fast forwarding
The assertions already have produces(mutation) and
produces(dht::decorated_key) overloads. Additional overload that accepts
a range of elements will allow to check if a range of mutations of
decorated keys is produced.
The same interface is exposed by mutation_reader_assertions.
produces(mutation_fragment::kind) is provided by
streamed_mutation_assertions and is going to be needed in order to
fully convert tests to the flat mutation readers.
Both fast_forward_to() overloads return a future which should be waited
for. Additionally, fast_forward_to(const dht::partition_range&) expects
the range to remain valid at least until the next call to
fast_forward_to(). The original mutation_reader_assertions guaranteed
that and so should flat_mutation_reader_assertions.
* seastar dc44656...ac78eec (3):
> json formatter: Add unsigned support to the json formatter
> Add missing usual smart-pointer methods to foreign_ptr
> future-util: remove use of forward references in some primitives
Recently, memtable flush in test requires storage service for tests,
or it fails with "Assertion `local_is_initialized()' failed".
storage_service_for_tests needs to run in a thread, that's why
flush_memtable was flattened.
Last but not least, we need to revert flushed memory account because
same memtable is used for all sstables in the perf test so as not
to trigger `_mt._flushed_memory <= _mt.occupancy().used_space()'
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20171205012853.21559-1-raphaelsc@scylladb.com>
In version 1.8.3 of JsonCpp shipped with Fedora 27, old FastWriter and
Reader classes from JsonCpp have been deprecated in favour of
newer/better ones: CharReaderBuilder/CharReader and
StreamWriterBuilder/StreamWriter.
This fix uses the new classes where available or resorts to old ones for
older versions of the library.
Fixes#2989
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
"Convert combined_mutation_reader into a flat_mutation_reader impl. For
now - in the name of incremental progress - all consumers are updated to
use the combined reader through the
mutation_reader_from_flat_mutation_reader adaptor. The combined reader also
uses all it's sub mutation_readers through the
flat_mutation_reader_from_mutation_reader adaptor."
* 'bdenes/flatten-combined-reader-v8' of https://github.com/denesb/scylla:
Add unit tests for the combined reader - selector interactions
Add flat_mutation_reader overload of make_combined_reader
Flatten the implementation of combined_mutation_reader
Add mutation_fragment_merger
mutation_fragment::apply(): handle partition start and end too
Add non-const overload of partition_start::partition_tombstone()
Make combined_mutation_reader a flat_mutation_reader
Move the mutation merging logic to combined_mutation_reader
Remove the unnecessary indirection of mutation_reader_merger::next()
Move the implementation of combined_mutation_reader into mutation_reader_merger
Remove unused mutation_and_reader::less_compare and operator<
Our external repos are already signed repo, so let's enable secure-apt.
Seems like more recent version of Ubuntu (tested on 18.04) does not accept
skipping GPG check, so we need it anyway in near future.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
