Commit Graph

9202 Commits

Author SHA1 Message Date
Duarte Nunes
f8d8dbdeb7 types: Don't wrap tombstone in an std::optional
All the callers of do_serialize_mutation_form pass a valid tombstone
that is converted into a non-empty optional. This happens even if the
tombstone is empty (tombstone::timestamp == api::missing_timestamp).

This patch fixes this by passing in a reference to the tombstone which
is convertible to bool, based on whether it is empty or not.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1460620528-3628-1-git-send-email-duarte@scylladb.com>
2016-04-20 09:22:01 +02:00
Duarte Nunes
40c1b29701 cql3: Implement contains relation
Although it doesn't work in the absence of secondary indexes,
now we provide the same error messages as origin when trying to use
the contains relation.

Fixes #1158

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1461088626-26958-1-git-send-email-duarte@scylladb.com>
2016-04-20 09:22:25 +03:00
Pekka Enberg
4ed702f0da Merge "Authorizer support" from Calle
"Conversion/implementation of "authorizer" code from origin, handling
 permissions management for users/resources.

 Default implementation keeps mapping of <user.resource>->{permissions}
 in a table, contents of which is cached for slightly quicker checks.

 Adds access control to all (existing) cql statements.
 Adds access management support to the CQL impl. (GRANT/REVOKE/LIST)

 Verified manually and with dtest auth_test.py. Note that several of these
 still fail due to (unrelated) unimplemented features, like index, types
 etc.

 Fixes #1138"
2016-04-19 15:00:38 +03:00
Calle Wilund
4c246b5cc3 scylla.yaml: Move authorizer/authenticator options to supported section 2016-04-19 11:49:06 +00:00
Calle Wilund
9ed25a970e Cql.g: Permission statements parsing 2016-04-19 11:49:06 +00:00
Calle Wilund
3b101c6e19 cql3::statements::drop_user_statement: Drop all permissions for user 2016-04-19 11:49:06 +00:00
Calle Wilund
14cc47d8b9 cql3::statements::revoke_statement: Initial conversion 2016-04-19 11:49:06 +00:00
Calle Wilund
4e1ef3c1bc cql3::statements::grant_statement: Initial conversion 2016-04-19 11:49:05 +00:00
Calle Wilund
04c37def3a cql3::statements::list_permissions_statement: Initial conversion 2016-04-19 11:49:05 +00:00
Calle Wilund
fe23447f6f cql3::statements::permission_altering_statement: Inital conversion
Alter permission base typ
2016-04-19 11:49:05 +00:00
Calle Wilund
add2111c0a cql3::statements::authorizarion_statement: Initial conversion
Auth cql base type
2016-04-19 11:49:05 +00:00
Calle Wilund
3906dc9f0d cql3::statements: Change check_access to future<> + implement 2016-04-19 11:49:05 +00:00
Calle Wilund
dac6cf69eb service::client_state: Add authorization checkers 2016-04-19 11:49:05 +00:00
Calle Wilund
072acc68da validation: Add KS validation + convinence methods
Looking up local db.
2016-04-19 11:49:05 +00:00
Calle Wilund
a7e1af1c06 db::config: Add permissions cache entries/mark auth/perm as used 2016-04-19 11:49:05 +00:00
Calle Wilund
36bb40c205 auth::auth: Add authorizer initialization + permissions getter
Create and init authorizer object on start. Create thread local
permissions cache to front end the actual authorizer.
2016-04-19 11:49:05 +00:00
Calle Wilund
03568d0325 tests::cql_test_env: Fake logged in user in case test requires is. 2016-04-19 11:49:05 +00:00
Calle Wilund
ead1c882f8 utils::loading_cache: Version of the LoadingCache type used in origin
Simple, expiring, cache of potentially limited number of entries.
2016-04-19 11:49:05 +00:00
Calle Wilund
956ee87e12 auth::authenticator: Change "protected_resources" to return reference
It it an immutable static value anyway.
2016-04-19 11:49:05 +00:00
Calle Wilund
1f0bbf2d9a auth::authorizer: Initial conversion
Main authorization endpoint. Default (and only) real authorizer
keeps a mapping resource -> permission sets in system table
2016-04-19 11:49:04 +00:00
Benoît Canet
e17795d2dd scylla_dev_mode_setup: Unify --developer-mode prompt and parsing
Fixes: #1194

Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1461002978-5379-2-git-send-email-benoit@scylladb.com>
2016-04-19 09:38:03 +03:00
Takuya ASADA
f6252be0c1 utils: fix compilation error on utils/exceptions.hh
It doesn't able to find std::system_error due to missing header.

Fixes #1202

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461006884-28316-1-git-send-email-syuu@scylladb.com>
2016-04-19 09:37:31 +03:00
Raphael S. Carvalho
bf03cd1ea6 sstables: kill unused code from size tiered strategy
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <485b1e49419cb052218ab4558f27270ce3bd03b4.1460761821.git.raphaelsc@scylladb.com>
2016-04-19 08:46:06 +03:00
Raphael S. Carvalho
29db5f5e1f sstables: move compaction strategy code to a new source file
Moving compaction strategy code from sstables/compaction.cc to
sstables/compaction_strategy.cc
That improves readability. Strategy code should be separated
from the generic compaction code.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <5af6fc8f7321351a071fc0ce03c80ffea21f8396.1460761821.git.raphaelsc@scylladb.com>
2016-04-19 08:45:43 +03:00
Pekka Enberg
a3a0404d33 Merge seastar upstream
* seastar 2185f37...2b3c363 (1):
  > net/tls: Fix compilation with older GnuTLS versions
2016-04-19 08:43:36 +03:00
Calle Wilund
c446fe50e6 tuple_hash: Add convinence operator for two arguments (non-pair) 2016-04-18 13:51:15 +00:00
Calle Wilund
f0d2efd206 data_value: Add constructor from unordered_set<> 2016-04-18 13:51:15 +00:00
Calle Wilund
690c7207fe cql3::untyped_result_set: Add get_set<> method
Gets a value as a, you guessed it, set.
2016-04-18 13:51:15 +00:00
Calle Wilund
443af44f24 log: Add output operator for std::exception&/std::system_error& 2016-04-18 13:51:15 +00:00
Calle Wilund
ca7d339110 auth::authenticated_user: Add copy/move constructors 2016-04-18 13:51:15 +00:00
Calle Wilund
d3a9650646 auth::permission_set: Add < operator 2016-04-18 13:51:15 +00:00
Calle Wilund
c93d114949 auth::permission: Add stringizers + move sets into namespace 2016-04-18 13:51:15 +00:00
Calle Wilund
6e09920f93 auth::data_resource: Fix to_string to match origin 2016-04-18 13:51:15 +00:00
Calle Wilund
bb96e5bd66 auth::data_resource: Move declaration of "resource_ids" 2016-04-18 13:51:15 +00:00
Takuya ASADA
2eb91421eb dist/ami: Show correct login message when scylla-ami-setup.service is still running
While scylla-ami-setup.service is running, login message says "run systemctl status scylla-server" to see status, but it actually never launched yet.

This patch fixes the message to notice RAID construction is running, and 'systemctl status scylla-ami-setup' is the correct way to see status.

Fixes #1035

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460660628-10103-2-git-send-email-syuu@scylladb.com>
2016-04-18 15:22:02 +03:00
Takuya ASADA
07a6057c03 dist/ami: fix incorrect service name on .bash_profile
Ubuntu's service name on .bash_profile is incorrect, fix it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460660628-10103-1-git-send-email-syuu@scylladb.com>
2016-04-18 15:21:48 +03:00
Tomasz Grabiec
45527fcffa Merge branch 'glommer/issue-1144-v5'
From Glauber:

There are current some outstanding issues with the throttling code. It's
easier to see them with the streaming code, but at least one of them is general.

One of them is related to situations in which the amount of memory available
leaves only one memtable fitting in memory. That would only happen with the
general code if we set the memtable cleanup threshold to 100 % - and I don't
even know if it is valid - but will happen quite often with the streaming code.
If that happens, we'll start throttling when that memtable is being written,
but won't be able to put anything else in its place - leading to unnecessary
throttling.

The second, and more serious, happens when we start throttling and the amount
of available memory is not at least 1MB. This can deadlock the database in
the sense that it will prevent any request from continuing, and in turn causing
a flush due to memtable size. It is a good practice anyway to always guarantee
progress.

Fixes #1144
2016-04-18 12:20:13 +02:00
Gleb Natapov
f3b515052b udt: fix error generation if accessed type is not udt
Fixes #1198
Message-Id: <1460884314-3717-2-git-send-email-gleb@scylladb.com>
2016-04-18 12:45:03 +03:00
Duarte Nunes
ece89069dd udt: Implement to_string() for selectable
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1460884314-3717-1-git-send-email-gleb@scylladb.com>
2016-04-18 12:44:48 +03:00
Pekka Enberg
edf7f098e2 Merge "Fix query of collection cell with all items deleted" from Tomek 2016-04-18 11:01:24 +03:00
Tomasz Grabiec
2e08d0f698 Merge branch 'dev/gleb/logging'
Logging improvements from Gleb.
2016-04-15 19:03:44 +02:00
Tomasz Grabiec
89bc32b020 tests: Add test for query of collection with deleted item 2016-04-15 18:14:05 +02:00
Tomasz Grabiec
c69d0a8e87 mutation_partition: Fix collection emptiness check
Broken by f15c380a4f.

This resulted in empty collection being returned in the results
instead of no collection.

Fixes org.apache.cassandra.cql3.validation.entities.CollectionsTest
from cassandra-unit-tests.
2016-04-15 18:14:05 +02:00
Tomasz Grabiec
b0d4782016 types: Add default argument values to is_any_live() 2016-04-15 18:14:05 +02:00
Avi Kivity
0de32ab120 Merge seastar upstream
* seastar 2aeb9dd...2185f37 (15):
  > reactor: avoid issuing systemwide memory barriers in parallel
  > Revert "Use sys_membarrier() when available"
  > Merge "Various exception-safety fixes" from Tomasz
  > future-util: make map reduce exception safe
  > collectd: do not give up after a failure
  > future-util: make repeat_until_value exception safe
  > rpc: do not block connection when unknown verbs is received
  > rpc: do not wait for a reply after timeout
  > rpc: move connection stats to base class
  > core/reactor: Handle io_submit failures inside flush_pending_aio
  > apps/iotune: add --fs-check option to use iotune for kernel version check
  > Merge "Some exception safety patches" from Paweł
  > tls: Fix conversion of dh_params::level to gnutls_sec_param_t
  > core: posix_thread: Mark start_routine as noexcept
  > fair_queue: better overflow protection
2016-04-15 16:06:53 +03:00
Pekka Enberg
3f2286d02e Merge "Delete compacted sstables atomically" from Avi
"If we compact sstables A, B into a new sstable C we must either delete both
A and B, or none of them.  This is because a tombstone in B may delete data
in A, and during compaction, both the tombstone and the data are removed.
If only B is deleted, then the data gets resurrected.

Non-atomic deletion occurs because the filesystem does not support atomic
deletion of multiple files; but the window for that is small and is not
addressed in this patchset.  Another case is when A is shared across
multiple shards (as is the case when changing shard count, or migrating
from existing Cassandra sstables).  This case is covered by this patchset.

Fixes #1181."
2016-04-14 22:04:15 +03:00
Glauber Costa
9c87ae3496 throttle: always release at least one request if we are below the limit
Our current throttling code releases one requests per 1MB of memory available
that we have. If we are below the memory limit, but not by 1MB or more, then
we will keep getting to unthrottle, but never really do anything.

If another memtable is close to the flushing point, those requests may be
exactly the ones that would make it flush. Without them, we'll freeze the
database.

In general, we need to always release at least one request to make sure that
progress is always achieved.

This fixes #1144

Signed-off-by: Glauber Costa <glauber@scylladb.com>
2016-04-14 13:13:15 -04:00
Gleb Natapov
9801d69d53 storage_proxy: add query result row count to brief format
Report number of rows in brief reporting format, but only if
we can count them without linearizing result's buffer.
2016-04-14 19:26:00 +03:00
Gleb Natapov
53993527ed storage_proxy: move verbose query result printing into separate logger
If query result is large tracing cannot be done since printing the result takes
too much time and space.
2016-04-14 19:26:00 +03:00
Gleb Natapov
46e5d05220 storage_proxy: cleanup query logging.
Since commit c1cffd06 logger catch errors internally, so no need to
catch most of them at the top level. Only those that can happen during
parameter evaluation can reach here. Change parameters to not throw
too.
2016-04-14 19:26:00 +03:00