Commit Graph

8065 Commits

Author SHA1 Message Date
Avi Kivity
cf8ab65fbc Merge seastar upstream
* seastar 43e64c2...6f9453d (2):
  > Merge "rpc resource accounting"
  > core: Introduce smp::invoke_on_all()
2016-01-14 14:28:27 +02:00
Asias He
826b6ed877 gossip: Print node status in handle_major_state_change
Message-Id: <1452768680-32355-1-git-send-email-asias@scylladb.com>
2016-01-14 14:22:37 +02:00
Asias He
e7a899f5f3 gossip: Enable debug msg for convcit
Kill one FIXME in convict

Message-Id: <1452768680-32355-2-git-send-email-asias@scylladb.com>
2016-01-14 14:22:36 +02:00
Gleb Natapov
647a09cd7b storage_proxy: improve mutation timeout logging
Message-Id: <20160114105359.GY6705@scylladb.com>
2016-01-14 12:00:35 +01:00
Pekka Enberg
733584c44d main: Start the API service as the last step
This reverts commit f0d68e4 ("main: start the http server in the first
step"). The service layer is not ready to serve clients before it's
fully up and running which causes early startup crashes everywhere.
Message-Id: <1452768015-22763-1-git-send-email-penberg@scylladb.com>
2016-01-14 12:55:50 +02:00
Tomasz Grabiec
1daaf909d7 Merge branch 'tgrabiec/row_cache_invalidate_fix'
Fixes for wrap-around range handling in row_cache.
2016-01-14 11:38:26 +01:00
Takuya ASADA
7479cde28b dist: extend root disk size to 10GB
Since default root disk size is too small for our purpose, it's better to extend.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452762325-5620-1-git-send-email-syuu@scylladb.com>
2016-01-14 11:29:00 +02:00
Pekka Enberg
90123197e1 service/client_state: Use anonymous user when authentication is disabled
If authentication is disabled, nobody calls login() to set the current
user. There's untranslated code in client_state constructor to do just
that.

Fixes "You have not logged in" errors when USE statement is executed
with authentication disabled.
Message-Id: <1452759946-13998-1-git-send-email-penberg@scylladb.com>
2016-01-14 09:29:33 +01:00
Avi Kivity
4143cf6385 Merge "Initial authenticator support" from Calle
"Add implementation of cassandra password authenticator, and user
password checking to CQL connections.

User/pwd are stored in system_auth table. Passwords are hashed
using glibc 'crypt_r'.

The latter is worth noting, as this is a difference compared to origin;
Origin uses Java bcrypt library for salt/hash, i.e. blowfish hashing.
Most glibc variants do _not_ have support for blowfish. To be 100%
compatible with imported origin tables we might need to add
bcrypt/blowfish sources into scylla (no packaged libs available afaict)

The code currently first attempts to use blowfish, if we happen to run
centos or Openwall, which has it compiled in. Otherwise we will fall
back to sha512, sha256 or even md5 depending on lib support.

To use:
* scylla.conf: authenticator=PasswordAuthenticator
* cqlsh -u cassandra -p cassandra

Not implemented (yet):
* "Authorizer", thus no KS/CF access checking
* CQL create/alter/delete user (create_user_statement etc). I.e. there is
  only a single user name; default "cassandra:cassandra" user/pwd combo"
2016-01-13 19:13:05 +02:00
Takuya ASADA
0511b02f90 dist: run scylla_prepare, scylla_stop on sudo
Since we changed uid on scylla-server.service to scylla, we need sudo for these scripts.

Fixes #783

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452704598-5292-1-git-send-email-syuu@scylladb.com>
2016-01-13 19:06:33 +02:00
Tomasz Grabiec
6b059fd828 row_cache: Guard against wrap-around range in make_reader() 2016-01-13 17:50:55 +01:00
Tomasz Grabiec
7fb0bc4e15 row_cache: Take the reclaim lock in invalidate()
It's needed to keep the iterators valid in case eviciton is triggered
somehwere in between. It probably isn't because destructors should not
allocate, but better be safe.
2016-01-13 17:50:55 +01:00
Tomasz Grabiec
5e05f63ee7 tests: Add more tests for row_cache::invalidate()
Regs #785.
2016-01-13 17:50:55 +01:00
Tomasz Grabiec
50cc0c162e row_cache: Make invalidate() handle wrap-around ranges
Currently for wrap around the "begin" iterator would not meet with the
"end" iterator, invoking undefined behavior in erase_and_dispose()
which results in a crash.

Fixes #785
2016-01-13 17:50:55 +01:00
Calle Wilund
8192384338 auth_test: Unit tests for auth objects 2016-01-13 15:37:39 +00:00
Calle Wilund
9e3295bc69 cql_test_env: Allow specifying db::config for the env 2016-01-13 15:35:37 +00:00
Calle Wilund
9ef05993ff config: Mark "authenticator" used + update description 2016-01-13 15:35:36 +00:00
Calle Wilund
1d811f1e8f transport::server: Add authentication support
If system autheticator object requires authentication, issue
a challenge to client, and process response.
2016-01-13 15:35:36 +00:00
Calle Wilund
1c30d37285 client_state: Add user object + login
Note: all actual authorization methods are still unimplemented.
2016-01-13 15:35:36 +00:00
Calle Wilund
4692f46b8d storage_service: Initialize auth system on start 2016-01-13 15:35:36 +00:00
Calle Wilund
9a4d45e19d auth::auth/authenticator: user storage and authentication
User db storage + login/pwd db using system tables.

Authenticator object is a global shard-shared singleton, assumed
to be completely immutable, thus safe.
Actual login authentication is done via locally created stateful object
(sasl challenge), that queries db.

Uses "crypt_r" for password hashing, vs. origins use of bcrypt.
Main reason is that bcrypt does not exist as any consistent package
that can be consumed, so to guarantee full compatibility we'd have
to include the source. Not hard, but at least initially more work than
worth.
2016-01-13 15:35:35 +00:00
Calle Wilund
00de63c920 cql3::query_processor: Add processing helpers for internal usage
syntactical sugar + "process" for internal, similar to 
execute_internal, but allowing querying the whole cluster, and optional
statement caching.
2016-01-13 15:35:21 +00:00
Calle Wilund
6a5f075107 batch_statement: Modify verify_batch_size to match current origin
Fixes #614

* Use warning threshold from config
* Don't throw exceptions. We're only supposed to warn.
* Try to actually estimate mutation data payload size, not
  number of mutations.
Message-Id: <1452615759-23213-1-git-send-email-calle@scylladb.com>
2016-01-13 12:26:49 +01:00
Calle Wilund
32e480025f cql3::query_options: Add constructors for internal processing 2016-01-13 08:49:01 +00:00
Calle Wilund
2e9ab3aff1 types.hh: Add data_type_for<bool> 2016-01-13 08:49:01 +00:00
Calle Wilund
40efd231b1 auth::authenticated_user: Object representing a named or anon user 2016-01-13 08:49:01 +00:00
Calle Wilund
51af2bcafd auth::permission: permissions for authorization
Not actually used yet. But some day...
2016-01-13 08:49:01 +00:00
Calle Wilund
6f708eae1c auth::data_resource: resource identifier for auth permissions 2016-01-13 08:49:01 +00:00
Calle Wilund
9c1d088718 exceptions: add authorization exceptions 2016-01-13 08:49:01 +00:00
Calle Wilund
cd4ae7a81e Merge branch 'master' of https://github.com/scylladb/scylla 2016-01-13 08:48:43 +00:00
Tomasz Grabiec
e88f41fb3f messaging_service: Move REPAIR_CHECKSUM_RANGE verb out of the streaming verbs group
Message-Id: <1452620321-17223-1-git-send-email-tgrabiec@scylladb.com>
2016-01-12 20:17:08 +02:00
Calle Wilund
8de95cdee8 paging bugfix: Allow reset/removal of "specific ck range"
Refs #752

Paged aggregate queries will re-use the partition_slice object,
thus when setting a specific ck range for "last pk", we will hit
an exception case.
Allow removing entries (actually only the one), and overwriting
(using schema equality for keys), so we maintain the interface
while allowing the pager code to re-set the ck range for previous
page pass.

[tgrabiec: commit log cleanup, fixed issue ref]

Message-Id: <1452616259-23751-1-git-send-email-calle@scylladb.com>
2016-01-12 17:45:57 +01:00
Calle Wilund
7d7d592665 batch_statement: Modify verify_batch_size to match current origin
Fixes #614

* Use warning threshold from config
* Don't throw exceptions. We're only supposed to warn.
* Try to actually estimate mutation data payload size, not
  number of mutations.
2016-01-12 16:30:31 +00:00
Calle Wilund
81e9dc0c2a paging bugfix: Ensure limit for single page is min(page size, limit left)
Fixes #752

We set row limit for query to be min of page size/remaining in limit,
but if we have a multinode query we might end up with more rows than asked
for, so must do this again in post-processing.
2016-01-12 16:30:30 +00:00
Calle Wilund
ea92d7d4fd paging bugfix: Allow reset/removal of "specific ck range"
Refs #792

Paged aggregate queries will re-use the partition_slice object,
thus when setting a specific ck range for "last pk", we will hit
an exception case.
Allow removing entries (actually only the one), and overwriting
(using schema equality for keys), so we maintain the interface
while allowing the pager code to re-set the ck range for previous
page pass. 

v2: 
* Changed to schema-equality checks so we sort of maintain a 
  sane api and behaviour, even with the 1-entry map
 
v3: 
* Renamed remove "contains" in specific_ranges, and made the calling
  code use more map-like logic, again to keep things cleaner
2016-01-12 16:30:30 +00:00
Calle Wilund
e50d8b6895 paging bugfix: Ensure limit for single page is min(page size, limit left)
Fixes #752

We set row limit for query to be min of page size/remaining in limit,
but if we have a multinode query we might end up with more rows than asked
for, so must do this again in post-processing.

Message-Id: <1452606935-12899-2-git-send-email-calle@scylladb.com>
2016-01-12 17:23:04 +02:00
Vlad Zolotarov
9232ad927f messaging_service::get_rpc_client(): fix the encryption logic
According to specification
(here https://wiki.apache.org/cassandra/InternodeEncryption)
when the internode encryption is set to `dc` the data passed between
DCs should be encrypted and similarly, when it's set to `rack`
the inter-rack traffic should encrypted.

Currently Scylla would encrypt the traffic inside a local DC in the
first case and inside the local RACK in the later one.

This patch fixes the encryption logic to follow the specification
above.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1452501794-23232-1-git-send-email-vladz@cloudius-systems.com>
2016-01-12 16:22:26 +02:00
Avi Kivity
4693197e37 Merge seastar upstream
* seastar fe7a49c...43e64c2 (1):
  > resource: fix failures on low-memory machines

Fixes #734.
2016-01-12 14:45:43 +02:00
Calle Wilund
5b9f196115 Merge branch 'master' of https://github.com/scylladb/scylla 2016-01-12 11:46:40 +00:00
Avi Kivity
39f81b95d6 main: make --developer-mode relax dma requirements
With Docker we might be running on a filesystem that does not support DMA
(aufs; or tmpfs on boot2docker), so let --developer-mode allow running
on those file systems.
Message-Id: <1452593083-25601-1-git-send-email-avi@scylladb.com>
2016-01-12 13:34:46 +02:00
Avi Kivity
d68026716e Merge seastar upstream
* seastar ad3577b...fe7a49c (2):
  > reactor: workaround tmpfs O_DIRECT vs O_EXCL bug
  > rpc: fix reordering between sending client's negotiation frame and user's data
2016-01-12 13:27:16 +02:00
Takuya ASADA
a1d1d0bd06 Revert "dist: prevent 'local rpm' AMI image update to older version of scylla package by yum update"
This reverts commit b28b8147a0.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452592877-29721-2-git-send-email-syuu@scylladb.com>
2016-01-12 12:26:09 +02:00
Takuya ASADA
5459df1e9e dist: renumber development version as 666.development
yum command think "development-xxxx.xxxx" is older than "0.x", so nightly package mistakenly update with release version.
To prevent this problem, we should add greater number prior to "development".
Also same on Ubuntu package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452592877-29721-1-git-send-email-syuu@scylladb.com>
2016-01-12 12:26:08 +02:00
Calle Wilund
fdda880920 Merge branch 'master' of https://github.com/scylladb/scylla 2016-01-12 10:17:22 +00:00
Avi Kivity
5809ed476f Merge "Orderly service startup for systemd"
Use systemd Type=notify to tell systemd about startup progress.

We can now use 'systemctl status scylla-server' to see where we are
in service startup, and 'systemctl start scylla-server' will wait until
either startup is complete, or we fail to start up.
2016-01-12 12:01:32 +02:00
Avi Kivity
3d5f6de683 main: notify systemd of startup progress
Send current startup stage via sd_notify STATUS variable; let it know that
startup is complete via READY=1.

Fixes #760.
2016-01-12 11:58:24 +02:00
Calle Wilund
1b54b9c2d8 Merge branch 'master' of https://github.com/scylladb/scylla 2016-01-12 09:02:05 +00:00
Calle Wilund
7f4985a017 commit log reader bugfix: Fix tried to read entries across chunk bounds
read_entry did not verify that current chunk has enough data left
for a minimal entry. Thus we could try to read an entry from the slack
left in a chunk, and get lost in the file (pos > next, skip very much
-> eof). And also give false errors about corruption.
Message-Id: <1452517700-599-1-git-send-email-calle@scylladb.com>
2016-01-12 10:29:07 +02:00
Tzach Livyatan
c5b332716c Fix AMI prompt from "nodetool --help" to "nodetool help"
Fixes #775

Signed-off-by: Tzach Livyatan <tzach@scylladb.com>
Message-Id: <1452586945-28738-1-git-send-email-tzach@scylladb.com>
2016-01-12 10:27:05 +02:00
Takuya ASADA
fc13b9eb66 dist: yum install epel-release before installing CentOS dependencies
Fixes #779

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452586442-19777-1-git-send-email-syuu@scylladb.com>
2016-01-12 10:24:56 +02:00