Commit Graph

8073 Commits

Author SHA1 Message Date
Pekka Enberg
bb0aeb9bb2 api/messaging_service: Fix heap-buffer-overflows in set_messaging_service()
Fix various issues in set_messaging_service() that caused
heap-buffer-overflows when JMX proxy connects to Scylla API:

  - Off-by-one error in 'num_verb' definition

  - Call to initializer list std::vector constructor variant that caused
    the vector to be two elements long.

  - Missing verb definitions from the Swagger definition that caused
    response vector to be too small.

Spotted by ASan.
Message-Id: <1453125439-16703-1-git-send-email-penberg@scylladb.com>
2016-01-20 10:29:27 +02:00
Takuya ASADA
d2c97d9620 dist: use our own CentOS7 Base image
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453241256-23338-4-git-send-email-syuu@scylladb.com>
2016-01-20 09:41:39 +02:00
Takuya ASADA
ddbe20f65c dist: stop ntpd before running ntpdate
New CentOS Base Image runs ntpd by default, so shutdown it before running ntpdate.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453241256-23338-3-git-send-email-syuu@scylladb.com>
2016-01-20 09:41:33 +02:00
Takuya ASADA
88bf12aa0b dist: disable SELinux only when it enabled
New CentOS7 Base Image disabled SELinux by default, and running 'setenforce 0' on the image causes error, we won't able to build AMI.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453241256-23338-2-git-send-email-syuu@scylladb.com>
2016-01-20 09:41:29 +02:00
Takuya ASADA
87fdf2ee0d dist: extend coredump size limit
16GB is not enough for some larger machines, so extend it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453115792-21989-2-git-send-email-syuu@scylladb.com>
2016-01-18 13:38:56 +02:00
Takuya ASADA
d6992189ed dist: preserve environment variable when running scylla_prepare on sudo
sysconfig parameters are passed via environment variables, but sudo resets it by default.
Need to keep them beyond sudo.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453115792-21989-1-git-send-email-syuu@scylladb.com>
2016-01-18 13:23:55 +02:00
Tomasz Grabiec
5bf1afa059 config: Set default logging level to info
Commit d7b403db1f changed the default in
logging::logger. It affected tests but not scylla binary, where it's
being overwritten in main.cc.
Message-Id: <1452777008-21708-1-git-send-email-tgrabiec@scylladb.com>
2016-01-14 15:12:28 +02:00
Tomasz Grabiec
b013ed6357 cql3: Disable ALTER TABLE unless experimental features are on 2016-01-14 14:32:15 +02:00
Tomasz Grabiec
d4d0dd9cda tests: cql_test_env: Enable experimental features 2016-01-14 14:32:10 +02:00
Tomasz Grabiec
5865a43400 config: Add 'experimental' switch 2016-01-14 14:32:05 +02:00
Pekka Enberg
b81292d5d2 release: prepare for 0.16 2016-01-14 13:21:50 +02:00
Gleb Natapov
647a09cd7b storage_proxy: improve mutation timeout logging
Message-Id: <20160114105359.GY6705@scylladb.com>
2016-01-14 12:00:35 +01:00
Pekka Enberg
733584c44d main: Start the API service as the last step
This reverts commit f0d68e4 ("main: start the http server in the first
step"). The service layer is not ready to serve clients before it's
fully up and running which causes early startup crashes everywhere.
Message-Id: <1452768015-22763-1-git-send-email-penberg@scylladb.com>
2016-01-14 12:55:50 +02:00
Tomasz Grabiec
1daaf909d7 Merge branch 'tgrabiec/row_cache_invalidate_fix'
Fixes for wrap-around range handling in row_cache.
2016-01-14 11:38:26 +01:00
Takuya ASADA
7479cde28b dist: extend root disk size to 10GB
Since default root disk size is too small for our purpose, it's better to extend.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452762325-5620-1-git-send-email-syuu@scylladb.com>
2016-01-14 11:29:00 +02:00
Pekka Enberg
90123197e1 service/client_state: Use anonymous user when authentication is disabled
If authentication is disabled, nobody calls login() to set the current
user. There's untranslated code in client_state constructor to do just
that.

Fixes "You have not logged in" errors when USE statement is executed
with authentication disabled.
Message-Id: <1452759946-13998-1-git-send-email-penberg@scylladb.com>
2016-01-14 09:29:33 +01:00
Avi Kivity
4143cf6385 Merge "Initial authenticator support" from Calle
"Add implementation of cassandra password authenticator, and user
password checking to CQL connections.

User/pwd are stored in system_auth table. Passwords are hashed
using glibc 'crypt_r'.

The latter is worth noting, as this is a difference compared to origin;
Origin uses Java bcrypt library for salt/hash, i.e. blowfish hashing.
Most glibc variants do _not_ have support for blowfish. To be 100%
compatible with imported origin tables we might need to add
bcrypt/blowfish sources into scylla (no packaged libs available afaict)

The code currently first attempts to use blowfish, if we happen to run
centos or Openwall, which has it compiled in. Otherwise we will fall
back to sha512, sha256 or even md5 depending on lib support.

To use:
* scylla.conf: authenticator=PasswordAuthenticator
* cqlsh -u cassandra -p cassandra

Not implemented (yet):
* "Authorizer", thus no KS/CF access checking
* CQL create/alter/delete user (create_user_statement etc). I.e. there is
  only a single user name; default "cassandra:cassandra" user/pwd combo"
2016-01-13 19:13:05 +02:00
Takuya ASADA
0511b02f90 dist: run scylla_prepare, scylla_stop on sudo
Since we changed uid on scylla-server.service to scylla, we need sudo for these scripts.

Fixes #783

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452704598-5292-1-git-send-email-syuu@scylladb.com>
2016-01-13 19:06:33 +02:00
Tomasz Grabiec
6b059fd828 row_cache: Guard against wrap-around range in make_reader() 2016-01-13 17:50:55 +01:00
Tomasz Grabiec
7fb0bc4e15 row_cache: Take the reclaim lock in invalidate()
It's needed to keep the iterators valid in case eviciton is triggered
somehwere in between. It probably isn't because destructors should not
allocate, but better be safe.
2016-01-13 17:50:55 +01:00
Tomasz Grabiec
5e05f63ee7 tests: Add more tests for row_cache::invalidate()
Regs #785.
2016-01-13 17:50:55 +01:00
Tomasz Grabiec
50cc0c162e row_cache: Make invalidate() handle wrap-around ranges
Currently for wrap around the "begin" iterator would not meet with the
"end" iterator, invoking undefined behavior in erase_and_dispose()
which results in a crash.

Fixes #785
2016-01-13 17:50:55 +01:00
Calle Wilund
8192384338 auth_test: Unit tests for auth objects 2016-01-13 15:37:39 +00:00
Calle Wilund
9e3295bc69 cql_test_env: Allow specifying db::config for the env 2016-01-13 15:35:37 +00:00
Calle Wilund
9ef05993ff config: Mark "authenticator" used + update description 2016-01-13 15:35:36 +00:00
Calle Wilund
1d811f1e8f transport::server: Add authentication support
If system autheticator object requires authentication, issue
a challenge to client, and process response.
2016-01-13 15:35:36 +00:00
Calle Wilund
1c30d37285 client_state: Add user object + login
Note: all actual authorization methods are still unimplemented.
2016-01-13 15:35:36 +00:00
Calle Wilund
4692f46b8d storage_service: Initialize auth system on start 2016-01-13 15:35:36 +00:00
Calle Wilund
9a4d45e19d auth::auth/authenticator: user storage and authentication
User db storage + login/pwd db using system tables.

Authenticator object is a global shard-shared singleton, assumed
to be completely immutable, thus safe.
Actual login authentication is done via locally created stateful object
(sasl challenge), that queries db.

Uses "crypt_r" for password hashing, vs. origins use of bcrypt.
Main reason is that bcrypt does not exist as any consistent package
that can be consumed, so to guarantee full compatibility we'd have
to include the source. Not hard, but at least initially more work than
worth.
2016-01-13 15:35:35 +00:00
Calle Wilund
00de63c920 cql3::query_processor: Add processing helpers for internal usage
syntactical sugar + "process" for internal, similar to 
execute_internal, but allowing querying the whole cluster, and optional
statement caching.
2016-01-13 15:35:21 +00:00
Calle Wilund
6a5f075107 batch_statement: Modify verify_batch_size to match current origin
Fixes #614

* Use warning threshold from config
* Don't throw exceptions. We're only supposed to warn.
* Try to actually estimate mutation data payload size, not
  number of mutations.
Message-Id: <1452615759-23213-1-git-send-email-calle@scylladb.com>
2016-01-13 12:26:49 +01:00
Calle Wilund
32e480025f cql3::query_options: Add constructors for internal processing 2016-01-13 08:49:01 +00:00
Calle Wilund
2e9ab3aff1 types.hh: Add data_type_for<bool> 2016-01-13 08:49:01 +00:00
Calle Wilund
40efd231b1 auth::authenticated_user: Object representing a named or anon user 2016-01-13 08:49:01 +00:00
Calle Wilund
51af2bcafd auth::permission: permissions for authorization
Not actually used yet. But some day...
2016-01-13 08:49:01 +00:00
Calle Wilund
6f708eae1c auth::data_resource: resource identifier for auth permissions 2016-01-13 08:49:01 +00:00
Calle Wilund
9c1d088718 exceptions: add authorization exceptions 2016-01-13 08:49:01 +00:00
Calle Wilund
cd4ae7a81e Merge branch 'master' of https://github.com/scylladb/scylla 2016-01-13 08:48:43 +00:00
Tomasz Grabiec
e88f41fb3f messaging_service: Move REPAIR_CHECKSUM_RANGE verb out of the streaming verbs group
Message-Id: <1452620321-17223-1-git-send-email-tgrabiec@scylladb.com>
2016-01-12 20:17:08 +02:00
Calle Wilund
8de95cdee8 paging bugfix: Allow reset/removal of "specific ck range"
Refs #752

Paged aggregate queries will re-use the partition_slice object,
thus when setting a specific ck range for "last pk", we will hit
an exception case.
Allow removing entries (actually only the one), and overwriting
(using schema equality for keys), so we maintain the interface
while allowing the pager code to re-set the ck range for previous
page pass.

[tgrabiec: commit log cleanup, fixed issue ref]

Message-Id: <1452616259-23751-1-git-send-email-calle@scylladb.com>
2016-01-12 17:45:57 +01:00
Calle Wilund
7d7d592665 batch_statement: Modify verify_batch_size to match current origin
Fixes #614

* Use warning threshold from config
* Don't throw exceptions. We're only supposed to warn.
* Try to actually estimate mutation data payload size, not
  number of mutations.
2016-01-12 16:30:31 +00:00
Calle Wilund
81e9dc0c2a paging bugfix: Ensure limit for single page is min(page size, limit left)
Fixes #752

We set row limit for query to be min of page size/remaining in limit,
but if we have a multinode query we might end up with more rows than asked
for, so must do this again in post-processing.
2016-01-12 16:30:30 +00:00
Calle Wilund
ea92d7d4fd paging bugfix: Allow reset/removal of "specific ck range"
Refs #792

Paged aggregate queries will re-use the partition_slice object,
thus when setting a specific ck range for "last pk", we will hit
an exception case.
Allow removing entries (actually only the one), and overwriting
(using schema equality for keys), so we maintain the interface
while allowing the pager code to re-set the ck range for previous
page pass. 

v2: 
* Changed to schema-equality checks so we sort of maintain a 
  sane api and behaviour, even with the 1-entry map
 
v3: 
* Renamed remove "contains" in specific_ranges, and made the calling
  code use more map-like logic, again to keep things cleaner
2016-01-12 16:30:30 +00:00
Calle Wilund
e50d8b6895 paging bugfix: Ensure limit for single page is min(page size, limit left)
Fixes #752

We set row limit for query to be min of page size/remaining in limit,
but if we have a multinode query we might end up with more rows than asked
for, so must do this again in post-processing.

Message-Id: <1452606935-12899-2-git-send-email-calle@scylladb.com>
2016-01-12 17:23:04 +02:00
Vlad Zolotarov
9232ad927f messaging_service::get_rpc_client(): fix the encryption logic
According to specification
(here https://wiki.apache.org/cassandra/InternodeEncryption)
when the internode encryption is set to `dc` the data passed between
DCs should be encrypted and similarly, when it's set to `rack`
the inter-rack traffic should encrypted.

Currently Scylla would encrypt the traffic inside a local DC in the
first case and inside the local RACK in the later one.

This patch fixes the encryption logic to follow the specification
above.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1452501794-23232-1-git-send-email-vladz@cloudius-systems.com>
2016-01-12 16:22:26 +02:00
Avi Kivity
4693197e37 Merge seastar upstream
* seastar fe7a49c...43e64c2 (1):
  > resource: fix failures on low-memory machines

Fixes #734.
2016-01-12 14:45:43 +02:00
Calle Wilund
5b9f196115 Merge branch 'master' of https://github.com/scylladb/scylla 2016-01-12 11:46:40 +00:00
Avi Kivity
39f81b95d6 main: make --developer-mode relax dma requirements
With Docker we might be running on a filesystem that does not support DMA
(aufs; or tmpfs on boot2docker), so let --developer-mode allow running
on those file systems.
Message-Id: <1452593083-25601-1-git-send-email-avi@scylladb.com>
2016-01-12 13:34:46 +02:00
Avi Kivity
d68026716e Merge seastar upstream
* seastar ad3577b...fe7a49c (2):
  > reactor: workaround tmpfs O_DIRECT vs O_EXCL bug
  > rpc: fix reordering between sending client's negotiation frame and user's data
2016-01-12 13:27:16 +02:00
Takuya ASADA
a1d1d0bd06 Revert "dist: prevent 'local rpm' AMI image update to older version of scylla package by yum update"
This reverts commit b28b8147a0.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452592877-29721-2-git-send-email-syuu@scylladb.com>
2016-01-12 12:26:09 +02:00