Commit Graph

7259 Commits

Author SHA1 Message Date
Takuya ASADA
368ee17d53 dist: prevent error building ubuntu package on EC2
fixes #491

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-06 22:06:55 +02:00
Gleb Natapov
120ae08082 storage_proxy: fix race during write completion
If write timeout and last acknowledgement needed for CL happen simultaneously
_ready can be sent to be exceptional by the timeout handler, but since
removal of the response handler happens in continuation it may be
reordered with last ack processing and there _ready will be set again
which will trigger assert. Fix it by removing the handler immediately,
no need to wait for continuation. It makes code simpler too.
2015-11-05 19:35:18 +02:00
Paweł Dziepak
64f1c2866c lsa: free segment in trim_emergency_reserve_to_max()
_emergency_reserve is an intrusive containers and it doesn't care about
segment lifetime.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-05 18:04:38 +02:00
Amnon Heiman
5f1cae67ff API: compaction_manager should catch the field reference by value
The get_cm_stats gets a pointer to a field in the stats object. It
should capture it by value or segmentation falut may occure when the
caller gets out of scope.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-05 14:17:12 +02:00
Avi Kivity
c9f02ce42f Merge seastar upstream
* seastar 05b41b8...5c10d3e (3):
  > reactor: more conservative signal handling
  > output_stream: add write(temporary_buffer) variant
  > input_stream: add read() method
2015-11-05 13:20:26 +02:00
Takuya ASADA
b50a6a1521 dist: add missing dist/ubuntu/changelog.in
Missing file for d2dd6b90a9

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-05 11:09:32 +02:00
Glauber Costa
958abc17a0 sstables: be more descriptive with filename parsing errors
Currently, we don't let the user know even what is the filename that failed.
That information should be included in the message.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-11-05 10:45:59 +02:00
Raphael S. Carvalho
279a8f13ee sstables: compaction: handle failure on read fiber
This assert (in write fiber) would fail if read fiber failed
because the variable done will not be set to true.
The use of assert is very bad, because it prevents scylla
from proceeding, which is possible.
To solve it, let's trigger an exception if done is not true.
We do have code that will wait for both read and write fibers,
and catch exceptions, if any.

Closes #523.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-11-04 16:48:01 +01:00
Tomasz Grabiec
598f4104fd query_processor: Use the correct client_state
Since 4641dfff24, query_state keeps a
copy of client_state, not a reference. Therefore _cl is no longer
updated by queries using _qp. Fix by using the client_state from _qp.

Fixes #525.
2015-11-04 12:43:26 +02:00
Takuya ASADA
ec06fd77a8 dist: split debug symbol to sepalate package on ubuntu
Fixes #524

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-04 12:35:35 +02:00
Asias He
0304567c09 storage_service: Implement get_new_source_ranges
It is needed by restore_replica_count.
2015-11-04 12:21:00 +02:00
Takuya ASADA
d2dd6b90a9 dist: support ./SCYLLA-VERSION-GEN on ubuntu package
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-04 11:50:38 +02:00
Avi Kivity
a41ffb3f4b Merge "support initial_token config option" from Asias
"This is needed by some tests to specify specific tokens."
2015-11-04 11:42:39 +02:00
Paweł Dziepak
a553c9236e transport: do not send response with unknown protocol version
All responses sent from the server have protocol version set to
connection::_version which is set to the version used by the client
in its first message. However, if the protocol version used by the
client is unuspported or invalide the server should use the latest
version it recognizes.

This solves problem with version negotiation with Java driver. The
driver first sends a request in the latest version it recognizes, if
that fails it retries with the version that server has used in the error
message. If that fails as well it gives up. However, since Scylla always
responds with the same version that the client has used the negotiation
always fails if the client supports more protocol version than the
server.

Refs #317.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-04 08:59:23 +02:00
Asias He
ed313160c2 storage_service: Add initial_token config option support 2015-11-04 10:42:17 +08:00
Asias He
20ecb0bede database: Introduce get_initial_tokens
Get initial tokens specified by the initial_token in scylla.conf.

E.g.,

   --initial-token "-1112521204969569328,1117992399013959838"
   --initial-token "1117992399013959838"

It can be multiple tokens split by comma.
2015-11-04 10:40:12 +08:00
Asias He
16596385ee token: Handle minimum token correctly in long_token
Fixes:

Exiting on unhandled exception of type 'runtime_exception': runtime
error: Invalid token. Should have size 8, has size 0
2015-11-04 09:01:06 +08:00
Avi Kivity
a060610347 Merge seastar upstream
* seastar 258daf9...05b41b8 (5):
  > resource: fix system memory reservation
  > Add .gitattributes to improve 'git diff' output
  > README: remove irrelevant OSv instructions
  > Add invocation of ninja in Ubuntu section of README
  > collectd: remove incorrect license
2015-11-04 00:18:06 +02:00
Avi Kivity
012f13eb53 Merge "Adding describering API" from Amnon
"This series adds the missing functionality that the nodetool describering would work.

It import the missing functionality from origin.
After this patch the API:
GET /storage_service/describe_ring/{keyspace}

will be available"
2015-11-03 16:54:13 +02:00
Amnon Heiman
ef3c6b2647 API: Add describe ring API implementation
This patch chanages the API to support describe ring instead of describe
ring jmx that will be implemented in the jmx server.

The API will return a list of objects instead of string.

An additional api was added as the equivelent to the jmx call with an
empty param.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 16:33:39 +02:00
Amnon Heiman
3809565dbd storage_service: Add method implementation from origin
This patch adds the following methods implementation:
getRpcaddress
getRangeToAddressMap
getRangeToAddressMapInLocalDC
describeRing
getAllRanges

Those methods are used as part of the describe_ring method
implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 16:33:20 +02:00
Gleb Natapov
28bb6a3efe messaging_service: fix hanging reference access
Do not pass reference to an on-stack objects to a function that uses
its parameters asynchronously.
2015-11-03 12:10:38 +01:00
Tomasz Grabiec
7a2ac628a0 Fix compilation of types_test 2015-11-03 11:02:35 +01:00
Avi Kivity
74faaa4698 types: implement date_type::from_string()
Luckily all the hard work was already done for timestamp_type.

Fixes #522.
2015-11-03 10:33:20 +01:00
Amnon Heiman
b77ec2bd6a Importing token_range and endpoint_details from origin
The storage server uses the token_range in origin to return inforamtion
about the ring.

This import the structures. The functionality in origin is redundant in
this case and was not imported.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-03 10:17:05 +02:00
Asias He
dafdac6731 ami: Improve scylla raid setup
Use all the disks except the one for rootfs for RAID0 which stores
scylla data. If only one disk is available warn the user since currently
our AMI's rootfs is not XFS.

[fedora@ip-172-31-39-189 ~]$ cat WARN.TXT
WARN: Scylla is not using XFS to store data. Performance will suffer.

Tested on AWS with 1 disk, 2 disks, 7 disk case.

(cherry picked from commit 49d6cba471)
2015-11-03 08:16:09 +08:00
Avi Kivity
389e035d19 Update scylla-ami submodule
* dist/ami/files/scylla-ami c6ddbea...3f37184 (1):
  > Update reflector URL

(cherry picked from commit 440b403089)
2015-11-03 08:12:51 +08:00
Takuya ASADA
a0feaf801c dist: add scylla.repo to fetch scylla rpms on ami
Mistakenly didn't included on yum repository for AMI patchset, but it's needed

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
(cherry picked from commit 8587c4d6b3)
2015-11-03 08:01:09 +08:00
Takuya ASADA
a5a69279ef dist: update packages on ec2 instance first bootup
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
(cherry picked from commit aaeccdee60)
2015-11-03 07:50:43 +08:00
Takuya ASADA
db634dfe01 dist: use scylla repo on ami, instead of locally built rpms
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
(cherry picked from commit 8b98fe5a1c)
2015-11-03 07:50:33 +08:00
Takuya ASADA
63d16f5816 dist: move inline script to setup-ami.sh
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
(cherry picked from commit 2863b8098f)
2015-11-03 07:50:12 +08:00
Amnon Heiman
e562736e82 API: A workaround for nodetool cleanup
The nodetool cleanup command is used in many of the tests, because the
API call is not implemented it causes the tests to fail.

This is a workaround until the cleanup will be implemented, the method
return successfuly.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 17:38:48 +02:00
Amnon Heiman
80e5289359 Adding API as an unimplemented cause
Normally an API call that is not implemented should fail, there are
cases that as a workaround an API call is stub, in those cases a warning
is added to indicate that the API is not implemented.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 17:38:48 +02:00
Avi Kivity
c9b8c368bd Merge "Adding missing statistics to enable nodetool netstats" from Amnon
"The series adds read repair and storage_service server statistics."
2015-11-02 17:10:43 +02:00
Amnon Heiman
fc7ea6207c API: messaging service, Add getter for the pending and completed server
This patch adds getter for the messeging service server pending and sent
messages.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:47 +02:00
Amnon Heiman
55d7ddafe5 API: Add repond complete to the messaging service swagger
This patch do the following:
It adds a getter for the completed respond messages (i.e. the total
messages that were sent by the server)

It replaces the return mapping for the statistics to use the key, value
notation that is used in the jmx side.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:36 +02:00
Amnon Heiman
afb921a643 API: add read repair stats to the storage_proxy
This adds the API to the storage_proxy read repair stats that replaces
the stub implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:17:22 +02:00
Amnon Heiman
b6034572dc storage_proxy: Add read repair statistics
This adds the read repair statistics to he storage_proxy stats and adds
to its implementation incrementing the counters value.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:16:40 +02:00
Amnon Heiman
d5d0653210 messaging_service: Add a function that goes over all the server stats
The API needs to get the stats from the rpc server, that is hidden from the
messaging service API.

This patch adds a foreach function that goes over all the server stats
without exposing the server implementation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-02 16:15:52 +02:00
Avi Kivity
9c09f43311 Merge "add [background] read/write statistics" from Gleb
"The main objective of the series is to introduce statistics about ongoing
read/writes and especially those that are done in the background (acknowledged,
but uncompleted), but it contains some cleanups as well."
2015-11-02 15:46:07 +02:00
Avi Kivity
e2b661bb78 sstables: fix compile error 2015-11-02 15:32:44 +02:00
Gleb Natapov
9af55b3a7b storage_proxy: add ongoing read statistics
Add statistics for ongoing reads and ongoing background reads. Read is
a background one if it was acknowledged, but there still work to do to
complete it.
2015-11-02 15:02:13 +02:00
Gleb Natapov
463d330a24 storage_proxy: do not capture 'this' needlessly. 2015-11-02 15:01:16 +02:00
Gleb Natapov
5067d027ba storage_proxy: export statistics via collectd 2015-11-02 15:01:16 +02:00
Gleb Natapov
287cf894a0 storage_proxy: count background mutation writes
Count how many writes are running in the background. Write is a
background write if it was acknowledged, but there still work to do to
complete it.
2015-11-02 15:01:12 +02:00
Gleb Natapov
afa61cb87c storage_proxy: separate methods from members with access specifier 2015-11-02 15:00:47 +02:00
Gleb Natapov
62e5ad27d8 storage_proxy: devirtualize total_block_for()
total_block_for() is defined only in base class, so make it non virtual.
2015-11-02 15:00:47 +02:00
Gleb Natapov
59d8f9f392 storage_proxy: move proxy pointer to write response handler
We will need it later there.
2015-11-02 15:00:47 +02:00
Gleb Natapov
ca0177f5cc storage_proxy: simplify timeout handling
Use _cl_achieved instead of calculating if there is enough acks.
2015-11-02 15:00:47 +02:00
Gleb Natapov
4838de008f storage_proxy: get rid of semaphore in mutation write response handler
Acks are counted in _cl_acks, so there is no need to count them once
more in semaphore, just make promise available when there is enough of
them.
2015-11-02 15:00:47 +02:00