Commit Graph

7768 Commits

Author SHA1 Message Date
Vlad Zolotarov
33552829b2 core: use steady_clock where monotinic clock is required
Use steady_clock instead of high_resolution_clock where monotonic
clock is required. high_resolution_clock is essentially a
system_clock (Wall Clock) therefore may not to be assumed monotonic
since Wall Clock may move backwards due to time/date adjustments.

Fixes issue #638

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-12-27 18:07:53 +02:00
Avi Kivity
f3980f1fad Merge seastar upstream
* seastar 51154f7...8b2171e (9):
  > memcached: avoid a collision of an expiration with time_point(-1).
  > tutorial: minor spelling corrections etc.
  > tutorial: expand semaphores section
  > Merge "Use steady_clock where monotonic clock is required" from Vlad
  > Merge "TLS fixes + RPC adaption" from Calle
  > do_with() optimization
  > tutorial: explain limiting parallelism using semaphores
  > submit_io: change pending flushes criteria
  > apps: remove defunct apps/seastar

Adjust code to use steady_clock instead of high_resolution_clock.
2015-12-27 14:40:20 +02:00
Avi Kivity
0687d7401d Merge "storage_service updates" from Asias
"
- Fix erase of new_replica_endpoints in get_changed_ranges_for_leaving
- Introduce ntroduce ring_delay_ms option
"
2015-12-27 12:46:37 +02:00
Nadav Har'El
06f8dd4eb2 repair: job id must start at 1
This patch fixes a bug where the *first* run of "nodetool repair" always
returned immediately, instead of waiting for the repair to complete.

Repair operations are asynchronous: Starting a repair returns a numeric
id, which can then be used to query for the repair's completion, and this
is what "nodetool repair" does (through our JMX layer). We started with
the repair ID "0", the next one is "1", and so on.

The problem is that "nodetool repair", when it sees 0 being returned,
treats it not as a regular repair ID, but rather as an answer that
there is nothing to repair - printing a message to that effect and *not*
waiting for the repair (which was correctly started) to complete.

The trivial fix is to start our repair IDs at 1, instead of 0.
We currently do not return 0 in any case (we don't know there is nothing
to repair before we actually start the work, and parameter errors
cause an exception, not a return of 0).

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2015-12-27 12:42:26 +02:00
Avi Kivity
93aeedf403 Merge "Fixes for CentOS/RHEL support" from Takuya
"Recent changes on scripts causes error on CentOS/RHEL, this patchset fixes it."
2015-12-27 12:21:29 +02:00
Glauber Costa
e299127e81 main: check if options file can be read.
If we can't open the file, we will fail with a misterious error. It is a costumary
scenario, though, since people who are unaware or have just forgotten about seastar's
restriction of direct io access may put those files in tmpfs and other mount points.

We have a direct_io check that is designed exactly for this purpose, so as to give
the user a better error message. This patch makes use of it.

Fixes #644

Signed-off-by: Glauber Costa <glauber@scylladb.com>
2015-12-27 12:20:40 +02:00
Asias He
f57ba6902b storage_service: Introduce ring_delay_ms option
It is hard-coded as 30 seconds at the moment.

Usage:
$ scylla --ring-delay-ms 5000

Time a node waits to hear from other nodes before joining the ring in
milliseconds.

Same as -Dcassandra.ring_delay_ms in cassandra.
2015-12-25 15:08:22 +08:00
Asias He
9c07ed8db6 storage_service: Fix erase new_replica_endpoints in get_changed_ranges_for_leaving
We need to calculate begin() and end() in the loop since elements in
new_replica_endpoints might be removed.

Refs #700
2015-12-25 15:08:22 +08:00
Asias He
88846bc816 storage_service: Add more debug info in decommission
It is useful to debug decommission issue.
2015-12-25 15:08:22 +08:00
Asias He
19f1875682 gossip: Print endpoint_state_map debug info in trace level
This generates too many logs with debug level. Make it trace level.
2015-12-25 15:08:22 +08:00
Nadav Har'El
06ab43a7ee murmur3 partitioner: fix midpoint() algorithm
The midpoint() algorithm to find a token between two tokens doesn't
work correctly in case of wraparound. The code tried to handle this
case, but did it wrong. So this patch fixes the midpoint() algorithm,
and adds clearer comments about why the fixed algorithm is correct.

This patch also modifies two midpoint() tests in partitioner_test,
which were incorrect - they verified that midpoint() returns some expected
values, but expected values were wrong!

We also add to the test a more fundemental test of midpoint() correctness,
which doesn't check the midpoint against a known value (which is easy to
get wrong, like indeed happened); Rather we simply check that the midpoint
is really inside the range (according to the token ordering operator).
This simple test failed with the old implementation of midpoint() and
passes with the new one.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2015-12-24 17:19:49 +02:00
Avi Kivity
3392f02b54 Merge "Make date parser more liberal" from Paweł
"This series makes date and time parsing more liberal so that Scylla
accepts the same date formats the origin does.

Fixes #521."
2015-12-24 17:18:04 +02:00
Takuya ASADA
bf9547b1c4 dist: support RHEL on scylla_install
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-24 18:48:30 +09:00
Takuya ASADA
bb0880f024 dist: use /etc/os-release instead of /etc/redhat-release
Since other scripts using /etc/os-release, it is better to use same one here.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-24 18:48:30 +09:00
Takuya ASADA
b6df28f3d5 dist: use $ID instead of $NAME to detect type of distribution
$NAME is full name of distribution, for script it is too long.
$ID is shortened one, which is more useful.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-24 18:48:30 +09:00
Takuya ASADA
0a4b68d35e dist: support CentOS yum repository
Fixes #671

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-24 18:48:30 +09:00
Takuya ASADA
8f4e90b87a dist: use tsc clocksource on AMI
Stop using xen clocksource, use tsc clocksource instead.
Fixes #462

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-22 22:29:32 +02:00
Amnon Heiman
b0856f7acf API: Init value for cf_map reduce should be of type int64_t
The helper function for summing statistic over the column family are
template function that infer the return type acording to the type of the
Init param.

In the API the return value should be int64_t, passing an integer would
cause a number wrap around.

A partial output from the nodetool cfstats after the fix

nodetool cfstats keyspace1
Keyspace: keyspace1
	Read Count: 0
	Read Latency: NaN ms.
	Write Count: 4050000
	Write Latency: 0.009178098765432099 ms.
	Pending Flushes: 0
		Table: standard1
		SSTable count: 12
		Space used (live): 1118617445
		Space used (total): 23336562465

Fixes #682

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-22 17:33:13 +02:00
Tomasz Grabiec
88f5da5d1d Merge branch 'calle/paging_fixes' from seastar-dev.git
From Calle:

Fixes #589
Query should not return dangling static row in partition without any
regular/ck columns if a CK restriction is applied.

Refs #650
Fixes bug in CK range code for paging, and removes CK use for tables with not
clustering -> way simpler code. Also removed lots of workaround code no longer
required.

Note that this patch set does not fully fix #650/paging since bug #663 causes
duplicate rows. Still almost there though.
2015-12-22 11:22:42 +01:00
Avi Kivity
926d340661 logger: be robust when exceptions are thrown while stringifying args
Instead of propagating the exception, swallow it and print it out in
the log message.

Fixes #672.
2015-12-21 19:58:08 +01:00
Paweł Dziepak
cf949e98cb tests/types: add more tests for date and time parsing
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-21 15:34:17 +01:00
Paweł Dziepak
633a13f7b3 types: timestamp_from_string: accept more date formats
Boost::date_time doesn't accept some of the date and time formats that
the origin do (e.g. 2013-9-22 or 2013-009-22).

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-21 15:30:35 +01:00
Calle Wilund
f118222b2d query_pagers: Remove unneeded clustering + remove static workaround
Refs #640

* Remove use of cluster key range for tables without CK
  Checking CK existance once and use the info allows us to remove some
  stupid complexity in checking for "last key" match
* With fix for #589 we can also remove some superfluous code to
  compensate for that issue, and make "partition end" simper
* Remove extra row in CK case. Not needed anymore

End result is that pager now more or less only relies on adapted query
ranges.
2015-12-21 14:19:45 +00:00
Calle Wilund
72a079d196 paging_state: Make clustering key optional 2015-12-21 14:19:45 +00:00
Calle Wilund
c868d22d0c db/serializer: Add support for optional<T> to be serialized
template spacialization.

Simply just wraps underlying type serialization and adds a "bool"
check mark first in stream.
2015-12-21 14:19:45 +00:00
Calle Wilund
803b58620f data_output: specialize serialized_size for bool to ensure sync with write 2015-12-21 14:19:45 +00:00
Paweł Dziepak
d41807cb66 types: timestamp_from_string(): restore indentation
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-21 15:17:50 +01:00
Paweł Dziepak
873ed78358 types: catch parsing errors in timestamp_from_string()
timestamp_from_string() is used by both timestamp and date types, so it
is better to move the try { } catch { } to the functions itself instead
of expecting its callers to catch exceptions.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-21 15:14:36 +01:00
Takuya ASADA
0d1ef007d3 dist: skip mounting RAID if it's already mounted
On AMI, scylla-server fails to systemctl restart because scylla_prepare tries to mount /var/lib/scylla even it's already mounted.
This patch fixes the issue.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-21 15:50:09 +02:00
Avi Kivity
c3d0ae822d Merge seastar upstream
* seastar b44d729...51154f7 (6):
  > semaphore: add with_semaphore()
  > scripts: posix_net_conf.sh: don't transform wide CPU mask
  > resource: fix build for systems without HWLOC
  > build: link libasan before all other libraries
  > Use sys_membarrier() when available
  > build: add missing library (boost_filesystem)
2015-12-21 14:45:57 +02:00
Calle Wilund
8c17e9e26c mutation_partition: Do not return static row if CK range does not match
Fixes #589

If we got no rows, but have live static columns, we should only
give them back IFF we did not have any CK restrictions.
If ck:s exist, and we have a restriction on them, we either have maching
rows, or return nothing, since cql does not allow "is null".
2015-12-21 10:38:48 +00:00
Pekka Enberg
98454b13b9 cql3: Remove some ifdef'd code 2015-12-21 10:38:48 +00:00
Pekka Enberg
c6541b4cc2 cql3: Remove untranslated IMeasurableMemory code from column_identifier
We will not be using it so just remove the untranslated code.
2015-12-21 10:38:48 +00:00
Pekka Enberg
81d72afd85 cql3: Move delete_statement implementation to source file 2015-12-21 10:38:48 +00:00
Pekka Enberg
cd58ea3b96 cql3: Move modification_statement implementation to source file 2015-12-21 10:38:48 +00:00
Pekka Enberg
bcd602d3f8 cql3: Move parsed_statement implementation to source file 2015-12-21 10:38:48 +00:00
Pekka Enberg
44ba4857eb cql3: Move property_definitions implementation to source file 2015-12-21 10:38:48 +00:00
Pekka Enberg
2759473c7a cql3: Move select_statement implementation to source file 2015-12-21 10:38:48 +00:00
Pekka Enberg
7a5d6818a3 cql3: Move update_statement implementation to source file 2015-12-21 10:38:48 +00:00
Paweł Dziepak
9aa24860d7 test/sstables: add more key_reader tests
This patch introduces a test for reading keys from a single sstable with
the range begining and end being the keys present in the index summary.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-21 10:38:48 +00:00
Paweł Dziepak
2fd7caafa0 sstables: respect range inclusiveness in key_reader
When choosing a relevant range of buckets it wasn't taken into account
whether the range bounds are inclusive or not. That may have resulted in
more buckets being read than necessary which was a condition not
expected by the code responsible from looking for a relevant keys inside
the buckets.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-21 10:38:48 +00:00
Raphael S. Carvalho
d8e810686a sstables: remove outdated comment
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-21 10:38:48 +00:00
Raphael S. Carvalho
99710ae0e6 db: fix indentation
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-21 10:38:48 +00:00
Raphael S. Carvalho
e1edc2111c sstables: fix comment describing sstable::mark_for_deletion
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-21 10:38:48 +00:00
Raphael S. Carvalho
22ac260059 db: add missing sstable::mark_for_deletion call
If a sstable doesn't belong to current shard, mark_for_deletion
should be called for the deletion manager to still work.
It doesn't mean that the sstable will be deleted, but that the
sstable is not relevant to the current shard, thus it can be
deleted by the deletion manager in the future.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-21 10:38:48 +00:00
Tomasz Grabiec
324ad43be1 Merge branch 'penberg/cql-cleanups/v1' from seastar-dev.git
Another round of cleanups to the CQL code from Pekka.
2015-12-18 17:36:45 +01:00
Tomasz Grabiec
0862d2f531 Merge branch 'pdziepak/fix-sstables-key_reader-663/v2'
From Paweł:

"This series fixes sstables::key_reader not respecting range inclusiveness
if the bounds were the keys that were present in the index summary.

Fixes #663."
2015-12-18 17:35:09 +01:00
Paweł Dziepak
b39d1fb1fc test/sstables: add more key_reader tests
This patch introduces a test for reading keys from a single sstable with
the range begining and end being the keys present in the index summary.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-18 17:24:29 +01:00
Paweł Dziepak
18b8d7cccc sstables: respect range inclusiveness in key_reader
When choosing a relevant range of buckets it wasn't taken into account
whether the range bounds are inclusive or not. That may have resulted in
more buckets being read than necessary which was a condition not
expected by the code responsible from looking for a relevant keys inside
the buckets.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-18 17:24:26 +01:00
Pekka Enberg
eeadf601e6 Merge "cleanups and improvements" from Raphael 2015-12-18 13:45:11 +02:00