Commit Graph

7647 Commits

Author SHA1 Message Date
Tomasz Grabiec
e445e4785c storage_proxy: Extract get_restricted_ranges() as a free function
To make it directly testable.
2015-12-16 13:09:01 +01:00
Tomasz Grabiec
756624ef18 Remove dead code 2015-12-16 13:09:01 +01:00
Tomasz Grabiec
eb27fb1f6b range: Introduce equal() 2015-12-16 13:09:01 +01:00
Tomasz Grabiec
a78f4656e8 Introduce ring_position_less_comparator 2015-12-15 18:00:55 +01:00
Avi Kivity
8fc7583224 Merge seastar upstream
* seastar 5b9e3da...294ea30 (9):
  > Merge "IO queues" from Glauber
  > reactor: increment check_direct_io_support to also deal with files
  > Merge "SSL/TLS initial certificate validation" from Calle
  > tutorial.md: remove inaccurate statements about x86
  > build: verify that the installed compiler is up to date
  > build: complain if fossil version of gnutls is installed
  > build: fix debian naming of gnutls-devel package
  > build: add configure-time check for gnutls-devel
  > tutorial.md: introduction to asynchrnous programming
2015-12-15 16:50:16 +02:00
Tomasz Grabiec
305c2b0880 frozen_mutation: Introduce decorated_key() helper
Requested by Asias for use in streaming code.
2015-12-15 15:16:04 +02:00
Tomasz Grabiec
179b587d62 Abstract timestamp creation behind new_timestamp()
Replace db_clock::now_in_usec() and db_clock::now() * 1000 accesses
where the intent is to create a new auto-generate cell timestamp with
a call to new_timestamp(). Now the knowledge of how to create timestamps
is in a single place.
2015-12-15 15:16:04 +02:00
Avi Kivity
8abd013601 Merge 2015-12-15 15:00:49 +02:00
Avi Kivity
fb8a4f6c1b Merge " implement get_compactions API" from Raphael
"get_compactions returns progress information for each compaction
running in the system. It can be accessed using swagger UI.
'nodetool compactionstats' is not working yet because of some
pending work in the nodetool side."
2015-12-15 14:59:49 +02:00
Paweł Dziepak
71f92c4d14 mutation_partition: do not move rows_entry::_link
Apparently, link hook copy constructor is a no-op and move contructor
doesn't exist so the code is correct, but that explicit move makes code
needlessly confusing.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-15 13:22:23 +01:00
Paweł Dziepak
59245e7913 row_cache: add functions for invalidating entries in cache
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-15 13:21:11 +01:00
Raphael S. Carvalho
833a78e9f7 api: implement get_compactions
get_compactions returns progress information about each
ongoing compaction.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:50:36 -02:00
Raphael S. Carvalho
193ede68f3 compaction: register and deregister compaction_stats
That's important for compaction stats API that will need stats
data of each ongoing compaction.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:50:32 -02:00
Raphael S. Carvalho
e74dcc86bd compaction_manager: introduce list of compaction_stats
This list will store compaction_stats for each ongoing compaction.
That's why register and deregister methods are provided.
This change is important for compaction stats API that needs data
of each ongoing compaction, such as progress, ks, cf, etc.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:50:28 -02:00
Raphael S. Carvalho
a26fb15d1a db: add method to get compaction manager from cf
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:50:20 -02:00
Raphael S. Carvalho
1fba394dd0 sstables: store keyspace and cf in compaction_stats
The reason behind this change is that we will need ks and cf
for the compaction stats API.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:50:02 -02:00
Raphael S. Carvalho
ac1a67c8bc sstables: move compaction_stats to header file
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:49:45 -02:00
Avi Kivity
a2eac711cf Merge "compaction history support" from Raphael
"This patchset will make Scylla update the system table
COMPACTION_HISTORY whenever a compaction job finishes.
Functions were added to both update and retrieve the
content of this system table. Compaction history API
is also enabled in this series."
2015-12-15 13:22:14 +02:00
Raphael S. Carvalho
87fbe29cf9 api: add support to compaction history
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:00:21 -02:00
Takuya ASADA
9c5afb8e58 dist: add scylla-gdb.py on scylla-server-debuginfo rpm package
It will place at /usr/src/debug/scylla-server-development/scylla-gdb.py
Fixes #604

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-15 12:13:17 +02:00
Raphael S. Carvalho
a2fb0ec9a3 sstables: update compaction history at the end of compaction
When compaction job finishes, call function to update the system
table COMPACTION_HISTORY. That's also needed for the compaction
history API.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-14 14:20:03 -02:00
Raphael S. Carvalho
433ed60ca3 db: add method to get compaction history
This method is intended to return content of the system table
COMPACTION_HISTORY as a vector of compaction_history_entry.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-14 14:19:04 -02:00
Raphael S. Carvalho
f3beacac28 db: add method to update the system table COMPACTION_HISTORY
It's supposed to be called at the end of compaction.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-14 13:47:10 -02:00
Tomasz Grabiec
0865ecde17 storage_proxy: Fix range splitting
There is a check whose intent was to detect wrap around during walk of
the ring tokens by comparing the split point with minimum token, which
is supposed to be inserted by the ring iterator. It assumed that when
we encounter it, the range is a wrap around. It doesn't hold when
minimum token is part of the token metadata or set of tokens is empty.

In such case, a full range would be split into 3 overlapping full
ranges. The fix is to drop the assumption and instead ensure that
ranges do not wrap around by unwrapping them if necessary.

Fixes #655.
2015-12-14 16:05:54 +02:00
Takuya ASADA
3b7693feda dist: add package dependency to gnutls library
Now Seastar depends to gnutls, we need to add it on .rpm/.deb package dependency.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-14 13:28:28 +02:00
Pekka Enberg
ba09c545fc dist/docker: Enable SMP support
Now that Scylla has a sleep mode, we can enable SMP support again.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-12-14 13:23:30 +02:00
Avi Kivity
fd14cb3743 mutation_partition: fix leak in move assignment operator
The default move assignment operator calls boost::intrusive::set's move
assignment operator, which leaks, because it does not believe it owns
the data.

Fix by providing a custom implementation.
2015-12-14 10:33:19 +01:00
Asias He
9781e0d34d storage_service: Make bootstrapping/leaving/moving log more consistent
It is useful for test code to grep the log.
2015-12-11 13:57:40 +02:00
Tomasz Grabiec
1991fd5ca2 Merge branch 'pdziepak/fix-clustering-key-comparison/v2' fom seastar-dev.git
From Paweł:

This series fixes comparison of byte order comparable clustering keys.

Fixes #645.
2015-12-11 12:51:02 +01:00
Paweł Dziepak
3a73496817 tests/cql: add test for ordering clustering keys
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-11 12:05:25 +01:00
Paweł Dziepak
8cab343895 compound: fix compare() of prefixable types
All components of prefixable compound type are preceeded by their
length what makes them not byte order comparable.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-11 12:04:31 +01:00
Paweł Dziepak
8fd4b9f911 schema: remove _clustering_key_prefix_type
All clustering keys are now prefixable.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-11 10:47:24 +01:00
Paweł Dziepak
bb9a71f70c thrift: let class_from_compound_type() accept prefixable types
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-11 10:45:56 +01:00
Pekka Enberg
0d8a02453e types: Fix frozen collection type names
Frozen collection type names must be wrapped in FrozenType so that we
are able to store the types correctly in system tables.

This fixes #646 and fixes #580.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-12-11 10:41:11 +01:00
Pekka Enberg
63bdeb65f2 cql3: Implement maps::literal::test_assignment() function
The test_assignement() function is invoked via the Cassandra unit tests
so we might as well implement it.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-12-11 09:35:13 +01:00
Asias He
57ee9676c2 storage_service: Fix default ring_delay time
It is 30 seconds instead of 5 seconds by default. To align with c*.

Pleas note, after this a node will takes at least 30 seconds to complete
a bootstrap.
2015-12-11 09:05:19 +02:00
Avi Kivity
b3cd672d97 Merge seastar upstream
* seastar ad07a2e...5b9e3da (2):
  > Merge "rpc cleanups and improvements" from Gleb
  > shared_future: Add missing include
2015-12-10 18:11:59 +02:00
Paweł Dziepak
9d482532f4 tests/lsa: reduce the size of large allocation
Originally, large allocation test case attempted to allocate an object
as big as halft of the space used by the lsa. That failed when the test
was executed with lower amount of memory available mainly due to the
memory fragmentation caused by previous test cases.

This patches reduces the size of the large allocation to 3/8 of the
total space used by the lsa which is still a lot but seems to make the
test pass even with as little memory as 64MB per shard.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 13:16:43 +01:00
Avi Kivity
d425aacaeb release: copy version string into heap
If we get a core dump from a user, it is important to be able to
identify its version.  Copy the release string into the heap (which is
copied into the code dump), so we can search for it using the "strings"
or "ident" commands.

Reviewed-by: Nadav Har'El <nyh@scylladb.com>
2015-12-10 13:12:40 +02:00
Lucas Meneghel Rodrigues
2167173251 utils/logalloc.cc - Declare member minimum_size from segment_zone struct
This fixes compile error:

In function `logalloc::segment_zone::segment_zone()':
/home/lmr/Code/scylla/utils/logalloc.cc:412: undefined reference to `logalloc::segment_zone::minimum_size'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-12-10 12:54:34 +02:00
Asias He
b7d10b710e streaming: Propagate fail to send PREPARE_DONE_MESSAGE exception
Otherwise the stream_plan will not be marked as failed state.
2015-12-10 12:38:00 +02:00
Paweł Dziepak
ec453c5037 managed_bytes: fix potentially unaligned accesses
blob_storage defined with attribute packed which makes its alignment
requirement equal 1. This means that its members may be unaligned.
GCC is obviously aware of that and will generate appropriate code
(and not generate ubsan checks). However, there are few places where
members of blob_storage are accessed via pointers, these have to be
wrapped by unaligned_cast<> to let the compiler know that the location
pointed to may be not aligned properly.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 11:59:54 +02:00
Tomasz Grabiec
43498b3158 Merge branch 'pdziepak/fix-partial-clustering-keys/v1' from seastar-dev.git
Form Paweł:

This series fixes support for clustering keys which trailing components
are null. The solution is to use clustering_key_prefix instead of
clustering_key everywhere.

Fixes #515.
2015-12-10 10:43:12 +01:00
Paweł Dziepak
66ff1421f0 tests/cql: add test for clustering keys with empty components
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:47:07 +01:00
Paweł Dziepak
64f50a4f40 db: make clustering_key a prefix
Schemas using compact storage can have clustering keys with the trailing
components not set and effectively being a clustering key prefixes
instead of full clustering keys.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:47 +01:00
Paweł Dziepak
77c7ed6cc5 keys: add prefix_equality_less_compare for prefixes
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Paweł Dziepak
220a3b23c0 keys: allow creating partial views of prefixes
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Paweł Dziepak
3c16ab080a sstables: do not assume clustering_key has the proper format
In case of non-compound dense tables the column name is just the value
of the clustering key (which has only one component). Current code just
casts clustering_key to bytes_view which works because there is no
additional metadata in single element clustering keys.
However, that may change when the internal representation of clustering
key is changed so explicitly extract the proper component.

This change will become necessary when clustering_key is replaced by
clustering_key_prefix.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Paweł Dziepak
5f1e9fd88f mutation_partition: remove unused find_entry()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Paweł Dziepak
3287022000 cql3: do not assume that clustering key is full
In case of schemas that use compact storage it is possible that trailing
components of clustering keys are not set.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00