The gocql driver assumes that there's a result metadata section in the
PREPARED message. Technically, Scylla is not at fault here as the CQL
specification explicitly states in Section 4.2.5.4. ("Prepared") that the
section may be empty:
- <result_metadata> is defined exactly as <metadata> but correspond to the
metadata for the resultSet that execute this query will yield. Note that
<result_metadata> may be empty (have the No_metadata flag and 0 columns, See
section 4.2.5.2) and will be for any query that is not a Select. There is
in fact never a guarantee that this will non-empty so client should protect
themselves accordingly. The presence of this information is an
However, Cassandra always populates the section so lets do that as well.
Fixes#912.
Message-Id: <1456317082-31688-1-git-send-email-penberg@scylladb.com>
* dist/ami/files/scylla-ami 398b1aa...d4a0e18 (3):
> Sort service running order (scylla-ami-setup.service -> scylla-io-setup.service -> scylla-server.service)
> Drop --ami and --disk-count parameters
> dist: pass the number of disks to set io params
In each gossip round, i.e., gossiper::run(), we do:
1) send syn message
2) peer node: receive syn message, send back ack message
3) process ack message in handle_ack_msg
apply_state_locally
mark_alive
send_gossip_echo
handle_major_state_change
on_restart
mark_alive
send_gossip_echo
mark_dead
on_dead
on_join
apply_new_states
do_on_change_notifications
on_change
4) send back ack2 message
5) peer node: process ack2 message
apply_state_locally
At the moment, syn is "wait" message, it times out in 3 seconds. In step
3, all the registered gossip callbacks are called which might take
significant amount of time to complete.
In order to reduce the gossip round latency, we make syn "no-wait" and
do not run the handle_ack_msg insdie the gossip::run(). As a result, we
will not get a ack message as the return value of a syn message any
more, so a GOSSIP_DIGEST_ACK message verb is introduced.
With this patch, the gossip message exchange is now async. It is useful
when some nodes are down in the cluster. We will not delay the gossip
round, which is supposed to run every second, 3*n seconds (n = 1-3,
since it talks to 1-3 peer nodes in each gossip round) or even
longer (considering the time to run gossip callbacks).
Later, we can make talking to the 1-3 peer nodes in parallel to reduce
latency even more.
Refs: #900
We will soon switch to use no-wait message for gossip. GOSSIP_DIGEST_SYN
will no longer return GOSSIP_DIGEST_ACK message. So we need a standalone
verb for GOSSIP_DIGEST_ACK.
The problem is we initialize _last_interpret when failure_detector
object is constructed. When interpret() runs for the first time, the
_last_interpret value is not the last time we run interpret() but the
time we initialize failure_detector object.
Fix by initializing _last_interpret inside interpret().
[Thu Feb 18 02:40:04 2016] INFO [shard 0] storage_service - Node 127.0.0.1 state jump to normal
[Thu Feb 18 02:40:04 2016] INFO [shard 0] storage_service - NORMAL: node is now in normal status
[Thu Feb 18 02:40:04 2016] INFO [shard 0] gossip - Waiting for gossip to settle before accepting client requests...
[Thu Feb 18 02:40:12 2016] INFO [shard 0] gossip - No gossip backlog; proceeding
Starting listening for CQL clients on 127.0.0.1:9042...
[Thu Feb 18 02:40:12 2016] INFO [shard 0] gossip - Node 127.0.0.2 is now part of the cluster
[Thu Feb 18 02:40:12 2016] INFO [shard 0] gossip - InetAddress 127.0.0.2 is now UP
[Thu Feb 18 02:40:13 2016] INFO [shard 0] gossip - do_gossip_to_live_member: Favor newly added node 127.0.0.2
[Thu Feb 18 02:40:13 2016] WARN [shard 0] failure_detector - Not marking nodes down due to local pause of 9091 > 5000 (milliseconds)
"Add scylla-io-setup.service to configure max-io-requests and num-io-queues on first boot.
Moved SCYLLA_IO configuration code from scylla_sysconfig_setup to scylla-io-setup.service, revert commits related it.
On scylla-io-setup.service, autodetect Amazon EC2 instead of using AMI variable on sysconfig."
While serialization vector it is sometimes required to rollback some of
the serialized elements.
vector_position is the equivalent to the bytes_ostream position struct.
It holds information about the current position in a serialized vector,
the position in the bufffer and the current number of elements
serialized.
It will allow to rollback to the current point.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1456041750-1505-2-git-send-email-amnon@scylladb.com>
Fixes#927.
The new visiting code builds cell instances using
atomic_cell::make_*() factory methods, which won't work in LSA context
because they depend on managed_bytes storage to be linearized. It may
not be since large blob support. This worked before because we created
cells from views before which works in all contexts.
Fix by constructing them in standard allocator context.
Message-Id: <1456234064-13608-2-git-send-email-tgrabiec@scylladb.com>
The line:
boost::apply_visitor(atomic_cell_or_collection_visitor(std::move(visitor), id, col), cell);
is executed in a loop, so visitor could be used after being
moved-from. This may not always be allowed for some visitors. Also,
vistors may keep state, which should be preserved for the whole
visitation.
This doesn't fix any issue right now.
Message-Id: <1456234064-13608-1-git-send-email-tgrabiec@scylladb.com>
The 'clear' function explicitly clears the screen and repaints it which
causes really annoying flicker. Use 'erase' to make scyllatop more
pleasant on the eyes.
Message-Id: <1456229348-2194-1-git-send-email-penberg@scylladb.com>
It is often the case that the there is useful debugging information
printed by the test before it hangs. It is annoying to see just "TIMED
OUT" in jenkins. Print the output always when it is available.
In addition to that, we should not interpret all exceptions thrown
from communicate() as timeouts. For example, currently ^C sent to the
script misleadingly results in "TIMED OUT" to be printed.
Message-Id: <1456174992-21909-1-git-send-email-tgrabiec@scylladb.com>
Every native scalar function is already tagged whether they're pure or
not but because we don't implement the is_pure() function, all functions
end up being advertised as pure. This means that functions like now()
that are *not* pure, end up being evaluated only once.
Fixes#571.
Message-Id: <1456227171-461-1-git-send-email-penberg@scylladb.com>
In same cases we may have a lot of empty partitions whose tombstones
expired, and there is no point in including them in the results.
This was found to cause performance issues for workloads using batch
updates. system.batchlog table would accumulate a lot of deletes over
time. It has gc_grace_seconds set to 0 so most of the tombstones would
be expired. mutation queries done by batchlog manager were still
returning all partitions present in memtables which caused mutation
queries result to be inflated. This in turn was causing
mutation_result_merger to take a long time to process them.
We don't support the 'CREATE TYPE' statement for now. The user-visible
error message, however, is unreadable because our CQL parser doesn't
even recognize the statement.
cqlsh:ks1> CREATE TYPE config (url text);
SyntaxException: <ErrorMessage code=2000 [Syntax error in CQL query] message=" : cannot match to any predicted input...
Implement just enough of 'CREATE TYPE' parsing to be able to report a
human readable error message if someone tries to execute such
statements:
cqlsh:ks1> CREATE TYPE config (url text);
ServerError: <ErrorMessage code=0000 [Server error] message="User-defined types are not supported yet">
Message-Id: <1456148719-9473-2-git-send-email-penberg@scylladb.com>
As explained in commit 0ff0c55 ("transport: server: 'short' should be
unsigned"), "short" type is always unsigned in the CQL binary protocol.
Therefore, drop the read_unsigned_short() variant altogether and just
use read_short() everywhere.
Message-Id: <1456133171-1433-1-git-send-email-penberg@scylladb.com>
This may result in errors during reading like the following one:
runtime error: Unexpected marker. Found k, expected \x01\n)'
The error above happened when executing limits.py:max_key_length_test dtest.
After change the exception will happen during writing and will be clearer.
Refs #807.
This patch doesn't deal with the problem of ensuring that we will
never hit those errors, which is very desirable. We shouldn't ack a
write if we can't persist it to sstables.
Message-Id: <1456130045-2364-1-git-send-email-tgrabiec@scylladb.com>
Change the name used with class_registrator from "EverywhereReplicationStrategy"
(used in the initial patch from CASSANDRA-826 JIRA) to "EverywhereStrategy"
as it is in the current DCE code.
With this change one will be able to create an instance of
everywhere_replication_strategy class by giving either
an "org.apache.cassandra.locator.EverywhereStrategy" (full name) or
an "EverywhereStrategy" (short name) as a replication strategy name.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1456081258-937-1-git-send-email-vladz@cloudius-systems.com>
This strategy would ignore an RF configuration and would
always try to replicate on all cluster nodes.
This means that its get_replication_factor() would return a
number of currently "known" nodes in the cluster and
if a cluster is currently bootstrapping this value obviously may
change in time for the same key. Therefore using this strategy
should be done with caution.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1456074333-15014-3-git-send-email-vladz@cloudius-systems.com>
Return a number of currently known endpoints when
it's needed in a fast path flow.
Calling a get_all_endpoints().size() for that matter
would not be fast enough because of the unordered_set->vector transformation
we don't need.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1456074333-15014-2-git-send-email-vladz@cloudius-systems.com>
Current algorithm is O(N^2) where N is the column count. This causes
limits.py:TestLimits.max_columns_and_query_parameters_test to timeout
because CREATE TABLE statement takes too long.
This change replaces it with an algorithm of O(N)
complexity. _defined_names are already sorted so if any duplicates
exist, they must be next to each other.
Message-Id: <1456058447-5080-1-git-send-email-tgrabiec@scylladb.com>
When find_uuid() fails Scylla would terminate with:
Exiting on unhandled exception of type 'std::out_of_range': _Map_base::at
But we are supposed to ignore directories for unknown column
families. The try {} catch block is doing just that when
no_such_column_family is thrown from the find_column_family() call
which follows find_uuid(). Fix by converting std::out_of_range to
no_such_column_family.
Message-Id: <1456056280-3933-1-git-send-email-tgrabiec@scylladb.com>
When there is a lot of chunks we may get stack overflow.
This seems to fix issue #906, a memory corruption during schema
merge. I suspect that what causes corruption there is overflowing of
the stack allocated for the seastar thread. Those stacks don't have
red zones which would catch overflow.
Message-Id: <1456056288-3983-1-git-send-email-tgrabiec@scylladb.com>
before:
^CINFO [shard 0] compaction_manager - Asked to stop
INFO [shard 0] compaction_manager - compaction task handler stopped due to shutdown
INFO [shard 0] compaction_manager - compaction task handler stopped due to shutdown
INFO [shard 1] compaction_manager - Asked to stop
INFO [shard 2] compaction_manager - Asked to stop
INFO [shard 1] compaction_manager - compaction task handler stopped due to shutdown
INFO [shard 2] compaction_manager - compaction task handler stopped due to shutdown
INFO [shard 3] compaction_manager - Asked to stop
INFO [shard 1] compaction_manager - compaction task handler stopped due to shutdown
INFO [shard 2] compaction_manager - compaction task handler stopped due to shutdown
INFO [shard 3] compaction_manager - compaction task handler stopped due to shutdown
INFO [shard 3] compaction_manager - compaction task handler stopped due to shutdown
after:
^CINFO [shard 0] compaction_manager - Asked to stop
INFO [shard 0] compaction_manager - Stopped
INFO [shard 1] compaction_manager - Asked to stop
INFO [shard 2] compaction_manager - Asked to stop
INFO [shard 3] compaction_manager - Asked to stop
INFO [shard 1] compaction_manager - Stopped
INFO [shard 2] compaction_manager - Stopped
INFO [shard 3] compaction_manager - Stopped
`compaction_manager - compaction task handler stopped due to shutdown` is still printed
in debug level
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <535d5ad40102571a3d5d36257342827989e8f0f4.1455835407.git.raphaelsc@scylladb.com>
