Refs #752
Paged aggregate queries will re-use the partition_slice object,
thus when setting a specific ck range for "last pk", we will hit
an exception case.
Allow removing entries (actually only the one), and overwriting
(using schema equality for keys), so we maintain the interface
while allowing the pager code to re-set the ck range for previous
page pass.
[tgrabiec: commit log cleanup, fixed issue ref]
Message-Id: <1452616259-23751-1-git-send-email-calle@scylladb.com>
Fixes#752
We set row limit for query to be min of page size/remaining in limit,
but if we have a multinode query we might end up with more rows than asked
for, so must do this again in post-processing.
Message-Id: <1452606935-12899-2-git-send-email-calle@scylladb.com>
According to specification
(here https://wiki.apache.org/cassandra/InternodeEncryption)
when the internode encryption is set to `dc` the data passed between
DCs should be encrypted and similarly, when it's set to `rack`
the inter-rack traffic should encrypted.
Currently Scylla would encrypt the traffic inside a local DC in the
first case and inside the local RACK in the later one.
This patch fixes the encryption logic to follow the specification
above.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1452501794-23232-1-git-send-email-vladz@cloudius-systems.com>
With Docker we might be running on a filesystem that does not support DMA
(aufs; or tmpfs on boot2docker), so let --developer-mode allow running
on those file systems.
Message-Id: <1452593083-25601-1-git-send-email-avi@scylladb.com>
yum command think "development-xxxx.xxxx" is older than "0.x", so nightly package mistakenly update with release version.
To prevent this problem, we should add greater number prior to "development".
Also same on Ubuntu package.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452592877-29721-1-git-send-email-syuu@scylladb.com>
Use systemd Type=notify to tell systemd about startup progress.
We can now use 'systemctl status scylla-server' to see where we are
in service startup, and 'systemctl start scylla-server' will wait until
either startup is complete, or we fail to start up.
read_entry did not verify that current chunk has enough data left
for a minimal entry. Thus we could try to read an entry from the slack
left in a chunk, and get lost in the file (pos > next, skip very much
-> eof). And also give false errors about corruption.
Message-Id: <1452517700-599-1-git-send-email-calle@scylladb.com>
"Our domain objects have schema version dependent format, for efficiency
reasons. The data structures which map between columns and values rely on
column ids, which are consecutive integers. For example, we store cells in a
vector where index into the vector is an implicit column id identifying table
column of the cell. When columns are added or removed the column ids may
shift. So, to access mutations or query results one needs to know the version
of the schema corresponding to it.
In case of query results, the schema version to which it conforms will always
be the version which was used to construct the query request. So there's no
change in the way query result consumers operate to handle schema changes. The
interfaces for querying needed to be extended to accept schema version and do
the conversions if necessary.
Shard-local interfaces work with a full definition of schema version,
represented by the schema type (usually passed as schema_ptr). Schema versions
are identified across shards and nodes with a UUID (table_schema_version
type). We maintain schema version registry (schema_registry) to avoid fetching
definitions we already know about. When we get a request using unknown schema,
we need to fetch the definition from the source, which must know it, to obtain
a shard-local schema_ptr for it.
Because mutation representation is schema version dependent, mutations of
different versions don't necessarily commute. When a column is dropped from
schema, the dropped column is no longer representable in the new schema. It is
generally fine to not hold data for dropped columns, the intent behind
dropping a column is to lose the data in that column. However, when merging an
incoming mutation with an existing mutation both of which have different
schema versions, we'd have to choose which schema should be considered
"latest" in order not to loose data. Schema changes can be made concurrently
in the cluster and initiated on different nodes so there is not always a
single notion of latest schema. However, schema changes are commutative and by
merging changes nodes eventually agree on the version. For example adding
column A (version X) on one node and adding column B (version Y) on another
eventually results in a schema version with both A and B (version Z). We
cannot tell which version among X and Y is newer, but we can tell that version
Z is newer than both X and Y. So the solution to the problem of merging
conflicting mutations could be to ensure that such merge is performed using
the schema which is superior to schemas of both mutations.
The approach taken in the series for ensuring this is as follows. When a node
receives a mutation of an unknown schema version it first performs a schema
merge with the source of that mutation. Schema merge makes sure that current
node's version is superior to the schema of incoming mutation. Once the
version is synced with, it is remembered as such and won't be synced with on
later mutations. Because of this bookkeeping, schema versions must be
monotonic; we don't want table altering to result in any earlier version
because that would cause nodes to avoid syncing with them. The version is a
cryptographically-secure hash of schema mutations, which should fulfill this
purpose in practice.
TODO: It's possible that the node is already performing a sync triggered by
broadcasted schema mutations. To avoid triggering a second sync needlessly, the
schema merging should mark incoming versions as being synced with.
Each table shard keeps track of its current schema version, which is
considered to be superior to all versions which are going to be applied to it.
All data sources for given column family within a shard have the same notion
of current schema version. Individual entries in cache and memtables may be at
earlier versions but this is hidden behind the interface. The entries are
upgraded to current version lazily on access. Sstables are immutable, so they
don't need to track current version. Like any other data source, they can be
queried with any schema version.
Note, the series triggered a bug in demangler:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68700"
* seastar d0bf6f8...ad3577b (9):
> httpd: close connection before deleting it
> reactor: support for non-O_DIRECT capable filesystems
> tests: modernize linecount
> IO queues: destruct within reactor's destructor
> tests: Use dnsdomainname in mkcert.gmk
> tests: memcached: workaround a possible race between flush_all and read
> apps: memcached: reduce the error during the expiration time translation
> timer: add missing #include
> core: do not call open_file_dma directly
Fixes#757.
The MOTD banner now printed upon .bash_profile execution,
if scylla is running, ends with a 'tput sgr0'. That command
appends an extra '[m' at the beginning of the output of any
following command. The automation scripts don't like this.
So let's add an 'echo' at the end of that path to add a newline,
avoiding the condition described above, and another one at the
'ScyllaDB is not started' path, for symmetry. I'm doing this
as it seems easier than having to develop heuristics to know
whether to remove or not that character.
CC: Shlomi Livne <slivne@scylladb.com>
Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
Message-Id: <1452216044-28374-1-git-send-email-lmr@scylladb.com>
Exceptions originated by an unimplemented to_string() methods
may interrupt the query() flow if not intercepted. Don't let it
happen.
Fixes issue #768
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
We want the statements to be removed before we ack the schema change,
otherwise it will race with all future operations.
Since the subscriber will be invoked on each shard, there is no need
to broadcast to all shards, we can just handle current shard.
Replicates https://issues.apache.org/jira/browse/CASSANDRA-7910 :
"Prepare a statement with a wildcard in the select clause.
2. Alter the table - add a column
3. execute the prepared statement
Expected result - get all the columns including the new column
Actual result - get the columns except the new column"
Currently the notify_*() method family broadcasts to all shards, so
schema merging code invokes them only on shard 0, to avoid doubling
notifications. We can simplify this by making the notify_*() methods
per-instance and thus shard-local.
