Compare commits

...

75 Commits

Author SHA1 Message Date
Nadav Har'El
3d22f42cf9 Merge 'select statement: verify EXECUTE permissions only for non native functions' from Eliran Sinvani
Commit 62458b8e4f introduced the enforcement of EXECUTE permissions of functions in cql select. However, according to the reference in #12869, the permissions should be enforced only on UDFs and UDAs.
The code does not distinguish between the two so the permissions are also unintenionally enforced also on native function. This commit introduce the distinction and only enforces the permissions on non native functions.

Fixes #16526

Manually verified (before and after change) with the reproducer supplied in #16526 and also with some the `min` and `max` native functions.
Also added test that checks for regression on native functions execution and verified that it fails on authorization before
the fix and passes after the fix.

Closes scylladb/scylladb#16556

* github.com:scylladb/scylladb:
  test.py: Add test for native functions permissions
  select statement: verify EXECUTE permissions only for non native functions

(cherry picked from commit fc71c34597)
2023-12-27 14:30:52 +02:00
Avi Kivity
8ca5794756 Merge 'cql: fix regression in SELECT * GROUP BY' from Nadav Har'El
This short series fixes a regression from Scylla 5.2 to Scylla 5.4 in "SELECT * GROUP BY" - this query was supposed to return just a single row from each partition (the first one in clustering order), but after the expression rewrite started to wrongly return all rows.

The series also includes a regression test that verifies that this query works doesn't work correctly before this series, but works with this patch - and also works as expected in Scylla 5.2 and in Cassadra.

Fixes #16531.

Closes scylladb/scylladb#16559

* github.com:scylladb/scylladb:
  test/cql-pytest: check that most aggregators don't take "*"
  cql-pytest: add reproducer for GROUP BY regression
  cql: fix regression in SELECT * GROUP BY

(cherry picked from commit 3968fc11bf)
2023-12-26 10:47:06 +02:00
Anna Stuchlik
abeeefb427 doc: add Raft verification to 5.4 upgrade
This commit adds the Raft verification step
to the 5.2-to-5.4 upgrade guide.

It is V2 of https://github.com/scylladb/scylladb/pull/16347.

Closes scylladb/scylladb#16481
2023-12-20 11:43:01 +01:00
Botond Dénes
9c482ff262 Update tools/java submodule
* tools/java 9387ac10...fcfe7b7c (1):
  > Merge "build: take care of old libthrift" from Piotr Grabowski

Fixes: scylladb/scylla-tools-java#352

Closes scylladb/scylladb#16463
2023-12-19 17:38:11 +02:00
Takuya ASADA
bfc98d1909 dist: fix local-fs.target dependency
systemd man page says:

systemd-fstab-generator(3) automatically adds dependencies of type Before= to
all mount units that refer to local mount points for this target unit.

So "Before=local-fs.taget" is the correct dependency for local mount
points, but we currently specify "After=local-fs.target", it should be
fixed.

Also replaced "WantedBy=multi-user.target" with "WantedBy=local-fs.target",
since .mount are not related with multi-user but depends local
filesystems.

Fixes #8761

Closes scylladb/scylladb#15647

(cherry picked from commit a23278308f)
2023-12-19 13:14:22 +02:00
Botond Dénes
2cef52aeaa Update tools/java submodule
* tools/java f9cce789...9387ac10 (2):
  > build: update logback dependency
  > build: update `netty` dependency

Fixes: https://github.com/scylladb/scylla-tools-java/issues/363
Fixes: https://github.com/scylladb/scylla-tools-java/issues/364

Closes scylladb/scylladb#16442
2023-12-18 17:13:05 +02:00
Alexey Novikov
a55561fc64 When add duration field to UDT check whether this UDT is used in some clustering key
Having values of the duration type is not allowed for clustering
columns, because duration can't be ordered. This is correctly validated
when creating a table but do not validated when we alter the type.

Fixes #12913

Closes scylladb/scylladb#16022

(cherry picked from commit bd73536b33)
2023-12-18 14:22:25 +02:00
Raphael S. Carvalho
7288bdfe09 sstables: Fix update of tombstone GC settings to have immediate effect
After "repair: Get rid of the gc_grace_seconds", the sstable's schema (mode,
gc period if applicable, etc) is used to estimate the amount of droppable
data (or determine full expiration = max_deletion_time < gc_before).
It could happen that the user switched from timeout to repair mode, but
sstables will still use the old mode, despite the user asked for a new one.
Another example is when you play with value of grace period, to prevent
data resurrection if repair won't be able to run in a timely manner.
The problem persists until all sstables using old GC settings are recompacted
or node is restarted.
To fix this, we have to feed latest schema into sstable procedures used
for expiration purposes.

Fixes #15643.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>

Closes scylladb/scylladb#15746

(cherry picked from commit fded314e46)
2023-12-18 14:14:02 +02:00
Eliran Sinvani
ac7ed6857a use_statement: Covert an exception to a future exception
The use statement execution code can throw if the keyspace is
doesn't exist, this can be a problem for code that will use
execute in a fiber since the exception will break the fiber even
if `then_wrapped` is used.

Fixes #14449

Signed-off-by: Eliran Sinvani <eliransin@scylladb.com>

Closes scylladb/scylladb#14394

(cherry picked from commit c5956957f3)
2023-12-15 13:03:05 +02:00
Nadav Har'El
bc8ff68cf6 cql: fix SELECT toJson() or SELECT JSON of time column
The implementation of "SELECT TOJSON(t)" or "SELECT JSON t" for a column
of type "time" forgot to put the time string in quotes. The result was
invalid JSON. This is patch is a one-liner fixing this bug.

This patch also removes the "xfail" marker from one xfailing test
for this issue which now starts to pass. We also add a second test for
this issue - the existing test was for "SELECT TOJSON(t)", and the second
test shows that "SELECT JSON t" had exactly the same bug - and both are
fixed by the same patch.

We also had a test translated from Cassandra which exposed this bug,
but that test continues to fail because of other bugs, so we just
need to update the xfail string.

The patch also fixes one C++ test, test/boost/json_cql_query_test.cc,
which enshrined the *wrong* behavior - JSON output that isn't even
valid JSON - and had to be fixed. Unlike the Python tests, the C++ test
can't be run against Cassandra, and doesn't even run a JSON parser
on the output, which explains how it came to enshrine wrong output
instead of helping to discover the bug.

Fixes #7988

Signed-off-by: Nadav Har'El <nyh@scylladb.com>

Closes scylladb/scylladb#16121

(cherry picked from commit 8d040325ab)
2023-12-15 11:41:47 +02:00
Israel Fruchter
0974ef893e docker: put cqlsh configuration in correct place
since always we were putting cqlsh configuration into `~/.cqlshrc`
acording to commit from 8 years ago [1], this path is deprecated.

until this commit [2], actully remove this path from cqlsh code

as part of moving to scylla-cqlsh, we got [2], and didn't
notice until the first release with it.

this change write the configuration into `~/.casssndra/cqlshrc`
as this is the default place cqlsh is looking.

[1]: 13ea8a6669/bin/cqlsh.py (L264)
[2]: 2024ea4796
Fixes: scylladb/scylladb#16329

Closes scylladb/scylladb#16340

(cherry picked from commit 514ef48d75)
2023-12-14 14:15:40 +02:00
Botond Dénes
9fc4c265a5 Merge 'mutation_query: properly send range tombstones in reverse queries' from Michał Chojnowski
reconcilable_result_builder passes range tombstone changes to _rt_assembler
using table schema, not query schema.
This means that a tombstone with bounds (a; b), where a < b in query schema
but a > b in table schema, will not be emitted from mutation_query.

This is a very serious bug, because it means that such tombstones in reverse
queries are not reconciled with data from other replicas.
If *any* queried replica has a row, but not the range tombstone which deleted
the row, the reconciled result will contain the deleted row.

In particular, range deletes performed while a replica is down will not
later be visible to reverse queries which select this replica, regardless of the
consistency level.

As far as I can see, this doesn't result in any persistent data loss.
Only in that some data might appear resurrected to reverse queries,
until the relevant range tombstone is fully repaired.

This series fixes the bug and adds a minimal reproducer test.

Fixes #10598

Closes scylladb/scylladb#16003

* github.com:scylladb/scylladb:
  mutation_query_test: test that range tombstones are sent in reverse queries
  mutation_query: properly send range tombstones in reverse queries

(cherry picked from commit 65e42e4166)
2023-12-14 12:52:51 +02:00
Botond Dénes
0518e47daf Update tools/java submodule
* tools/java 3764ae94...f9cce789 (1):
  > Merge "print more informative error when fail to parse sstable generation" from Kefu Chai

Fixes: scylladb/scylla-tools-java#360
2023-12-12 09:49:35 +02:00
Yaron Kaikov
1e8eb6172a build_docker.sh: Upgrade package during creation and remove sshd service
When scanning our latest docker image using `trivy` (command: `trivy
image docker.io/scylladb/scylla-nightly:latest`), it shows we have OS
packages which are out of date.

Also removing `openssh-server` and `openssh-client` since we don't use
it for our docker images

Fixes: https://github.com/scylladb/scylladb/issues/16222

Closes scylladb/scylladb#16224

(cherry picked from commit 7ce6962141)

Closes scylladb/scylladb#16359
2023-12-11 10:56:46 +02:00
Paweł Zakrzewski
14814c972e auth: fix error message when consistency level is not met
Propagate `exceptions::unavailable_exception` error message to the
client such as cqlsh.

Fixes #2339

(cherry picked from commit 400aa2e932)
2023-12-07 14:49:20 +02:00
Jenkins Promoter
7a67db594a Update ScyllaDB version to: 5.4.1 2023-12-06 16:58:35 +02:00
Anna Stuchlik
5434fcb5a8 doc: replace the OSS-only link on the Raft page
This commit replaces the link to the OSS-only page
(the 5.2-to-5.4 upgrade guide not present in
the Enterprise docs) on the Raft page.

While providing the link to the specific upgrade
guide is more user-friendly, it causes build failures
of the Enterprise documentation. I've replaced
it with the link to the general Upgrade section.

The ".. only:: opensource" directive used to wrap
the OSS-only content correctly excludes the content
form the Enterprise docs - but it doesn't prevent
build warnings.

This commit must be backported to branch-5.4 to
prevent errors in all versions.

Closes scylladb/scylladb#16176

(cherry picked from commit 24d5dbd66f)
2023-12-06 13:19:03 +02:00
Nadav Har'El
b4ef2248cc Backport fixes for nodetool commands with Alternator GSI in the database
Fixes #16153

* jmx 166599f...f45067f (3):
  > ColumnFamilyStore: only quote table names if necessary
  > APIBuilder: allow quoted scope names
  > ColumnFamilyStore: don't fail if there is a table with ":" in its name

* java dfbf3726ee...3764ae94db (1):
  > NodeProbe: allow addressing table name with colon in it

Signed-off-by: Nadav Har'El <nyh@scylladb.com>

Closes scylladb/scylladb#16294
2023-12-06 10:46:54 +02:00
Anna Stuchlik
21996e12ae doc: enabling experimental Raft-managed topology
This commit adds a short paragraph to the Raft
page to explain how to enable consistent
topology updates with Raft - an experimental
feature in version 5.4.

The paragraph should satisfy the requirements
for version 5.4. The Raft page will be
rewritten in the next release when consistent
topology changes with Raft will be GA.

Fixes https://github.com/scylladb/scylladb/issues/15080

Requires backport to branch-5.4.

Closes scylladb/scylladb#16273

(cherry picked from commit 409e20e5ab)
2023-12-06 08:57:46 +02:00
Anna Stuchlik
df7b96a092 doc: add metric upgrade info to the 5.4 upgrade
This commit adds the information about metrics
update to the 5.2-to-5.4 upgrade guide.

Fixes https://github.com/scylladb/scylladb/issues/15966

Closes scylladb/scylladb#16161

(cherry picked from commit 97244eb68e)
2023-12-05 15:17:28 +02:00
Anna Stuchlik
5df85094d9 doc: fix rollback in the 4.6-to-5.0 upgrade guide
This commit fixes the rollback procedure in
the 4.6-to-5.0 upgrade guide:
- The "Restore system tables" step is removed.
- The "Restore the configuration file" command
  is fixed.
- The "Gracefully shutdown ScyllaDB" command
  is fixed.

In addition, there are the following updates
to be in sync with the tests:

- The "Backup the configuration file" step is
  extended to include a command to backup
  the packages.
- The Rollback procedure is extended to restore
  the backup packages.
- The Reinstallation section is fixed for RHEL.

Refs https://github.com/scylladb/scylladb/issues/11907

This commit must be backported to branch-5.4, branch-5.2, and branch-5.1

Closes scylladb/scylladb#16155

(cherry picked from commit 1e80bdb440)
2023-12-05 15:09:59 +02:00
Anna Stuchlik
a0ca8900e1 doc: fix rollback in the 5.0-to-5.1 upgrade guide
This commit fixes the rollback procedure in
the 5.0-to-5.1 upgrade guide:
- The "Restore system tables" step is removed.
- The "Restore the configuration file" command
  is fixed.
- The "Gracefully shutdown ScyllaDB" command
  is fixed.

In addition, there are the following updates
to be in sync with the tests:

- The "Backup the configuration file" step is
  extended to include a command to backup
  the packages.
- The Rollback procedure is extended to restore
  the backup packages.
- The Reinstallation section is fixed for RHEL.

Also, I've the section removed the rollback
section for images, as it's not correct or
relevant.

Refs https://github.com/scylladb/scylladb/issues/11907

This commit must be backported to branch-5.4, branch-5.2, and branch-5.1

Closes scylladb/scylladb#16154

(cherry picked from commit 7ad0b92559)
2023-12-05 15:07:58 +02:00
Anna Stuchlik
98bd287177 doc: fix rollback in the 5.1-to-5.2 upgrade guide
This commit fixes the rollback procedure in
the 5.1-to-5.2 upgrade guide:
- The "Restore system tables" step is removed.
- The "Restore the configuration file" command
  is fixed.
- The "Gracefully shutdown ScyllaDB" command
  is fixed.

In addition, there are the following updates
to be in sync with the tests:

- The "Backup the configuration file" step is
  extended to include a command to backup
  the packages.
- The Rollback procedure is extended to restore
  the backup packages.
- The Reinstallation section is fixed for RHEL.

Also, I've the section removed the rollback
section for images, as it's not correct or
relevant.

Refs https://github.com/scylladb/scylladb/issues/11907

This commit must be backported to branch-5.4 and branch-5.2.

Closes scylladb/scylladb#16152

(cherry picked from commit 91cddb606f)
2023-12-05 14:57:24 +02:00
Anna Stuchlik
c4a249022f doc: fix rollback for RHEL (install) in 5.4
This commit fixes the installation command
in the Rollback section for RHEL/Centos
in the 5.2-5.4 upgrade guide.

It's a follow-up to https://github.com/scylladb/scylladb/pull/16114
where the command was not updated.

Refs https://github.com/scylladb/scylladb/issues/11907

This commit must be backported to branch-5.4.

Closes scylladb/scylladb#16156

(cherry picked from commit 52c2698978)
2023-12-05 14:56:20 +02:00
Jenkins Promoter
58a89e7a42 Update ScyllaDB version to: 5.4.0 2023-12-04 15:00:40 +02:00
Botond Dénes
1a0424db01 Update ./tools/jmx submodule
* ./tools/jmx 9a03d4fa...166599f0 (1):
  > StorageService: Normalize endpoint inetaddress strings to java form

Fixes: scylladb/scylladb#16039
2023-12-04 12:53:13 +02:00
Botond Dénes
6d7919041b Merge 'row_cache: abort on exteral_updater::execute errors' from Benny Halevy
Currently the cache updaters aren't exception safe
yet they are intended to be.

Instead of allowing exceptions from
`external_updater::execute` escape `row_cache::update`,
abort using `on_fatal_internal_error`.

Future changes should harden all `execute` implementations
to effectively make them `noexcept`, then the pure virtual
definition can be made `noexcept` to cement that.

\Fixes scylladb/scylladb#15576

\Closes scylladb/scylladb#15577

* github.com:scylladb/scylladb:
  row_cache: abort on exteral_updater::execute errors
  row_cache: do_update: simplify _prev_snapshot_pos setup

(cherry picked from commit 4a0f16474f)

Closes scylladb/scylladb#16256
2023-12-03 20:52:03 +02:00
Jenkins Promoter
17c15f6222 Update ScyllaDB version to: 5.4.0-rc4 2023-12-03 20:28:53 +02:00
Michał Chojnowski
91d1c9153b position_in_partition: make operator= exception-safe
The copy assignment operator of _ck can throw
after _type and _bound_weight have already been changed.
This leaves position_in_partition in an inconsistent state,
potentially leading to various weird symptoms.

The problem was witnessed by test_exception_safety_of_reads.
Specifically: in cache_flat_mutation_reader::add_to_buffer,
which requires the assignment to _lower_bound to be exception-safe.

The easy fix is to perform the only potentially-throwing step first.

Fixes #15822

Closes scylladb/scylladb#15864

(cherry picked from commit 93ea3d41d8)
2023-11-30 15:00:39 +02:00
Avi Kivity
95364e2454 Update seastar submodule (spins on epoll)
* seastar bab1625cf3...95a38bb0c6 (1):
  > epoll: Avoid spinning on aborted connections

Fixes #12774
Fixes #7753
Fixes #13337
2023-11-30 14:07:17 +02:00
Avi Kivity
6d779f58a9 Update seastar submodule to scylla-seastar.git
This lets us backport seastar patches into branch-5.4.
2023-11-30 14:07:17 +02:00
Jenkins Promoter
b956646ba2 Update ScyllaDB version to: 5.4.0-rc3 2023-11-29 14:29:54 +02:00
Anna Stuchlik
62b93018ac doc: add experimental support for object storage
This commit adds information on how to enable
object storage for a keyspace.

The "Keyspace storage options" section already
existed in the doc, but it was not valid as
the support was only added in version 5.4

The scope of this commit:
- Update the "Keyspace storage options" section.
- Add the information about object storage support
  to the Data Definition> CREATE KEYSPACE section
  * Marked as "Experimental".
  * Excluded from the Enterprise docs with the
    .. only:: opensource directive.

This commit must be backported to branch-5.4,
as support for object storage was added
in version 5.4.

Closes scylladb/scylladb#16081

(cherry picked from commit bfe19c0ed2)
2023-11-29 08:39:12 +02:00
Piotr Grabowski
b0410c9391 install-dependencies.sh: update node_exporter to 1.7.0
Update node_exporter to 1.7.0.

The previous version (1.6.1) was flagged by security scanners (such as
Trivy) with HIGH-severity CVE-2023-39325. 1.7.0 release fixed that
problem.

[Botond: regenerate frozen toolchain]

Fixes #16085

Closes scylladb/scylladb#16086

Closes scylladb/scylladb#16090

(cherry picked from commit 321459ec51)

[avi: regenerate frozen toolchain]
2023-11-27 16:48:30 +00:00
Botond Dénes
6f073dfa54 Update ./tools/jmx and ./tools/java submodules
* ./tools/jmx 8d15342e...9a03d4fa (1):
  > Merge "scylla-apiclient: update several Java dependencies" from Piotr Grabowski

* ./tools/java 3c09ab97...dfbf3726 (1):
  > Merge 'build: update several dependencies' from Piotr Grabowski

Update build dependencies which were flagged by security scanners.

Refs: scylladb/scylla-jmx#220
Refs: scylladb/scylla-tools-java#351

Closes scylladb/scylladb#16149
2023-11-23 18:34:24 +02:00
Anna Stuchlik
a24b53e6bb doc: fix rollback in the 5.2-to-5.4 upgrade guide
This commit fixes the rollback procedure in
the 5.2-to-5.4 upgrade guide:
- The "Restore system tables" step is removed.
- The "Restore the configuration file" command
  is fixed.
- The "Gracefully shutdown ScyllaDB" command
  is fixed.

In addition, there are the following updates
to be in sync with the tests:

- The "Backup the configuration file" step is
  extended to include a command to backup
  the packages.
- The Rollback procedure is extended to restore
  the backup packages.
- The Reinstallation section is fixed for RHEL.

Also, I've removed the optional step to enable
consistent schema management from the list of
steps - the appropriate section has already
been removed, but it remained in the procedure
description, which was misleading.

Refs https://github.com/scylladb/scylladb/issues/11907

This commit must be backported to branch-5.4

Closes scylladb/scylladb#16114

(cherry picked from commit 3751acce42)
2023-11-23 10:10:20 +02:00
Anna Mikhlin
219adcea71 release: prepare for 5.4.0-rc2 2023-11-21 13:48:38 +00:00
Nadav Har'El
6d01d01deb Merge 'Materialize_views: don't construct global_schema_ptr from views schemas that lacks base information' from Eliran Sinvani
This miniset addresses two potential conversions to `global_schema_ptr` of incomplete materialized views schemas.
One of them was completely unnecessary and also is a "chicken and an egg" problem where on the sync schema procedure itself a view schema was converted to `global_schema_ptr` solely for the purposes of logging. This can create a
"hickup" in the materialized views updates if they are comming from a node with a different mv schema.
The reason why sometimes a synced schema can have no base info is because of deactivision and reactivision of the schema inside the `schema_registry` which doesn't restore the base information due to lack of context.
When a schema is synced the problem becomes easy since we can just use the latest base information from the database.

Fixes #14011

Closes scylladb/scylladb#14861

* github.com:scylladb/scylladb:
  migration manager: fix incomplete mv schemas returned from get_schema_for_write
  migration_manager: do not globalize potentially incomplete schema

(cherry picked from commit 5752dc875b)
2023-11-21 09:53:24 +02:00
Botond Dénes
b1f54efc2d gms,service: add a feature to protect the usage of allow_mutation_read_page_without_live_row
allow_mutation_read_page_without_live_row is a new option in the
partition_slice::option option set. In a mixed clusters, old nodes
possibly don't know this new option, so its usage must be protected by a
cluster feature. This patch does just that.

Fixes: #15795

Closes scylladb/scylladb#15890

(cherry picked from commit f53961248d)
2023-11-21 09:49:04 +02:00
Botond Dénes
bc1202aab2 api/storage_service: start/stop native transport in the statement sg
Currently, it is started/stopped in the streaming/maintenance sg, which
is what the API itself runs in.
Starting the native transport in the streaming sg, will lead to severely
degraded performance, as the streaming sg has significantly less
CPU/disk shares and reader concurrency semaphore resources.
Furthermore, it will lead to multi-paged reads possibly switching
between scheduling groups mid-way, triggering an internal error.

To fix, use `with_scheduling_group()` for both starting and stopping
native transport. Technically, it is only strictly necessary for
starting, but I added it for stop as well for consistency.

Also apply the same treatment to RPC (Thrift). Although no one uses it,
best to fix it, just to be on the safe side.

I think we need a more systematic approach for solving this once and for
all, like passing the scheduling group to the protocol server and have
it switch to it internally. This allows the server to always run on the
correct scheduling group, not depending on the caller to remember using
it. However, I think this is best done in a follow-up, to keep this
critical patch small and easily backportable.

Fixes: #15485

Closes scylladb/scylladb#16019

(cherry picked from commit dfd7981fa7)
2023-11-20 19:47:49 +02:00
Takuya ASADA
2cb709461c scylla_post_install.sh: detect RHEL correctly
$ID_LIKE = "rhel" works only on RHEL compatible OSes, not for RHEL
itself.
To detect RHEL correctly, we also need to check $ID = "rhel".

Fixes #16040

Closes scylladb/scylladb#16041

(cherry picked from commit 338a9492c9)
2023-11-20 19:36:00 +02:00
Tomasz Grabiec
44c72f6e56 Merge 'Multishard mutation query test fix misses expectations' from Botond Dénes
There are two tests, test_read_all and test_read_with_partition_row_limits, which asserts on every page as well
as at the end that there are no misses whatsoever. This is incorrect, because it is possible that on a given page, not all shards participate and thus there won't be a saved reader on every shard. On the subsequent page, a shard without a reader may produce a miss. This is fine. Refine the asserts, to check that we have only as much misses, as many
shards we have without readers on them.

Fixes: https://github.com/scylladb/scylladb/issues/14087

Closes scylladb/scylladb#15806

* github.com:scylladb/scylladb:
  test/boost/multishard_mutation_query_test: fix querier cache misses expectations
  test/lib/test_utils: add require_* variants for all comparators

(cherry picked from commit 457d170078)
2023-11-19 19:34:44 +02:00
Marcin Maliszkiewicz
6943447c6a db: view: run local materialized view mutations on a separate smp service group
When base write triggers mv write and it needs to be send to another
shard it used the same service group and we could end up with a
deadlock.

This fix affects also alternator's secondary indexes.

Testing was done using (yet) not committed framework for easy alternator
performance testing: https://github.com/scylladb/scylladb/pull/13121.
I've changed hardcoded max_nonlocal_requests config in scylla from 5000 to 500 and
then ran:

./build/release/scylla perf-alternator-workloads --workdir /tmp/scylla-workdir/ --smp 2 \
--developer-mode 1 --alternator-port 8000 --alternator-write-isolation forbid --workload write_gsi \
--duration 60 --ring-delay-ms 0 --skip-wait-for-gossip-to-settle 0 --continue-after-error true --concurrency 2000

Without the patch when scylla is overloaded (i.e. number of scheduled futures being close to max_nonlocal_requests) after couple seconds
scylla hangs, cpu usage drops to zero, no progress is made. We can confirm we're hitting this issue by seeing under gdb:

p seastar::get_smp_service_groups_semaphore(2,0)._count
$1 = 0

With the patch I wasn't able to observe the problem, even with 2x
concurrency. I was able to make the process hang with 10x concurrency
but I think it's hitting different limit as there wasn't any depleted
smp service group semaphore and it was happening also on non mv loads.

Fixes https://github.com/scylladb/scylladb/issues/15844

Closes scylladb/scylladb#15845

(cherry picked from commit 020a9c931b)
2023-11-19 18:47:11 +02:00
Anna Stuchlik
b259bb43c6 doc: mark the link to upgrade guide as OSS-only
This commit adds the .. only:: opensource directive
to the Raft page to exclude the link to the 5.2-to-5.4
upgrade guide from the Enterprise documentation.

The Raft page belongs to both OSS and Enterprise
documentation sets, while the upgrade guide
is OSS-only. This causes documentation build
issues in the Enterprise repository, for example,
https://github.com/scylladb/scylla-enterprise/pull/3242.

As a rule, all OSS-only links should be provided
by using the .. only:: opensource directive.

This commit must be backported to branch-5.4
to prevent errors in the documentation for
ScyllaDB Enterprise 2024.1

(backport)

Closes scylladb/scylladb#16064

(cherry picked from commit ca22de4843)
2023-11-17 11:00:04 +02:00
Botond Dénes
88e96def63 migration_manager: also reload schema on enabling digest_insensitive_to_expiry
Currently, when said feature is enabled, we recalcuate the schema
digest. But this feature also influences how table versions are
calculated, so it has to trigger a recalculation of all table versions,
so that we can guarantee correct versions.
Before, this used to happen by happy accident. Another feature --
table_digest_insensitive_to_expiry -- used to take care of this, by
triggering a table version recalulation. However this feature only takes
effect if digest_insensitive_to_expiry is also enabled. This used to be
the case incidently, by the time the reload triggered by
table_digest_insensitive_to_expiry ran, digest_insensitive_to_expiry was
already enabled. But this was not guaranteed whatsoever and as we've
recently seen, any change to the feature list, which changes the order
in which features are enabled, can cause this intricate balance to
break.
This patch makes digest_insensitive_to_expiry also kick off a schema
reload, to eliminate our dependence on (unguaranteed) feature order, and
to guarantee that table schemas have a correct version after all features
are enabled. In fact, all schema feature notification handlers now kick
off a full schema reload, to ensure bugs like this don't creep in, in
the future.

Fixes: #16004

Closes scylladb/scylladb#16013

(cherry picked from commit 22381441b0)
2023-11-16 17:46:53 +02:00
Kamil Braun
187e275147 system_keyspace: use system memory for system.raft table
`system.raft` was using the "user memory pool", i.e. the
`dirty_memory_manager` for this table was set to
`database::_dirty_memory_manager` (instead of
`database::_system_dirty_memory_manager`).

This meant that if a write workload caused memory pressure on the user
memory pool, internal `system.raft` writes would have to wait for
memtables of user tables to get flushed before the write would proceed.

This was observed in SCT longevity tests which ran a heavy workload on
the cluster and concurrently, schema changes (which underneath use the
`system.raft` table). Raft would often get stuck waiting many seconds
for user memtables to get flushed. More details in issue #15622.
Experiments showed that moving Raft to system memory fixed this
particular issue, bringing the waits to reasonable levels.

Currently `system.raft` stores only one group, group 0, which is
internally used for cluster metadata operations (schema and topology
changes) -- so it makes sense to keep use system memory.

In the future we'd like to have other groups, for strongly consistent
tables. These groups should use the user memory pool. It means we won't
be able to use `system.raft` for them -- we'll just have to use a
separate table.

Fixes: scylladb/scylladb#15622

Closes scylladb/scylladb#15972

(cherry picked from commit f094e23d84)
2023-11-16 12:51:03 +01:00
Kamil Braun
7926e4e7eb Merge 'raft topology: handle abort exceptions better in fence_previous_coordinator' from Piotr Dulikowski
When topology coordinator tries to fence the previous coordinator it
performs a group0 operation. The current topology coordinator might be
aborted in the meantime, which will result in a `raft::request_aborted`
exception being thrown. After the fix to scylladb/scylladb#15728 was
merged, the exception is caught, but then `sleep_abortable` is called
which immediately throws `abort_requested_exception` as it uses the same
abort source as the group0 operation. The `fence_previous_coordinator`
function which does all those things is not supposed to throw
exceptions, if it does - it causes `raft_state_monitor_fiber` to exit,
completely disabling the topology coordinator functionality on that
node.

Modify the code in the following way:

- Catch `abort_requested_exception` thrown from `sleep_abortable` and
  exit the function if it happens. In addition to the described issue,
it will also handle the case when abort is requested while
`sleep_abortable` happens,
- Catch `raft::request_aborted` thrown from group0 operation, log the
  exception with lower verbosity and exit the function explicitly.

Finally, wrap both `fence_previous_coordinator` and `run` functions in a
`try` block with `on_fatal_internal_error` in the catch handler in order
to implement the behavior that adding `noexcept` was originally supposed
to introduce.

Fixes: scylladb/scylladb#15747

Closes scylladb/scylladb#15948

* github.com:scylladb/scylladb:
  raft topology: catch and abort on exceptions from topology_coordinator::run
  Revert "storage_service: raft topology: mark topology_coordinator::run function as noexcept"
  raft topology: don't print an error when fencing previous coordinator is aborted
  raft topology: handle abort exceptions from sleeping in fence_previous_coordinator

(cherry picked from commit 07e9522d6c)
2023-11-16 12:40:19 +01:00
Anna Stuchlik
23e4762baa doc: remove DateTieredCompactionStrategy
This commit removes support for DateTieredCompactionStrategy
from the documentation.

Support for DTCS was removed in 5.4, so this commit
must be backported to branch-5.4.

Refs https://github.com/scylladb/scylladb/issues/15869#issuecomment-1784181274

The information is already added to the 5.2-to-5.4
upgrade guide: https://github.com/scylladb/scylladb/pull/15988

(backport)

Closes scylladb/scylladb#16061

(cherry picked from commit e8129d9a0c)
2023-11-16 10:21:39 +02:00
Anna Stuchlik
1dad9cdfdf doc: document full support for RBNO
This commit updates the Repair-Based Node
Operations page. In particular:
- Information about RBNO enabled for all
  node operations is added (before 5.4, RBNO
  was enabled for the replace operation, while
  it was experimental for others).
- The content is rewritten to remove redundant
  information about previous versions.

The improvement is part of the 5.4 release.
This commit must be backported to branch-5.4

Closes scylladb/scylladb#16015

(cherry picked from commit 8a4a8f077a)
2023-11-15 16:58:33 +02:00
Botond Dénes
1bee785734 Merge '[Backport 5.4]: docs: render options with role' from Kefu Chai
since we use `sphinx_multiversion` for building multiple versions of document. and in #15860, we changed the way how options are rendered, so the same change should be applied to the branch which includes the option list.

to address the conflicts, in addition to #15860, the depended PRs are also backported. so, in this pull request, following PRs are backported:

- #15827
- #15782
- #15860

Closes scylladb/scylladb#16030

* github.com:scylladb/scylladb:
  docs: add divider using CSS
  docs: extract _clean_description as a filter
  docs: render option with role
  docs: update cofig params design
  docs: parse source files right into rst
2023-11-14 13:02:23 +02:00
Kefu Chai
df61c2c2ce docs: add divider using CSS
instead of hardwiring the formatting in the html code, do this using
CSS, more flexible this way.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
(cherry picked from commit ff12f1f678)
2023-11-14 13:59:35 +08:00
Kefu Chai
20b5896b7a docs: extract _clean_description as a filter
would be better to split the parser from the formatter. in future,
we can apply more filter on top of the exiting one.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
(cherry picked from commit 1694a7addc)
2023-11-14 13:59:35 +08:00
Kefu Chai
eff8157cea docs: render option with role
so we can cross-reference them with the syntax like

:confval:`alternator_timeout_in_ms`.

or even render an option like:

.. confval:: alternator_timeout_in_ms

in order to make the headerlink of the option visible,
a new CSS rule is added.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
(cherry picked from commit 9ddc639237)
2023-11-14 13:59:35 +08:00
David Garcia
043dd5cc12 docs: update cofig params design
Closes scylladb/scylladb#15827

(cherry picked from commit a5519c7c1f)
2023-11-14 13:59:35 +08:00
Kefu Chai
3f66f18f85 docs: parse source files right into rst
so we can render the rst without writing a temporary YAML.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
(cherry picked from commit 53dfb5661d)
2023-11-14 13:59:33 +08:00
Kamil Braun
9c7454993f docs: mention Raft getting enabled when upgrading to 5.4
Fixes: scylladb/scylladb#15952

Closes scylladb/scylladb#16000

(cherry picked from commit 3036a80334)
2023-11-13 10:03:41 +02:00
Anna Stuchlik
7c38cd9359 doc: fix info about in 5.4 upgrade guide
This commit fixes the information about
Raft-based consistent cluster management
in the 5.2-to-5.4 upgrade guide.

This a follow-up to https://github.com/scylladb/scylladb/pull/15880 and must be backported to branch-5.4.

In addition, it adds information about removing
DateTieredCompactionStrategy to the 5.2-to-5.4
upgrade guide, including the guideline to
migrate to TimeWindowCompactionStrategy.

Closes scylladb/scylladb#15988

(cherry picked from commit ca0f5f39b5)
2023-11-13 10:03:29 +02:00
Anna Stuchlik
66898b2144 doc: update cqlsh compatibility with Python
This commit updates the cqlsh compatibility
with Python to Python 3.

In addition it:
- Replaces "Cassandra" with "ScyllaDB" in
  the description of cqlsh.
  The previous description was outdated, as
  we no longer can talk about using cqlsh
  released with Cassandra.
- Replaces occurrences of "Scylla" with "ScyllaDB".
- Adds additional locations of cqlsh (Docker Hub
  and PyPI), as well as the link to the scylla-cqlsh
  repository.

Closes scylladb/scylladb#16016

(cherry picked from commit 8d618bbfc6)
2023-11-13 10:03:12 +02:00
Botond Dénes
9e33771e1b repair/repair.cc: do_repair_ranges(): prevent stalls when skipping ranges
We have observed do_repair_ranges() receiving tens of thousands of
ranges to repairs on occasion. do_repair_ranges() repairs all ranges in
parallel, with parallel_for_each(). This is normally fine, as the lambda
inside parallel_for_each() takes a semaphore and this will result in
limited concurrency.
However, in some instances, it is possible that most of these ranges are
skipped. In this case the lambda will become synchronous, only logging a
message. This can cause stalls beacuse there are no opportunities to
yield. Solve this by adding an explicit yield to prevent this.

Fixes: #14330

Closes scylladb/scylladb#15879

(cherry picked from commit 90a8489809)
2023-11-08 20:50:07 +02:00
David Garcia
b25859d6de docs: add dynamic version on aws images extension
Closes scylladb/scylladb#15940

(cherry picked from commit afaeb30930)
2023-11-08 08:38:03 +02:00
Botond Dénes
38a3fd4708 Merge 'doc: add the upgrade guide from 5.2 to 5.4' from Anna Stuchlik
This PR adds the 5.2-5.4 upgrade guide.
In addition, it removes the redundant upgrade guide from 5.2 to 5.3 (as 5.3 was skipped), as well as some mentions of version 5.3.

This PR must be backported to branch-5.4.

Closes scylladb/scylladb#15880

* github.com:scylladb/scylladb:
  doc: add the upgrade guide from 5.2 to 5.4
  doc: remove version "5.3" from the docs
  doc: remove the 5.2-to-5.3 upgrade guide

(cherry picked from commit 74f68a472f)
2023-11-07 17:00:08 +02:00
Pavel Emelyanov
66be0fc1eb Merge 'api: failure_detector: invoke on shard 0' from Kamil Braun
These APIs may return stale or simply incorrect data on shards
other than 0. Newer versions of Scylla are better at maintaining
cross-shard consistency, but we need a simple fix that can be easily and
without risk be backported to older versions; this is the fix.

Add a simple test to check that the `failure_detector/endpoints`
API returns nonzero generation.

Fixes: scylladb/scylladb#15816

Closes scylladb/scylladb#15970

* github.com:scylladb/scylladb:
  test: rest_api: test that generation is nonzero in `failure_detector/endpoints`
  api: failure_detector: fix indentation
  api: failure_detector: invoke on shard 0

(cherry picked from commit 9443253f3d)
2023-11-07 14:50:41 +01:00
Anna Stuchlik
4345b26eb2 doc: update package installation in version 5.4
This commit updates the package installation
instructions in version 5.4.
- It updates the variables to include "5.4"
  as the version name.
- It adds the information for the newly supported
  Rocky/RHEL 9 - a new EPEL download link is required.

Closes scylladb/scylladb#15963

(cherry picked from commit 1e0cbfe522)
2023-11-07 10:33:45 +02:00
Anna Stuchlik
c4e8557afa doc: add OS support in version 5.4
This commit adds OS support information
in version 5.4 (removing the non-released
version 5.3).

In particular, it adds support for Oracle Linux
and Amazon Linux.

Also, it removes support for outdated versions.

Closes scylladb/scylladb#15923

(cherry picked from commit 3756705520)
2023-11-07 10:33:28 +02:00
Anna Stuchlik
6d91d560ec doc: add the latest AWS image info to Installation
This commit adds the AWS image information for
the latest patch release to the Launch on AWS
page in the installation section.

This is a follow-up PR required to finalize
the AWS installation docs and should be
backported to branch-5.4.

Related:
https://github.com/scylladb/scylladb/pull/14153
https://github.com/scylladb/scylladb/pull/15651

Closes scylladb/scylladb#15867

(cherry picked from commit d4b1e8441a)
2023-11-07 09:58:22 +02:00
Kamil Braun
9c37f5e02f Merge 'raft: topology: outside topology-on-raft mode, make sure not to use its RPCs' from Piotr Dulikowski
Topology on raft is still an experimental feature. The RPC verbs
introduced in that mode shouldn't be used when it's disabled, otherwise
we lose the right to make breaking changes to those verbs.

First, make sure that the aforementioned verbs are not sent outside the
mode. It turns out that `raft_pull_topology_snapshot` could be sent
outside topology-on-raft mode - after the PR, it no longer can.

Second, topology-on-raft mode verbs are now not registered at all on the
receiving side when the mode is disabled.

Additionally tested by running `topology/` tests with
`consistent_cluster_management: True` but with experimental features
disabled.

Fixes: scylladb/scylladb#15862

Closes scylladb/scylladb#15917

* github.com:scylladb/scylladb:
  storage_service: fix indentation
  raft: topology: only register verbs in topology-on-raft mode
  raft: topology: only pull topology snapshot in topology-on-raft mode

(cherry picked from commit 5cf18b18b2)
2023-11-03 09:19:21 +01:00
Raphael S. Carvalho
ed7b3e2325 replica: Fix major compaction semantics by performing off-strategy first
Major compaction semantics is that all data of a table will be compacted
together, so user can expect e.g. a recently introduced tombstone to be
compacted with the data it shadows.
Today, it can happen that all data in maintenance set won't be included
for major, until they're promoted into main set by off-strategy.
So user might be left wondering why major is not having the expected
effect.
To fix this, let's perform off-strategy first, so data in maintenance
set will be made available by major. A similar approach is done for
data in memtable, so flush is performed before major starts.
The only exception will be data in staging, which cannot be compacted
until view building is done with it, to avoid inconsistency in view
replicas.
The serialization in comapaction manager of reshape jobs guarantee
correctness if there's an ongoing off-strategy on behalf of the
table.

Fixes #11915.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>

Closes scylladb/scylladb#15792

(cherry picked from commit ea6c281b9f)
2023-10-30 12:36:48 +02:00
Avi Kivity
1100a0b176 Merge 'cql3: mutation_fragments_select_statement: keep erm alive for duration of the query' from Botond Dénes
Said statement keeps a reference to erm indirectly, via a topology node pointer, but doesn't keep erm alive. This can result in use-after-free. Furthermore, it allows for vnodes being pulled from under the query's feet, as it is running.
To prevent this, keep the erm alive for the duration of the query.
Also, use `host_id` instead of `node`, the node pointer is not needed really, as the statement only uses the host id from it.

Fixes: #15802

Closes scylladb/scylladb#15808

* github.com:scylladb/scylladb:
  cql3: mutation_fragments_select_statement: use host_id instead of node
  cql3: mutation_fragments_select_statement: pin erm reference

(cherry picked from commit 782c6a208a)
2023-10-30 10:17:19 +02:00
Gleb Natapov
2aa29763af storage_service: raft topology: do not throw error from fence_previous_coordinator()
Throwing error kills the topology coordinator monitor fiber. Instead we
retry the operation until it succeeds or the node looses its leadership.
This is fine before for the operation to succeed quorum is needed and if
the quorum is not available the node should relinquish its leadership.

Fixes #15728

(cherry picked from commit 65bf5877e7)
2023-10-29 13:59:19 +02:00
Anna Mikhlin
24efacf90d release: prepare for 5.4.0-rc1 2023-10-26 22:44:14 +03:00
David Garcia
1639a468df docs: add latest option to aws_images extension
rollback only latest

Closes scylladb/scylladb#15651

(cherry picked from commit 51466dcb23)
2023-10-26 13:10:44 +03:00
Anna Stuchlik
a0766ac236 doc: fix the Reference page layout
This commit fixes the layout of the Reference
page. Previously, the toctree level was "2",
which made the page hard to navigate.
This PR changes the level to "1".

In addition, the capitalization of page
titles is fixed.

This is a follow-up PR to the ones that
created and updated the Reference section.
It must be backported to branch-5.4.

Closes scylladb/scylladb#15830

(cherry picked from commit e223624e2e)
2023-10-26 11:30:15 +03:00
Botond Dénes
fa0f382a82 Merge 'Initialize datadir for system and non-system keyspaces the same way' from Pavel Emelyanov
When populating system keyspace the sstable_directory forgets to create upload/ subdir in the tables' datadir because of the way it's invoked from distributed loader. For non-system keyspaces directories are created in table::init_storage() which is self-contained and just creates the whole layout regardless of what.

This PR makes system keyspace's tables use table::init_storage() as well so that the datadir layout is the same for all on-disk tables.

Test included.

fixes: #15708
closes: scylladb/scylla-manager#3603

Closes scylladb/scylladb#15723

* github.com:scylladb/scylladb:
  test: Add test for datadir/ layout
  sstable_directory: Indentation fix after previous patch
  db,sstables: Move storage init for system keyspace to table creation

(cherry picked from commit 7f81957437)
2023-10-25 12:13:03 +03:00
Anna Stuchlik
37fd8a4c36 doc:remove upgrade guides for unsupported versions
This commit:
- Removes upgrade guides for versions older than 5.0.
  The oldest one is from version 4.6 to 5.0.
- Adds the redirections for the removed pages.

Closes scylladb/scylladb#15709
2023-10-19 10:00:23 +03:00
Yaron Kaikov
83f7d0073a release: prepare for 5.4.0-rc0 2023-10-18 14:38:53 +03:00
258 changed files with 1932 additions and 10715 deletions

2
.gitmodules vendored
View File

@@ -1,6 +1,6 @@
[submodule "seastar"]
path = seastar
url = ../seastar
url = ../scylla-seastar
ignore = dirty
[submodule "swagger-ui"]
path = swagger-ui

View File

@@ -78,7 +78,7 @@ fi
# Default scylla product/version tags
PRODUCT=scylla
VERSION=5.4.0-dev
VERSION=5.4.1
if test -f version
then

View File

@@ -18,37 +18,43 @@ namespace fd = httpd::failure_detector_json;
void set_failure_detector(http_context& ctx, routes& r, gms::gossiper& g) {
fd::get_all_endpoint_states.set(r, [&g](std::unique_ptr<request> req) {
std::vector<fd::endpoint_state> res;
res.reserve(g.num_endpoints());
g.for_each_endpoint_state([&] (const gms::inet_address& addr, const gms::endpoint_state& eps) {
fd::endpoint_state val;
val.addrs = fmt::to_string(addr);
val.is_alive = g.is_alive(addr);
val.generation = eps.get_heart_beat_state().get_generation().value();
val.version = eps.get_heart_beat_state().get_heart_beat_version().value();
val.update_time = eps.get_update_timestamp().time_since_epoch().count();
for (const auto& [as_type, app_state] : eps.get_application_state_map()) {
fd::version_value version_val;
// We return the enum index and not it's name to stay compatible to origin
// method that the state index are static but the name can be changed.
version_val.application_state = static_cast<std::underlying_type<gms::application_state>::type>(as_type);
version_val.value = app_state.value();
version_val.version = app_state.version().value();
val.application_state.push(version_val);
}
res.emplace_back(std::move(val));
return g.container().invoke_on(0, [] (gms::gossiper& g) {
std::vector<fd::endpoint_state> res;
res.reserve(g.num_endpoints());
g.for_each_endpoint_state([&] (const gms::inet_address& addr, const gms::endpoint_state& eps) {
fd::endpoint_state val;
val.addrs = fmt::to_string(addr);
val.is_alive = g.is_alive(addr);
val.generation = eps.get_heart_beat_state().get_generation().value();
val.version = eps.get_heart_beat_state().get_heart_beat_version().value();
val.update_time = eps.get_update_timestamp().time_since_epoch().count();
for (const auto& [as_type, app_state] : eps.get_application_state_map()) {
fd::version_value version_val;
// We return the enum index and not it's name to stay compatible to origin
// method that the state index are static but the name can be changed.
version_val.application_state = static_cast<std::underlying_type<gms::application_state>::type>(as_type);
version_val.value = app_state.value();
version_val.version = app_state.version().value();
val.application_state.push(version_val);
}
res.emplace_back(std::move(val));
});
return make_ready_future<json::json_return_type>(res);
});
return make_ready_future<json::json_return_type>(res);
});
fd::get_up_endpoint_count.set(r, [&g](std::unique_ptr<request> req) {
int res = g.get_up_endpoint_count();
return make_ready_future<json::json_return_type>(res);
return g.container().invoke_on(0, [] (gms::gossiper& g) {
int res = g.get_up_endpoint_count();
return make_ready_future<json::json_return_type>(res);
});
});
fd::get_down_endpoint_count.set(r, [&g](std::unique_ptr<request> req) {
int res = g.get_down_endpoint_count();
return make_ready_future<json::json_return_type>(res);
return g.container().invoke_on(0, [] (gms::gossiper& g) {
int res = g.get_down_endpoint_count();
return make_ready_future<json::json_return_type>(res);
});
});
fd::get_phi_convict_threshold.set(r, [] (std::unique_ptr<request> req) {
@@ -56,11 +62,13 @@ void set_failure_detector(http_context& ctx, routes& r, gms::gossiper& g) {
});
fd::get_simple_states.set(r, [&g] (std::unique_ptr<request> req) {
std::map<sstring, sstring> nodes_status;
g.for_each_endpoint_state([&] (const gms::inet_address& node, const gms::endpoint_state&) {
nodes_status.emplace(node.to_sstring(), g.is_alive(node) ? "UP" : "DOWN");
return g.container().invoke_on(0, [] (gms::gossiper& g) {
std::map<sstring, sstring> nodes_status;
g.for_each_endpoint_state([&] (const gms::inet_address& node, const gms::endpoint_state&) {
nodes_status.emplace(node.to_sstring(), g.is_alive(node) ? "UP" : "DOWN");
});
return make_ready_future<json::json_return_type>(map_to_key_value<fd::mapper>(nodes_status));
});
return make_ready_future<json::json_return_type>(map_to_key_value<fd::mapper>(nodes_status));
});
fd::set_phi_convict_threshold.set(r, [](std::unique_ptr<request> req) {
@@ -71,13 +79,15 @@ void set_failure_detector(http_context& ctx, routes& r, gms::gossiper& g) {
});
fd::get_endpoint_state.set(r, [&g] (std::unique_ptr<request> req) {
auto state = g.get_endpoint_state_ptr(gms::inet_address(req->param["addr"]));
if (!state) {
return make_ready_future<json::json_return_type>(format("unknown endpoint {}", req->param["addr"]));
}
std::stringstream ss;
g.append_endpoint_state(ss, *state);
return make_ready_future<json::json_return_type>(sstring(ss.str()));
return g.container().invoke_on(0, [req = std::move(req)] (gms::gossiper& g) {
auto state = g.get_endpoint_state_ptr(gms::inet_address(req->param["addr"]));
if (!state) {
return make_ready_future<json::json_return_type>(format("unknown endpoint {}", req->param["addr"]));
}
std::stringstream ss;
g.append_endpoint_state(ss, *state);
return make_ready_future<json::json_return_type>(sstring(ss.str()));
});
});
fd::get_endpoint_phi_values.set(r, [](std::unique_ptr<request> req) {

View File

@@ -250,17 +250,21 @@ future<json::json_return_type> set_tables_tombstone_gc(http_context& ctx, const
}
void set_transport_controller(http_context& ctx, routes& r, cql_transport::controller& ctl) {
ss::start_native_transport.set(r, [&ctl](std::unique_ptr<http::request> req) {
ss::start_native_transport.set(r, [&ctx, &ctl](std::unique_ptr<http::request> req) {
return smp::submit_to(0, [&] {
return ctl.start_server();
return with_scheduling_group(ctx.db.local().get_statement_scheduling_group(), [&ctl] {
return ctl.start_server();
});
}).then([] {
return make_ready_future<json::json_return_type>(json_void());
});
});
ss::stop_native_transport.set(r, [&ctl](std::unique_ptr<http::request> req) {
ss::stop_native_transport.set(r, [&ctx, &ctl](std::unique_ptr<http::request> req) {
return smp::submit_to(0, [&] {
return ctl.request_stop_server();
return with_scheduling_group(ctx.db.local().get_statement_scheduling_group(), [&ctl] {
return ctl.request_stop_server();
});
}).then([] {
return make_ready_future<json::json_return_type>(json_void());
});
@@ -282,17 +286,21 @@ void unset_transport_controller(http_context& ctx, routes& r) {
}
void set_rpc_controller(http_context& ctx, routes& r, thrift_controller& ctl) {
ss::stop_rpc_server.set(r, [&ctl](std::unique_ptr<http::request> req) {
ss::stop_rpc_server.set(r, [&ctx, &ctl](std::unique_ptr<http::request> req) {
return smp::submit_to(0, [&] {
return ctl.request_stop_server();
return with_scheduling_group(ctx.db.local().get_statement_scheduling_group(), [&ctl] {
return ctl.request_stop_server();
});
}).then([] {
return make_ready_future<json::json_return_type>(json_void());
});
});
ss::start_rpc_server.set(r, [&ctl](std::unique_ptr<http::request> req) {
ss::start_rpc_server.set(r, [&ctx, &ctl](std::unique_ptr<http::request> req) {
return smp::submit_to(0, [&] {
return ctl.start_server();
return with_scheduling_group(ctx.db.local().get_statement_scheduling_group(), [&ctl] {
return ctl.start_server();
});
}).then([] {
return make_ready_future<json::json_return_type>(json_void());
});

View File

@@ -245,6 +245,8 @@ future<authenticated_user> password_authenticator::authenticate(
std::throw_with_nested(exceptions::authentication_exception(e.what()));
} catch (exceptions::authentication_exception& e) {
std::throw_with_nested(e);
} catch (exceptions::unavailable_exception& e) {
std::throw_with_nested(exceptions::authentication_exception(e.get_message()));
} catch (...) {
std::throw_with_nested(exceptions::authentication_exception("authentication failed"));
}

View File

@@ -1800,7 +1800,7 @@ get_fully_expired_sstables(const table_state& table_s, const std::vector<sstable
int64_t min_timestamp = std::numeric_limits<int64_t>::max();
for (auto& sstable : overlapping) {
auto gc_before = sstable->get_gc_before_for_fully_expire(compaction_time, table_s.get_tombstone_gc_state());
auto gc_before = sstable->get_gc_before_for_fully_expire(compaction_time, table_s.get_tombstone_gc_state(), table_s.schema());
if (sstable->get_max_local_deletion_time() >= gc_before) {
min_timestamp = std::min(min_timestamp, sstable->get_stats_metadata().min_timestamp);
}
@@ -1819,7 +1819,7 @@ get_fully_expired_sstables(const table_state& table_s, const std::vector<sstable
// SStables that do not contain live data is added to list of possibly expired sstables.
for (auto& candidate : compacting) {
auto gc_before = candidate->get_gc_before_for_fully_expire(compaction_time, table_s.get_tombstone_gc_state());
auto gc_before = candidate->get_gc_before_for_fully_expire(compaction_time, table_s.get_tombstone_gc_state(), table_s.schema());
clogger.debug("Checking if candidate of generation {} and max_deletion_time {} is expired, gc_before is {}",
candidate->generation(), candidate->get_stats_metadata().max_local_deletion_time, gc_before);
// A fully expired sstable which has an ancestor undeleted shouldn't be compacted because

View File

@@ -51,7 +51,7 @@ std::vector<compaction_descriptor> compaction_strategy_impl::get_cleanup_compact
}));
}
bool compaction_strategy_impl::worth_dropping_tombstones(const shared_sstable& sst, gc_clock::time_point compaction_time, const tombstone_gc_state& gc_state) {
bool compaction_strategy_impl::worth_dropping_tombstones(const shared_sstable& sst, gc_clock::time_point compaction_time, const table_state& t) {
if (_disable_tombstone_compaction) {
return false;
}
@@ -62,7 +62,7 @@ bool compaction_strategy_impl::worth_dropping_tombstones(const shared_sstable& s
if (db_clock::now()-_tombstone_compaction_interval < sst->data_file_write_time()) {
return false;
}
auto gc_before = sst->get_gc_before_for_drop_estimation(compaction_time, gc_state);
auto gc_before = sst->get_gc_before_for_drop_estimation(compaction_time, t.get_tombstone_gc_state(), t.schema());
return sst->estimate_droppable_tombstone_ratio(gc_before) >= _tombstone_threshold;
}

View File

@@ -64,7 +64,7 @@ public:
// Check if a given sstable is entitled for tombstone compaction based on its
// droppable tombstone histogram and gc_before.
bool worth_dropping_tombstones(const shared_sstable& sst, gc_clock::time_point compaction_time, const tombstone_gc_state& gc_state);
bool worth_dropping_tombstones(const shared_sstable& sst, gc_clock::time_point compaction_time, const table_state& t);
virtual std::unique_ptr<compaction_backlog_tracker::impl> make_backlog_tracker() const = 0;

View File

@@ -51,15 +51,15 @@ compaction_descriptor leveled_compaction_strategy::get_sstables_for_compaction(t
auto& sstables = manifest.get_level(level);
// filter out sstables which droppable tombstone ratio isn't greater than the defined threshold.
auto e = boost::range::remove_if(sstables, [this, compaction_time, &table_s] (const sstables::shared_sstable& sst) -> bool {
return !worth_dropping_tombstones(sst, compaction_time, table_s.get_tombstone_gc_state());
return !worth_dropping_tombstones(sst, compaction_time, table_s);
});
sstables.erase(e, sstables.end());
if (sstables.empty()) {
continue;
}
auto& sst = *std::max_element(sstables.begin(), sstables.end(), [&] (auto& i, auto& j) {
auto gc_before1 = i->get_gc_before_for_drop_estimation(compaction_time, table_s.get_tombstone_gc_state());
auto gc_before2 = j->get_gc_before_for_drop_estimation(compaction_time, table_s.get_tombstone_gc_state());
auto gc_before1 = i->get_gc_before_for_drop_estimation(compaction_time, table_s.get_tombstone_gc_state(), table_s.schema());
auto gc_before2 = j->get_gc_before_for_drop_estimation(compaction_time, table_s.get_tombstone_gc_state(), table_s.schema());
return i->estimate_droppable_tombstone_ratio(gc_before1) < j->estimate_droppable_tombstone_ratio(gc_before2);
});
return sstables::compaction_descriptor({ sst }, sst->get_sstable_level());

View File

@@ -243,7 +243,7 @@ size_tiered_compaction_strategy::get_sstables_for_compaction(table_state& table_
for (auto&& sstables : buckets | boost::adaptors::reversed) {
// filter out sstables which droppable tombstone ratio isn't greater than the defined threshold.
auto e = boost::range::remove_if(sstables, [this, compaction_time, &table_s] (const sstables::shared_sstable& sst) -> bool {
return !worth_dropping_tombstones(sst, compaction_time, table_s.get_tombstone_gc_state());
return !worth_dropping_tombstones(sst, compaction_time, table_s);
});
sstables.erase(e, sstables.end());
if (sstables.empty()) {

View File

@@ -366,7 +366,7 @@ time_window_compaction_strategy::get_next_non_expired_sstables(table_state& tabl
// if there is no sstable to compact in standard way, try compacting single sstable whose droppable tombstone
// ratio is greater than threshold.
auto e = boost::range::remove_if(non_expiring_sstables, [this, compaction_time, &table_s] (const shared_sstable& sst) -> bool {
return !worth_dropping_tombstones(sst, compaction_time, table_s.get_tombstone_gc_state());
return !worth_dropping_tombstones(sst, compaction_time, table_s);
});
non_expiring_sstables.erase(e, non_expiring_sstables.end());
if (non_expiring_sstables.empty()) {

View File

@@ -433,12 +433,17 @@ protected:
}
};
::shared_ptr<selection> selection::wildcard(schema_ptr schema) {
// Return a list of columns that "SELECT *" should show - these are all
// columns except potentially some that are is_hidden_from_cql() (currently,
// those can be the "virtual columns" used in materialized views).
// The list points to column_definition objects in the given schema_ptr,
// which can be used only as long as the caller keeps the schema_ptr alive.
std::vector<const column_definition*> selection::wildcard_columns(schema_ptr schema) {
auto columns = schema->all_columns_in_select_order();
// filter out hidden columns, which should not be seen by the
// user when doing "SELECT *". We also disallow selecting them
// individually (see column_identifier::new_selector_factory()).
auto cds = boost::copy_range<std::vector<const column_definition*>>(
return boost::copy_range<std::vector<const column_definition*>>(
columns |
boost::adaptors::filtered([](const column_definition& c) {
return !c.is_hidden_from_cql();
@@ -446,7 +451,10 @@ protected:
boost::adaptors::transformed([](const column_definition& c) {
return &c;
}));
return simple_selection::make(schema, std::move(cds), true);
}
::shared_ptr<selection> selection::wildcard(schema_ptr schema) {
return simple_selection::make(schema, wildcard_columns(schema), true);
}
::shared_ptr<selection> selection::for_columns(schema_ptr schema, std::vector<const column_definition*> columns) {

View File

@@ -118,6 +118,7 @@ public:
}
static ::shared_ptr<selection> wildcard(schema_ptr schema);
static std::vector<const column_definition*> wildcard_columns(schema_ptr schema);
static ::shared_ptr<selection> for_columns(schema_ptr schema, std::vector<const column_definition*> columns);
// Adds a column to the selection and result set. Returns an index within the result set row.

View File

@@ -135,6 +135,18 @@ user_type alter_type_statement::add_or_alter::do_add(data_dictionary::database d
throw exceptions::invalid_request_exception(format("Cannot add new field to type {}: maximum number of fields reached", _name));
}
if (_field_type->is_duration()) {
auto&& ks = db.find_keyspace(keyspace());
for (auto&& schema : ks.metadata()->cf_meta_data() | boost::adaptors::map_values) {
for (auto&& column : schema->clustering_key_columns()) {
if (column.type->references_user_type(_name.get_keyspace(), _name.get_user_type_name())) {
throw exceptions::invalid_request_exception(format("Cannot add new field to type {} because it is used in the clustering key column {} of table {}.{} where durations are not allowed",
_name.to_cql_string(), column.name_as_text(), schema->ks_name(), schema->cf_name()));
}
}
}
}
std::vector<bytes> new_names(to_update->field_names());
new_names.push_back(_field_name->name());
std::vector<data_type> new_types(to_update->field_types());

View File

@@ -226,7 +226,8 @@ future<> select_statement::check_access(query_processor& qp, const service::clie
}
if (!_selection->is_trivial()) {
std::vector<::shared_ptr<functions::function>> used_functions = _selection->used_functions();
for (const auto& used_function : used_functions) {
auto not_native = [] (::shared_ptr<functions::function> func) { return !func->is_native(); };
for (const auto& used_function : used_functions | std::ranges::views::filter(not_native)) {
sstring encoded_signature = auth::encode_signature(used_function->name().name, used_function->arg_types());
co_await state.has_function_access(used_function->name().keyspace, encoded_signature, auth::permission::EXECUTE);
}
@@ -1660,7 +1661,7 @@ schema_ptr mutation_fragments_select_statement::generate_output_schema(schema_pt
future<exceptions::coordinator_result<service::storage_proxy_coordinator_query_result>>
mutation_fragments_select_statement::do_query(
const locator::node* this_node,
locator::host_id this_node,
service::storage_proxy& sp,
schema_ptr schema,
lw_shared_ptr<query::read_command> cmd,
@@ -1670,7 +1671,7 @@ mutation_fragments_select_statement::do_query(
auto res = co_await replica::mutation_dump::dump_mutations(sp.get_db(), schema, _underlying_schema, partition_ranges, *cmd, optional_params.timeout(sp));
service::replicas_per_token_range last_replicas;
if (this_node) {
last_replicas.emplace(dht::token_range::make_open_ended_both_sides(), std::vector<locator::host_id>{this_node->host_id()});
last_replicas.emplace(dht::token_range::make_open_ended_both_sides(), std::vector<locator::host_id>{this_node});
}
co_return service::storage_proxy_coordinator_query_result{std::move(res), std::move(last_replicas), {}};
}
@@ -1731,12 +1732,17 @@ mutation_fragments_select_statement::do_execute(query_processor& qp, service::qu
auto timeout_duration = get_timeout(state.get_client_state(), options);
auto timeout = db::timeout_clock::now() + timeout_duration;
auto& tbl = qp.proxy().local_db().find_column_family(_underlying_schema);
// Since this query doesn't go through storage-proxy, we have to take care of pinning erm here.
auto erm_keepalive = tbl.get_effective_replication_map();
if (!aggregate && !_restrictions_need_filtering && (page_size <= 0
|| !service::pager::query_pagers::may_need_paging(*_schema, page_size,
*command, key_ranges))) {
return do_query({}, qp.proxy(), _schema, command, std::move(key_ranges), cl,
{timeout, state.get_permit(), state.get_client_state(), state.get_trace_state(), {}, {}})
.then(wrap_result_to_error_message([&, this] (service::storage_proxy_coordinator_query_result&& qr) {
.then(wrap_result_to_error_message([&, this, erm_keepalive] (service::storage_proxy_coordinator_query_result&& qr) {
cql3::selection::result_set_builder builder(*_selection, now);
query::result_view::consume(*qr.query_result, std::move(slice),
cql3::selection::result_set_builder::visitor(builder, *_schema, *_selection));
@@ -1745,16 +1751,14 @@ mutation_fragments_select_statement::do_execute(query_processor& qp, service::qu
}));
}
const locator::node* this_node = nullptr;
locator::host_id this_node;
{
auto& tbl = qp.proxy().local_db().find_column_family(_underlying_schema);
auto& erm = tbl.get_effective_replication_map();
auto& topo = erm->get_topology();
this_node = topo.this_node();
auto& topo = erm_keepalive->get_topology();
this_node = topo.this_node()->host_id();
auto state = options.get_paging_state();
if (state && !state->get_last_replicas().empty()) {
auto last_host = state->get_last_replicas().begin()->second.front();
if (last_host != this_node->host_id()) {
if (last_host != this_node) {
const auto last_node = topo.find_node(last_host);
throw exceptions::invalid_request_exception(format(
"Moving between coordinators is not allowed in SELECT FROM MUTATION_FRAGMENTS() statements, last page's coordinator was {}{}",
@@ -1774,7 +1778,10 @@ mutation_fragments_select_statement::do_execute(query_processor& qp, service::qu
command,
std::move(key_ranges),
_restrictions_need_filtering ? _restrictions : nullptr,
std::bind_front(&mutation_fragments_select_statement::do_query, this, this_node));
[this, erm_keepalive, this_node] (service::storage_proxy& sp, schema_ptr schema, lw_shared_ptr<query::read_command> cmd, dht::partition_range_vector partition_ranges,
db::consistency_level cl, service::storage_proxy_coordinator_query_options optional_params) {
return do_query(this_node, sp, std::move(schema), std::move(cmd), std::move(partition_ranges), cl, std::move(optional_params));
});
if (_selection->is_trivial() && !_restrictions_need_filtering && !_per_partition_limit) {
return p->fetch_page_generator_result(page_size, now, timeout, _stats).then(wrap_result_to_error_message([this, p = std::move(p)] (result_generator&& generator) {
@@ -1901,6 +1908,21 @@ std::unique_ptr<prepared_statement> select_statement::prepare(data_dictionary::d
// Force aggregation if GROUP BY is used. This will wrap every column x as first(x).
if (!_group_by_columns.empty()) {
aggregation_depth = std::max(aggregation_depth, 1u);
if (prepared_selectors.empty()) {
// We have a "SELECT * GROUP BY". If we leave prepared_selectors
// empty, below we choose selection::wildcard() for SELECT *, and
// forget to do the "levellize" trick needed for the GROUP BY.
// So we need to set prepared_selectors. See #16531.
auto all_columns = selection::selection::wildcard_columns(schema);
std::vector<::shared_ptr<selection::raw_selector>> select_all;
select_all.reserve(all_columns.size());
for (const column_definition *cdef : all_columns) {
auto name = ::make_shared<cql3::column_identifier::raw>(cdef->name_as_text(), true);
select_all.push_back(::make_shared<selection::raw_selector>(
expr::unresolved_identifier(std::move(name)), nullptr));
}
prepared_selectors = selection::raw_selector::to_prepared_selectors(select_all, *schema, db, keyspace());
}
}
for (auto& ps : prepared_selectors) {

View File

@@ -19,10 +19,7 @@
#include "index/secondary_index_manager.hh"
#include "exceptions/exceptions.hh"
#include "exceptions/coordinator_result.hh"
namespace locator {
class node;
} // namespace locator
#include "locator/host_id.hh"
namespace service {
class client_state;
@@ -341,7 +338,7 @@ public:
private:
future<exceptions::coordinator_result<service::storage_proxy_coordinator_query_result>>
do_query(
const locator::node* this_node,
locator::host_id this_node,
service::storage_proxy& sp,
schema_ptr schema,
lw_shared_ptr<query::read_command> cmd,

View File

@@ -56,7 +56,11 @@ future<> use_statement::check_access(query_processor& qp, const service::client_
future<::shared_ptr<cql_transport::messages::result_message>>
use_statement::execute(query_processor& qp, service::query_state& state, const query_options& options, std::optional<service::group0_guard> guard) const {
state.get_client_state().set_keyspace(qp.db().real_database(), _keyspace);
try {
state.get_client_state().set_keyspace(qp.db().real_database(), _keyspace);
} catch(...) {
return make_exception_future<::shared_ptr<cql_transport::messages::result_message>>(std::current_exception());
}
auto result =::make_shared<cql_transport::messages::result_message::set_keyspace>(_keyspace);
return make_ready_future<::shared_ptr<cql_transport::messages::result_message>>(result);
}

View File

@@ -502,7 +502,7 @@ struct to_json_string_visitor {
sstring operator()(const tuple_type_impl& t) { return to_json_string_aux(t, bv); }
sstring operator()(const user_type_impl& t) { return to_json_string_aux(t, bv); }
sstring operator()(const simple_date_type_impl& t) { return quote_json_string(t.to_string(bv)); }
sstring operator()(const time_type_impl& t) { return t.to_string(bv); }
sstring operator()(const time_type_impl& t) { return quote_json_string(t.to_string(bv)); }
sstring operator()(const empty_type_impl& t) { return "null"; }
sstring operator()(const duration_type_impl& t) {
auto v = t.deserialize(bv);

View File

@@ -1879,8 +1879,7 @@ std::vector<schema_ptr> system_keyspace::all_tables(const db::config& cfg) {
static bool maybe_write_in_user_memory(schema_ptr s) {
return (s.get() == system_keyspace::batchlog().get()) || (s.get() == system_keyspace::paxos().get())
|| s == system_keyspace::v3::scylla_views_builds_in_progress()
|| s == system_keyspace::raft();
|| s == system_keyspace::v3::scylla_views_builds_in_progress();
}
future<> system_keyspace::make(
@@ -1888,6 +1887,7 @@ future<> system_keyspace::make(
replica::database& db) {
for (auto&& table : system_keyspace::all_tables(db.get_config())) {
co_await db.create_local_system_table(table, maybe_write_in_user_memory(table), erm_factory);
co_await db.find_column_family(table).init_storage();
}
}

View File

@@ -1689,7 +1689,7 @@ future<> view_update_generator::mutate_MV(
auto mut_ptr = remote_endpoints.empty() ? std::make_unique<frozen_mutation>(std::move(mut.fm)) : std::make_unique<frozen_mutation>(mut.fm);
tracing::trace(tr_state, "Locally applying view update for {}.{}; base token = {}; view token = {}",
mut.s->ks_name(), mut.s->cf_name(), base_token, view_token);
local_view_update = _proxy.local().mutate_locally(mut.s, *mut_ptr, tr_state, db::commitlog::force_sync::no).then_wrapped(
local_view_update = _proxy.local().mutate_mv_locally(mut.s, *mut_ptr, tr_state, db::commitlog::force_sync::no).then_wrapped(
[s = mut.s, &stats, &cf_stats, tr_state, base_token, view_token, my_address, mut_ptr = std::move(mut_ptr),
units = sem_units.split(sem_units.count())] (future<>&& f) {
--stats.writes;

View File

@@ -62,8 +62,7 @@ ExternalSizeMax=1024G
[Unit]
Description=Save coredump to scylla data directory
Conflicts=umount.target
Before=scylla-server.service
After=local-fs.target
Before=local-fs.target scylla-server.service
DefaultDependencies=no
[Mount]
@@ -73,7 +72,7 @@ Type=none
Options=bind
[Install]
WantedBy=multi-user.target
WantedBy=local-fs.target
'''[1:-1]
with open('/etc/systemd/system/var-lib-systemd-coredump.mount', 'w') as f:
f.write(dot_mount)

View File

@@ -257,19 +257,19 @@ if __name__ == '__main__':
dev_type = 'realpath'
LOGGER.error(f'Failed to detect uuid, using {dev_type}: {mount_dev}')
after = 'local-fs.target'
after = ''
wants = ''
if raid and args.raid_level != '0':
after += f' {md_service}'
wants = f'\nWants={md_service}'
after = wants = 'md_service'
opt_discard = ''
if args.online_discard:
opt_discard = ',discard'
unit_data = f'''
[Unit]
Description=Scylla data directory
Before=scylla-server.service
After={after}{wants}
Before=local-fs.target scylla-server.service
After={after}
Wants={wants}
DefaultDependencies=no
[Mount]
@@ -279,7 +279,7 @@ Type=xfs
Options=noatime{opt_discard}
[Install]
WantedBy=multi-user.target
WantedBy=local-fs.target
'''[1:-1]
with open(f'/etc/systemd/system/{mntunit_bn}', 'w') as f:
f.write(unit_data)

View File

@@ -64,7 +64,6 @@ bcp "${packages[@]}" packages/
bcp dist/docker/etc etc/
bcp dist/docker/scylla-housekeeping-service.sh /scylla-housekeeping-service.sh
bcp dist/docker/sshd-service.sh /sshd-service.sh
bcp dist/docker/scyllasetup.py /scyllasetup.py
bcp dist/docker/commandlineparser.py /commandlineparser.py
@@ -74,10 +73,11 @@ bcp dist/docker/scylla_bashrc /scylla_bashrc
run apt-get -y clean expire-cache
run apt-get -y update
run apt-get -y upgrade
run apt-get -y install dialog apt-utils
run bash -ec "echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections"
run bash -ec "rm -rf /etc/rsyslog.conf"
run apt-get -y install hostname supervisor openssh-server openssh-client openjdk-11-jre-headless python2 python3 python3-yaml curl rsyslog sudo
run apt-get -y install hostname supervisor openjdk-11-jre-headless python2 python3 python3-yaml curl rsyslog sudo
run bash -ec "echo LANG=C.UTF-8 > /etc/default/locale"
run bash -ec "dpkg -i packages/*.deb"
run apt-get -y clean all

View File

@@ -1,6 +0,0 @@
[program:sshd]
command=/sshd-service.sh
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

View File

@@ -75,7 +75,8 @@ class ScyllaSetup:
hostname = self._listenAddress
else:
hostname = subprocess.check_output(['hostname', '-i']).decode('ascii').strip()
with open("%s/.cqlshrc" % home, "w") as cqlshrc:
self._run(["mkdir", "-p", "%s/.cassandra" % home])
with open("%s/.cassandra/cqlshrc" % home, "w") as cqlshrc:
cqlshrc.write("[connection]\nhostname = %s\n" % hostname)
def set_housekeeping(self):

View File

@@ -1,15 +0,0 @@
#!/bin/bash
if [ ! -f /run/sshd ]; then
mkdir -p /run/sshd
fi
if [ ! -f /etc/ssh/ssh_host_ed25519_key ]; then
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ''
fi
if [ ! -f /etc/ssh/ssh_host_rsa_key ]; then
ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ''
fi
/usr/sbin/sshd -D

View File

@@ -91,7 +91,7 @@ redirects: setup
# Preview commands
.PHONY: preview
preview: setup
$(POETRY) run sphinx-autobuild -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml --host $(PREVIEW_HOST) --port 5500 --ignore '_data/*'
$(POETRY) run sphinx-autobuild -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml --host $(PREVIEW_HOST) --port 5500 --ignore *.csv --ignore *.yaml
.PHONY: multiversionpreview
multiversionpreview: multiversion

View File

@@ -118,6 +118,7 @@ class AMIVersionsTemplateDirective(Directive):
option_spec = {
"version": directives.unchanged,
"exclude": directives.unchanged,
"only_latest": directives.flag,
}
def _extract_version_from_filename(self, filename):
@@ -144,10 +145,28 @@ class AMIVersionsTemplateDirective(Directive):
version = self._extract_version_from_filename(filename)
return tuple(map(int, version.split("."))) if version else (0,)
def _get_current_version(self, current_version, stable_version):
prefix = 'branch-'
version = current_version
if current_version.startswith(prefix):
version = current_version
elif not stable_version.startswith(prefix):
LOGGER.error("Invalid stable_version format in conf.py. It should start with 'branch-'")
else:
version = stable_version
return version.replace(prefix, '')
def run(self):
app = self.state.document.settings.env.app
version_pattern = self.options.get("version", "")
current_version = os.environ.get('SPHINX_MULTIVERSION_NAME', '')
stable_version = app.config.smv_latest_version
version_pattern = self._get_current_version(current_version, stable_version)
version_options = self.options.get("version", "")
if version_options:
version_pattern = version_options
exclude_patterns = self.options.get("exclude", "").split(",")
download_directory = os.path.join(
@@ -169,6 +188,8 @@ class AMIVersionsTemplateDirective(Directive):
LOGGER.warning(
f"No files match in directory '{download_directory}' with version pattern '{version_pattern}'."
)
elif "only_latest" in self.options:
files = [files[0]]
output = []
for file in files:

View File

@@ -1,14 +1,23 @@
import os
import re
import yaml
from typing import Any, Dict, List
import jinja2
from sphinx import addnodes
from sphinx.application import Sphinx
from sphinxcontrib.datatemplates.directive import DataTemplateYAML
from sphinx.directives import ObjectDescription
from sphinx.util import logging, status_iterator, ws_re
from sphinx.util.docfields import Field
from sphinx.util.docutils import switch_source_input, SphinxDirective
from sphinx.util.nodes import make_id, nested_parse_with_titles
from sphinx.jinja2glue import BuiltinTemplateLoader
from docutils import nodes
from docutils.parsers.rst import directives
from docutils.statemachine import StringList
CONFIG_FILE_PATH = "../db/config.cc"
CONFIG_HEADER_FILE_PATH = "../db/config.hh"
DESTINATION_PATH = "_data/db_config.yaml"
logger = logging.getLogger(__name__)
class DBConfigParser:
@@ -47,42 +56,18 @@ class DBConfigParser:
"""
COMMENT_PATTERN = r"/\*.*?\*/|//.*?$"
def __init__(self, config_file_path, config_header_file_path, destination_path):
all_properties = {}
def __init__(self, config_file_path, config_header_file_path):
self.config_file_path = config_file_path
self.config_header_file_path = config_header_file_path
self.destination_path = destination_path
def _create_yaml_file(self, destination, data):
current_data = None
try:
with open(destination, "r") as file:
current_data = yaml.safe_load(file)
except FileNotFoundError:
pass
if current_data != data:
os.makedirs(os.path.dirname(destination), exist_ok=True)
with open(destination, "w") as file:
yaml.dump(data, file)
@staticmethod
def _clean_description(description):
return (
description.replace("\\n", "")
.replace('<', '&lt;')
.replace('>', '&gt;')
.replace("\n", "<br>")
.replace("\\t", "- ")
.replace('"', "")
)
def _clean_comments(self, content):
return re.sub(self.COMMENT_PATTERN, "", content, flags=re.DOTALL | re.MULTILINE)
def _parse_group(self, group_match, config_group_content):
group_name = group_match.group(1).strip()
group_description = self._clean_description(group_match.group(2).strip()) if group_match.group(2) else ""
group_description = group_match.group(2).strip() if group_match.group(2) else ""
current_group = {
"name": group_name,
@@ -111,14 +96,16 @@ class DBConfigParser:
config_matches = re.findall(self.CONFIG_CC_REGEX_PATTERN, content, re.DOTALL)
for match in config_matches:
name = match[1].strip()
property_data = {
"name": match[1].strip(),
"name": name,
"value_status": match[4].strip(),
"default": match[5].strip(),
"liveness": "True" if match[3] else "False",
"description": self._clean_description(match[6].strip()),
"description": match[6].strip(),
}
properties.append(property_data)
DBConfigParser.all_properties[name] = property_data
return properties
@@ -135,7 +122,7 @@ class DBConfigParser:
if property_data["name"] == property_key:
property_data["type"] = match[0].strip()
def _parse_db_properties(self):
def parse(self):
groups = []
with open(self.config_file_path, "r", encoding='utf-8') as file:
@@ -158,26 +145,170 @@ class DBConfigParser:
return groups
def run(self, app: Sphinx):
dest_path = os.path.join(app.builder.srcdir, self.destination_path)
parsed_properties = self._parse_db_properties()
self._create_yaml_file(dest_path, parsed_properties)
@classmethod
def get(cls, name: str):
return DBConfigParser.all_properties[name]
class DBConfigTemplateDirective(DataTemplateYAML):
option_spec = DataTemplateYAML.option_spec.copy()
option_spec["value_status"] = directives.unchanged_required
def _make_context(self, data, config, env):
context = super()._make_context(data, config, env)
context["value_status"] = self.options.get("value_status")
return context
def setup(app: Sphinx):
db_parser = DBConfigParser(
CONFIG_FILE_PATH, CONFIG_HEADER_FILE_PATH, DESTINATION_PATH
def readable_desc(description: str) -> str:
return (
description.replace("\\n", "")
.replace('<', '&lt;')
.replace('>', '&gt;')
.replace("\n", "<br>")
.replace("\\t", "- ")
.replace('"', "")
)
app.connect("builder-inited", db_parser.run)
app.add_directive("scylladb_config_template", DBConfigTemplateDirective)
def maybe_add_filters(builder):
env = builder.templates.environment
if 'readable_desc' not in env.filters:
env.filters['readable_desc'] = readable_desc
class ConfigOption(ObjectDescription):
has_content = True
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = False
# TODO: instead of overriding transform_content(), render option properties
# as a field list.
doc_field_types = [
Field('type',
label='Type',
has_arg=False,
names=('type',)),
Field('default',
label='Default value',
has_arg=False,
names=('default',)),
Field('liveness',
label='Liveness',
has_arg=False,
names=('liveness',)),
]
def handle_signature(self,
sig: str,
signode: addnodes.desc_signature) -> str:
signode.clear()
signode += addnodes.desc_name(sig, sig)
# normalize whitespace like XRefRole does
return ws_re.sub(' ', sig)
@property
def env(self):
document = self.state.document
return document.settings.env
def before_content(self) -> None:
maybe_add_filters(self.env.app.builder)
def _render(self, name) -> str:
item = DBConfigParser.get(name)
if item is None:
raise self.error(f'Option "{name}" not found!')
builder = self.env.app.builder
template = self.config.scylladb_cc_properties_option_tmpl
return builder.templates.render(template, item)
def transform_content(self,
contentnode: addnodes.desc_content) -> None:
name = self.arguments[0]
# the source is always None here
_, lineno = self.get_source_info()
source = f'scylla_config:{lineno}:<{name}>'
fields = StringList(self._render(name).splitlines(),
source=source, parent_offset=lineno)
with switch_source_input(self.state, fields):
self.state.nested_parse(fields, 0, contentnode)
def add_target_and_index(self,
name: str,
sig: str,
signode: addnodes.desc_signature) -> None:
node_id = make_id(self.env, self.state.document, self.objtype, name)
signode['ids'].append(node_id)
self.state.document.note_explicit_target(signode)
entry = f'{name}; configuration option'
self.indexnode['entries'].append(('pair', entry, node_id, '', None))
std = self.env.get_domain('std')
std.note_object(self.objtype, name, node_id, location=signode)
class ConfigOptionList(SphinxDirective):
has_content = False
required_arguments = 2
optional_arguments = 0
final_argument_whitespace = True
option_spec = {
'template': directives.path,
'value_status': directives.unchanged_required,
}
@property
def env(self):
document = self.state.document
return document.settings.env
def _resolve_src_path(self, path: str) -> str:
rel_filename, filename = self.env.relfn2path(path)
self.env.note_dependency(filename)
return filename
def _render(self, context: Dict[str, Any]) -> str:
builder = self.env.app.builder
template = self.options.get('template')
if template is None:
self.error(f'Option "template" not specified!')
return builder.templates.render(template, context)
def _make_context(self) -> Dict[str, Any]:
header = self._resolve_src_path(self.arguments[0])
source = self._resolve_src_path(self.arguments[1])
db_parser = DBConfigParser(source, header)
value_status = self.options.get("value_status")
return dict(data=db_parser.parse(),
value_status=value_status)
def run(self) -> List[nodes.Node]:
maybe_add_filters(self.env.app.builder)
rendered = self._render(self._make_context())
contents = StringList(rendered.splitlines())
node = nodes.section()
node.document = self.state.document
nested_parse_with_titles(self.state, contents, node)
return node.children
def setup(app: Sphinx) -> Dict[str, Any]:
app.add_config_value(
'scylladb_cc_properties_option_tmpl',
default='db_option.tmpl',
rebuild='html',
types=[str])
app.add_object_type(
'confgroup',
'confgroup',
objname='configuration group',
indextemplate='pair: %s; configuration group',
doc_field_types=[
Field('example',
label='Example',
has_arg=False)
])
app.add_object_type(
'confval',
'confval',
objname='configuration option')
app.add_directive_to_domain('std', 'confval', ConfigOption, override=True)
app.add_directive('scylladb_config_list', ConfigOptionList)
return {
"version": "0.1",
"parallel_read_safe": True,
"parallel_write_safe": True,
}

View File

@@ -17,10 +17,22 @@
}
.content blockquote li p {
margin-bottom: 10px;
margin-bottom: 5px;
}
h3 .pre {
font-size: 16px;
font-weight: bold;
}
}
hr {
max-width: 100%;
}
dl dt:hover > a.headerlink {
visibility: visible;
}
dl.confval {
border-bottom: 1px solid #cacaca;
}

View File

@@ -8,22 +8,12 @@
{% if group.description %}
.. raw:: html
<p>{{ group.description }}</p>
<p>{{ group.description | readable_desc }}</p>
{% endif %}
{% for item in group.properties %}
{% if item.value_status == value_status %}
``{{ item.name }}``
{{ '=' * (item.name|length + 4) }}
.. raw:: html
<p>{{ item.description }}</p>
{% if item.type %}* **Type:** ``{{ item.type }}``{% endif %}
{% if item.default %}* **Default value:** ``{{ item.default }}``{% endif %}
{% if item.liveness %}* **Liveness** :term:`* <Liveness>` **:** ``{{ item.liveness }}``{% endif %}
.. confval:: {{ item.name }}
{% endif %}
{% endfor %}
{% endif %}

7
docs/_templates/db_option.tmpl vendored Normal file
View File

@@ -0,0 +1,7 @@
.. raw:: html
<p>{{ description | readable_desc }}</p>
{% if type %}* **Type:** ``{{ type }}``{% endif %}
{% if default %}* **Default value:** ``{{ default }}``{% endif %}
{% if liveness %}* **Liveness** :term:`* <Liveness>` **:** ``{{ liveness }}``{% endif %}

View File

@@ -1,2 +1,107 @@
### a dictionary of redirections
#old path: new path
#old path: new path
# Removed the outdated upgrade guides
/stable/upgrade/upgrade-opensource/upgrade-guide-from-ubuntu-14-to-16.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.x.y-to-2.x.z/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.x.y-to-2.x.z/upgrade-guide-from-2.x.y-to-2.x.z-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.x.y-to-2.x.z/upgrade-guide-from-2.x.y-to-2.x.z-ubuntu.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.x.y-to-2.x.z/upgrade-guide-from-2.x.y-to-2.x.z-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.1-to-2.2/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.1-to-2.2/upgrade-guide-from-2.1-to-2.2-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.1-to-2.2/upgrade-guide-from-2.1-to-2.2-ubuntu.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.1-to-2.2/upgrade-guide-from-2.1-to-2.2-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.1-to-2.2/metric-update-2.1-to-2.2.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.2-to-2.3/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.2-to-2.3/upgrade-guide-from-2.2-to-2.3-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.2-to-2.3/upgrade-guide-from-2.2-to-2.3-ubuntu.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.2-to-2.3/upgrade-guide-from-2.2-to-2.3-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.2-to-2.3/upgrade-guide-from-2.2-to-2.3-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.2-to-2.3/metric-update-2.2-to-2.3.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.3-to-3.0/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.3-to-3.0/upgrade-guide-from-2.3-to-3.0-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.3-to-3.0/upgrade-guide-from-2.3-to-3.0-ubuntu.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.3-to-3.0/upgrade-guide-from-2.3-to-3.0-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.3-to-3.0/upgrade-guide-from-2.3-to-3.0-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.3-to-3.0/upgrade-guide-from-2.3-to-3.0-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-2.3-to-3.0/metric-update-2.3-to-3.0.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.0-to-3.1/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.0-to-3.1/upgrade-guide-from-3.0-to-3.1-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.0-to-3.1/upgrade-guide-from-3.0-to-3.1-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.0-to-3.1/upgrade-guide-from-3.0-to-3.1-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.0-to-3.1/upgrade-guide-from-3.0-to-3.1-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.0-to-3.1/metric-update-3.0-to-3.1.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.1-to-3.2/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.1-to-3.2/upgrade-guide-from-3.1-to-3.2-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.1-to-3.2/upgrade-guide-from-3.1-to-3.2-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.1-to-3.2/upgrade-guide-from-3.1-to-3.2-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.1-to-3.2/upgrade-guide-from-3.1-to-3.2-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.1-to-3.2/metric-update-3.1-to-3.2.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.2-to-3.3/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.2-to-3.3/upgrade-guide-from-3.2-to-3.3-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.2-to-3.3/upgrade-guide-from-3.2-to-3.3-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.2-to-3.3/upgrade-guide-from-3.2-to-3.3-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.2-to-3.3/upgrade-guide-from-3.2-to-3.3-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.2-to-3.3/metric-update-3.2-to-3.3.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.3-to-4.0/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.3-to-4.0/upgrade-guide-from-3.3-to-4.0-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.3-to-4.0/upgrade-guide-from-3.3-to-4.0-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.3-to-4.0/upgrade-guide-from-3.3-to-4.0-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.3-to-4.0/upgrade-guide-from-3.3-to-4.0-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.3-to-4.0/metric-update-3.3-to-4.0.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.x.y-to-3.x.z/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.x.y-to-3.x.z/upgrade-guide-from-3.x.y-to-3.x.z-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.x.y-to-3.x.z/upgrade-guide-from-3.x.y-to-3.x.z-ubuntu.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-3.x.y-to-3.x.z/upgrade-guide-from-3.x.y-to-3.x.z-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/upgrade-guide-from-4.0-to-4.1-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/upgrade-guide-from-4.0-to-4.1-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/upgrade-guide-from-4.0-to-4.1-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/upgrade-guide-from-4.0-to-4.1-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/alternator.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/metric-update-4.0-to-4.1.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.x.y-to-4.x.z/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.x.y-to-4.x.z/upgrade-guide-from-4.x.y-to-4.x.z-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.x.y-to-4.x.z/upgrade-guide-from-4.x.y-to-4.x.z-ubuntu.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.x.y-to-4.x.z/upgrade-guide-from-4.x.y-to-4.x.z-debian.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.1-to-4.2/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.1-to-4.2/upgrade-guide-from-4.1-to-4.2-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.1-to-4.2/upgrade-guide-from-4.1-to-4.2-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.1-to-4.2/upgrade-guide-from-4.1-to-4.2-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.1-to-4.2/upgrade-guide-from-4.1-to-4.2-debian-9.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.1-to-4.2/upgrade-guide-from-4.1-to-4.2-debian-10.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.1-to-4.2/metric-update-4.1-to-4.2.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.2-to-4.3/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.2-to-4.3/upgrade-guide-from-4.2-to-4.3-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.2-to-4.3/upgrade-guide-from-4.2-to-4.3-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.2-to-4.3/upgrade-guide-from-4.2-to-4.3-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.2-to-4.3/upgrade-guide-from-4.2-to-4.3-debian-9.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.2-to-4.3/upgrade-guide-from-4.2-to-4.3-debian-10.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.2-to-4.3/metric-update-4.2-to-4.3.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.3-to-4.4/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.3-to-4.4/upgrade-guide-from-4.3-to-4.4-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.3-to-4.4/upgrade-guide-from-4.3-to-4.4-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.3-to-4.4/upgrade-guide-from-4.3-to-4.4-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.3-to-4.4/upgrade-guide-from-4.3-to-4.4-ubuntu-20-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.3-to-4.4/upgrade-guide-from-4.3-to-4.4-debian-9.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.3-to-4.4/upgrade-guide-from-4.3-to-4.4-debian-10.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.3-to-4.4/metric-update-4.3-to-4.4.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.4-to-4.5/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.4-to-4.5/upgrade-guide-from-4.4-to-4.5-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.4-to-4.5/upgrade-guide-from-4.4-to-4.5-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.4-to-4.5/upgrade-guide-from-4.4-to-4.5-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.4-to-4.5/upgrade-guide-from-4.4-to-4.5-ubuntu-20-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.4-to-4.5/upgrade-guide-from-4.4-to-4.5-debian-9.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.4-to-4.5/upgrade-guide-from-4.4-to-4.5-debian-10.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.4-to-4.5/metric-update-4.4-to-4.5.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.5-to-4.6/index.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.5-to-4.6/upgrade-guide-from-4.5-to-4.6-rpm.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.5-to-4.6/upgrade-guide-from-4.5-to-4.6-ubuntu-16-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.5-to-4.6/upgrade-guide-from-4.5-to-4.6-ubuntu-18-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.5-to-4.6/upgrade-guide-from-4.5-to-4.6-ubuntu-20-04.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.5-to-4.6/upgrade-guide-from-4.5-to-4.6-debian-9.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.5-to-4.6/upgrade-guide-from-4.5-to-4.6-debian-10.html: /stable/upgrade/index.html
/stable/upgrade/upgrade-opensource/upgrade-guide-from-4.5-to-4.6/metric-update-4.5-to-4.6.html: /stable/upgrade/index.html

View File

@@ -8,8 +8,7 @@ Scylla implements the following compaction strategies in order to reduce :term:`
* `Size-tiered compaction strategy (STCS)`_ - triggered when the system has enough (four by default) similarly sized SSTables.
* `Leveled compaction strategy (LCS)`_ - the system uses small, fixed-size (by default 160 MB) SSTables distributed across different levels.
* `Incremental Compaction Strategy (ICS)`_ - shares the same read and write amplification factors as STCS, but it fixes its 2x temporary space amplification issue by breaking huge sstables into SSTable runs, which are comprised of a sorted set of smaller (1 GB by default), non-overlapping SSTables.
* `Time-window compaction strategy (TWCS)`_ - designed for time series data; replaced Date-tiered compaction.
* `Date-tiered compaction strategy (DTCS)`_ - designed for time series data.
* `Time-window compaction strategy (TWCS)`_ - designed for time series data.
This document covers how to choose a compaction strategy and presents the benefits and disadvantages of each one. If you want more information on compaction in general or on any of these strategies, refer to the :doc:`Compaction Overview </kb/compaction>`. If you want an explanation of the CQL commands used to create a compaction strategy, refer to :doc:`Compaction CQL Reference </cql/compaction>` .
@@ -78,7 +77,6 @@ ICS is only available in ScyllaDB Enterprise. See the `ScyllaDB Enetrpise docume
Time-window Compaction Strategy (TWCS)
======================================
Time-window compaction strategy was introduced in Cassandra 3.0.8 for time-series data as a replacement for `Date-tiered Compaction Strategy (DTCS)`_.
Time-Window Compaction Strategy compacts SSTables within each time window using `Size-tiered Compaction Strategy (STCS)`_.
SSTables from different time windows are never compacted together. You set the :ref:`TimeWindowCompactionStrategy <time-window-compactionstrategy-twcs>` parameters when you create a table using a CQL command.
@@ -87,9 +85,8 @@ SSTables from different time windows are never compacted together. You set the :
Time-window Compaction benefits
-------------------------------
* Keeps entries according to a time range, making searches for data within a given range easy to do, resulting in better read performance
* Improves over DTCS in that it reduces the number to huge compactions
* Allows you to expire an entire SSTable at once (using a TTL) as the data is already organized within a time frame
* Keeps entries according to a time range, making searches for data within a given range easy to do, resulting in better read performance.
* Allows you to expire an entire SSTable at once (using a TTL) as the data is already organized within a time frame.
Time-window Compaction deficits
-------------------------------
@@ -102,14 +99,6 @@ Set the parameters for :ref:`Time-window Compaction <time-window-compactionstrat
Use the table in `Which strategy is best`_ to determine if this is the right strategy for your needs.
.. _DTCS1:
Date-tiered Compaction Strategy (DTCS)
======================================
Date-Tiered Compaction is designed for time series data. This strategy was introduced with Cassandra 2.1.
It is only suitable for time-series data. This strategy is not recommended and has been replaced by :ref:`Time-window Compaction Strategy <TWCS1>`.
.. _which-strategy-is-best:
Which strategy is best

View File

@@ -37,7 +37,12 @@ Enabling Raft
.. note::
In ScyllaDB 5.2 and ScyllaDB Enterprise 2023.1 Raft is Generally Available and can be safely used for consistent schema management.
In further versions, it will be mandatory.
It will get enabled by default when you upgrade your cluster to ScyllaDB 5.4 or 2024.1.
If needed, you can explicitly prevent it from getting enabled upon upgrade.
.. only:: opensource
See :doc:`the upgrade guide from 5.2 to 5.4 </upgrade/index>` for details.
ScyllaDB Open Source 5.2 and later, and ScyllaDB Enterprise 2023.1 and later come equipped with a procedure that can setup Raft-based consistent cluster management in an existing cluster. We refer to this as the **Raft upgrade procedure** (do not confuse with the :doc:`ScyllaDB version upgrade procedure </upgrade/index/>`).
@@ -214,6 +219,36 @@ of nodes in the cluster is available. The following examples illustrate how Raft
In summary, Raft makes schema changes safe, but it requires that a quorum of nodes in the cluster is available.
.. _raft-topology-changes:
.. only:: opensource
Consistent Topology with Raft :label-caution:`Experimental`
-----------------------------------------------------------------
ScyllaDB can use Raft to manage cluster topology. With Raft-managed topology
enabled, all topology operations are internally sequenced in a consistent
way. A centralized coordination process ensures that topology metadata is
synchronized across the nodes on each step of a topology change procedure.
This makes topology updates fast and safe, as the cluster administrator can
trigger many topology operations concurrently, and the coordination process
will safely drive all of them to completion. For example, multiple nodes can
be bootstrapped concurrently, which couldn't be done with the old
gossip-based topology.
Support for Raft-managed topology is experimental and must be explicitly
enabled in the ``scylla.yaml`` configuration file by specifying
the ``consistent-topology-changes`` option:
.. code::
experimental_features:
- consistent-topology-changes
As with other experimental features in ScyllaDB, you should not enable this
feature in production clusters due to insufficient stability. The feature
is undergoing backward-incompatible changes that may prevent upgrading
the cluster.
.. _raft-handling-failures:

View File

@@ -19,8 +19,6 @@ The following compaction strategies are supported by Scylla:
* Time-window Compaction Strategy (`TWCS`_)
* Date-tiered Compaction Strategy (DTCS) - use `TWCS`_ instead
This page concentrates on the parameters to use when creating a table with a compaction strategy. If you are unsure which strategy to use or want general information on the compaction strategies which are available to Scylla, refer to :doc:`Compaction Strategies </architecture/compaction/compaction-strategies>`.
Common options

View File

@@ -79,27 +79,66 @@ and to the TRUNCATE data definition query.
In addition, the timeout parameter can be applied to SELECT queries as well.
```eval_rst
.. _keyspace-storage-options:
```
## Keyspace storage options
Storage options allows specifying the storage format assigned to a keyspace.
The default storage format is `LOCAL`, which simply means storing all the sstables
in a local directory.
Experimental support for `S3` storage format is also added. This option is not fully
implemented yet, but it will allow storing sstables in a shared, S3-compatible object store.
<!---
This section must be moved to Data Definition> CREATE KEYSPACE
when support for object storage is GA.
--->
Storage options can be specified via `CREATE KEYSPACE` or `ALTER KEYSPACE` statement
and it's formatted as a map of options - similarly to how replication strategy is handled.
By default, SStables of a keyspace are stored in a local directory.
As an alternative, you can configure your keyspace to be stored
on Amazon S3 or another S3-compatible object store.
Examples:
```cql
CREATE KEYSPACE ks
WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }
AND STORAGE = { 'type' : 'S3', 'bucket' : '/tmp/b1', 'endpoint' : 'localhost' } ;
Support for object storage is experimental and must be explicitly
enabled in the ``scylla.yaml`` configuration file by specifying
the ``keyspace-storage-options`` option:
```
experimental_features:
- keyspace-storage-options
```
With support for object storage enabled, add your endpoint configuration
to ``scylla.yaml``:
1. Create an ``object-storage-config-file.yaml`` file with a description of
allowed endpoints, for example:
```
endpoints:
- name: $endpoint_address_or_domain_name
port: $port_number
https: optional True or False
aws_region: optional region name, e.g. us-east-1
aws_access_key_id: optional AWS access key ID
aws_secret_access_key: optional AWS secret access key
aws_session_token: optional AWS session token
```
1. Specify the ``object-storage-config-file`` option in your ``scylla.yaml``,
providing ``object-storage-config-file.yaml`` as the value:
```
object-storage-config-file: object-storage-config-file.yaml
```
Now you can configure your object storage when creating a keyspace:
```cql
ALTER KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }
AND STORAGE = { 'type' : 'S3', 'bucket': '/tmp/b2', 'endpoint' : 'localhost' } ;
CREATE KEYSPACE with STORAGE = { 'type': 'S3', 'endpoint': '$endpoint_name', 'bucket': '$bucket' }
```
**Example**
```cql
CREATE KEYSPACE ks
WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy', 'replication_factor' : 3 }
AND STORAGE = { 'type' : 'S3', 'bucket' : '/tmp/b1', 'endpoint' : 'localhost' } ;
```
Storage options can be inspected by checking the new system schema table: `system_schema.scylla_keyspaces`:

View File

@@ -6,18 +6,26 @@
CQLSh: the CQL shell
--------------------
cqlsh is a command line shell for interacting with Cassandra through CQL (the Cassandra Query Language). It is shipped
with every Cassandra package and can be found in the bin/ directory alongside the Cassandra executable. cqlsh utilizes
the Python native protocol driver and connects to the single node specified on the command line.
cqlsh is a command line shell for interacting with ScyllaDB through CQL
(the Cassandra Query Language). It is shipped with every ScyllaDB package
and can be found in the ``bin/`` directory. In addition, it is available on
`Docker Hub <https://hub.docker.com/r/scylladb/scylla-cqlsh>`_ and in
the `Python Package Index (PyPI) <https://pypi.org/project/scylla-cqlsh/>`_.
cqlsh utilizes the Python native protocol driver and connects to the single
node specified on the command line.
See the `scylla-cqlsh <https://github.com/scylladb/scylla-cqlsh>`_ repository
on GitHub for usage examples.
Compatibility
^^^^^^^^^^^^^
cqlsh is compatible with Python 2.7.
cqlsh is compatible with Python 3.8 - Python 3.11.
In general, a given version of cqlsh is only guaranteed to work with the version of Cassandra that it was released with.
In some cases, cqlsh may work with older or newer versions of Cassandra, but this is not officially supported.
A given version of cqlsh is only guaranteed to work with the version of ScyllaDB that it was released with.
cqlsh may work with older or newer versions of ScyllaDB without any guarantees.
Optional Dependencies
@@ -72,13 +80,13 @@ Options:
``/usr/bin/google-chrome-stable %s``).
``--ssl``
Use SSL when connecting to Cassandra
Use SSL when connecting to ScyllaDB.
``-u`` ``--user``
Username to authenticate against Cassandra with
Username to authenticate against ScyllaDB.
``-p`` ``--password``
The password to authenticate against Cassandra with should
The password to authenticate against ScyllaDB, which should
be used in conjunction with ``--user``
``-k`` ``--keyspace``
@@ -162,17 +170,17 @@ consistency ``ALL`` is not guaranteed to be enough).
SHOW VERSION
~~~~~~~~~~~~
This command is useful if you want to check which Cassandra version is compatible with your Scylla version.
This command is useful if you want to check which Cassandra version is compatible with your ScyllaDB version.
Note that the two standards are not 100% identical and this command is simply a comparison tool.
If you want to display your current Scylla Version, refer to :ref:`Check your current version of Scylla <check-your-current-version-of-scylla>`.
If you want to display your current ScyllaDB version, refer to :ref:`Check your current version of Scylla <check-your-current-version-of-scylla>`.
The display shows:
* The cqlsh tool version that you're using
* The Apache Cassandra version that your version of Scylla is most compatible with
* The CQL protocol standard that your version of Scylla is most compatible with
* The native protocol standard that your version of Scylla is most compatible with
* The Apache Cassandra version that your version of ScyllaDB is most compatible with
* The CQL protocol standard that your version of ScyllaDB is most compatible with
* The native protocol standard that your version of ScyllaDB is most compatible with
Example:
@@ -191,7 +199,7 @@ Returns:
SHOW HOST
~~~~~~~~~
Prints the IP address and port of the Cassandra node that cqlsh is connected to in addition to the cluster name.
Prints the IP address and port of the ScyllaDB node that cqlsh is connected to in addition to the cluster name.
Example:
@@ -324,7 +332,7 @@ contents of a single column are large.
LOGIN
~~~~~
Authenticate as a specified Cassandra user for the current session.
Authenticate as a specified ScyllaDB user for the current session.
`Usage`::

View File

@@ -198,6 +198,18 @@ An example that excludes a datacenter while using ``replication_factor``::
DESCRIBE KEYSPACE excalibur
CREATE KEYSPACE excalibur WITH replication = {'class': 'NetworkTopologyStrategy', 'DC1': '3'} AND durable_writes = true;
.. only:: opensource
Keyspace storage options :label-caution:`Experimental`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
By default, SStables of a keyspace are stored locally.
As an alternative, you can configure your keyspace to be stored
on Amazon S3 or another S3-compatible object store.
See :ref:`Keyspace storage options <keyspace-storage-options>` for details.
.. _use-statement:
USE
@@ -687,19 +699,12 @@ Compaction options
The ``compaction`` options must at least define the ``'class'`` sub-option, which defines the compaction strategy class
to use. The default supported class are ``'SizeTieredCompactionStrategy'``,
``'LeveledCompactionStrategy'``, ``'IncrementalCompactionStrategy'``, and ``'DateTieredCompactionStrategy'``
``'LeveledCompactionStrategy'``, and ``'IncrementalCompactionStrategy'``.
Custom strategy can be provided by specifying the full class name as a :ref:`string constant
<constants>`.
All default strategies support a number of common options, as well as options specific to
the strategy chosen (see the section corresponding to your strategy for details: :ref:`STCS <stcs-options>`, :ref:`LCS <lcs-options>`, and :ref:`TWCS <twcs-options>`). DTCS is not recommended, and TWCS should be used instead.
.. ``'Date Tiered Compaction Strategy is not recommended and has been replaced by Time Window Compaction Stragegy.'`` (:ref:`TWCS <TWCS>`) (the
.. is also supported but is deprecated and ``'TimeWindowCompactionStrategy'`` should be
.. preferred instead).
the strategy chosen (see the section corresponding to your strategy for details: :ref:`STCS <stcs-options>`, :ref:`LCS <lcs-options>`, and :ref:`TWCS <twcs-options>`).
.. _cql-compression-options:

View File

@@ -5,7 +5,7 @@ Wasm support for user-defined functions
This document describes the details of Wasm language support in user-defined functions (UDF). The language ``wasm`` is one of the possible languages to use, besides Lua, to implement these functions. To learn more about User-defined functions in ScyllaDB, click :ref:`here <udfs>`.
.. note:: Until ScyllaDB 5.2, the Wasm language was called ``xwasm``. This name is replaced with ``wasm`` in ScyllaDB 5.3.
.. note:: Until ScyllaDB 5.2, the Wasm language was called ``xwasm``. This name is replaced with ``wasm`` in ScyllaDB 5.4.
How to generate a correct Wasm UDF source code
----------------------------------------------

View File

@@ -1,10 +1,11 @@
.. |UBUNTU_SCYLLADB_LIST| replace:: scylla-5.2.list
.. |CENTOS_SCYLLADB_REPO| replace:: scylla-5.2.repo
.. |UBUNTU_SCYLLADB_LIST| replace:: scylla-5.4.list
.. |CENTOS_SCYLLADB_REPO| replace:: scylla-5.4.repo
.. The |RHEL_EPEL| variable needs to be adjuster per release, depening on support for RHEL.
.. 5.2 supports Rocky/RHEL 8
.. When RHEL 9 is supported, add https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
.. |RHEL_EPEL| replace:: https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
.. 5.2 supports Rocky/RHEL 8 only
.. 5.4 supports Rocky/RHEL 8 and 9
.. |RHEL_EPEL_8| replace:: https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
.. |RHEL_EPEL_9| replace:: https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
======================================
Install ScyllaDB Linux Packages
@@ -113,14 +114,21 @@ Install ScyllaDB
sudo yum install epel-release
RHEL:
Rocky/RHEL 8
.. code-block:: console
:substitutions:
sudo yum -y install |RHEL_EPEL|
sudo yum -y install |RHEL_EPEL_8|
Rocky/RHEL 9
.. code-block:: console
:substitutions:
sudo yum -y install |RHEL_EPEL_9|
#. Add the ScyllaDB RPM repository to your system.
.. code-block:: console

View File

@@ -8,8 +8,14 @@ as-a-service, see `ScyllaDB Cloud documentation <https://cloud.docs.scylladb.com
Launching Instances from ScyllaDB AMI
---------------------------------------
#. Go to `Amazon EC2 AMIs ScyllaDB <https://www.scylladb.com/download/?platform=aws#open-source>`_ in ScyllaDB's download center,
choose your region, and click the **Node** link to open the EC2 instance creation wizard.
#. Choose your region, and click the **Node** link to open the EC2 instance creation wizard.
The following table shows the latest patch release. See :doc:`AWS Images </reference/aws-images/>` for earlier releases.
.. scylladb_aws_images_template::
:exclude: rc,dev
:only_latest:
#. Choose the instance type. See :ref:`Cloud Instance Recommendations for AWS <system-requirements-aws>` for the list of recommended instances.
Other instance types will work, but with lesser performance. If you choose an instance type other than the recommended ones, make sure to run the :ref:`scylla_setup <system-configuration-scripts>` script.

View File

@@ -5,8 +5,6 @@ The following matrix shows which Linux distributions, containers, and images are
Where *supported* in this scope means:
.. REMOVE IN FUTURE VERSIONS - Remove information about versions from the notes below in version 5.2.
- A binary installation package is available to `download <https://www.scylladb.com/download/>`_.
- The download and install procedures are tested as part of ScyllaDB release process for each version.
- An automated install is included from :doc:`ScyllaDB Web Installer for Linux tool </getting-started/installation-common/scylla-web-installer>` (for latest versions)
@@ -27,39 +25,18 @@ ScyllaDB Open Source
The recommended OS for ScyllaDB Open Source is Ubuntu 22.04.
+----------------------------+----------------------------------+-----------------------------+---------+---------------+
| Linux Distributions | Ubuntu | Debian | CentOS /| Rocky / |
| | | | RHEL | RHEL |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| ScyllaDB Version / Version | 14.04| 16.04| 18.04|20.04 |22.04 | 8 | 9 | 10 | 11 | 7 | 8 | 9 |
+============================+======+======+======+======+======+======+======+=======+=======+=========+=======+=======+
| 5.3 | |x| | |x| | |x| | |v| | |v| | |x| | |x| | |v| | |v| | |v| | |v| | |v| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 5.2 | |x| | |x| | |x| | |v| | |v| | |x| | |x| | |v| | |v| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 5.1 | |x| | |x| | |v| | |v| | |v| | |x| | |x| | |v| | |v| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 5.0 | |x| | |x| | |v| | |v| | |v| | |x| | |x| | |v| | |v| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 4.6 | |x| | |v| | |v| | |v| | |x| | |x| | |v| | |v| | |x| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 4.5 | |x| | |v| | |v| | |v| | |x| | |x| | |v| | |v| | |x| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 4.4 | |x| | |v| | |v| | |v| | |x| | |x| | |v| | |v| | |x| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 4.3 | |x| | |v| | |v| | |v| | |x| | |x| | |v| | |v| | |x| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 4.2 | |x| | |v| | |v| | |x| | |x| | |x| | |v| | |v| | |x| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 4.1 | |x| | |v| | |v| | |x| | |x| | |x| | |v| | |v| | |x| | |v| | |v| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
| 4.0 | |x| | |v| | |v| | |x| | |x| | |x| | |v| | |x| | |x| | |v| | |x| | |x| |
+----------------------------+------+------+------+------+------+------+------+-------+-------+---------+-------+-------+
All releases are available as a Docker container, EC2 AMI, and a GCP image (GCP image from version 4.3). Since
version 5.2, the ScyllaDB AMI/Image OS for ScyllaDB Open Source is based on Ubuntu 22.04.
+----------------------------+-------------+---------------+---------+---------------+--------------+--------------+
| Linux Distributions |Ubuntu | Debian | CentOS /| Rocky / | Oracle Linux | Amazon Linux |
| | | | RHEL | RHEL | | |
+----------------------------+------+------+-------+-------+---------+-------+-------+-------+------+--------------+
| ScyllaDB Version / Version |20.04 |22.04 | 10 | 11 | 7 | 8 | 9 | 7.6 | 8.1 | 2 |
+============================+======+======+=======+=======+=========+=======+=======+=======+======+==============+
| 5.4 | |v| | |v| | |v| | |v| | |v| | |v| | |v| | |v| | |v| | |v| |
+----------------------------+------+------+-------+-------+---------+-------+-------+-------+------+--------------+
| 5.2 | |v| | |v| | |v| | |v| | |v| | |v| | |x| | |x| | |x| | |x| |
+----------------------------+------+------+-------+-------+---------+-------+-------+-------+------+--------------+
All releases are available as a Docker container and EC2 AMI, GCP, and Azure images.
ScyllaDB Enterprise

View File

@@ -44,8 +44,7 @@ A compaction strategy is what determines which of the SSTables will be compacted
* `Size-tiered compaction strategy (STCS)`_ - (default setting) triggered when the system has enough similarly sized SSTables.
* `Leveled compaction strategy (LCS)`_ - the system uses small, fixed-size (by default 160 MB) SSTables divided into different levels and lowers both Read and Space Amplification.
* :ref:`Incremental compaction strategy (ICS) <incremental-compaction-strategy-ics>` - :label-tip:`ScyllaDB Enterprise` Uses runs of sorted, fixed size (by default 1 GB) SSTables in a similar way that LCS does, organized into size-tiers, similar to STCS size-tiers. If you are an Enterprise customer ICS is an updated strategy meant to replace STCS. It has the same read and write amplification, but has lower space amplification due to the reduction of temporary space overhead is reduced to a constant manageable level.
* `Time-window compaction strategy (TWCS)`_ - designed for time series data and puts data in time order. This strategy replaced Date-tiered compaction. TWCS uses STCS to prevent accumulating SSTables in a window not yet closed. When the window closes, TWCS works towards reducing the SSTables in a time window to one.
* `Date-tiered compaction strategy (DTCS)`_ - designed for time series data, but TWCS should be used instead.
* `Time-window compaction strategy (TWCS)`_ - designed for time series data and puts data in time order. TWCS uses STCS to prevent accumulating SSTables in a window not yet closed. When the window closes, TWCS works towards reducing the SSTables in a time window to one.
How to Set a Compaction Strategy
................................
@@ -125,7 +124,7 @@ ICS is only available in ScyllaDB Enterprise. See the `ScyllaDB Enetrpise docume
Time-window Compaction Strategy (TWCS)
--------------------------------------
Time-window compaction strategy was introduced as a replacement for `Date-tiered Compaction Strategy (DTCS)`_ for handling time series workloads. Time-Window Compaction Strategy compacts SSTables within each time window using `Size-tiered Compaction Strategy (STCS)`_. SSTables from different time windows are never compacted together.
Time-Window Compaction Strategy is designed for handling time series workloads. It compacts SSTables within each time window using `Size-tiered Compaction Strategy (STCS)`_. SSTables from different time windows are never compacted together.
.. include:: /rst_include/warning-ttl-twcs.rst
@@ -148,22 +147,6 @@ The primary motivation for TWCS is to separate data on disk by timestamp and to
While TWCS tries to minimize the impact of commingled data, users should attempt to avoid this behavior. Specifically, users should avoid queries that explicitly set the timestamp. It is recommended to run frequent repairs (which streams data in such a way that it does not become commingled), and disable background read repair by setting the tables :ref:`read_repair_chance <create-table-general-options>` and :ref:`dclocal_read_repair_chance <create-table-general-options>` to ``0``.
Date-tiered compaction strategy (DTCS)
--------------------------------------
Date-Tiered Compaction is designed for time series data. It is only suitable for time-series data. This strategy has been replaced by `Time-window compaction strategy (TWCS)`_.
Date-tiered compaction strategy works as follows:
* First it sorts the SSTables by time and then compacts adjacent (time-wise) SSTables.
* This results in SSTables whose sizes increase exponentially as they grow older.
For example, at some point we can have the last minute of data in one SSTable (by default, base_time_seconds = 60), another minute before that in another SSTable, then the 4 minutes before that in one SSTable, then the 4 minutes before that, then an SSTable of the 16 minutes before that, and so on. This structure can easily be maintained by compaction, very similar to size-tiered compaction. When there are 4 (the default value for min_threshold) small (one-minute) consecutive SSTables, they are compacted into one 4-minute SSTable. When there are 4 of the bigger SSTables one after another (time-wise), they are merged into a 16-minute SSTable, and so on.
Antique SSTables older than ``max_SSTable_age_days`` (by default 365 days) are not compacted as doing these compactions would not be useful for most queries, the process would be very slow, and the compaction would require huge amounts of temporary disk space.
Changing Compaction Strategies or Properties
--------------------------------------------

View File

@@ -70,7 +70,6 @@ To display the log classes (output changes with each version so your display may
cql_server
storage_proxy
cache
DateTieredCompactionStrategy
schema_tables
rpc
compaction_manager

View File

@@ -1,25 +1,36 @@
Repair Based Node Operations
****************************
====================================
Repair-Based Node Operations (RBNO)
====================================
Scylla has two use cases for transferring data between nodes:
In ScyllaDB, data is transferred between nodes during:
- Topology changes, like adding and removing nodes.
- Repair, a background process to compare and sync data between nodes.
* Topology changes via node operations, such as adding or removing nodes.
* Repair - a row-level background process to compare and sync data between nodes.
Up to Scylla 4.6, the two used different underline logic. In later releases, the same data transferring logic used for repair is also used for topology changes, making it more robust, reliable, and safer for data consistency. In particular, node operations can restart from the same point it stopped without sending data that has been synced, a significant time-saver when adding or removing large nodes.
In 4.6, Repair Based Node Operations (RBNO) is enabled by default only for replace node operation.
Example from scylla.yaml:
By default, the row-level repair mechanism used for the repair process is also
used during node operations (instead of streaming). We refer to it as
Repair-Based Node Operations (RBNO).
.. code:: yaml
enable_repair_based_node_ops: true
allowed_repair_based_node_ops: replace
RBNO is more robust, reliable, and safer for data consistency than streaming.
In particular, a failed node operation can resume from the point it stopped -
without sending data that has already been synced, which is a significant
time-saver when adding or removing large nodes. In addition, with RBNO enabled,
you don't need to run repair befor or after node operations, such as replace
or removenode.
To enable other operations (experimental), add them as a comma-separated list to allowed_repair_based_node_ops. Available operations are:
RBNO is enabled for the following node operations:
* bootstrap
* replace
* removenode
* decommission
* rebuild
* removenode
* replace
The following configuration options can be used to enable or disable RBNO:
* ``enable_repair_based_node_ops= true|false`` - Enables or disables RBNO.
* ``allowed_repair_based_node_ops= "replace,removenode,rebuild,bootstrap,decommission"`` -
Specifies the node operations for which the RBNO mechanism is enabled.
See :doc:`Configuration Parameters </reference/configuration-parameters/>` for details.

View File

@@ -1,7 +1,6 @@
==========
AWS images
AWS Images
==========
.. scylladb_aws_images_template::
:version: 5.2
:exclude: rc,dev

View File

@@ -1,10 +1,10 @@
========================
Configuration parameters
Configuration Parameters
========================
This section contains a list of properties that can be configured in ``scylla.yaml`` - the main configuration file for ScyllaDB.
In addition, properties that support live updates (liveness) can be updated via the ``system.config`` virtual table or the REST API.
.. scylladb_config_template:: ../_data/db_config.yaml
.. scylladb_config_list:: ../../db/config.hh ../../db/config.cc
:template: db_config.tmpl
:value_status: Used

View File

@@ -37,9 +37,6 @@ Glossary
Quorum
Quorum is a *global* consistency level setting across the entire cluster including all data centers. See :doc:`Consistency Levels </cql/consistency>`.
Date-tiered compaction strategy (DTCS)
:abbr:`DTCS (Date-tiered compaction strategy)` is designed for time series data, but should not be used. Use :term:`Time-Window Compaction Strategy`. See :doc:`Compaction Strategies</architecture/compaction/compaction-strategies/>`.
Entropy
A state where data is not consistent. This is the result when replicas are not synced and data is random. Scylla has measures in place to be antientropic. See :doc:`Scylla Anti-Entropy </architecture/anti-entropy/index>`.
@@ -151,7 +148,7 @@ Glossary
A collection of columns fetched by row. Columns are ordered by Clustering Key. See :doc:`Ring Architecture </architecture/ringarchitecture/index>`.
Time-window compaction strategy
TWCS is designed for time series data and replaced Date-tiered compaction. See :doc:`Compaction Strategies</architecture/compaction/compaction-strategies/>`.
TWCS is designed for time series data. See :doc:`Compaction Strategies</architecture/compaction/compaction-strategies/>`.
Token
A value in a range, used to identify both nodes and partitions. Each node in a Scylla cluster is given an (initial) token, which defines the end of the range a node handles. See :doc:`Ring Architecture </architecture/ringarchitecture/index>`.

View File

@@ -3,7 +3,7 @@ Reference
===============
.. toctree::
:maxdepth: 2
:maxdepth: 1
:glob:
/reference/*

View File

@@ -1,170 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 1.6 to 1.7 for |OS|
=============================================================================
This document is a step by step procedure for upgrading from Scylla 1.6 to Scylla 1.7, and rollback to 1.6 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.6.x to Scylla version 1.7.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* drain node and backup the data
* check your current release
* backup configuration file
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 1.7 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.6
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 1.6.x version, stop right here! This guide only covers 1.6.x to 1.7.y upgrades.
To upgrade:
1. Update the |APT|_ to **1.7**
2. Upgrade java to 1.8 on Ubuntu 14.04 and Debian 8, which is requested by Scylla 1.7
* |ENABLE_APT_REPO|
* sudo apt-get update
* sudo apt-get install -y |JESSIE_BACKPORTS|
* sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
3. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04 and Debian 8) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 1.7.x to 1.6.y. Apply this procedure if an upgrade from 1.6 to 1.7 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 1.7
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.6, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |APT|_ to **1.6**
3. install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.6 /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,190 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 1.7 to 2.0 for |OS|
=============================================================================
This document is a step by step procedure for upgrading from Scylla 1.7 to Scylla 2.0, and rollback to 1.7 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.7.x (x >= 4) to Scylla version 2.0.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* check cluster schema
* drain node and backup the data
* backup configuration file
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 2.0 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.7
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 1.7.x version, stop right here! This guide only covers 1.7.x to 2.0.y upgrades.
To upgrade:
1. Update the |APT|_ to **2.0**
2. Upgrade java to 1.8 on Ubuntu 14.04 and Debian 8, which is requested by Scylla 2.0
* |ENABLE_APT_REPO|
* sudo apt-get update
* sudo apt-get install -y |JESSIE_BACKPORTS|
* sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
3. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04 and Debian 8) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on :doc:`Scylla Metrics Update - Scylla 1.7 to 2.0<metric-update-1.7-to-2.0>`
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 2.0.x to 1.7.y (y >= 4). Apply this procedure if an upgrade from 1.7 to 2.0 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 2.0
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.7, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |APT|_ to **1.7**
3. install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.7 /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, 2.0 uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,157 +0,0 @@
======================================================================
Upgrade Guide - Scylla 1.x.y to 1.x.z for |OS|
======================================================================
This document is a step by step procedure for upgrading from Scylla 1.x.y to Scylla 1.x.z.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.x.y to Scylla version 1.x.z on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* drain node and backup the data
* check your current release
* backup configuration file and deb packages
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 1.x.z features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file and deb packages
------------------------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.x.z
If you install scylla by apt, you can find the deb packages in ``/var/cache/apt/``, backup them to ``scylla_1.x.y_backup`` directory which will be used in rollback.
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 1.x.y version, stop right here! This guide only covers 1.x.y to 1.x.z upgrades.
To upgrade:
1. Update the |APT|_ to **1.x**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 1.x.z to 1.x.y. Apply this procedure if an upgrade from 1.x.y to 1.x.z failed before completing on all nodes. Use this procedure only for nodes you upgraded to 1.x.z
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.x.y, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
Install the old release from backuped deb packages
--------------------------------------------------
1. install
.. code:: sh
sudo apt-get remove scylla\* -y
sudo apt-get install scylla_1.x.y_backup/scylla*.deb
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.x.z /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,201 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 2.0 to 2.1 for |OS|
=============================================================================
This document is a step by step procedure for upgrading from Scylla 2.0 to Scylla 2.1, and rollback to 2.0 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 2.0.x to Scylla version 2.1.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 2.1 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-2.0
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 2.0.x version, stop right here! This guide only covers 2.0.x to 2.1.y upgrades.
To upgrade:
1. Update the |APT|_ to **2.1**, and enable scylla/ppa repo
.. code:: sh
Debian 8:
sudo apt-get install gnupg-curl -y
sudo apt-key adv --fetch-keys https://download.opensuse.org/repositories/home:/scylladb:/scylla-3rdparty-jessie/Debian_8.0/Release.key
sudo sh -c "echo 'deb http://download.opensuse.org/repositories/home:/scylladb:/scylla-3rdparty-jessie/Debian_8.0/ /' > /etc/apt/sources.list.d/scylla-3rdparty.list"
Ubuntu 14/16:
sudo add-apt-repository -y ppa:scylladb/ppa
2. Upgrade java to 1.8 on Ubuntu 14.04 and Debian 8, which is requested by Scylla 2.1
* |ENABLE_APT_REPO|
* sudo apt-get update
* sudo apt-get install -y |JESSIE_BACKPORTS|
* sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
3. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04 and Debian 8) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on :doc:`Scylla Metrics Update - Scylla 2.0 to 2.1<metric-update-2.0-to-2.1>`
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 2.1.x to 2.0.y. Apply this procedure if an upgrade from 2.0 to 2.1 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 2.1
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 2.0, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |APT|_ to **2.0**
3. install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-2.0 /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, 2.1 uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,200 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 2.1 to 2.2 for |OS|
=============================================================================
This document is a step by step procedure for upgrading from Scylla 2.1 to Scylla 2.2, and rollback to 2.1 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 2.1.x to Scylla version 2.2.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 2.2 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
for conf in $(cat /var/lib/dpkg/info/scylla-*server.conffiles /var/lib/dpkg/info/scylla-*conf.conffiles /var/lib/dpkg/info/scylla-*jmx.conffiles | grep -v init ); do sudo cp -v $conf $conf.backup-2.1; done
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 2.1.x version, stop right here! This guide only covers 2.1.x to 2.2.y upgrades.
To upgrade:
1. Update the |APT|_ to **2.2**, and enable scylla/ppa repo
.. code:: sh
Debian 8:
sudo apt-get install gnupg-curl -y
sudo apt-key adv --fetch-keys https://download.opensuse.org/repositories/home:/scylladb:/scylla-3rdparty-jessie/Debian_8.0/Release.key
sudo sh -c "echo 'deb http://download.opensuse.org/repositories/home:/scylladb:/scylla-3rdparty-jessie/Debian_8.0/ /' > /etc/apt/sources.list.d/scylla-3rdparty.list"
Ubuntu 14/16:
sudo add-apt-repository -y ppa:scylladb/ppa
2. Upgrade java to 1.8 on Ubuntu 14.04 and Debian 8, which is requested by Scylla 2.2
* |ENABLE_APT_REPO|
* sudo apt-get update
* sudo apt-get install -y |JESSIE_BACKPORTS|
* sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
3. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04 and Debian 8) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on :doc:`Scylla Metrics Update - Scylla 2.1 to 2.2<metric-update-2.1-to-2.2>`
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 2.2.x to 2.1.y. Apply this procedure if an upgrade from 2.1 to 2.2 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 2.2
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 2.1, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |APT|_ to **2.1**
3. install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
for conf in $(cat /var/lib/dpkg/info/scylla-*server.conffiles /var/lib/dpkg/info/scylla-*conf.conffiles /var/lib/dpkg/info/scylla-*jmx.conffiles | grep -v init ); do sudo cp -v $conf.backup-2.1 $conf; done
sudo systemctl daemon-reload (Ubuntu 16.04 and Debian 8)
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, 2.2 uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,203 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 2.2 to 2.3 for |OS|
=============================================================================
This document is a step by step procedure for upgrading from Scylla 2.2 to Scylla 2.3, and rollback to 2.2 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 2.2.x to Scylla version 2.3.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 2.3 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
for conf in $(cat /var/lib/dpkg/info/scylla-*server.conffiles /var/lib/dpkg/info/scylla-*conf.conffiles /var/lib/dpkg/info/scylla-*jmx.conffiles | grep -v init ); do sudo cp -v $conf $conf.backup-2.2; done
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 2.2.x version, stop right here! This guide only covers 2.2.x to 2.3.y upgrades.
To upgrade:
1. Update the |APT|_ to **2.3**, and enable scylla/ppa repo
.. code:: sh
Debian 8:
sudo apt-get install gnupg-curl -y
sudo apt-key adv --fetch-keys https://download.opensuse.org/repositories/home:/scylladb:/scylla-3rdparty-jessie/Debian_8.0/Release.key
sudo sh -c "echo 'deb http://download.opensuse.org/repositories/home:/scylladb:/scylla-3rdparty-jessie/Debian_8.0/ /' > /etc/apt/sources.list.d/scylla-3rdparty.list"
Ubuntu 14/16:
sudo add-apt-repository -y ppa:scylladb/ppa
2. Upgrade java to 1.8 on Ubuntu 14.04 and Debian 8, which is requested by Scylla 2.3
* |ENABLE_APT_REPO|
* sudo apt-get update
* sudo apt-get install -y |JESSIE_BACKPORTS|
* sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
3. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04 and Debian 8) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on :doc:`Scylla Metrics Update - Scylla 2.2 to 2.3<metric-update-2.2-to-2.3>`
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 2.3.x to 2.2.y. Apply this procedure if an upgrade from 2.2 to 2.3 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 2.3
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 2.2, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |APT|_ to **2.2**
3. install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
for conf in $(cat /var/lib/dpkg/info/scylla-*server.conffiles /var/lib/dpkg/info/scylla-*conf.conffiles /var/lib/dpkg/info/scylla-*jmx.conffiles | grep -v init ); do sudo cp -v $conf.backup-2.2 $conf; done
sudo systemctl daemon-reload (Ubuntu 16.04 and Debian 8)
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, 2.3 uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,218 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 2.3 to 3.0 for |OS|
=============================================================================
This document is a step by step procedure for upgrading from Scylla 2.3 to Scylla 3.0, and rollback to 2.3 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 2.3.x to Scylla version 3.0.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 3.0 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
for conf in $(cat /var/lib/dpkg/info/scylla-*server.conffiles /var/lib/dpkg/info/scylla-*conf.conffiles /var/lib/dpkg/info/scylla-*jmx.conffiles | grep -v init ); do sudo cp -v $conf $conf.backup-2.3; done
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 2.3.x version, stop right here! This guide only covers 2.3.x to 3.0.y upgrades.
To upgrade:
1. Update the |APT|_ to **3.0**, and enable scylla/ppa repo
.. code:: sh
Debian 8:
sudo apt-get install gnupg-curl -y
sudo apt-key adv --fetch-keys https://download.opensuse.org/repositories/home:/scylladb:/scylla-3rdparty-jessie/Debian_8.0/Release.key
sudo sh -c "echo 'deb http://download.opensuse.org/repositories/home:/scylladb:/scylla-3rdparty-jessie/Debian_8.0/ /' > /etc/apt/sources.list.d/scylla-3rdparty.list"
Ubuntu 14/16:
sudo add-apt-repository -y ppa:scylladb/ppa
2. Upgrade java to 1.8 on Ubuntu 14.04 and Debian 8, which is requested by Scylla 3.0
* |ENABLE_APT_REPO|
* sudo apt-get update
* sudo apt-get install -y |JESSIE_BACKPORTS|
* sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
3. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
4. Upgrade node_exporter
.. code:: sh
sudo service node_exporter stop
sudo rm /usr/bin/node_exporter
sudo node_exporter_install
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04 and Debian 8) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on :doc:`Scylla Metrics Update - Scylla 2.3 to 3.0<metric-update-2.3-to-3.0>`
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 3.0.x to 2.3.y. Apply this procedure if an upgrade from 2.3 to 3.0 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 3.0
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 2.3, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |APT|_ to **2.3**
3. install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
for conf in $(cat /var/lib/dpkg/info/scylla-*server.conffiles /var/lib/dpkg/info/scylla-*conf.conffiles /var/lib/dpkg/info/scylla-*jmx.conffiles | grep -v init ); do sudo cp -v $conf.backup-2.3 $conf; done
sudo systemctl daemon-reload (Ubuntu 16.04 and Debian 8)
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, 3.0 uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
Install old node_exporter
-------------------------
.. code:: sh
sudo service node_exporter stop
sudo rm /usr/bin/node_exporter
sudo node_exporter_install
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,155 +0,0 @@
======================================================================
Upgrade Guide - Scylla 2.x.y to 2.x.z for |OS|
======================================================================
This document is a step by step procedure for upgrading from Scylla 2.x.y to Scylla 2.x.z.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 2.x.y to Scylla version 2.x.z on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Drain node and backup the data
* Check your current release
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 2.x.z features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-2.x.z
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 2.x.y version, stop right here! This guide only covers 2.x.y to 2.x.z upgrades.
To upgrade:
1. Update the |APT|_ to **2.x**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 2.x.z to 2.x.y. Apply this procedure if an upgrade from 2.x.y to 2.x.z failed before completing on all nodes. Use this procedure only for nodes you upgraded to 2.x.z
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 2.x.y, you will:
* Drain the node and stop Scylla
* Downgrade to previous release
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
Downgrade to previous release
-----------------------------
1. install
.. code:: sh
sudo apt-get install scylla=2.x.y\* scylla-server=2.x.y\* scylla-jmx=2.x.y\* scylla-tools=2.x.y\* scylla-tools-core=2.x.y\* scylla-kernel-conf=2.x.y\* scylla-conf=2.x.y\*
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-2.x.z /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,161 +0,0 @@
======================================================================
Upgrade Guide - Scylla 3.x.y to 3.x.z for |OS|
======================================================================
This document is a step by step procedure for upgrading from Scylla 3.x.y to Scylla 3.x.z.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 3.x.y to Scylla version 3.x.z on the following platform:
* |OS|
.. include:: /upgrade/_common/note-ubuntu14.rst
.. include:: /upgrade/upgrade-opensource/upgrade-guide-from-3.x.y-to-3.x.z/_common/note_3.1.0_to_3.1.1.rst
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Drain node and backup the data
* Check your current release
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 3.x.z features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-3.x.z
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a 3.x.y version, stop right here! This guide only covers 3.x.y to 3.x.z upgrades.
To upgrade:
1. Update the |APT|_ to **3.x**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04 and Ubuntu 18.04) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 3.x.z to 3.x.y. Apply this procedure if an upgrade from 3.x.y to 3.x.z failed before completing on all nodes. Use this procedure only for nodes you upgraded to 3.x.z
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 3.x.y, you will:
* Drain the node and stop Scylla
* Downgrade to previous release
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
Downgrade to previous release
-----------------------------
1. install
.. code:: sh
sudo apt-get install scylla=3.x.y\* scylla-server=3.x.y\* scylla-jmx=3.x.y\* scylla-tools=3.x.y\* scylla-tools-core=3.x.y\* scylla-kernel-conf=3.x.y\* scylla-conf=3.x.y\*
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-3.x.z /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,158 +0,0 @@
======================================================================
Upgrade Guide - ScyllaDB |FROM| to |TO| for |OS|
======================================================================
This document is a step by step procedure for upgrading from Scylla |FROM| to Scylla |TO|.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: |FROM| to Scylla version |TO| on the following platform:
* |OS|
.. include:: /upgrade/_common/note-ubuntu14.rst
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Drain node and backup the data
* Check your current release
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new |TO| features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-4.x.z
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a |FROM| version, stop right here! This guide only covers |FROM| to |TO| upgrades.
To upgrade:
1. Update the |APT|_ to **4.x**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04 and Ubuntu 18.04) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release |TO| to |FROM|. Apply this procedure if an upgrade from |FROM| to |TO| failed before completing on all nodes. Use this procedure only for nodes you upgraded to |TO|.
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to |FROM|, you will:
* Drain the node and stop Scylla
* Downgrade to previous release
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
Downgrade to previous release
-----------------------------
1. install
.. code:: sh
sudo apt-get install scylla=4.x.y\* scylla-server=4.x.y\* scylla-jmx=4.x.y\* scylla-tools=4.x.y\* scylla-tools-core=4.x.y\* scylla-kernel-conf=4.x.y\* scylla-conf=4.x.y\*
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-4.x.z /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,183 +0,0 @@
=============================================================================================================
Upgrade Guide - |SCYLLA_NAME| |SRC_VERSION| to |NEW_VERSION| for Red Hat Enterprise Linux 7 or CentOS 7
=============================================================================================================
This document is a step by step procedure for upgrading from |SCYLLA_NAME| |SRC_VERSION| to |SCYLLA_NAME| |NEW_VERSION|, and rollback to |SRC_VERSION| if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: |SRC_VERSION|.x or later to |SCYLLA_NAME| version |NEW_VERSION|.y, on the following platforms:
* Red Hat Enterprise Linux, version 7 and later
* CentOS, version 7 and later
* No longer provide packages for Fedora
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new |NEW_VERSION| features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes. See `sctool <https://manager.docs.scylladb.com/stable/sctool/index.html>`_ for suspending Scylla Manager (only available Scylla Enterprise) scheduled or running repairs.
* Not to apply schema changes
.. note:: Before upgrading, make sure to use |SCYLLA_MONITOR|_ or newer, for the Dashboards.
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-src
Stop Scylla
-----------
.. code:: sh
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``rpm -qa | grep scylla-server``. You should use the same version in case you want to :ref:`rollback <rollback-procedure>` the upgrade. If you are not running a |SRC_VERSION|.x version, stop right here! This guide only covers |SRC_VERSION|.x to |NEW_VERSION|.y upgrades.
To upgrade:
1. Update the |SCYLLA_REPO|_ to |NEW_VERSION|
2. install
.. code:: sh
sudo yum update scylla\* -y
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in the log.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on |Scylla_METRICS|_
.. _rollback-procedure:
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from |SCYLLA_NAME| release |NEW_VERSION|.x to |SRC_VERSION|.y. Apply this procedure if an upgrade from |SRC_VERSION| to |NEW_VERSION| failed before completing on all nodes. Use this procedure only for nodes you upgraded to |NEW_VERSION|
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to |SRC_VERSION|, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the |SCYLLA_REPO|_ to |SRC_VERSION|
3. Install
.. parsed-literal::
\ sudo yum clean all
\ sudo rm -rf /var/cache/yum
\ sudo yum remove scylla\\*tools-core
\ sudo yum downgrade scylla\\* -y
\ sudo yum install |PKG_NAME|
\
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-src| /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, |NEW_VERSION| uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,197 +0,0 @@
=============================================================================
Upgrade Guide - |SCYLLA_NAME| |SRC_VERSION| to |NEW_VERSION| for |OS|
=============================================================================
This document is a step by step procedure for upgrading from |SCYLLA_NAME| |SRC_VERSION| to |SCYLLA_NAME| |NEW_VERSION|, and rollback to |SRC_VERSION| if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: |SRC_VERSION|.x or later to |SCYLLA_NAME| version |NEW_VERSION|.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new |NEW_VERSION| features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes. See `sctool <https://manager.docs.scylladb.com/stable/sctool/index.html>`_ for suspending Scylla Manager (only available Scylla Enterprise) scheduled or running repairs.
* Not to apply schema changes
.. note:: Before upgrading, make sure to use |SCYLLA_MONITOR|_ or newer, for the Dashboards.
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-|SRC_VERSION|
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a |SRC_VERSION|.x version, stop right here! This guide only covers |SRC_VERSION|.x to |NEW_VERSION|.y upgrades.
To upgrade:
1. Update the |SCYLLA_REPO|_ to |NEW_VERSION|, and enable scylla/ppa repo
.. code:: sh
Ubuntu 16:
sudo add-apt-repository -y ppa:scylladb/ppa
2. Config java to 1.8, which is requested by |SCYLLA_NAME| |NEW_VERSION|
* sudo apt-get update
* sudo apt-get install -y |OPENJDK|
* sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
3. Install
.. parsed-literal::
\ sudo apt-get update
\ sudo apt-get dist-upgrade |PKG_NAME|
\
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (by ``journalctl _COMM=scylla``) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on |Scylla_METRICS|_
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from |SCYLLA_NAME| release |NEW_VERSION|.x to |SRC_VERSION|.y. Apply this procedure if an upgrade from |SRC_VERSION| to |NEW_VERSION| failed before completing on all nodes. Use this procedure only for nodes you upgraded to |NEW_VERSION|
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to |SRC_VERSION|, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |SCYLLA_REPO|_ to |SRC_VERSION|
3. install
.. parsed-literal::
\ sudo apt-get update
\ sudo apt-get remove scylla\* -y
\ sudo apt-get install |PKG_NAME|
\
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-|SRC_VERSION| /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, |NEW_VERSION| uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,196 +0,0 @@
=============================================================================================================
Upgrade Guide - |SCYLLA_NAME| |SRC_VERSION| to |NEW_VERSION| for Red Hat Enterprise Linux 7 or CentOS 7
=============================================================================================================
This document is a step by step procedure for upgrading from |SCYLLA_NAME| |SRC_VERSION| to |SCYLLA_NAME| |NEW_VERSION|, and rollback to |SRC_VERSION| if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: |SRC_VERSION|.x or later to |SCYLLA_NAME| version |NEW_VERSION|.y, on the following platforms:
* Red Hat Enterprise Linux, version 7 and later
* CentOS, version 7 and later
* No longer provide packages for Fedora
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new |NEW_VERSION| features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes. See `sctool <https://manager.docs.scylladb.com/stable/sctool/index.html>`_ for suspending Scylla Manager (only available Scylla Enterprise) scheduled or running repairs.
* Not to apply schema changes
.. note:: Before upgrading, make sure to use |SCYLLA_MONITOR|_ or newer, for the Dashboards.
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-src
Stop Scylla
-----------
.. code:: sh
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``rpm -qa | grep scylla-server``. You should use the same version in case you want to :ref:`rollback <rollback-procedure>` the upgrade. If you are not running a |SRC_VERSION|.x version, stop right here! This guide only covers |SRC_VERSION|.x to |NEW_VERSION|.y upgrades.
To upgrade:
1. Update the |SCYLLA_REPO|_ to |NEW_VERSION|
2. install
.. code:: sh
sudo yum update scylla\* -y
.. note::
Alternator users upgrading from Scylla 4.0 to 4.1, need to set :doc:`default isolation level </upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/alternator>`
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in the log.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on |Scylla_METRICS|_
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from |SCYLLA_NAME| release |NEW_VERSION|.x to |SRC_VERSION|.y. Apply this procedure if an upgrade from |SRC_VERSION| to |NEW_VERSION| failed before completing on all nodes. Use this procedure only for nodes you upgraded to |NEW_VERSION|
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to |SRC_VERSION|, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Reload systemd configuration
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the |SCYLLA_REPO|_ to |SRC_VERSION|
3. Install
.. parsed-literal::
\ sudo yum clean all
\ sudo rm -rf /var/cache/yum
\ sudo yum remove scylla\\*tools-core
\ sudo yum downgrade scylla\\* -y
\ sudo yum install |PKG_NAME|
\
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-src| /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, |NEW_VERSION| uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
Reload systemd configuration
---------------------------------
Require to reload the unit file if the systemd unit file is changed.
.. code:: sh
sudo systemctl daemon-reload
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,213 +0,0 @@
=============================================================================
Upgrade Guide - |SCYLLA_NAME| |SRC_VERSION| to |NEW_VERSION| for |OS|
=============================================================================
This document is a step by step procedure for upgrading from |SCYLLA_NAME| |SRC_VERSION| to |SCYLLA_NAME| |NEW_VERSION|, and rollback to |SRC_VERSION| if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: |SRC_VERSION|.x or later to |SCYLLA_NAME| version |NEW_VERSION|.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new |NEW_VERSION| features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes. See `sctool <https://manager.docs.scylladb.com/stable/sctool/index.html>`_ for suspending Scylla Manager (only available Scylla Enterprise) scheduled or running repairs.
* Not to apply schema changes
.. note:: Before upgrading, make sure to use |SCYLLA_MONITOR|_ or newer, for the Dashboards.
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-|SRC_VERSION|
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a |SRC_VERSION|.x version, stop right here! This guide only covers |SRC_VERSION|.x to |NEW_VERSION|.y upgrades.
To upgrade:
1. Update the |SCYLLA_REPO|_ to |NEW_VERSION|, and enable scylla/ppa repo
.. code:: sh
Ubuntu 16:
sudo add-apt-repository -y ppa:scylladb/ppa
2. Config java to 1.8, which is requested by |SCYLLA_NAME| |NEW_VERSION|
* sudo apt-get update
* sudo apt-get install -y |OPENJDK|
* sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
3. Install
.. parsed-literal::
\ sudo apt-get update
\ sudo apt-get dist-upgrade |PKG_NAME|
\
Answer y to the first two questions.
.. note::
Alternator users upgrading from Scylla 4.0 to 4.1, need to set :doc:`default isolation level </upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/alternator>`
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (by ``journalctl _COMM=scylla``) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on |Scylla_METRICS|_
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from |SCYLLA_NAME| release |NEW_VERSION|.x to |SRC_VERSION|.y. Apply this procedure if an upgrade from |SRC_VERSION| to |NEW_VERSION| failed before completing on all nodes. Use this procedure only for nodes you upgraded to |NEW_VERSION|
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to |SRC_VERSION|, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Restore system tables
* Reload systemd configuration
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |SCYLLA_REPO|_ to |SRC_VERSION|
3. install
.. parsed-literal::
\ sudo apt-get update
\ sudo apt-get remove scylla\* -y
\ sudo apt-get install |PKG_NAME|
\
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-|SRC_VERSION| /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, |NEW_VERSION| uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
Reload systemd configuration
----------------------------
Require to reload the unit file if the systemd unit file is changed.
.. code:: sh
sudo systemctl daemon-reload
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,192 +0,0 @@
=============================================================================================================
Upgrade Guide - |SCYLLA_NAME| |SRC_VERSION| to |NEW_VERSION| for |OS|
=============================================================================================================
This document is a step by step procedure for upgrading from |SCYLLA_NAME| |SRC_VERSION| to |SCYLLA_NAME| |NEW_VERSION|, and rollback to |SRC_VERSION| if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: |SRC_VERSION|.x or later to |SCYLLA_NAME| version |NEW_VERSION|.y, on the following platforms:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
Upgrading your Scylla version is a rolling procedure that does not require a full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
* Check the cluster's schema
* Drain the node and backup the data
* Backup the configuration file
* Stop the Scylla service
* Download and install new Scylla packages
* Start the Scylla service
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade, it is highly recommended:
* Not to use new |NEW_VERSION| features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes. See `sctool <https://manager.docs.scylladb.com/stable/sctool/index.html>`_ for suspending Scylla Manager (only available Scylla Enterprise) scheduled or running repairs.
* Not to apply schema changes
.. note:: Before upgrading, make sure to use |SCYLLA_MONITOR|_ or newer, for the Dashboards.
Upgrade steps
=============
Check the cluster schema
------------------------
Make sure that all nodes have the schema synched prior to upgrade as any schema disagreement between the nodes causes the upgrade to fail.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is **highly recommended** to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to an external backup device.
When the upgrade is complete (for all nodes), remove the snapshot by running ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of disk space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-src
Stop Scylla
-----------
.. include:: /rst_include/scylla-commands-stop-index.rst
Download and install the new release
------------------------------------
Before upgrading, check what Scylla version you are currently running with ``rpm -qa | grep scylla-server``. You should use the same version as this version in case you want to :ref:`rollback <rollback-procedure>` the upgrade. If you are not running a |SRC_VERSION|.x version, stop right here! This guide only covers |SRC_VERSION|.x to |NEW_VERSION|.y upgrades.
To upgrade:
1. Update the |SCYLLA_REPO|_ to |NEW_VERSION|
2. Install the new Scylla version
.. code:: sh
sudo yum clean all
sudo yum update scylla\* -y
.. note::
Alternator users upgrading from Scylla 4.0 to 4.1, need to set :doc:`default isolation level </upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/alternator>`
Start the node
--------------
.. include:: /rst_include/scylla-commands-start-index.rst
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check the Scylla version. Validate that the version matches the one you upgraded to.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in the log.
4. Check again after two minutes, to validate no new issues are introduced.
Once you are sure the node upgrade was successful, move to the next node in the cluster.
* More on |Scylla_METRICS|_
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from |SCYLLA_NAME| release |NEW_VERSION|.x to |SRC_VERSION|.y. Apply this procedure if an upgrade from |SRC_VERSION| to |NEW_VERSION| failed before completing on all nodes. Use this procedure only for the nodes that you upgraded to |NEW_VERSION|
Scylla rollback is a rolling procedure that does **not** require a full cluster shutdown.
For each of the nodes rollback to |SRC_VERSION|, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Reload the systemd configuration
* Restart the Scylla service
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the rollback was successful and that the node is up and running with the old version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
.. include:: /rst_include/scylla-commands-stop-index.rst
Download and install the new release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the |SCYLLA_REPO|_ to |SRC_VERSION|
3. Install
.. parsed-literal::
\ sudo yum clean all
\ sudo rm -rf /var/cache/yum
\ sudo yum remove scylla\\*tools-core
\ sudo yum downgrade scylla\\* -y
\ sudo yum install |PKG_NAME|
\
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-src| /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, |NEW_VERSION| uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
Reload systemd configuration
---------------------------------
Require to reload the unit file if the systemd unit file is changed.
.. code:: sh
sudo systemctl daemon-reload
Start the node
--------------
.. include:: /rst_include/scylla-commands-start-index.rst
Validate
--------
Check the upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster. Keep in mind that the version you want to see on your node is the old version, which you noted at the beginning of the procedure.

View File

@@ -1,203 +0,0 @@
=============================================================================
Upgrade Guide - |SCYLLA_NAME| |SRC_VERSION| to |NEW_VERSION| for |OS|
=============================================================================
This document is a step by step procedure for upgrading from |SCYLLA_NAME| |SRC_VERSION| to |SCYLLA_NAME| |NEW_VERSION|, and rollback to |SRC_VERSION| if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: |SRC_VERSION|.x or later to |SCYLLA_NAME| version |NEW_VERSION|.y on the following platform:
* |OS|
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* Check cluster schema
* Drain node and backup the data
* Backup configuration file
* Stop Scylla
* Download and install new Scylla packages
* Start Scylla
* Validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new |NEW_VERSION| features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes. See `sctool <https://manager.docs.scylladb.com/stable/sctool/index.html>`_ for suspending Scylla Manager (only available Scylla Enterprise) scheduled or running repairs.
* Not to apply schema changes
.. note:: Before upgrading, make sure to use |SCYLLA_MONITOR|_ or newer, for the Dashboards.
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-|SRC_VERSION|
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to |ROLLBACK|_ the upgrade. If you are not running a |SRC_VERSION|.x version, stop right here! This guide only covers |SRC_VERSION|.x to |NEW_VERSION|.y upgrades.
To upgrade:
1. Update the |SCYLLA_REPO|_ to |NEW_VERSION|
2. Install
.. parsed-literal::
\ sudo apt-get clean all
\ sudo apt-get update
\ sudo apt-get dist-upgrade |PKG_NAME|
\
Answer y to the first two questions.
.. note::
Alternator users upgrading from Scylla 4.0 to 4.1, need to set :doc:`default isolation level </upgrade/upgrade-opensource/upgrade-guide-from-4.0-to-4.1/alternator>`
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (by ``journalctl _COMM=scylla``) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on |Scylla_METRICS|_
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from |SCYLLA_NAME| release |NEW_VERSION|.x to |SRC_VERSION|.y. Apply this procedure if an upgrade from |SRC_VERSION| to |NEW_VERSION| failed before completing on all nodes. Use this procedure only for nodes you upgraded to |NEW_VERSION|
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to |SRC_VERSION|, you will:
* Drain the node and stop Scylla
* Retrieve the old Scylla packages
* Restore the configuration file
* Restore system tables
* Reload systemd configuration
* Restart Scylla
* Validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the |SCYLLA_REPO|_ to |SRC_VERSION|
3. install
.. parsed-literal::
\ sudo apt-get update
\ sudo apt-get remove scylla\* -y
\ sudo apt-get install |PKG_NAME|
\
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-|SRC_VERSION| /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, |NEW_VERSION| uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
Reload systemd configuration
----------------------------
Require to reload the unit file if the systemd unit file is changed.
.. code:: sh
sudo systemctl daemon-reload
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -58,9 +58,14 @@ When the upgrade is completed on all nodes, remove the snapshot with the ``nodet
Backup the configuration file
------------------------------
Back up the ``scylla.yaml`` configuration file and the ScyllaDB packages
in case you need to rollback the upgrade.
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-src
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup
sudo cp /etc/yum.repos.d/scylla.repo ~/scylla.repo-backup
Stop ScyllaDB
---------------
@@ -69,7 +74,7 @@ Stop ScyllaDB
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``rpm -qa | grep scylla-server``. You should use the same version as this version in case you want to :ref:`rollback <rollback-procedure>` the upgrade. If you are not running a |SRC_VERSION|.x version, stop right here! This guide only covers |SRC_VERSION|.x to |NEW_VERSION|.y upgrades.
Before upgrading, check what version you are running now using ``rpm -qa | grep scylla-server``. You should use the same version as this version in case you want to :ref:`rollback <rollback-procedure-v4>` the upgrade. If you are not running a |SRC_VERSION|.x version, stop right here! This guide only covers |SRC_VERSION|.x to |NEW_VERSION|.y upgrades.
To upgrade:
@@ -98,6 +103,8 @@ Once you are sure the node upgrade was successful, move to the next node in the
See |Scylla_METRICS|_ for more information..
.. _rollback-procedure-v4:
Rollback Procedure
==================
@@ -122,29 +129,33 @@ Rollback Steps
==============
Gracefully shutdown ScyllaDB
-----------------------------
.. code:: sh
nodetool drain
.. include:: /rst_include/scylla-commands-stop-index.rst
nodetool snapshot
sudo service scylla-server stop
Download and install the old release
Restore and install the old release
------------------------------------
#. Remove the old repo file.
#. Restore the |SRC_VERSION| packages backed up during the upgrade.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
sudo cp ~/scylla.repo-backup /etc/yum.repos.d/scylla.repo
sudo chown root.root /etc/yum.repos.d/scylla.repo
sudo chmod 644 /etc/yum.repos.d/scylla.repo
#. Update the |SCYLLA_REPO|_ to |SRC_VERSION|.
#. Install:
.. code:: console
sudo yum clean all
sudo rm -rf /var/cache/yum
sudo yum remove scylla\\*tools-core
sudo yum downgrade scylla\\* -y
sudo yum install scylla
sudo yum downgrade scylla-\*cqlsh -y
sudo yum remove scylla-\*cqlsh -y
sudo yum downgrade scylla\* -y
sudo yum install scylla -y
Restore the configuration file
@@ -153,18 +164,7 @@ Restore the configuration file
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-src | /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot because |NEW_VERSION| uses a different set of system tables. See :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>` for details.
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo cp /etc/scylla/scylla.yaml-backup /etc/scylla/scylla.yaml
Reload systemd configuration
---------------------------------

View File

@@ -60,9 +60,13 @@ When the upgrade is completed on all nodes, remove the snapshot with the ``nodet
Backup the configuration file
------------------------------
Back up the ``scylla.yaml`` configuration file and the ScyllaDB packages
in case you need to rollback the upgrade.
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-src
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup
sudo cp /etc/apt/sources.list.d/scylla.list ~/scylla.list-backup
Gracefully stop the node
------------------------

View File

@@ -44,7 +44,6 @@ For each of the nodes you rollback to |SRC_VERSION|, you will:
* Drain the node and stop Scylla
* Retrieve the old ScyllaDB packages
* Restore the configuration file
* Restore system tables
* Reload systemd configuration
* Restart ScyllaDB
* Validate the rollback success
@@ -59,17 +58,19 @@ Gracefully shutdown ScyllaDB
.. code:: sh
nodetool drain
nodetool snapshot
sudo service scylla-server stop
Download and install the old release
Restore and install the old release
------------------------------------
#. Remove the old repo file.
#. Restore the |SRC_VERSION| packages backed up during the upgrade.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
sudo cp ~/scylla.list-backup /etc/apt/sources.list.d/scylla.list
sudo chown root.root /etc/apt/sources.list.d/scylla.list
sudo chmod 644 /etc/apt/sources.list.d/scylla.list
#. Update the |SCYLLA_REPO|_ to |SRC_VERSION|.
#. Install:
.. code-block::
@@ -85,18 +86,7 @@ Restore the configuration file
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-src | /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from the previous snapshot because |NEW_VERSION| uses a different set of system tables. See :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>` for reference.
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo chown -R scylla:scylla /var/lib/scylla/data/keyspace_name/table_name-UUID/
sudo cp /etc/scylla/scylla.yaml-backup /etc/scylla/scylla.yaml
Reload systemd configuration
----------------------------

View File

@@ -34,7 +34,7 @@ The following example shows the upgrade path for a 3-node cluster from version 4
#. Upgrade all three nodes to version 4.6.
Upgrading to each patch version by following the :doc:`Scylla Maintenance Release Upgrade Guide </upgrade/upgrade-opensource/upgrade-guide-from-4.x.y-to-4.x.z/index>`
Upgrading to each patch version by following the Maintenance Release Upgrade Guide
is optional. However, we recommend upgrading to the latest patch release for your version before upgrading to a new version.
For example, upgrade to patch 4.4.8 before upgrading to version 4.5.

View File

@@ -13,7 +13,7 @@ ScyllaDB follows the ``MAJOR.MINOR.PATCH`` `semantic versioning <https://semver.
ScyllaDB Open Source:
* ``MAJOR`` versions: 4.y, 5.y
* ``MINOR`` versions: 5.2.z, 5.3.z
* ``MINOR`` versions: 5.2.z, 5.4.z
* ``PATCH`` versions: 5.2.1, 5.2.2

View File

@@ -4,31 +4,13 @@ Upgrade ScyllaDB Open Source
.. toctree::
:hidden:
ScyllaDB 5.2 to 5.3 <upgrade-guide-from-5.2-to-5.3/index>
ScyllaDB 5.2 to 5.4 <upgrade-guide-from-5.2-to-5.4/index>
ScyllaDB 5.1 to 5.2 <upgrade-guide-from-5.1-to-5.2/index>
ScyllaDB 5.0 to 5.1 <upgrade-guide-from-5.0-to-5.1/index>
ScyllaDB 5.x maintenance release <upgrade-guide-from-5.x.y-to-5.x.z/index>
ScyllaDB 4.6 to 5.0 <upgrade-guide-from-4.6-to-5.0/index>
ScyllaDb 4.5 to 4.6 <upgrade-guide-from-4.5-to-4.6/index>
ScyllaDB 4.4 to 4.5 <upgrade-guide-from-4.4-to-4.5/index>
ScyllaDB 4.3 to 4.4 <upgrade-guide-from-4.3-to-4.4/index>
ScyllaDB 4.2 to 4.3 <upgrade-guide-from-4.2-to-4.3/index>
ScyllaDB 4.1 to 4.2 <upgrade-guide-from-4.1-to-4.2/index>
ScyllaDB 4.x maintenance release <upgrade-guide-from-4.x.y-to-4.x.z/index>
ScyllaDB 4.0 to 4.1 <upgrade-guide-from-4.0-to-4.1/index>
ScyllaDB 3.x maintenance release <upgrade-guide-from-3.x.y-to-3.x.z/index>
ScyllaDB 3.3 to 4.0 <upgrade-guide-from-3.3-to-4.0/index>
ScyllaDB 3.2 to 3.3 <upgrade-guide-from-3.2-to-3.3/index>
ScyllaDB 3.1 to 3.2 <upgrade-guide-from-3.1-to-3.2/index>
ScyllaDB 3.0 to 3.1 <upgrade-guide-from-3.0-to-3.1/index>
ScyllaDB 2.3 to 3.0 <upgrade-guide-from-2.3-to-3.0/index>
ScyllaDB 2.2 to 2.3 <upgrade-guide-from-2.2-to-2.3/index>
ScyllaDB 2.1 to 2.2 <upgrade-guide-from-2.1-to-2.2/index>
ScyllaDB 2.x maintenance release <upgrade-guide-from-2.x.y-to-2.x.z/index>
Older versions <upgrade-archive>
Ubuntu 14.04 to 16.04 <upgrade-guide-from-ubuntu-14-to-16>
.. panel-box::
:title: Upgrade ScyllaDB Open Source
@@ -38,27 +20,9 @@ Upgrade ScyllaDB Open Source
Procedures for upgrading to a newer version of ScyllaDB Open Source.
* :doc:`Upgrade Guide - ScyllaDB 5.2 to 5.3 <upgrade-guide-from-5.2-to-5.3/index>`
* :doc:`Upgrade Guide - ScyllaDB 5.2 to 5.4 <upgrade-guide-from-5.2-to-5.4/index>`
* :doc:`Upgrade Guide - ScyllaDB 5.1 to 5.2 <upgrade-guide-from-5.1-to-5.2/index>`
* :doc:`Upgrade Guide - ScyllaDB 5.0 to 5.1 <upgrade-guide-from-5.0-to-5.1/index>`
* :doc:`Upgrade Guide - ScyllaDB 5.x maintenance releases <upgrade-guide-from-5.x.y-to-5.x.z/index>`
* :doc:`Upgrade Guide - ScyllaDB 4.6 to 5.0 <upgrade-guide-from-4.6-to-5.0/index>`
* :doc:`Upgrade Guide - ScyllaDB 4.5 to 4.6 <upgrade-guide-from-4.5-to-4.6/index>`
* :doc:`Upgrade Guide - ScyllaDB 4.4 to 4.5 <upgrade-guide-from-4.4-to-4.5/index>`
* :doc:`Upgrade Guide - ScyllaDB 4.3 to 4.4 <upgrade-guide-from-4.3-to-4.4/index>`
* :doc:`Upgrade Guide - ScyllaDB 4.2 to 4.3 <upgrade-guide-from-4.2-to-4.3/index>`
* :doc:`Upgrade Guide - ScyllaDB 4.1 to 4.2 <upgrade-guide-from-4.1-to-4.2/index>`
* :doc:`Upgrade Guide - ScyllaDB 4.x maintenance release <upgrade-guide-from-4.x.y-to-4.x.z/index>`
* :doc:`Upgrade Guide - ScyllaDB 4.0 to 4.1 <upgrade-guide-from-4.0-to-4.1/index>`
* :doc:`Upgrade Guide - ScyllaDB 3.x maintenance release <upgrade-guide-from-3.x.y-to-3.x.z/index>`
* :doc:`Upgrade Guide - ScyllaDB 3.3 to 4.0 <upgrade-guide-from-3.3-to-4.0/index>`
* :doc:`Upgrade Guide - ScyllaDB 3.2 to 3.3 <upgrade-guide-from-3.2-to-3.3/index>`
* :doc:`Upgrade Guide - ScyllaDB 3.1 to 3.2 <upgrade-guide-from-3.1-to-3.2/index>`
* :doc:`Upgrade Guide - ScyllaDB 3.0 to 3.1 <upgrade-guide-from-3.0-to-3.1/index>`
* :doc:`Upgrade Guide - ScyllaDB 2.3 to 3.0 <upgrade-guide-from-2.3-to-3.0/index>`
* :doc:`Upgrade Guide - ScyllaDB 2.2 to 2.3 <upgrade-guide-from-2.2-to-2.3/index>`
* :doc:`Upgrade Guide - ScyllaDB 2.1 to 2.2 <upgrade-guide-from-2.1-to-2.2/index>`
* :doc:`Upgrade Guide - ScyllaDB 2.x maintenance release <upgrade-guide-from-2.x.y-to-2.x.z/index>`
* :doc:`Upgrade Guide - older versions <upgrade-archive>`
* :doc:`Upgrade Guide - Ubuntu 14.04 to 16.04 <upgrade-guide-from-ubuntu-14-to-16>`
* :ref:`Upgrade Unified Installer (relocatable executable) install <unified-installed-upgrade>`

View File

@@ -1,59 +0,0 @@
==========================================
Upgrade Scylla Opensource - older versions
==========================================
.. toctree::
:hidden:
Scylla 2.0 to 2.1 <upgrade-guide-from-2.0-to-2.1/index>
Scylla 1.7 to 2.0 <upgrade-guide-from-1.7-to-2.0/index>
Scylla 1.6 to 1.7 <upgrade-guide-from-1.6-to-1.7/index>
Scylla 1.5 to 1.6 <upgrade-guide-from-1.5-to-1.6/index>
Scylla 1.4 to 1.5 <upgrade-guide-from-1.4-to-1.5/index>
Scylla 1.3 to 1.4 <upgrade-guide-from-1.3-to-1.4/index>
Scylla 1.2 to 1.3 <upgrade-guide-from-1.2-to-1.3/index>
Scylla 1.1 to 1.2 <upgrade-guide-from-1.1-to-1.2/index>
Scylla 1.0 to 1.1 <upgrade-guide-from-1.0-to-1.1/index>
Scylla 1.x maintenance release <upgrade-guide-from-1.x.y-to-1.x.z/index>
.. raw:: html
<div class="panel callout radius animated">
<div class="row">
<div class="medium-3 columns">
<h5 id="getting-started">Upgrade Scylla Open Source</h5>
</div>
<div class="medium-9 columns">
Procedures for upgrading to a newer version of Scylla Open Source. For the latest open source versions see :doc:`here </upgrade/upgrade-opensource/index>`.
* :doc:`Upgrade Guide - Scylla 2.0 to 2.1 </upgrade/upgrade-opensource/upgrade-guide-from-2.0-to-2.1/index>`
* :doc:`Upgrade Guide - Scylla 1.x maintenance release </upgrade/upgrade-opensource/upgrade-guide-from-1.x.y-to-1.x.z/index>`
* :doc:`Upgrade Guide - Scylla 1.7 to 2.0 </upgrade/upgrade-opensource/upgrade-guide-from-1.7-to-2.0/index>`
* :doc:`Upgrade Guide - Scylla 1.6 to 1.7 </upgrade/upgrade-opensource/upgrade-guide-from-1.6-to-1.7/index>`
* :doc:`Upgrade Guide - Scylla 1.5 to 1.6 </upgrade/upgrade-opensource/upgrade-guide-from-1.5-to-1.6/index>`
* :doc:`Upgrade Guide - Scylla 1.4 to 1.5 </upgrade/upgrade-opensource/upgrade-guide-from-1.4-to-1.5/index>`
* :doc:`Upgrade Guide - Scylla 1.3 to 1.4 </upgrade/upgrade-opensource/upgrade-guide-from-1.3-to-1.4/index>`
* :doc:`Upgrade Guide - Scylla 1.2 to 1.3 </upgrade/upgrade-opensource/upgrade-guide-from-1.2-to-1.3/index>`
* :doc:`Upgrade Guide - Scylla 1.1 to 1.2 </upgrade/upgrade-opensource/upgrade-guide-from-1.1-to-1.2/index>`
* :doc:`Upgrade Guide - Scylla 1.0 to 1.1 </upgrade/upgrade-opensource/upgrade-guide-from-1.0-to-1.1/index>`
.. raw:: html
</div>
</div>
</div>

View File

@@ -1,16 +0,0 @@
=================================
Upgrade - Scylla 1.0 to 1.1
=================================
.. toctree::
:maxdepth: 2
:hidden:
Fedora, Red Hat Enterprise Linux, and CentOS <upgrade-guide-from-1.0-to-1.1-rpm>
Ubuntu <upgrade-guide-from-1.0-to-1.1-ubuntu>
Upgrade guides are available for:
* :doc:`Upgrade Scylla from 1.0.x to 1.1.y on Fedora, Red Hat Enterprise Linux, and CentOS <upgrade-guide-from-1.0-to-1.1-rpm>`
* :doc:`Upgrade Scylla from 1.0.x to 1.1.y on Ubuntu <upgrade-guide-from-1.0-to-1.1-ubuntu>`

View File

@@ -1,183 +0,0 @@
==================================================================
Upgrade Guide - Scylla 1.0 to 1.1 for Red Hat Enterprise or CentOS
==================================================================
This document is a step by step procedure for upgrading from Scylla 1.0
to Scylla 1.1, and rollback to 1.0 if required.
Applicable versions
-------------------
This guide covers upgrading Scylla from the following versions: 1.0.x to
Scylla version 1.1.y, on the following platforms:
- Fedora 22
- Red Hat Enterprise Linux, version 7 and later
- CentOS, version 7 and later
Upgrade Procedure
-----------------
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
- backup the data
- check your current release
- backup configuration file
- download and install new Scylla packages
- gracefully restart Scylla
- validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
**During** the rolling upgrade it is highly recommended:
- Not to use new 1.1 features
- Not to run administration functions, like repairs, refresh, rebuild
or add or remove nodes
- Not to apply schema changes
Backup the data
~~~~~~~~~~~~~~~
Before any major procedure, like an upgrade, it is recommended to backup
all the data to an external device. In Scylla, backup is done using the
``nodetool snapshot`` command. For **each** node in the cluster, run the
following command:
.. code:: sh
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all
the directories having this name under ``/var/lib/scylla`` to a backup
device.
Upgrade steps
-------------
Backup configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.0
sudo rm -rf /etc/yum.repos.d/scylla.repo
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before upgrading, check what version you are running now using
``rpm -qa | grep scylla-server``. You should use the same version in
case you want to
:ref:`rollback <upgrade-1.0-1.1-rpm-rollback-procedure>` the
upgrade. If you are not running a 1.0.x version, stop right here! This
guide only covers 1.0.x to 1.1.y upgrades.
To upgrade:
1. Update the `Scylla RPM repo <http://www.scylladb.com/download/#fndtn-RPM>`_ to
**1.1**
2. install
.. code:: sh
sudo yum clean all
sudo yum update scylla-server scylla-jmx scylla-tools -y
Gracefully restart the node
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo systemctl restart scylla-server.service
Validate
~~~~~~~~
1. Check cluster status with ``nodetool status`` and make sure **all**
nodes, including the one you just upgraded, are in UN status.
2. Use ``journalctl _COMM=scylla`` to check there are no new errors in
the log.
3. Check again after 2 minutes, to validate no new issues are
introduced.
Once you are sure the node upgrade is successful, move to the next node
in the cluster.
.. _upgrade-1.0-1.1-rpm-rollback-procedure:
Rollback Procedure
------------------
The following procedure describes a rollback from Scylla release 1.1.x
to 1.0.y. Apply this procedure if an upgrade from 1.0 to 1.1 failed
before completing on all nodes. Use this procedure only for nodes you
upgraded to 1.1
Scylla rollback is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes rollback to 1.0, you will:
- retrieve the old Scylla packages
- drain the node
- restore the configuration file
- restart Scylla
- validate the rollback success
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
Rollback steps
--------------
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the `Scylla RPM repo <http://www.scylladb.com/download/#fndtn-RPM>`_ to
**1.0**
3. Install
.. code:: sh
sudo yum clean all
sudo yum downgrade scylla-server scylla-jmx scylla-tools -y
Gracefully shutdown Scylla
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server stop
Restore the configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.0 /etc/scylla/scylla.yaml
Start the node
~~~~~~~~~~~~~~
.. code:: sh
sudo systemctl start scylla-server.service
Validate
~~~~~~~~
Check upgrade instruction above for validation. Once you are sure the
node rollback is successful, move to the next node in the cluster.

View File

@@ -1,185 +0,0 @@
============================================
Upgrade Guide - Scylla 1.0 to 1.1 for Ubuntu
============================================
This document is a step by step procedure for upgrading from Scylla 1.0
to Scylla 1.1, and rollback to 1.0 if required.
Applicable versions
-------------------
This guide covers upgrading Scylla from the following versions: 1.0.x to
Scylla version 1.1.y on the following platform:
- Ubuntu 14.04
Upgrade Procedure
-----------------
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
- backup the data
- check your current release
- backup configuration file
- download and install new Scylla packages
- gracefully restart Scylla
- validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
**During** the rolling upgrade it is highly recommended:
- Not to use new 1.1 features
- Not to run administration functions, like repairs, refresh, rebuild
or add or remove nodes
- Not to apply schema changes
Backup the data
~~~~~~~~~~~~~~~
Before any major procedure, like an upgrade, it is recommended to backup
all the data to an external device. In Scylla, backup is done using the
``nodetool snapshot`` command. For **each** node in the cluster, run the
following command:
.. code:: sh
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all
the directories having this name under ``/var/lib/scylla`` to a backup
device.
Upgrade steps
-------------
Backup configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.0
sudo rm -rf /etc/apt/sources.list.d/scylla.list
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before upgrading, check what version you are running now using
``dpkg -s scylla-server``. You should use the same version in case you
want to :ref:`rollback <upgrade-1.0-1.1-ubuntu-rollback-procedure>`
the upgrade. If you are not running a 1.0.x version, stop right here!
This guide only covers 1.0.x to 1.1.y upgrades.
To upgrade:
1. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to
**1.1**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get upgrade scylla-server scylla-jmx scylla-tools
Answer y to the first two questions and 'n' when asked to overwrite
``scylla.yaml``.
Gracefully restart the node
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server restart
Validate
~~~~~~~~
1. Check cluster status with ``nodetool status`` and make sure **all**
nodes, including the one you just upgraded, are in UN status.
2. Check ``/var/log/upstart/scylla-server.log`` and ``/var/log/syslog``
to validate there are no errors.
3. Check again after 2 minutes, to validate no new issues are
introduced.
Once you are sure the node upgrade is successful, move to the next node
in the cluster.
.. _upgrade-1.0-1.1-ubuntu-rollback-procedure:
Rollback Procedure
------------------
The following procedure describes a rollback from Scylla release 1.1.x
to 1.0.y. Apply this procedure if an upgrade from 1.0 to 1.1 failed
before completing on all nodes. Use this procedure only for nodes you
upgraded to 1.1
Scylla rollback is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes rollback to 1.0, you will:
- retrieve the old Scylla packages
- drain the node
- restore the configuration file
- restart Scylla
- validate the rollback success
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
Rollback steps
--------------
download and install the old release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to
**1.0**
3. install
::
sudo apt-get update
sudo apt-get remove --assume-yes scylla-server scylla-jmx scylla-tools
sudo apt-get install scylla-server scylla-jmx scylla-tools
Answer y to the first two questions and 'n' when asked to overwrite
``scylla.yaml``.
Gracefully shutdown Scylla
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo systemctl stop scylla-server.service
Restore the configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.0 /etc/scylla/scylla.yaml
Start the node
~~~~~~~~~~~~~~
.. code:: sh
sudo service scylla-server start
Validate
~~~~~~~~
Check upgrade instruction above for validation. Once you are sure the
node rollback is successful, move to the next node in the cluster.

View File

@@ -1,33 +0,0 @@
=================================
Upgrade - Scylla 1.1 to 1.2
=================================
.. toctree::
:maxdepth: 2
:hidden:
Fedora, Red Hat Enterprise Linux, and CentOS <upgrade-guide-from-1.1-to-1.2-rpm>
Ubuntu <upgrade-guide-from-1.1-to-1.2-ubuntu>
.. raw:: html
<div class="panel callout radius animated">
<div class="row">
<div class="medium-3 columns">
<h5 id="getting-started">Upgrade Scylla from 1.1 to 1.2</h5>
</div>
<div class="medium-9 columns">
Upgrade guides are available for:
* :doc:`Upgrade Scylla from 1.1.x to 1.2.y on Fedora, Red Hat Enterprise Linux, and CentOS <upgrade-guide-from-1.1-to-1.2-rpm>`
* :doc:`Upgrade Scylla from 1.1.x to 1.2.y on Ubuntu <upgrade-guide-from-1.1-to-1.2-ubuntu>`
.. raw:: html
</div>
</div>
</div>

View File

@@ -1,185 +0,0 @@
==================================================================
Upgrade Guide - Scylla 1.1 to 1.2 for Red Hat Enterprise or CentOS
==================================================================
This document is a step by step procedure for upgrading from Scylla 1.1
to Scylla 1.2, and rollback to 1.1 if required.
Applicable versions
-------------------
This guide covers upgrading Scylla from the following versions: 1.1.x to
Scylla version 1.2.y, on the following platforms:
- Fedora 22
- Red Hat Enterprise Linux, version 7 and later
- CentOS, version 7 and later
Upgrade Procedure
-----------------
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
- backup the data
- check your current release
- backup configuration file
- download and install new Scylla packages
- gracefully restart Scylla
- validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
**During** the rolling upgrade it is highly recommended:
- Not to use new 1.2 features
- Not to run administration functions, like repairs, refresh, rebuild
or add or remove nodes
- Not to apply schema changes
Backup the data
~~~~~~~~~~~~~~~
Before any major procedure, like an upgrade, it is recommended to backup
all the data to an external device. In Scylla, backup is done using the
``nodetool snapshot`` command. For **each** node in the cluster, run the
following command:
.. code:: sh
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all
the directories having this name under ``/var/lib/scylla`` to a backup
device.
Upgrade steps
-------------
Backup configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.1
sudo rm -rf /etc/yum.repos.d/scylla.repo
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before upgrading, check what version you are running now using
``rpm -qa | grep scylla-server``. You should use the same version in
case you want to
:ref:`rollback <upgrade-1.1-1.2-rpm-rollback-procedure>` the
upgrade. If you are not running a 1.1.x version, stop right here! This
guide only covers 1.1.x to 1.2.y upgrades.
To upgrade:
1. Update the `Scylla RPM repo <http://www.scylladb.com/download/#fndtn-RPM>`_ to
**1.2**
2. install
.. code:: sh
sudo yum clean all
sudo yum update scylla-server scylla-jmx scylla-tools -y
sudo yum install scylla
Gracefully restart the node
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo systemctl restart scylla-server.service
Validate
~~~~~~~~
1. Check cluster status with ``nodetool status`` and make sure **all**
nodes, including the one you just upgraded, are in UN status.
2. Use ``journalctl _COMM=scylla`` to check there are no new errors in
the log.
3. Check again after 2 minutes, to validate no new issues are
introduced.
Once you are sure the node upgrade is successful, move to the next node
in the cluster.
.. _upgrade-1.1-1.2-rpm-rollback-procedure:
Rollback Procedure
------------------
The following procedure describes a rollback from Scylla release 1.2.x
to 1.1.y. Apply this procedure if an upgrade from 1.1 to 1.2 failed
before completing on all nodes. Use this procedure only for nodes you
upgraded to 1.2
Scylla rollback is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes rollback to 1.1, you will:
- retrieve the old Scylla packages
- drain the node
- restore the configuration file
- restart Scylla
- validate the rollback success
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
Rollback steps
--------------
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the `Scylla RPM repo <http://www.scylladb.com/download/#fndtn-RPM>`_ to
**1.1**
3. Install
.. code:: sh
sudo yum clean all
sudo yum remove scylla
sudo yum downgrade scylla-server scylla-jmx scylla-tools -y
Gracefully shutdown Scylla
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server stop
Restore the configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.1 /etc/scylla/scylla.yaml
Start the node
~~~~~~~~~~~~~~
.. code:: sh
sudo systemctl start scylla-server.service
Validate
~~~~~~~~
Check upgrade instruction above for validation. Once you are sure the
node rollback is successful, move to the next node in the cluster.

View File

@@ -1,191 +0,0 @@
============================================
Upgrade Guide - Scylla 1.1 to 1.2 for Ubuntu
============================================
This document is a step by step procedure for upgrading from Scylla 1.1
to Scylla 1.2, and rollback to 1.1 if required.
Applicable versions
-------------------
This guide covers upgrading Scylla from the following versions: 1.1.x to
Scylla version 1.2.y on the following platform:
- Ubuntu 14.04
Upgrade Procedure
-----------------
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
- backup the data
- check your current release
- backup configuration file
- download and install new Scylla packages
- gracefully restart Scylla
- validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
**During** the rolling upgrade it is highly recommended:
- Not to use new 1.2 features
- Not to run administration functions, like repairs, refresh, rebuild
or add or remove nodes
- Not to apply schema changes
Backup the data
~~~~~~~~~~~~~~~
Before any major procedure, like an upgrade, it is recommended to backup
all the data to an external device. In Scylla, backup is done using the
``nodetool snapshot`` command. For **each** node in the cluster, run the
following command:
.. code:: sh
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all
the directories having this name under ``/var/lib/scylla`` to a backup
device.
Upgrade steps
-------------
Backup configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.1
sudo rm -rf /etc/apt/sources.list.d/scylla.list
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before upgrading, check what version you are running now using
``dpkg -s scylla-server``. You should use the same version in case you
want to :ref:`rollback <upgrade-1.1-1.2-ubuntu-rollback-procedure>`
the upgrade. If you are not running a 1.1.x version, stop right here!
This guide only covers 1.1.x to 1.2.y upgrades.
To upgrade:
1. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to
**1.2**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla-server scylla-jmx scylla-tools
Answer 'y'
.. code:: sh
sudo apt-get install scylla
Answer y to the first two questions and 'n' when asked to overwrite
``scylla.yaml``.
Gracefully restart the node
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server restart
Validate
~~~~~~~~
1. Check cluster status with ``nodetool status`` and make sure **all**
nodes, including the one you just upgraded, are in UN status.
2. Check ``/var/log/upstart/scylla-server.log`` and ``/var/log/syslog``
to validate there are no errors.
3. Check again after 2 minutes, to validate no new issues are
introduced.
Once you are sure the node upgrade is successful, move to the next node
in the cluster.
.. _upgrade-1.1-1.2-ubuntu-rollback-procedure:
Rollback Procedure
------------------
The following procedure describes a rollback from Scylla release 1.2.x
to 1.1.y. Apply this procedure if an upgrade from 1.1 to 1.2 failed
before completing on all nodes. Use this procedure only for nodes you
upgraded to 1.2
Scylla rollback is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes rollback to 1.1, you will:
- retrieve the old Scylla packages
- drain the node
- restore the configuration file
- restart Scylla
- validate the rollback success
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
Rollback steps
--------------
download and install the old release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to
**1.1**
3. install
::
sudo apt-get update
sudo apt-get remove --assume-yes scylla-conf scylla-server scylla-jmx scylla-tools
sudo apt-get install scylla-server scylla-jmx scylla-tools
Answer y to the first question and 'n' when asked to overwrite
``scylla.yaml``.
Gracefully shutdown Scylla
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server stop
Restore the configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.1 /etc/scylla/scylla.yaml
Start the node
~~~~~~~~~~~~~~
.. code:: sh
sudo service scylla-server start
Validate
~~~~~~~~
Check upgrade instruction above for validation. Once you are sure the
node rollback is successful, move to the next node in the cluster.

View File

@@ -1,32 +0,0 @@
=================================
Upgrade Guide - Scylla 1.2 to 1.3
=================================
.. toctree::
:maxdepth: 2
:hidden:
Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.2-to-1.3-rpm>
Ubuntu <upgrade-guide-from-1.2-to-1.3-ubuntu>
.. raw:: html
<div class="panel callout radius animated">
<div class="row">
<div class="medium-3 columns">
<h5 id="getting-started">Upgrade Scylla from 1.2 to 1.3</h5>
</div>
<div class="medium-9 columns">
Upgrade guides are available for:
* :doc:`Upgrade Scylla from 1.2.x to 1.3.y on Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.2-to-1.3-rpm>`
* :doc:`Upgrade Scylla from 1.2.x to 1.3.y on Ubuntu <upgrade-guide-from-1.2-to-1.3-ubuntu>`
.. raw:: html
</div>
</div>
</div>

View File

@@ -1,186 +0,0 @@
==================================================================
Upgrade Guide - Scylla 1.2 to 1.3 for Red Hat Enterprise or CentOS
==================================================================
This document is a step by step procedure for upgrading from Scylla 1.2
to Scylla 1.3, and rollback to 1.2 if required.
Applicable versions
-------------------
This guide covers upgrading Scylla from the following versions: 1.2.x to
Scylla version 1.3.y, on the following platforms:
- Red Hat Enterprise Linux, version 7 and later
- CentOS, version 7 and later
- No longer provide packages for Fedora
Upgrade Procedure
-----------------
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
- backup the data
- check your current release
- backup configuration file
- download and install new Scylla packages
- gracefully restart Scylla
- validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
**During** the rolling upgrade it is highly recommended:
- Not to use new 1.3 features
- Not to run administration functions, like repairs, refresh, rebuild
or add or remove nodes
- Not to apply schema changes
Backup the data
~~~~~~~~~~~~~~~
Before any major procedure, like an upgrade, it is recommended to backup
all the data to an external device. In Scylla, backup is done using the
``nodetool snapshot`` command. For **each** node in the cluster, run the
following command:
.. code:: sh
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all
the directories having this name under ``/var/lib/scylla`` to a backup
device.
Upgrade steps
-------------
Backup configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.2
sudo rm -rf /etc/yum.repos.d/scylla.repo
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before upgrading, check what version you are running now using
``rpm -qa | grep scylla-server``. You should use the same version in
case you want to
:ref:`rollback <upgrade-1.2-1.3-rpm-rollback-procedure>` the
upgrade. If you are not running a 1.2.x version, stop right here! This
guide only covers 1.2.x to 1.3.y upgrades.
To upgrade:
1. Update the `Scylla RPM repo <http://www.scylladb.com/download/#fndtn-RPM>`_ to
**1.3**
2. install
.. code:: sh
sudo yum clean all
sudo yum update scylla scylla-server scylla-jmx scylla-tools scylla-conf scylla-kernel-conf -y
Gracefully restart the node
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server restart
Validate
~~~~~~~~
1. Check cluster status with ``nodetool status`` and make sure **all**
nodes, including the one you just upgraded, are in UN status.
2. Use
``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"``
to check scylla version.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in
the log.
4. Check again after 2 minutes, to validate no new issues are
introduced.
Once you are sure the node upgrade is successful, move to the next node
in the cluster.
.. _upgrade-1.2-1.3-rpm-rollback-procedure:
Rollback Procedure
------------------
The following procedure describes a rollback from Scylla release 1.3.x
to 1.2.y. Apply this procedure if an upgrade from 1.2 to 1.3 failed
before completing on all nodes. Use this procedure only for nodes you
upgraded to 1.3
Scylla rollback is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes rollback to 1.2, you will:
- retrieve the old Scylla packages
- drain the node
- restore the configuration file
- restart Scylla
- validate the rollback success
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
Rollback steps
--------------
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the `Scylla RPM repo <http://www.scylladb.com/download/#fndtn-RPM>`_ to
**1.2**
3. Install
.. code:: sh
sudo yum clean all
sudo yum downgrade scylla scylla-server scylla-jmx scylla-tools scylla-conf scylla-kernel-conf -y
Gracefully shutdown Scylla
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server stop
Restore the configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.2 /etc/scylla/scylla.yaml
Start the node
~~~~~~~~~~~~~~
.. code:: sh
sudo service scylla-server start
Validate
~~~~~~~~
Check upgrade instruction above for validation. Once you are sure the
node rollback is successful, move to the next node in the cluster.

View File

@@ -1,187 +0,0 @@
============================================
Upgrade Guide - Scylla 1.2 to 1.3 for Ubuntu
============================================
This document is a step by step procedure for upgrading from Scylla 1.2
to Scylla 1.3, and rollback to 1.2 if required.
Applicable versions
-------------------
This guide covers upgrading Scylla from the following versions: 1.2.x to
Scylla version 1.3.y on the following platform:
- Ubuntu 14.04
- Ubuntu 16.04
Upgrade Procedure
-----------------
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
- backup the data
- check your current release
- backup configuration file
- download and install new Scylla packages
- gracefully restart Scylla
- validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
**During** the rolling upgrade it is highly recommended:
- Not to use new 1.3 features
- Not to run administration functions, like repairs, refresh, rebuild
or add or remove nodes
- Not to apply schema changes
Backup the data
~~~~~~~~~~~~~~~
Before any major procedure, like an upgrade, it is recommended to backup
all the data to an external device. In Scylla, backup is done using the
``nodetool snapshot`` command. For **each** node in the cluster, run the
following command:
.. code:: sh
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all
the directories having this name under ``/var/lib/scylla`` to a backup
device.
Upgrade steps
-------------
Backup configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.2
sudo rm -rf /etc/apt/sources.list.d/scylla.list
Download and install the new release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before upgrading, check what version you are running now using
``dpkg -s scylla-server``. You should use the same version in case you
want to :ref:`rollback <upgrade-1.2-1.3-ubuntu-rollback-procedure>`
the upgrade. If you are not running a 1.2.x version, stop right here!
This guide only covers 1.2.x to 1.3.y upgrades.
To upgrade:
1. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to
**1.3**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Gracefully restart the node
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server restart
Validate
~~~~~~~~
1. Check cluster status with ``nodetool status`` and make sure **all**
nodes, including the one you just upgraded, are in UN status.
2. Use
``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"``
to check scylla version.
3. Check ``/var/log/upstart/scylla-server.log`` and ``/var/log/syslog``
to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are
introduced.
Once you are sure the node upgrade is successful, move to the next node
in the cluster.
.. _upgrade-1.2-1.3-ubuntu-rollback-procedure:
Rollback Procedure
------------------
The following procedure describes a rollback from Scylla release 1.3.x
to 1.2.y. Apply this procedure if an upgrade from 1.2 to 1.3 failed
before completing on all nodes. Use this procedure only for nodes you
upgraded to 1.3
Scylla rollback is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes rollback to 1.2, you will:
- retrieve the old Scylla packages
- drain the node
- restore the configuration file
- restart Scylla
- validate the rollback success
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
Rollback steps
--------------
download and install the old release
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to
**1.2**
3. install
::
sudo apt-get update
sudo apt-get remove scylla scylla-server scylla-jmx scylla-tools scylla-conf scylla-kernel-conf -y
sudo apt-get install scylla
Answer y to the first two questions.
Gracefully shutdown Scylla
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
nodetool drain
sudo service scylla-server stop
Restore the configuration file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.2 /etc/scylla/scylla.yaml
Start the node
~~~~~~~~~~~~~~
.. code:: sh
sudo service scylla-server start
Validate
~~~~~~~~
Check upgrade instruction above for validation. Once you are sure the
node rollback is successful, move to the next node in the cluster.

View File

@@ -1,34 +0,0 @@
=================================
Upgrade Guide - Scylla 1.3 to 1.4
=================================
.. toctree::
:maxdepth: 2
:hidden:
Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.3-to-1.4-rpm>
Ubuntu <upgrade-guide-from-1.3-to-1.4-ubuntu>
.. raw:: html
<div class="panel callout radius animated">
<div class="row">
<div class="medium-3 columns">
<h5 id="getting-started">Upgrade Scylla from 1.3 to 1.4</h5>
</div>
<div class="medium-9 columns">
Upgrade guides are available for:
* :doc:`Upgrade Scylla from 1.3.x to 1.4.y on Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.3-to-1.4-rpm>`
* :doc:`Upgrade Scylla from 1.3.x to 1.4.y on Ubuntu <upgrade-guide-from-1.3-to-1.4-ubuntu>`
.. raw:: html
</div>
</div>
</div>

View File

@@ -1,202 +0,0 @@
======================================================================
Upgrade Guide - Scylla 1.3 to 1.4 for Red Hat Enterprise 7 or CentOS 7
======================================================================
This document is a step by step procedure for upgrading from Scylla 1.3
to Scylla 1.4, and rollback to 1.3 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.3.x to
Scylla version 1.4.y, on the following platforms:
- Red Hat Enterprise Linux, version 7 and later
- CentOS, version 7 and later
- No longer provide packages for Fedora
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
- drain node and backup the data
- check your current release
- backup configuration file
- stop Scylla
- download and install new Scylla packages
- start Scylla
- validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
**During** the rolling upgrade it is highly recommended:
- Not to use new 1.4 features
- Not to run administration functions, like repairs, refresh, rebuild
or add or remove nodes
- Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup
all the data to an external device. In Scylla, backup is done using the
``nodetool snapshot`` command. For **each** node in the cluster, run the
following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all
the directories having this name under ``/var/lib/scylla`` to a backup
device.
When the upgrade is complete (all nodes), the snapshot should be removed
by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of
space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.3
Stop Scylla
-----------
.. code:: sh
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using
``rpm -qa | grep scylla-server``. You should use the same version in
case you want to
:ref:`rollback <upgrade-1.3-1.4-rpm-rollback-procedure>` the
upgrade. If you are not running a 1.3.x version, stop right here! This
guide only covers 1.3.x to 1.4.y upgrades.
To upgrade:
1. Update the `Scylla RPM repo <http://www.scylladb.com/download/centos_rpm>`_ to
**1.4**
2. install
.. code:: sh
sudo yum update scylla\* -y
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all**
nodes, including the one you just upgraded, are in UN status.
2. Use
``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"``
to check scylla version.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in
the log.
4. Check again after 2 minutes, to validate no new issues are
introduced.
Once you are sure the node upgrade is successful, move to the next node
in the cluster.
.. _upgrade-1.3-1.4-rpm-rollback-procedure:
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 1.4.x
to 1.3.y. Apply this procedure if an upgrade from 1.3 to 1.4 failed
before completing on all nodes. Use this procedure only for nodes you
upgraded to 1.4
Scylla rollback is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes rollback to 1.3, you will:
- drain the node and stop Scylla
- retrieve the old Scylla packages
- restore the configuration file
- restart Scylla
- validate the rollback success
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the `Scylla RPM repo <http://www.scylladb.com/download/centos_rpm>`_ to
**1.3**
3. Install
.. code:: sh
sudo yum clean all
sudo yum downgrade scylla\* -y
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.3 /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
Check upgrade instruction above for validation. Once you are sure the
node rollback is successful, move to the next node in the cluster.

View File

@@ -1,201 +0,0 @@
===========================================================
Upgrade Guide - Scylla 1.3 to 1.4 for Ubuntu 14.04 or 16.04
===========================================================
This document is a step by step procedure for upgrading from Scylla 1.3
to Scylla 1.4, and rollback to 1.3 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.3.x to
Scylla version 1.4.y on the following platform:
- Ubuntu 14.04
- Ubuntu 16.04
Upgrade Procedure
=================
A Scylla upgrade is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes in the cluster, you will:
- drain node and backup the data
- check your current release
- backup configuration file
- stop Scylla
- download and install new Scylla packages
- start Scylla
- validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
**During** the rolling upgrade it is highly recommended:
- Not to use new 1.4 features
- Not to run administration functions, like repairs, refresh, rebuild
or add or remove nodes
- Not to apply schema changes
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup
all the data to an external device. In Scylla, backup is done using the
``nodetool snapshot`` command. For **each** node in the cluster, run the
following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all
the directories having this name under ``/var/lib/scylla`` to a backup
device.
When the upgrade is complete (all nodes), the snapshot should be removed
by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of
space.
Upgrade steps
=============
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.3
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using
``dpkg -s scylla-server``. You should use the same version in case you
want to :ref:`rollback <upgrade-1.3-1.4-ubuntu-rollback-procedure>`
the upgrade. If you are not running a 1.3.x version, stop right here!
This guide only covers 1.3.x to 1.4.y upgrades.
To upgrade:
1. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to
**1.4**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all**
nodes, including the one you just upgraded, are in UN status.
2. Use
``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"``
to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log``
for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu
16.04) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are
introduced.
Once you are sure the node upgrade is successful, move to the next node
in the cluster.
.. _upgrade-1.3-1.4-ubuntu-rollback-procedure:
Rollback Procedure
==================
The following procedure describes a rollback from Scylla release 1.4.x
to 1.3.y. Apply this procedure if an upgrade from 1.3 to 1.4 failed
before completing on all nodes. Use this procedure only for nodes you
upgraded to 1.4
Scylla rollback is a rolling procedure which does **not** require full
cluster shutdown. For each of the nodes rollback to 1.3, you will:
- drain the node and stop Scylla
- retrieve the old Scylla packages
- restore the configuration file
- restart Scylla
- validate the rollback success
Apply the following procedure **serially** on each node. Do not move to
the next node before validating the node is up and running with the new
version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to
**1.3**
3. install
::
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.3 /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the
node rollback is successful, move to the next node in the cluster.

View File

@@ -1,32 +0,0 @@
=================================
Upgrade Guide - Scylla 1.4 to 1.5
=================================
.. toctree::
:maxdepth: 2
:hidden:
Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.4-to-1.5-rpm>
Ubuntu <upgrade-guide-from-1.4-to-1.5-ubuntu>
.. raw:: html
<div class="panel callout radius animated">
<div class="row">
<div class="medium-3 columns">
<h5 id="getting-started">Upgrade Scylla from 1.4 to 1.5</h5>
</div>
<div class="medium-9 columns">
Upgrade guides are available for:
* :doc:`Upgrade Scylla from 1.4.x to 1.5.y on Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.4-to-1.5-rpm>`
* :doc:`Upgrade Scylla from 1.4.x to 1.5.y on Ubuntu <upgrade-guide-from-1.4-to-1.5-ubuntu>`
.. raw:: html
</div>
</div>
</div>

View File

@@ -1,157 +0,0 @@
======================================================================
Upgrade Guide - Scylla 1.4 to 1.5 for Red Hat Enterprise 7 or CentOS 7
======================================================================
This document is a step by step procedure for upgrading from Scylla 1.4 to Scylla 1.5, and rollback to 1.4 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.4.x to Scylla version 1.5.y, on the following platforms:
* Red Hat Enterprise Linux, version 7 and later
* CentOS, version 7 and later
* No longer provide packages for Fedora
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes in the cluster, you will:
* drain node and backup the data
* check your current release
* backup configuration file
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 1.5 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.4
Stop Scylla
-----------
.. code:: sh
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``rpm -qa | grep scylla-server``. You should use the same version in case you want to :ref:`rollback <upgrade-1.4-1.5-rpm-rollback-procedure>` the upgrade. If you are not running a 1.4.x version, stop right here! This guide only covers 1.4.x to 1.5.y upgrades.
To upgrade:
1. Update the `Scylla RPM repo <http://www.scylladb.com/download/centos_rpm>`_ to **1.5**
2. install
.. code:: sh
sudo yum update scylla\* -y
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in the log.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
.. _upgrade-1.4-1.5-rpm-rollback-procedure:
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 1.5.x to 1.4.y. Apply this procedure if an upgrade from 1.4 to 1.5 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 1.5
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.4, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the `Scylla RPM repo <http://www.scylladb.com/download/centos_rpm>`_ to **1.4**
3. Install
.. code:: sh
sudo yum clean all
sudo yum downgrade scylla\* -y
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.4 /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,160 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 1.4 to 1.5 for Ubuntu 14.04 or 16.04
=============================================================================
This document is a step by step procedure for upgrading from Scylla 1.4 to Scylla 1.5, and rollback to 1.4 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.4.x to Scylla version 1.5.y on the following platform:
* Ubuntu 14.04
* Ubuntu 16.04
Upgrade Procedure
=================
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
* drain node and backup the data
* check your current release
* backup configuration file
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 1.5 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Upgrade steps
=============
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.4
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to :ref:`rollback <upgrade-1.4-1.5-ubuntu-rollback-procedure>` the upgrade. If you are not running a 1.4.x version, stop right here! This guide only covers 1.4.x to 1.5.y upgrades.
To upgrade:
1. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to **1.5**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
.. _upgrade-1.4-1.5-ubuntu-rollback-procedure:
Rollback Procedure
==================
The following procedure describes a rollback from Scylla release 1.5.x to 1.4.y. Apply this procedure if an upgrade from 1.4 to 1.5 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 1.5
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.4, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to **1.4**
3. install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.4 /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,30 +0,0 @@
=================================
Upgrade Guide - Scylla 1.5 to 1.6
=================================
.. toctree::
:maxdepth: 2
:hidden:
Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.5-to-1.6-rpm>
Ubuntu <upgrade-guide-from-1.5-to-1.6-ubuntu>
.. raw:: html
<div class="panel callout radius animated">
<div class="row">
<div class="medium-3 columns">
<h5 id="getting-started">Upgrade Scylla from 1.5 to 1.6</h5>
</div>
<div class="medium-9 columns">
Upgrade guides are available for:
* :doc:`Upgrade Scylla from 1.5.x to 1.6.y on Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.5-to-1.6-rpm>`
* :doc:`Upgrade Scylla from 1.5.x to 1.6.y on Ubuntu <upgrade-guide-from-1.5-to-1.6-ubuntu>`
.. raw:: html
</div>
</div>
</div>

View File

@@ -1,156 +0,0 @@
======================================================================
Upgrade Guide - Scylla 1.5 to 1.6 for Red Hat Enterprise 7 or CentOS 7
======================================================================
This document is a step by step procedure for upgrading from Scylla 1.5 to Scylla 1.6, and rollback to 1.5 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.5.x to Scylla version 1.6.y, on the following platforms:
* Red Hat Enterprise Linux, version 7 and later
* CentOS, version 7 and later
* No longer provide packages for Fedora
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
* drain node and backup the data
* check your current release
* backup configuration file
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 1.6 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.5
Stop Scylla
-----------
.. code:: sh
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``rpm -qa | grep scylla-server``. You should use the same version in case you want to :ref:`rollback <upgrade-1.5-1.6-rpm-rollback-procedure>` the upgrade. If you are not running a 1.5.x version, stop right here! This guide only covers 1.5.x to 1.6.y upgrades.
To upgrade:
1. Update the `Scylla RPM repo <http://www.scylladb.com/download/centos_rpm>`_ to **1.6**
2. install
.. code:: sh
sudo yum update scylla\* -y
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in the log.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
.. _upgrade-1.5-1.6-rpm-rollback-procedure:
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 1.6.x to 1.5.y. Apply this procedure if an upgrade from 1.5 to 1.6 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 1.6
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.5, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the `Scylla RPM repo <http://www.scylladb.com/download/centos_rpm>`_ to **1.5**
3. Install
.. code:: sh
sudo yum clean all
sudo yum downgrade scylla\* -y
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.5 /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,160 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 1.5 to 1.6 for Ubuntu 14.04 or 16.04
=============================================================================
This document is a step by step procedure for upgrading from Scylla 1.5 to Scylla 1.6, and rollback to 1.5 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.5.x to Scylla version 1.6.y on the following platform:
* Ubuntu 14.04
* Ubuntu 16.04
Upgrade Procedure
=================
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
* drain node and backup the data
* check your current release
* backup configuration file
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 1.6 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Upgrade steps
=============
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.5
Gracefully stop the node
------------------------
.. code:: sh
sudo service scylla-server stop
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``dpkg -s scylla-server``. You should use the same version in case you want to :ref:`rollback <upgrade-1.5-1.6-ubuntu-rollback-procedure>` the upgrade. If you are not running a 1.5.x version, stop right here! This guide only covers 1.5.x to 1.6.y upgrades.
To upgrade:
1. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to **1.6**
2. Install
.. code:: sh
sudo apt-get update
sudo apt-get dist-upgrade scylla
Answer y to the first two questions.
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Check scylla-server log (check ``/var/log/upstart/scylla-server.log`` for Ubuntu 14.04, execute ``journalctl _COMM=scylla`` for Ubuntu 16.04) and ``/var/log/syslog`` to validate there are no errors.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
.. _upgrade-1.5-1.6-ubuntu-rollback-procedure:
Rollback Procedure
==================
The following procedure describes a rollback from Scylla release 1.6.x to 1.5.y. Apply this procedure if an upgrade from 1.5 to 1.6 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 1.6
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.5, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo service scylla-server stop
download and install the old release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/apt/sources.list.d/scylla.list
2. Update the `Scylla deb repo <http://www.scylladb.com/download/#fndtn-deb>`_ to **1.5**
3. install
.. code:: sh
sudo apt-get update
sudo apt-get remove scylla\* -y
sudo apt-get install scylla
Answer y to the first two questions.
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.5 /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo service scylla-server start
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,37 +0,0 @@
=================================
Upgrade Guide - Scylla 1.6 to 1.7
=================================
.. toctree::
:maxdepth: 2
:hidden:
Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.6-to-1.7-rpm>
Ubuntu <upgrade-guide-from-1.6-to-1.7-ubuntu>
Debian <upgrade-guide-from-1.6-to-1.7-debian>
.. raw:: html
<div class="panel callout radius animated">
<div class="row">
<div class="medium-3 columns">
<h5 id="getting-started">Upgrade Scylla from 1.6 to 1.7</h5>
</div>
<div class="medium-9 columns">
Upgrade guides are available for:
* :doc:`Upgrade Scylla from 1.6.x to 1.7.y on Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.6-to-1.7-rpm>`
* :doc:`Upgrade Scylla from 1.6.x to 1.7.y on Ubuntu <upgrade-guide-from-1.6-to-1.7-ubuntu>`
* :doc:`Upgrade Scylla from 1.6.x to 1.7.y on Debian <upgrade-guide-from-1.6-to-1.7-debian>`
.. raw:: html
</div>
</div>
</div>

View File

@@ -1,8 +0,0 @@
.. |OS| replace:: Debian 8
.. |ROLLBACK| replace:: rollback
.. _ROLLBACK: /upgrade/upgrade-opensource/upgrade-guide-from-1.6-to-1.7/upgrade-guide-from-1.6-to-1.7-debian/#upgrade-1-6-1-7-rollback-procedure
.. |APT| replace:: Scylla deb repo
.. _APT: http://www.scylladb.com/download/#fndtn-Debian
.. |ENABLE_APT_REPO| replace:: echo 'deb http://http.debian.net/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list
.. |JESSIE_BACKPORTS| replace:: -t jessie-backports openjdk-8-jre-headless
.. include:: /upgrade/_common/upgrade-guide-from-1.6-to-1.7-ubuntu-and-debian.rst

View File

@@ -1,157 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 1.6 to 1.7 for Red Hat Enterprise 7 or CentOS 7
=============================================================================
This document is a step by step procedure for upgrading from Scylla 1.6 to Scylla 1.7, and rollback to 1.6 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.6.x to Scylla version 1.7.y, on the following platforms:
* Red Hat Enterprise Linux, version 7 and later
* CentOS, version 7 and later
* No longer provide packages for Fedora
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
* drain node and backup the data
* check your current release
* backup configuration file
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 1.7 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.6
Stop Scylla
-----------
.. code:: sh
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``rpm -qa | grep scylla-server``. You should use the same version in case you want to :ref:`rollback <upgrade-1.6-1.7-rpm-rollback-procedure>` the upgrade. If you are not running a 1.6.x version, stop right here! This guide only covers 1.6.x to 1.7.y upgrades.
To upgrade:
1. Update the `Scylla RPM repo <http://www.scylladb.com/download/centos_rpm>`_ to **1.7**
2. install
.. code:: sh
sudo yum update scylla\* -y
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in the log.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
.. _upgrade-1.6-1.7-rpm-rollback-procedure:
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 1.7.x to 1.6.y. Apply this procedure if an upgrade from 1.6 to 1.7 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 1.7
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.6, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the `Scylla RPM repo <http://www.scylladb.com/download/centos_rpm>`_ to **1.6**
3. Install
.. code:: sh
sudo yum clean all
sudo yum downgrade scylla\* -y
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.6 /etc/scylla/scylla.yaml
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,8 +0,0 @@
.. |OS| replace:: Ubuntu 14.04 or 16.04
.. |ROLLBACK| replace:: rollback
.. _ROLLBACK: /upgrade/upgrade-opensource/upgrade-guide-from-1.6-to-1.7/upgrade-guide-from-1.6-to-1.7-ubuntu/#upgrade-1-6-1-7-rollback-procedure
.. |APT| replace:: Scylla deb repo
.. _APT: http://www.scylladb.com/download/#fndtn-deb
.. |ENABLE_APT_REPO| replace:: sudo add-apt-repository -y ppa:openjdk-r/ppa
.. |JESSIE_BACKPORTS| replace:: openjdk-8-jre-headless
.. include:: /upgrade/_common/upgrade-guide-from-1.6-to-1.7-ubuntu-and-debian.rst

View File

@@ -1,43 +0,0 @@
=================================
Upgrade Guide - Scylla 1.7 to 2.0
=================================
.. toctree::
:maxdepth: 2
:hidden:
Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.7-to-2.0-rpm>
Ubuntu <upgrade-guide-from-1.7-to-2.0-ubuntu>
Debian <upgrade-guide-from-1.7-to-2.0-debian>
Metrics <metric-update-1.7-to-2.0>
.. raw:: html
<div class="panel callout radius animated">
<div class="row">
<div class="medium-3 columns">
<h5 id="getting-started">Upgrade Scylla from 1.7 to 2.0</h5>
</div>
<div class="medium-9 columns">
Upgrade guides are available for:
* :doc:`Upgrade Scylla from 1.7.x to 2.0.y on Red Hat Enterprise Linux and CentOS <upgrade-guide-from-1.7-to-2.0-rpm>`
* :doc:`Upgrade Scylla from 1.7.x to 2.0.y on Ubuntu <upgrade-guide-from-1.7-to-2.0-ubuntu>`
* :doc:`Upgrade Scylla from 1.7.x to 2.0.y on Debian <upgrade-guide-from-1.7-to-2.0-debian>`
* :doc:`Scylla Metrics Update - Scylla 1.7 to 2.0<metric-update-1.7-to-2.0>`
.. raw:: html
</div>
</div>
</div>

View File

@@ -1,66 +0,0 @@
========================================
Scylla Metric Update - Scylla 1.7 to 2.0
========================================
.. toctree::
:maxdepth: 2
:hidden:
The following metric names have changed
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* scylla_cache_evictions To scylla_cache_partition_evictions
* scylla_cache_hits To scylla_cache_partition_hits
* scylla_cache_insertions To scylla_cache_partition_insertions
* scylla_cache_merges To scylla_cache_partition_merges
* scylla_cache_misses To scylla_cache_partition_misses
* scylla_cache_removals To scylla_cache_partition_removals
* scylla_cache_total To scylla_cache_bytes_total
* scylla_cache_used To scylla_cache_bytes_used
The following metrics are no longer available
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* scylla_cache_uncached_wide_partitions
* scylla_cache_wide_partition_evictions
* scylla_cache_wide_partition_mispopulations
The following metrics are new in Scylla 2.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* scylla_cache_mispopulations
* scylla_cache_reads
* scylla_cache_active_reads
* scylla_cache_reads_with_misses
* scylla_cache_row_hits
* scylla_cache_row_insertions
* scylla_cache_row_misses
* scylla_cache_sstable_partition_skips
* scylla_cache_sstable_reader_recreations
* scylla_cache_sstable_row_skips
* scylla_column_family_live_disk_space
* scylla_column_family_live_sstable
* scylla_column_family_memtable_switch
* scylla_column_family_pending_compaction
* scylla_column_family_pending_tasks
* scylla_column_family_total_disk_space
* scylla_database_active_reads_streaming
* scylla_database_counter_cell_lock_acquisition
* scylla_database_counter_cell_lock_pending
* scylla_database_cpu_flush_quota
* scylla_database_queued_reads_streaming
* scylla_execution_stages_function_calls_enqueued
* scylla_execution_stages_function_calls_executed
* scylla_execution_stages_tasks_preempted
* scylla_execution_stages_tasks_scheduled
* scylla_scylladb_current_version
* scylla_sstables_index_page_blocks
* scylla_sstables_index_page_hits
* scylla_sstables_index_page_misses
* scylla_storage_proxy_coordinator_background_read_repairs
* scylla_storage_proxy_coordinator_foreground_read_repair
* scylla_storage_proxy_coordinator_read_latency
* scylla_storage_proxy_coordinator_write_latency
* scylla_storage_proxy_replica_received_counter_updates

View File

@@ -1,8 +0,0 @@
.. |OS| replace:: Debian 8
.. |ROLLBACK| replace:: rollback
.. _ROLLBACK: /upgrade/upgrade-opensource/upgrade-guide-from-1.7-to-2.0/upgrade-guide-from-1.7-to-2.0-debian/#rollback-procedure
.. |APT| replace:: Scylla deb repo
.. _APT: http://www.scylladb.com/download/debian8/
.. |ENABLE_APT_REPO| replace:: echo 'deb http://http.debian.net/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list
.. |JESSIE_BACKPORTS| replace:: -t jessie-backports openjdk-8-jre-headless
.. include:: /upgrade/_common/upgrade-guide-from-1.7-to-2.0-ubuntu-and-debian.rst

View File

@@ -1,178 +0,0 @@
=============================================================================
Upgrade Guide - Scylla 1.7 to 2.0 for Red Hat Enterprise Linux 7 or CentOS 7
=============================================================================
This document is a step by step procedure for upgrading from Scylla 1.7 to Scylla 2.0, and rollback to 1.7 if required.
Applicable versions
===================
This guide covers upgrading Scylla from the following versions: 1.7.x (x >= 4) to Scylla version 2.0.y, on the following platforms:
* Red Hat Enterprise Linux, version 7 and later
* CentOS, version 7 and later
* No longer provide packages for Fedora
Upgrade Procedure
=================
.. include:: /upgrade/_common/warning.rst
A Scylla upgrade is a rolling procedure which does not require full cluster shutdown. For each of the nodes in the cluster, serially (i.e. one at a time), you will:
* check cluster schema
* drain node and backup the data
* backup configuration file
* stop Scylla
* download and install new Scylla packages
* start Scylla
* validate that the upgrade was successful
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
**During** the rolling upgrade it is highly recommended:
* Not to use new 2.0 features
* Not to run administration functions, like repairs, refresh, rebuild or add or remove nodes
* Not to apply schema changes
Upgrade steps
=============
Check cluster schema
--------------------
Make sure that all nodes have the schema synched prior to upgrade, we won't survive an upgrade that has schema disagreement between nodes.
.. code:: sh
nodetool describecluster
Drain node and backup the data
------------------------------
Before any major procedure, like an upgrade, it is recommended to backup all the data to an external device. In Scylla, backup is done using the ``nodetool snapshot`` command. For **each** node in the cluster, run the following command:
.. code:: sh
nodetool drain
nodetool snapshot
Take note of the directory name that nodetool gives you, and copy all the directories having this name under ``/var/lib/scylla`` to a backup device.
When the upgrade is complete (all nodes), the snapshot should be removed by ``nodetool clearsnapshot -t <snapshot>``, or you risk running out of space.
Backup configuration file
-------------------------
.. code:: sh
sudo cp -a /etc/scylla/scylla.yaml /etc/scylla/scylla.yaml.backup-1.7
Stop Scylla
-----------
.. code:: sh
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
Before upgrading, check what version you are running now using ``rpm -qa | grep scylla-server``. You should use the same version in case you want to :ref:`rollback <upgrade-1.7-2.0-rpm-rollback-procedure>` the upgrade. If you are not running a 1.7.x version, stop right here! This guide only covers 1.7.x to 2.0.y upgrades.
To upgrade:
1. Update the `Scylla RPM repo <http://www.scylladb.com/download/?platform=centos>`_ to **2.0**
2. install
.. code:: sh
sudo yum update scylla\* -y
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
1. Check cluster status with ``nodetool status`` and make sure **all** nodes, including the one you just upgraded, are in UN status.
2. Use ``curl -X GET "http://localhost:10000/storage_service/scylla_release_version"`` to check scylla version.
3. Use ``journalctl _COMM=scylla`` to check there are no new errors in the log.
4. Check again after 2 minutes, to validate no new issues are introduced.
Once you are sure the node upgrade is successful, move to the next node in the cluster.
* More on :doc:`Scylla Metrics Update - Scylla 1.7 to 2.0<metric-update-1.7-to-2.0>`
.. _upgrade-1.7-2.0-rpm-rollback-procedure:
Rollback Procedure
==================
.. include:: /upgrade/_common/warning_rollback.rst
The following procedure describes a rollback from Scylla release 2.0.x to 1.7.y (y >= 4). Apply this procedure if an upgrade from 1.7 to 2.0 failed before completing on all nodes. Use this procedure only for nodes you upgraded to 2.0
Scylla rollback is a rolling procedure which does **not** require full cluster shutdown.
For each of the nodes rollback to 1.7, you will:
* drain the node and stop Scylla
* retrieve the old Scylla packages
* restore the configuration file
* restart Scylla
* validate the rollback success
Apply the following procedure **serially** on each node. Do not move to the next node before validating the node is up and running with the new version.
Rollback steps
==============
Gracefully shutdown Scylla
--------------------------
.. code:: sh
nodetool drain
sudo systemctl stop scylla-server
Download and install the new release
------------------------------------
1. Remove the old repo file.
.. code:: sh
sudo rm -rf /etc/yum.repos.d/scylla.repo
2. Update the `Scylla RPM repo <http://www.scylladb.com/download/?platform=centos>`_ to **1.7**
3. Install
.. code:: sh
sudo yum clean all
sudo yum remove scylla-tools-core
sudo yum downgrade scylla\* -y
sudo yum install scylla
Restore the configuration file
------------------------------
.. code:: sh
sudo rm -rf /etc/scylla/scylla.yaml
sudo cp -a /etc/scylla/scylla.yaml.backup-1.7 /etc/scylla/scylla.yaml
Restore system tables
---------------------
Restore all tables of **system** and **system_schema** from previous snapshot, 2.0 uses a different set of system tables. Reference doc: :doc:`Restore from a Backup and Incremental Backup </operating-scylla/procedures/backup-restore/restore/>`
.. code:: sh
cd /var/lib/scylla/data/keyspace_name/table_name-UUID/snapshots/<snapshot_name>/
sudo cp -r * /var/lib/scylla/data/keyspace_name/table_name-UUID/
Start the node
--------------
.. code:: sh
sudo systemctl start scylla-server
Validate
--------
Check upgrade instruction above for validation. Once you are sure the node rollback is successful, move to the next node in the cluster.

View File

@@ -1,8 +0,0 @@
.. |OS| replace:: Ubuntu 14.04 or 16.04
.. |ROLLBACK| replace:: rollback
.. _ROLLBACK: /upgrade/upgrade-opensource/upgrade-guide-from-1.7-to-2.0/upgrade-guide-from-1.7-to-2.0-ubuntu/#rollback-procedure
.. |APT| replace:: Scylla deb repo
.. _APT: http://www.scylladb.com/download/
.. |ENABLE_APT_REPO| replace:: sudo add-apt-repository -y ppa:openjdk-r/ppa
.. |JESSIE_BACKPORTS| replace:: openjdk-8-jre-headless
.. include:: /upgrade/_common/upgrade-guide-from-1.7-to-2.0-ubuntu-and-debian.rst

Some files were not shown because too many files have changed in this diff Show More