Commit Graph

16 Commits

Author SHA1 Message Date
Avi Kivity
fcb8d040e8 treewide: use Software Package Data Exchange (SPDX) license identifiers
Instead of lengthy blurbs, switch to single-line, machine-readable
standardized (https://spdx.dev) license identifiers. The Linux kernel
switched long ago, so there is strong precedent.

Three cases are handled: AGPL-only, Apache-only, and dual licensed.
For the latter case, I chose (AGPL-3.0-or-later and Apache-2.0),
reasoning that our changes are extensive enough to apply our license.

The changes we applied mechanically with a script, except to
licenses/README.md.

Closes #9937
2022-01-18 12:15:18 +01:00
Avi Kivity
a55b434a2b treewide: extent copyright statements to present day 2021-06-06 19:18:49 +03:00
Kamil Braun
e486e0f759 tree-wide: rename "cdc streams timestamp" to "cdc generation id"
Each CDC generation always has a timestamp, but the fact that the
timestamp identifies the generation is an implementation detail.
We abstract away from this detail by using a more generic naming scheme:
a generation "identifier" (whatever that is - a timestamp or something
else).

It's possible that a CDC generation will be identified by more than a
timestamp in the (near) future.

The actual string gossiped by nodes in their application state is left
as "CDC_STREAMS_TIMESTAMP" for backward compatibility.

Some stale comments have been updated.
2021-04-06 13:15:31 +02:00
Juliusz Stasiewicz
b150906d39 gossip: Added SNITCH_NAME to application_state
Snitch name needs to be exchanged within cluster once, on shadow
round, so joining nodes cannot use wrong snitch. The snitch names
are compared on bootstrap and on normal node start.

If the cluster already used mixed snitches, the upgrade to this
version will fail. In this case customer needs to add a node with
correct snitch for every node with the wrong snitch, then put
down the nodes with the wrong snitch and only then do the upgrade.

Fixes #6832

Closes #7739
2020-12-09 15:45:25 +02:00
Asias He
7ac9e0f2a1 gossip: Print CDC_STREAMS_TIMESTAMP correctly
I saw UNKNOWN application state in the logs:

INFO  2020-03-06 11:09:48,931 [shard 0] storage_service - Update
system.peers table: endpoint=127.0.0.2, app_state=CACHE_HITRATES, versioned_value=Value(,14)
INFO  2020-03-06 11:09:48,931 [shard 0] storage_service - Update
system.peers table: endpoint=127.0.0.2, app_state=SCHEMA_TABLES_VERSION, versioned_value=Value(3,15)
INFO  2020-03-06 11:09:48,931 [shard 0] storage_service - Update
system.peers table: endpoint=127.0.0.2, app_state=RPC_READY, versioned_value=Value(0,16)
INFO  2020-03-06 11:09:48,931 [shard 0] storage_service - Update
system.peers table: endpoint=127.0.0.2, app_state=VIEW_BACKLOG, versioned_value=Value(,17)
INFO  2020-03-06 11:09:48,931 [shard 0] storage_service - Update
system.peers table: endpoint=127.0.0.2, app_state=SHARD_COUNT, versioned_value=Value(1,30)
INFO  2020-03-06 11:09:48,931 [shard 0] storage_service - Update
system.peers table: endpoint=127.0.0.2, app_state=IGNOR_MSB_BITS, versioned_value=Value(12,31)
INFO  2020-03-06 11:09:48,931 [shard 0] storage_service - Update
system.peers table: endpoint=127.0.0.2, app_state=UNKNOWN, versioned_value=Value(1583371936128,20)

It turned out it was CDC_STREAMS_TIMESTAMP.

$ nodetool gossipinfo|grep 1583371936128
  X8:1583371936128
  X8:1583371936128

Fixes #5992
2020-03-15 11:51:35 +01:00
Piotr Jastrzebski
afe520ad77 gossip: Add application_state::IGNORE_MSB_BITS
We would like to share with other nodes
the value of ignore_msb_bits property used by the node.

This is needed because CDC will operate on
streams of changes. Each shard on each node
will have its own stream that will be identified
by a stream_id. Stream_id will be selected in
such a way that using stream_id as partition key
will locate partition identified by stream_id on
a node and shard that the stream belongs to.

To be able to generate such stream_id we need
to know ignore_msb_bits property value for each node.

IMPORTANT NOTE: At this point CDC does not support
topology changes. It will work only on a stable cluster.
Support for topology modifications will be added in
later steps.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2019-10-17 10:55:31 +02:00
Piotr Jastrzebski
a66d7cfe57 gossip: Add application_state::SHARD_COUNT
We would like to share with other nodes
the number of shards available at the node.

This is needed because CDC will operate on
streams of changes. Each shard on each node
will have its own stream that will be identified
by a stream_id. Stream_id will be selected in
such a way that using stream_id as partition key
will locate partition identified by stream_id on
a node and shard that the stream belongs to.

To be able to generate such stream_id we need
to know how many shards are on each node.

IMPORTANT NOTE: At this point CDC does not support
topology changes. It will work only on a stable cluster.
Support for topology modifications will be added in
later steps.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2019-10-17 10:55:31 +02:00
Duarte Nunes
8da6a31e75 service: Advertise view update backlog over gossip
This lays the groundwork for brokering a node's view update
backlog across the whole cluster. This is needed for when a
coordinator does not contact a given replica for a long time, and uses
a backlog view that is outdated and causes requests to be
unnecessarily delayed.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2018-12-19 22:38:30 +00:00
Asias He
67093324ad gms: Add application_state::RPC_READY
It is used to tell peer nodes that the cql server is ready and can
accept clients request.

Follow the same name which Cassandra uses.
2018-12-10 19:16:44 +08:00
Asias He
98e9049820 gossip: Print SCHEMA_TABLES_VERSION correctly
Found this when debugging gossip with debug print. The application state
SCHEMA_TABLES_VERSION was printed as UNKNOWN.
Message-Id: <d7616920d2e6516b5470a758bcf9c88f3d857381.1506391495.git.asias@scylladb.com>
2017-09-26 08:38:28 +02:00
Gleb Natapov
8ca1432b04 Distribute cache temperature over gossiper.
When a node start it does not have any information about cache temperature
of other nodes in the cluster and it is hard (if not impossible) to make
right guess. During cluster startup all nodes have cold caches, so there
is no point to redirect reads to other nodes even though local cache it
cold, but if only that node restarted than other nodes have populated
cache and reads should be redirected.

The node will get up-to-date information about other nodes caches,
but only after receiving first reply, until then it does not have the
information to make right decisions which may cause unwanted spikes
immediately after restart. Having cache temperature in gossiper helps
to solve the problem.
2017-06-13 09:57:14 +03:00
Avi Kivity
ebaeefa02b Merge seatar upstream (seastar namespace)
- introcduced "seastarx.hh" header, which does a "using namespace seastar";
 - 'net' namespace conflicts with seastar::net, renamed to 'netw'.
 - 'transport' namespace conflicts with seastar::transport, renamed to
   cql_transport.
 - "logger" global variables now conflict with logger global type, renamed
   to xlogger.
 - other minor changes
2017-05-21 12:26:15 +03:00
Asias He
a6080773b3 gossip: Add SUPPORTED_FEATURES application_state
It is used to negotiate cluster wide features.
2016-04-06 07:12:34 +08:00
Asias He
7acc9816d2 gossip: Handle unknown application_state when printing
In case an unknown application_state is received, we should be able to
handle it when printting.

Message-Id: <98d2307359292e90c8925f38f67a74b69e45bebe.1458553057.git.asias@scylladb.com>
2016-03-21 11:59:04 +02:00
Asias He
cc3073b42d gossip: cleanup application_state
Drop the unused one.

Message-Id: <4cc45164d55742951b618d2c7b1e8bdb997f005a.1452771260.git.asias@scylladb.com>
2016-01-14 19:01:51 +02:00
Asias He
687d88dd0c gossip: Add operator<< operator for application_state 2015-10-23 16:32:36 +08:00