Commit Graph

100 Commits

Author SHA1 Message Date
Benny Halevy
27e0aee358 docs/debugging.md: fix anchor links
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Message-Id: <20191229074136.13516-1-bhalevy@scylladb.com>
2019-12-29 16:26:26 +02:00
Avi Kivity
a18a921308 docs: maintainer.md: use command line to merge multi-commit pull requests
If you merge a pull request that contains multiple patches via
the github interface, it will document itself as the committer.

Work around this brain damage by using the command line.
2019-12-06 10:59:46 +01:00
Nadav Har'El
5b2f35a21a Merge "Redis: fix the options related to Redis API, fix the DEL and GET command"
Merged pull request https://github.com/scylladb/scylla/pull/5381 by
Peng Jian, fixing multiple small issues with Redis:

* Rename the options related to Redis API, and describe them clearly.
* Rename redis_transport_port to redis_port
* Rename redis_transport_port_ssl to redis_ssl_port
* Rename redis_default_database_count to redis_database_count
* Remove unnecessary option enable_redis_protocol
* Modify the default value of opition redis_read_consistency_level and redis_write_consistency_level to LOCAL_QUORUM

* Fix the DEL command: support to delete mutilple keys in one command.

* Fix the GET command: return the empty string when the required key is not exists.

* Fix the redis-test/test_del_non_existent_key: mark xfail.
2019-12-05 11:58:34 +02:00
Avi Kivity
91d3f2afce docs: maintainers.md: fix typo in git push --force-with-lease
Just one lease, not many.

Reported by Piotr Sarna.
2019-12-03 18:17:46 +01:00
fastio
039b83ad3b Redis API: Rename options related to Redis API, describe them clearly, and remove unnecessary one.
Rename option redis_transport_port to redis_port, which the redis transport listens on for clients.
Rename option redis_transport_port_ssl to redis_ssl_port, which the redis TLS transport listens on for clients.
Rename option redis_database_count. Set the redis dabase count.
Rename option redis_keyspace_opitons to redis_keyspace_replication_strategy_options. Set the replication strategy for redis keyspace.
Remove option enable_redis_protocol, which is unnecessary.

Fixes: #5335

Signed-off-by: Peng Jian <pengjian.uestc@gmail.com>
2019-12-03 17:13:35 +08:00
Avi Kivity
28355af134 docs: add maintainer's handbook (#5396)
This is a list of recipes used by maintainers to maintain
scylla.git.
2019-12-02 15:01:54 +02:00
Botond Dénes
cf24f4fe30 imr: move documentation to docs/
Where all the other documentation is, and hence where people would be
looking for it.

Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20191128144612.378244-1-bdenes@scylladb.com>
2019-11-28 16:47:52 +02:00
Peng Jian
708a42c284 Document: add docs/redis/redis.md
In this document, the detailed design and implementation of Redis API in
Scylla is provided.

Signed-off-by: Peng Jian <pengjian.uestc@gmail.com>
2019-11-20 04:46:33 +08:00
Avi Kivity
1fe062aed4 Merge "Add basic UDF support" from Rafael
"

This patch series adds only UDF support, UDA will be in the next patch series.

With this all CQL types are mapped to Lua. Right now we setup a new
lua state and copy the values for each argument and return. This will
be optimized once profiled.

We require --experimental to enable UDF in case there is some change
to the table format.
"

* 'espindola/udf-only-v4' of https://github.com/espindola/scylla: (65 commits)
  Lua: Document the conversions between Lua and CQL
  Lua: Implement decimal subtraction
  Lua: Implement decimal addition
  Lua: Implement support for returning decimal
  Lua: Implement decimal to string conversion
  Lua: Implement decimal to floating point conversion
  Lua: Implement support for decimal arguments
  Lua: Implement support for returning varint
  Lua: Implement support for returning duration
  Lua: Implement support for duration arguments
  Lua: Implement support for returning inet
  Lua: Implement support for inet arguments
  Lua: Implement support for returning time
  Lua: Implement support for time arguments
  Lua: Implement support for returning timeuuid
  Lua: Implement support for returning uuid
  Lua: Implement support for uuid and timeuuid arguments
  Lua: Implement support for returning date
  Lua: Implement support for date arguments
  Lua: Implement support for returning timestamp
  ...
2019-11-17 16:38:19 +02:00
Shlomi Livne
3e873812a4 Document backport queue and procedure (#5282)
This document adds information about how fixes are tracked to be
backported into releases and what is the procedure that is followed to
backport those fixes.

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
2019-11-17 01:45:24 -08:00
Rafael Ávila de Espíndola
10bcbaf348 Lua: Document the conversions between Lua and CQL
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-11-07 08:41:08 -08:00
Benny Halevy
76581e7f14 docs/debugging.md: fix gdb command for retrieving shared libraries information
This correct command is `info sharedlibrary`.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Message-Id: <20191027153541.27286-1-bhalevy@scylladb.com>
2019-10-27 18:15:09 +02:00
Piotr Sarna
280eb28324 docs: update alternator entry for authorization
The document now mentions that secret keys are extracted
from the system_auth.roles table.
2019-10-23 15:05:39 +02:00
Nadav Har'El
774f8aa4b8 docs/debugging.md: add guide on how to debug cores
Merged patch series from Botond Dénes:

This series extends the existing docs/debugging.md with a detailed guide
on how to debug Scylla coredumps. The intended target audience is
developers who are debugging their first core, hence the level of
details (hopefully enough). That said this should be just as useful for
seasoned debuggers just quickly looking up some snippet they can't
remember exactly. A Throubleshooting chapter is also added in this
series for commonly-met problems.

I decided to create this guide after myself having struggled for more
than a day on just opening(!) a coredump that was produced on Ubuntu.
As my main source, I used the How-to-debug-a-coredump page from the
internal wiki which contains many useful information on debugging
coredumps, however I found it to be missing some crucial information, as
well being very terse, thus being primarily useful for experienced
debuggers who can fill in the blanks. The reason I'm not extending said
wiki page is that I think this information should not be hidden in some
internal wiki page. Also, docs/debugging.md now seems to be a much
better base for such a document. This document was started as a
comprehensive debugging manual for beginners (but not just).

You will notice that the information on how to debug cores from
CentOS/Redhat are quite sparse. This is because I have no experience
with such cores, so for now the respective chapters are just stubs. I
intend to complete them in the future after having gained the necessary
experience and knowledge, however those being in possession of said
knowledge are more then welcome to send a patch. :)

Botond Dénes (4):
	docs/debugging.md: demote 'Starting GDB' and 'Using GDB'
	docs/debugging.md: fix formatting issues
	docs/debugging.md: add 'Debugging coredumps' subchapter
	docs/debugging.md: add 'Throubleshooting' subchapter

  docs/debugging.md | 240 +++++++++++++++++++++++++++++++++++++++++++---
  1 file changed, 228 insertions(+), 12 deletions(-)
2019-10-23 07:39:57 +03:00
Botond Dénes
4dff50b7a4 docs/debugging.md: add 'Throubleshooting' subchapter
To the 'Debugging Scylla with GDB' chapter.
2019-10-18 10:08:23 +03:00
Botond Dénes
77ea086975 docs/debugging.md: add 'Debugging coredumps' subchapter
To the 'Debuggin Scylla with GDB` chapter. The '### Debugging
relocatable binaries built with the toolchain' subchapter is demoted to
be just a section in this new subchapter. It is also renamed to
'Relocatable binaries'.
This subchapter intends to be a complete guide on how to debug coredumps
from how to obtain the correct version of all the binaries all the way
to how to correctly open the core with GDB.
2019-10-18 10:08:23 +03:00
Botond Dénes
71923577a4 docs/debugging.md: fix formatting issues 2019-10-15 14:40:24 +03:00
Botond Dénes
4babd116d8 docs/debugging.md: demote 'Starting GDB' and 'Using GDB'
They really belong to the 'Introduction' chapter, instead of being
separate chapters of their own.
2019-10-15 14:40:20 +03:00
Pekka Enberg
0c1dad0838 Merge "Misc documentation cleanup" from Botond
"Delete README-DPDK.md, move IDL.md to docs/ and fix
docs/review-checklist.md to point to scylla's coding style document,
instead of seastar's."

* 'documentation-cleanup/v3' of https://github.com/denesb/scylla:
  docs/review-checklist.md: point to scylla's coding-style.md instead of seastar's
  docs: mv coding-style.md docs/
  rm README-DPDK.md
  docs: mv IDL.md docs/
2019-10-15 12:53:49 +02:00
Nadav Har'El
33027a36b4 alternator: Add authorization
Merged patch set from Piotr Sarna:

Refs #5046

This commit adds handling "Authorization:" header in incoming requests.
The signature sent in the authorization is recomputed server-side
and compared with what the client sent. In case of a mismatch,
UnrecognizedClientException is returned.
The signature computation is based on boto3 Python implementation
and uses gnutls to compute HMAC hashes.

This series is rebased on a previous HTTPS series in order to ease
merging these two. As such, it depends on the HTTPS series being
merged first.

Tests: alternator(local, remote)

The series also comes with a simple authorization test and a docs update.

Piotr Sarna (6):
  alternator: migrate split() function to string_view
  alternator: add computing the auth signature
  config: add alternator_enforce_authorization entry
  alternator: add verifying the auth signature
  alternator-test: add a basic authorization test case
  docs: update alternator authorization entry

 alternator-test/test_authorization.py |  34 ++++++++
 configure.py                          |   1 +
 alternator/{server.hh => auth.hh}     |  22 ++---
 alternator/server.hh                  |   3 +-
 db/config.hh                          |   1 +
 alternator/auth.cc                    |  88 ++++++++++++++++++++
 alternator/server.cc                  | 112 +++++++++++++++++++++++---
 db/config.cc                          |   1 +
 main.cc                               |   2 +-
 docs/alternator/alternator.md         |   7 +-
 10 files changed, 241 insertions(+), 30 deletions(-)
 create mode 100644 alternator-test/test_authorization.py
 copy alternator/{server.hh => auth.hh} (58%)
 create mode 100644 alternator/auth.cc
2019-10-10 15:57:46 +03:00
Nadav Har'El
df62499710 docs/isolation.md: copy-edit
Minor spelling and syntax corrections. No new content or semantic changes.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20191010093457.20439-1-nyh@scylladb.com>
2019-10-10 15:17:28 +03:00
Piotr Sarna
6ca55d3c83 docs: update alternator authorization entry
The entry now contains a comment that computing a signature works,
but is still based on a hardcoded key.
2019-10-10 13:51:00 +02:00
Botond Dénes
e52712f82c docs: add README.md
Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20191010103926.34705-2-bdenes@scylladb.com>
2019-10-10 14:14:09 +03:00
Piotr Sarna
07ac3ea632 docs: update alternator entry on HTTPS
The HTTPS entry is updated - it's now supported, but still
misses the same features as HTTP - CRC headers, etc.
2019-10-03 19:10:30 +02:00
Nadav Har'El
c9aae13fae docs/alternator/getting-started.md: fix indentation in example code
The example Python code had wrong indentation, and wouldn't actually
work if naively copy-pasted. Noticed by Noam Hasson.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190929091440.28042-1-nyh@scylladb.com>
2019-09-30 13:03:29 +03:00
Botond Dénes
425cc0c104 doc: add debugging.md
A documentation file that is intended to be a place for anything
debugging related: getting started tutorial, tips and tricks and
advanced guides.
For now it contains a short introductions, some selected links to
more in-depth documentation and some trips and tricks that I could think
off the top of my head.
One of those tricks describes how to load cores obtained from
relocatable packages inside the `dbuild` container. I originally
intended to add that to `tools/toolchain/README.md` but was convinced
that `docs/debugging.md` would be a better place for this.

Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20190924133110.15069-1-bdenes@scylladb.com>
2019-09-24 20:18:45 +03:00
Dor Laor
7d639d058e Added comments about DAX, monitoring and more 2019-09-11 18:01:05 +03:00
Nadav Har'El
b58eadd6c9 alternator: update docs/alternator.md on GSI/LSI situation
Update docs/alternator.md on the current level of compatibility of our
GSI and LSI implementation vs. DynamoDB.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190904120730.12615-1-nyh@scylladb.com>
2019-09-11 18:01:05 +03:00
Eliran Sinvani
a6f600c54f Alternator: Add getting started document for alternator
This patch adds a getting started document for alternator,
it explains how to start up a cluster that has an alternator
API port open and how to test that it works using either an
application or some simple and minimal python scripts.
The goal of the document is to get a user to have an up and
running docker based cluster with alternator support in the
shortest time possible.
2019-09-11 18:01:05 +03:00
Eliran Sinvani
573ff2de35 move alternator.md to its own directory
As part of trying to make alternator more accessible
to users, we expect more documents to be created so
it seems like a good idea to give all of the alternator
docs their own directory.
2019-09-11 18:01:05 +03:00
Pekka Enberg
380a7be54b dist/docker: Add support for Alternator
This adds a "alternator-address" and "alternator-port" configuration
options to the Docker image, so people can enable Alternator with
"docker run" with:

  docker run --name some-scylla -d <image> --alternator-port=8080
Message-Id: <20190902110920.19269-1-penberg@scylladb.com>
2019-09-11 18:01:05 +03:00
Nadav Har'El
32b898db7b alternator: docs/alternator.md: link to a longer document
Add a link to a longer document (currently, around 40 pages) about
DynamoDB's features and how we implemented or may implement them in
Alternator.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190825121201.31747-2-nyh@scylladb.com>
2019-09-11 18:01:05 +03:00
Nadav Har'El
a5c3d11ccb alternator: document choice of RF
After changing the choice of RF in a previous patch, let's update the
relevant part of docs/alternator.md.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190825121201.31747-1-nyh@scylladb.com>
2019-09-11 18:01:05 +03:00
Nadav Har'El
d20ec9f492 alternator: expand docs/alternator.md
Expand docs/alternator.md with new sections about how to run Alternator,
and a very brief introduction to its design.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190818164628.12531-1-nyh@scylladb.com>
2019-09-11 18:01:05 +03:00
Nadav Har'El
ea41dd2cf8 alternator: docs/alternator.md more about filtering support
Give more details about what is, and what isn't, currently
supported in filtering of Scan (and Query) results.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190811094425.30951-1-nyh@scylladb.com>
2019-09-11 18:01:05 +03:00
Nadav Har'El
d74b203dee alternator: add docs/alternator.md
This adds a new document, docs/alternator.md, about Alternator.

The scope of this document should be expanded in the future. We begin
here by introducing Alternator and its current compatibility level with
Amazon DynamoDB, but it should later grow to explain the design of Alternator
and how it maps the DynamoDB data model onto Scylla's.

Whether this document should remain a short high-level overview, or a long
and detailed design document, remains an open question.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190805085340.17543-1-nyh@scylladb.com>
2019-09-11 18:01:04 +03:00
Nadav Har'El
2543760ee6 docs/metrics.md: document additional "lables"
Recently we started to use more the concept of metric labels - several
metrics which share the same name, but differ in the value of some label
such a "group" (for different scheduling groups).

This patch documents this feature in docs/metrics.md, gives the example of
scheduling groups, and explains a couple more relevant Promethueus syntax
tricks.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190909113803.15383-1-nyh@scylladb.com>
2019-09-09 15:15:57 +03:00
Rafael Ávila de Espíndola
278b6abb2b Improve documentation on the system.large_* tables
This clarifies that "rows" are clustering rows and that there is no
information about individual collection elements.

The patch also documents some properties common to all these tables.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20190820171204.48739-1-espindola@scylladb.com>
2019-08-21 10:36:25 +03:00
Pekka Enberg
7d4bf10d87 docs/building-packages.md: Document how to build Scylla packages
This documents the steps needed to build Scylla's Linux packages with
the relocatable package infrastructure we use today.

Message-Id: <20190807134017.4275-1-penberg@scylladb.com>
2019-08-08 14:11:35 +03:00
Piotr Sarna
1e54752167 docs: init system_schema_keyspace.md with column computations
The documentation file for system_schema keyspace is introduced,
and its first entry describes the column_computation table.
2019-07-19 11:58:42 +02:00
Rafael Ávila de Espíndola
67c624d967 Add documentation for large_rows and large_cells
Fixes #4552

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20190614151907.20292-1-espindola@scylladb.com>
2019-07-12 19:21:26 +03:00
Avi Kivity
187859ad78 review-checklist: mention that the guidelines are not absolute rules and can be overridden 2019-07-06 12:26:08 +03:00
Asias He
f686f0b9d6 docs: Add RPC stream doc for row level repair
This documents RPC stream usage in row level repair.
2019-07-03 08:09:57 +08:00
Avi Kivity
9d21341733 review-checklist.md: add common checks
- code style
 - naming
 - micro-performance
 - concurrency
 - unit-testing
 - templates and type erasure
 - singletons
2019-06-25 19:05:29 +03:00
Rafael Ávila de Espíndola
8672eddff2 Document the best practices for when to use asserts/exceptions/logs
The intention is just to document what is currently done. If someone
wants to propose changes, that can be done after the current practices
have been documented.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20190524135109.29436-1-espindola@scylladb.com>
2019-06-18 12:13:01 +03:00
Nadav Har'El
95bab04cf9 docs/metrics.md: "instance" label no longer comes from Scylla
Prometheus needs to remember which "instance" (node) each measurement
came from. But it doesn't actually need Scylla to tell it the instance
name - it knows which node it got each measurement from.

After Seastar commit 79281ef287
which fixed Seastar issue https://github.com/scylladb/seastar/issues/477,
the "instance" label on measurements no longer comes from Scylla but rather
is added by Prometheus. This patch corrects the documentation to explain the
current situation, instead of incorrectly saying that Scylla adds the
"instance" label itself.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20190602074629.14336-1-nyh@scylladb.com>
2019-06-06 12:42:30 +03:00
Piotr Jastrzebski
e91e1a1dde docs: add CorrectEmptyCounters to sstable-scylla-format
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2019-05-23 10:10:24 +02:00
Avi Kivity
987739898f docs: document SSTable Scylla.db component
Document the format and meaning of the various bits of the Scylla.db component.
Message-Id: <20190513081605.7394-1-avi@scylladb.com>
2019-05-14 16:00:23 +03:00
Tzach Livyatan
f444c949bd Fix the Dockerhub documentation for listen-address
Fix listen-address documention: it is used for internal communication, not for external clients

Signed-off-by: Tzach Livyatan <tzach@scylladb.com>
Message-Id: <20190410181409.16078-1-tzach@scylladb.com>
2019-04-11 11:53:40 +03:00
Piotr Sarna
a45022dbc7 docs: document index target serialization
Index target serialization format is extended for the purpose
of local indexing. Both new and old formats are described
in docs.
2019-03-20 09:51:46 +01:00