This PR extends #9209. It consists of 2 main points:
To enable parallelization of user-defined aggregates, reduction function was added to UDA definition. Reduction function is optional and it has to be scalar function that takes 2 arguments with type of UDA's state and returns UDA's state
All currently implemented native aggregates got their reducible counterpart, which return their state as final result, so it can be reduced with other result. Hence all native aggregates can now be distributed.
Local 3-node cluster made with current master. `node1` updated to this branch. Accessing node with `ccm <node-name> cqlsh`
I've tested belowed things from both old and new node:
- creating UDA with reduce function - not allowed
- selecting count(*) - distributed
- selecting other aggregate function - not distributed
Fixes: #10224Closes#10295
* github.com:scylladb/scylla:
test: add tests for parallelized aggregates
test: cql3: Add UDA REDUCEFUNC test
forward_service: enable multiple selection
forward_service: support UDA and native aggregate parallelization
cql3:functions: Add cql3::functions::functions::mock_get()
cql3: selection: detect parallelize reduction type
db,cql3: Move part of cql3's function into db
selection: detect if selectors factory contains only simple selectors
cql3: reducible aggregates
DB: Add `scylla_aggregates` system table
db,gms: Add SCYLLA_AGGREGATES schema features
CQL3: Add reduce function to UDA
gms: add UDA_NATIVE_PARALLELIZED_AGGREGATION feature
Fix https://github.com/scylladb/scylla-docs/issues/4040
Fix https://github.com/scylladb/scylla-docs/issues/4128
This PR adds the upgrade guides from ScyllaDB Enterprise 2021.1 to 2022.1. They are based on the previous guides.
Closes#11036
* github.com:scylladb/scylla:
doc: add the description of the new metrics in 2022.1
doc: remove the upgrade guide for Ubuntu 16.04 (no longer supported in version 2022.1)
doc: remove the outdated warning
Update docs/upgrade/_common/upgrade-guide-from-2021.1-to-2022.1-ubuntu-and-debian.rst
Update docs/upgrade/_common/upgrade_to_2022_warning.rst
doc: add a space on line 60 to fix the warning
doc: document metric update for 2022.1
doc: add the upgrade guide from 2021.1 to 2022.1
This series includes an assortment of loosely related improvements developed for a recent investigation. The changes include:
* Fix broken `std_deque` wrapper.
* Make `scylla smp-queues` fast.
* Teach `scylla smp-queues` to filter for both sender CPU (`--from`) and receiver CPU (`--to`) or both.
* Teach `scylla smp-queues` to make histogram over content of the queues -- i.e. the type of tasks in the smp queues.
* Teach `scylla smp-queues` to filter for tasks belonging to a certain scheduling group.
* Teach `scylla task_histogram` to include only tasks in the histogram.
* Teach `scylla task_histogram` to filter for tasks belonging to a certain scheduling group.
* Teach `scylla-fiber` to walk in both directions.
And some refactoring.
Fixes: https://github.com/scylladb/scylla/issues/7059Closes#11019
* github.com:scylladb/scylla:
docs/dev/debugging.md: update continuation chain traversal guide
scylla-gdb.py: scylla fiber: walk continuation chain in both directions
scylla-gdb.py: scylla fiber: allow passing analyzed pointers to _probe_pointer()
scylla-gdb.py: scylla fiber: hoist preparatory code out of _walk()
scylla-gdb.py: scylla task_histogram: add --scheduling-groups option
scylla-gdb.py: scylla task_histogram: add --filter-tasks option
scylla-gdb.py: scylla task_histogram: use histogram class
scylla-gdb.py: scylla-fiber: extract symbol matching logic
scylla-gdb.py: histogram: add limit feature
scylla-gdb.py: histogram: handle formatting errors
scylla-gdb.py: intrusive_slist: avoid infinite recursion in __len__()
scylla-gdb.py: scylla smp-queues: add --scheduling-group option
scylla-gdb.py: scylla smp-queues: add --content switch
scylla-gdb.py: smp-queue: add filtering capability
scylla-gdb.py: make scylla smp-queues fast
scylla-gdb.py: fix disagreement between std_deque len() and iter()
In order to allow our Scylla OSS customers the ability to select a version for their documentation, we are migrating the Scylla docs content to the Scylla OSS repository. This PR covers the following points of the [Migration Plan](https://docs.google.com/document/d/15yBf39j15hgUVvjeuGR4MCbYeArqZrO1ir-z_1Urc6A/edit#):
1. Creates a subdirectory for dev docs: /docs/dev
2. Moves the existing dev doc content in the scylla repo to /docs/dev, but keep Alternator docs in /docs.
3. Flattens the structure in /docs/dev (remove the subfolders).
4. Adds redirects from `scylla.docs.scylladb.com/<version>/<document>` to `https://github.com/scylladb/scylla/blob/master/docs/dev/<document>.md`
5. Excludes publishing docs for /docs/devs.
1. Enter the docs folder with `cd docs`.
2. Run `make redirects`.
3. Enter the docs folder and run `make preview`. The docs should build without warnings.
4. Open http://127.0.0.1:5500 in your browser. You shoul donly see the alternator docs.
5. Open http://127.0.0.1:5500/stable/design-notes/IDL.html in your browser. It should redirect you to https://github.com/scylladb/scylla/blob/master/docs/dev/IDL.md and raise a 404 error since this PR is not merged yet.
6. Surf the `docs/dev` folder. It should have all the scylla project internal docs without subdirectories.
Closes#10873
* github.com:scylladb/scylla:
Update docs/conf.py
Update docs/dev/protocols.md
Update docs/dev/README.md
Update docs/dev/README.md
Update docs/conf.py
Fix broken links
Remove source folder
Add redirections
Move dev docs to docs/dev
Currently in docs/alternator/compatibility.md experimental features
and unimplemented features are bunched together under one heading
("unimplemented features"). In this patch we separate them into two
sections. This makes the "unimplemented features" section shorter,
and also allows us to link to the new "experimental features" section
separately.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Closes#10893
Evaluating Python code from within gdb is priceless,
especially that all helper classes and functions sourced from
scylla-gdb.py can be used in there. This commit adds a paragraph
in debugging.md mentioning this tool.
Closes#10869
Adds a CQL protocol extension which introduces the rate_limit_error. The
new error code will be used to indicate that the operation failed due to
it exceeding the allowed per-partition rate limit.
The error code is supposed to be returned only if the corresponding CQL
extension is enabled by the client - if it's not enabled, then
Config_error will be returned in its stead.
The docs test dislike the gdbinit link because it refers out of
the source tree. Unconfuse the tests by removing the link. It's
sad, but the file is more easily used by referring to it rather
than viewing it, so give a hint about that too.
Closes#10650
