This pull request introduces global secondary-indexing for non-frozen collections.
The intent is to enable such queries:
```
CREATE TABLE test(int id, somemap map<int, int>, somelist<int>, someset<int>, PRIMARY KEY(id));
CREATE INDEX ON test(keys(somemap));
CREATE INDEX ON test(values(somemap));
CREATE INDEX ON test(entries(somemap));
CREATE INDEX ON test(values(somelist));
CREATE INDEX ON test(values(someset));
-- index on test(c) is the same as index on (values(c))
CREATE INDEX IF NOT EXISTS ON test(somelist);
CREATE INDEX IF NOT EXISTS ON test(someset);
CREATE INDEX IF NOT EXISTS ON test(somemap);
SELECT * FROM test WHERE someset CONTAINS 7;
SELECT * FROM test WHERE somelist CONTAINS 7;
SELECT * FROM test WHERE somemap CONTAINS KEY 7;
SELECT * FROM test WHERE somemap CONTAINS 7;
SELECT * FROM test WHERE somemap[7] = 7;
```
We use here all-familiar materialized views (MVs). Scylla treats all the
collections the same way - they're a list of pairs (key, value). In case
of sets, the value type is dummy one. In case of lists, the key type is
TIMEUUID. When describing the design, I will forget that there is more
than one collection type. Suppose that the columns in the base table
were as follows:
```
pkey int, ckey1 int, ckey2 int, somemap map<int, text>, PRIMARY KEY(pkey, ckey1, ckey2)
```
The MV schema is as follows (the names of columns which are not the same
as in base might be different). All the columns here form the primary
key.
```
-- for index over entries
indexed_coll (int, text), idx_token long, pkey int, ckey1 int, ckey2 int
-- for index over keys
indexed_coll int, idx_token long, pkey int, ckey1 int, ckey2 int
-- for index over values
indexed_coll text, idx_token long, pkey int, ckey1 int, ckey2 int, coll_keys_for_values_index int
```
The reason for the last additional column is that the values from a collection might not be unique.
Fixes#2962Fixes#8745Fixes#10707
This patch does not implement **local** secondary indexes for collection columns: Refs #10713.
Closes#10841
* github.com:scylladb/scylladb:
test/cql-pytest: un-xfail yet another passing collection-indexing test
secondary index: fix paging in map value indexing
test/cql-pytest: test for paging with collection values index
cql, view: rename and explain bytes_with_action
cql, index: make collection indexing a cluster feature
test/cql-pytest: failing tests for oversized key values in MV and SI
cql: fix secondary index "target" when column name has special characters
cql, index: improve error messages
cql, index: fix default index name for collection index
test/cql-pytest: un-xfail several collecting indexing tests
test/cql-pytest/test_secondary_index: verify that local index on collection fails.
docs/design-notes/secondary_index: add `VALUES` to index target list
test/cql-pytest/test_secondary_index: add randomized test for indexes on collections
cql-pytest/cassandra_tests/.../secondary_index_test: fix error message in test ported from Cassandra
cql-pytest/cassandra_tests/.../secondary_index_on_map_entries,select_test: test ported from Cassandra is expected to fail, since Scylla assumes that comparison with null doesn't throw error, just evaluates to false. Since it's not a bug, but expected behavior from the perspective of Scylla, we don't mark it as xfail.
test/boost/secondary_index_test: update for non-frozen indexes on collections
test/cql-pytest: Uncomment collection indexes tests that should be working now
cql, index: don't use IS NOT NULL on collection column
cql3/statements/select_statement: for index on values of collection, don't emit duplicate rows
cql/expr/expression, index/secondary_index_manager: needs_filtering and index_supports_expression rewrite to accomodate for indexes over collections
cql3, index: Use entries() indexes on collections for queries
cql3, index: Use keys() and values() indexes on collections for queries.
types/tuple: Use std::begin() instead of .begin() in tuple_type_impl::build_value_fragmented
cql3/statements/index_target: throw exception to signalize that we didn't miss returning from function
db/view/view.cc: compute view_updates for views over collections
view info: has_computed_column_depending_on_base_non_primary_key
column_computation: depends_on_non_primary_key_column
schema, index/secondary_index_manager: make schema for index-induced mv
index/secondary_index_manager: extract keys, values, entries types from collection
cql3/statements/: validate CREATE INDEX for index over a collection
cql3/statements/create_index_statement,index_target: rewrite index target for collection
column_computation.hh, schema.cc: collection_column_computation
column_computation.hh, schema.cc: compute_value interface refactor
Cql.g, treewide: support cql syntax `INDEX ON table(VALUES(collection))`
Fix https://github.com/scylladb/scylladb/issues/11197
This PR adds a new page where specifying workload attributes with service levels is described and adds it to the menu.
Also, I had to fix some links because of the warnings.
Closes#11209
* github.com:scylladb/scylladb:
doc: remove the reduntant space from index
doc: update the syntax for defining service level attributes
doc: rewording
doc: update the links to fix the warnings
doc: add the new page to the toctree
doc: add the descrption of specifying workload attributes with service levels
doc: add the definition of workloads to the glossary
Fix https://github.com/scylladb/scylla-doc-issues/issues/438
In addition, I've replaced "Scylla" with "ScyllaDB" on that page.
Closes#11281
* github.com:scylladb/scylladb:
doc: replace Scylla with ScyllaDB on the Fault Tolerance page
doc: fis the typo in the note
Fix https://github.com/scylladb/scylla-doc-issues/issues/857Closes#11253
* github.com:scylladb/scylladb:
doc: language improvemens to the Counrers page
doc: fix the external link
doc: clarify the disclaimer about reusing deleted counter column values
Fix https://github.com/scylladb/scylla-doc-issues/issues/867
Plus some language, formatting, and organization improvements.
Closes#11248
* github.com:scylladb/scylladb:
doc: language, formatting, and organization improvements
doc: add a disclaimer about not supporting local counters by SSTableLoader
Fix https://github.com/scylladb/scylla-doc-issues/issues/842
This PR changes the default for the `overprovisioned` option from `disabled` to `enabled`, according to https://github.com/scylladb/scylla-doc-issues/issues/842.
In addition, I've used this opportunity to replace "Scylla" with "ScyllaDB" on the updated page.
Closes#11256
* github.com:scylladb/scylladb:
doc: replace Scylla with ScyllaDB in the product name
doc: change the default for the overprovisioned option
Ubuntu 22.04 is supported by both ScyllaDB Open Source 5.0 and Enterprise 2022.1.
Closes#11227
* github.com:scylladb/scylladb:
doc: add the redirects from Ubuntu version specific to version generic pages
doc: remove version-speific content for Ubuntu and add the generic page to the toctree
doc: rename the file to include Ubuntu
doc: remove the version number from the document and add the link to Supported Versions
doc: add a generic page for Ubuntu
doc: add the upgrade guide from 5.0 to 2022.1 on Ubuntu 2022.1
For generating #include directives in the generated files,
so we don't have to hand-craft include the dependencies
in the right order.
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
In this PR, I have:
- removed the docs for Manager (including the sources for Manager 2.1 and the upgrade guides).
- added redirects to https://manager.docs.scylladb.com/.
- replaced the internal links with external links to https://manager.docs.scylladb.com/.
Closes#11162
* github.com:scylladb/scylladb:
doc: update the link to fix the warning about duplicate targets
Update docs/kb/gc-grace-seconds.rst
Update docs/_utils/redirects.yaml
doc: update the links to Manager
doc: add the link to manager.docs.scylladb.com to the toctree
doc: remove the docs for Manager - the Manager page, the guide for Manager 2.1, Manger upgrade guides
doc: add redirections from Manager 2.1 to the Manager docs
doc: add redirections to manager.docs.scylladb.com
Fix https://github.com/scylladb/scylla-docs/issues/4125
I've added the upgrade guides from 5.0 to 2022.1. They are based on the previous upgrade guides from Open Source to Enterprise.
Closes#11108
* github.com:scylladb/scylladb:
doc: apply feedback about scylla-enterprise-machine-image
doc: update the note about installing scylla-enterprise-machine-image
update the info about installing scylla-enterprise-machine-image during upgrade
doc: add the requirement to install scylla-enterprise-machine-image if the previous version was installed with an image
doc: update the info about metrics in 2022.1 compared to 5.0
doc: minor formatting and language fixes
doc: add the new guide to the toctree
doc: add the upgrade guide from 5.0 to 2022.1
I created this branch to remove the external docs (Manager, Monitoring, Operator) from the core ScyllaDB documentation.
However, to make reviewing easier, this PR only covers removing the docs for ScyllaDB Monitoring Stack. I'm going to send other PRs to cover Manager and Operator.
In this PR, I have:
- removed the docs for ScyllaDB Monitoring Stack (including the sources for old versions).
- added redirects to https://monitoring.docs.scylladb.com/.
- replaced the internal links with external links to https://monitoring.docs.scylladb.com/.
Closes#11151
* github.com:scylladb/scylladb:
doc: fix the link to the Monitoring Stack
doc: fix the links in the manager section
doc: add the external link to Monitoring Stack to the menu
doc: replace the links to Monitoring Stack
doc: add the redirections for Monitoring Stack
doc: delete the Monitoring Stack documentation form the ScyllaDB docs and remove it from the toctree
