scylladb

Files

Nadav Har'El e2c99436cf Merge 'cdc, vector_search: enable CDC when the index is created' from Dawid Pawlik

When a vector index is created in Scylla, it is initially built using a full scan of the database. After that, it stays up to date by tracking changes through CDC, which should be automatically enabled when the vector index is created.

When a user attempts to enable Vector Search (VS), the system checks whether Change Data Capture (CDC) is enabled and properly configured:

1. CDC is not enabled

    - CDC is automatically enabled with the minimum required TTL (Time-to-Live) for VS (24 hours) and the delta mode set to 'full' or post-image is enabled.

    - If the user later tries to reduce the CDC TTL below 24 hours or set delta mode to 'keys' with post-image disabled, the action fails.

    - Error message: Clearly states that CDC TTL must be at least 24 hours and delta mode must be set to 'full' or post-image must be enabled for VS to function.

2. CDC is already enabled

    - If CDC TTL is ≥ 24 hours and delta mode is set to 'full' or post-image is enabled: VS is enabled successfully.

    - If CDC TTL is < 24 hours or delta mode is set to 'keys' with post-image disabled: The VS enabling process fails.

    - Error message: Informs the user that CDC TTL must be at least 24 hours, delta mode must be set to 'full' or post-image must be enabled, and provides a link to documentation on how to update the TTL, delta mode, and post-image.

When a user attempts to disable CDC when VS is enabled, the action will fail and the user will be informed by error message that clearly states that VS needs to be disabled (vector indexes have to be dropped) first.

Full setup requirements and steps will be detailed in the documentation of Vector Search.

Co-authored-by: @smoczy123

Fixes: VECTOR-27
Fixes: VECTOR-25

Closes scylladb/scylladb#25179

* github.com:scylladb/scylladb:
  test/cqlpy: ensure Vector Search CDC options
  test/boost: adjust CDC boost tests for Vector Search
  test/cql: add Vector Search CDC enable/disable test
  cdc, vector_index: provide minimal option setup for Vector Search
  test/cqlpy: adjust describe table tests with CDC for Vector Search
  describe, cdc: adjust describe for cdc log tables
  cdc: enable CDC log when vector index is created
  test/cqlpy: run vector_index tests only on vnodes
  vector_index: check if vector index exists in schema

2025-08-26 23:01:32 +03:00

built_indexes_virtual_reader.hh

interval: rename start() to start_ref() (and end() etc).

2025-06-14 21:26:16 +03:00

CMakeLists.txt

vector_index: add custom index and vector index classes

2025-05-27 21:04:50 +02:00

secondary_index_manager.cc

vector_index: make parameter names case insensitive

2025-08-01 12:08:50 +02:00

secondary_index_manager.hh

treewide: avoid including gms/feature_service.hh from headers