caa0cbe328
Initially, tests for high availability were implemented in vector-store.git repository. High availability is currently implemented in scylladb.git repository so this repository should be the better place to store them. This commit copies these tests into the scylladb.git. The commit copies validator-vector-store/src/high_availability.rs (tests logic) and validator-tests/src/common.rs (utils for tests) into the local crate validator-scylla. The common.rs should be copied to be able for reviewer to see common test code and this code most likely be frequent to change - it will be hard to maintain one common version between two repositories. The commit updates also README for vector_search_validator; it shortly describe the validator modules. The commit updates reference to the latest vector-store.git master. As a next step on the vector-store.git high_availability.rs would be removed and common.rs moved from validator-tests into validator-vector-store. References: VECTOR-394 Closes scylladb/scylladb#27499
vector-search-validator tests for Scylla and Vector Store
vector-search-validator is a testing tool for validating the functionality of
integration between Scylla and Vector Store. Such integration depends on the
Scylla cluster and the Vector Store nodes and also on the DNS service. For this
reason we run vector-search-validator in a network and storage linux
namespace to separate it from the host environment. vector-search-validator
contains DNS server and all tests in one binary. It uses external scylla and
vector-store binaries.
Running tests
The test_validator.py::test_validator[test-case] is the entry point for
running the tests. It is parametrized with name of the test case. Available
test cases are taken dynamically from the vector-search-validator binary.
To run test with dev Scylla and test case cql run the following command in
the dbuild environment (non dbuild environment is not supported currently, as
test needs to have sudo permissions without password):
$ pytest --mode=dev test/vector_search_validator/test_validator.py::test_validator[cql]
To run all tests with dev Scylla run the following command:
$ pytest --mode=dev test/vector_search_validator/test_validator.py
$ pytest --mode=dev test/vector_search_validator/test_validator.py::test_validator
You can tests with custom filters supported by validator. To run filtered tests
with dev Scylla run the following command:
$ pytest --mode=dev test/vector_search_validator/test_validator.py --filters filter1,filter2,...
Logs are stored in
testlog/{mode}/vector_search_validator/{test-case}-{run_id}/ directory.
Development of test cases
vector-search-validator (in short validator) is divided into multiple
crates:
validator- a main crate that contains only the entry pointvalidator-scylla- contains implementation of the validator tests on the scylladb.git side. If you want to add/modify the tests implemented in the scylladb.git, you will work in this crate.vector-store.git/validator-engine- contains the core logic of the validator - overall test runner and implementation of actors for tests (dns server, scylla cluster, vector store cluster)vector-store.git/validator-tests- contains the core logic of the framework tests, provides base structures for tests and actor interfaces. In the future we should check if it is possible to merge it withvalidator-enginecrate.vector-store.git/validator-vector-store- contains implementation of the validator tests on the vector-store.git side. If you want to add/modify the tests implemented in the vector-store.git, you will work in this crate.