Files
scylladb/test/cql-pytest/run
Nadav Har'El de586ef856 test/cql-pytest: mechanism for tests requiring raft-based schema updates
Issue #8968 no longer exists when Raft-based schema updates are enabled
in Scylla (with --experimental-features=raft). Before we can close this
issue we need a way to re-run its test

        test_keyspace.py::test_concurrent_create_and_drop_keyspace

with Raft and see it pass. But we also want the tests to continue to run
by default the older raft-less schema updates - so that this mode doesn't
regress during the potentially-long duration that it's still the default!

The solution in this patch is:

1. Introduce a "--raft" option to test/cql-pytest/run, which runs the tests
   against a Scylla with the raft experimental feature, while the default is
   still to run without it.

2. Introduce a text fixture "fails_without_raft" which marks a test which
   is expected to fail with the old pre-raft code, but is expected to
   pass in the new code.

3. Mark the test test_concurrent_create_and_drop_keyspace with this new
   "fails_without_raft".

After this patch, running

        test/cql-pytest/run --raft
            test_keyspace.py::test_concurrent_create_and_drop_keyspace

Passes, which shows that issue 8968 was fixed (in Raft mode) - so we can say:
Fixes #8968

Running the same test without "--raft" still xfails (an expected failure).

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20220208162732.260888-1-nyh@scylladb.com>
2022-02-10 09:10:24 +03:00

45 lines
1.3 KiB
Python
Executable File

#!/usr/bin/env python3
import sys
import run # run.py in this directory
print('Scylla is: ' + run.scylla + '.')
ssl = '--ssl' in sys.argv
if ssl:
cmd = run.run_scylla_ssl_cql_cmd
check_cql = run.check_ssl_cql
else:
cmd = run.run_scylla_cmd
check_cql = run.check_cql
# If the "--raft" option is given, switch to the experimental Raft-based
# implementation of schema operations. Some tests are expected to fail
# when not in raft mode, so they use the fails_without_raft fixture
# that will cause them to xfail when raft isn't used.
if '--raft' in sys.argv:
sys.argv.remove('--raft')
def run_with_raft(pid, dir):
(c, e) = run_with_raft.orig_cmd(pid, dir)
return (c + ['--experimental-features=raft'], e)
run_with_raft.orig_cmd = cmd
cmd = run_with_raft
pid = run.run_with_temporary_dir(cmd)
ip = run.pid_to_ip(pid)
run.wait_for_services(pid, [
lambda: run.check_rest_api(ip),
lambda: check_cql(ip)
])
success = run.run_pytest(sys.path[0], ['--host', ip] + sys.argv[1:])
run.summary = 'Scylla tests pass' if success else 'Scylla tests failure'
exit(0 if success else 1)
# Note that the run.cleanup_all() function runs now, just like on any exit
# for any reason in this script. It will delete the temporary files and
# announce the failure or success of the test (printing run.summary).