mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-20 00:20:47 +00:00
b1e9c0b867
Add skip_reason_plugin.py — a framework-agnostic pytest plugin that provides typed skip markers (skip_bug, skip_not_implemented, skip_slow, skip_env) so that the reason a test is skipped is machine-readable in JUnit XML and Allure reports. Bare untyped pytest.mark.skip now triggers a warning (to become an error after full migration). Runtime skips via skip() are also enriched by parsing the [type] prefix from the skip message. The plugin is a class (SkipReasonPlugin) that receives the concrete SkipType enum and an optional report_callback from conftest.py, keeping it decoupled from allure and project-specific types. Extract SkipType enum and convenience runtime skip wrappers (skip_bug, skip_env, etc.) into test/pylib/skip_types.py so callers only need a single import instead of importing both SkipType and skip() separately. conftest.py imports SkipType from the new module and registers the plugin instance unconditionally (for all test runners). New files: - test/pylib/skip_reason_plugin.py: core plugin — typed marker processing, bare-skip warnings, JUnit/Allure report enrichment (including runtime skip() parsing via _parse_skip_type helper) - test/pylib/skip_types.py: SkipType enum and convenience wrappers (skip_bug, skip_not_implemented, skip_slow, skip_env) - test/pylib_test/test_skip_reason_plugin.py: 17 pytester-based test functions (51 cases across 3 build modes) covering markers, warnings, reports, callbacks, and skip_mode interaction Infrastructure changes: - test/conftest.py: import SkipType from skip_types, register SkipReasonPlugin with allure report callback - test/pylib/runner.py: set SKIP_TYPE_KEY/SKIP_REASON_KEY stash keys for skip_mode so the report hook can enrich JUnit/Allure with skip_type=mode without longrepr parsing - test/pytest.ini: register typed marker definitions (required for --strict-markers even when plugin is not loaded) Migrated test files (representative samples): - test/cluster/test_tablet_repair_scheduler.py: skip -> skip_bug (#26844), skip -> skip_not_implemented - test/cqlpy/.../timestamp_test.py: skip -> skip_slow - test/cluster/dtest/schema_management_test.py: skip -> skip_not_implemented - test/cluster/test_change_replication_factor_1_to_0.py: skip -> skip_bug (#20282) - test/alternator/conftest.py: skip -> skip_env - test/alternator/test_https.py: use skip_env() wrapper Fixes SCYLLADB-79 Closes scylladb/scylladb#29235
70 lines
4.0 KiB
INI
70 lines
4.0 KiB
INI
[pytest]
|
|
xfail_strict = true
|
|
|
|
addopts = --strict-markers --strict-config
|
|
|
|
asyncio_mode = auto
|
|
asyncio_default_fixture_loop_scope = session
|
|
|
|
junit_logging = all
|
|
junit_log_passing_tests = False
|
|
|
|
log_format = %(asctime)s.%(msecs)03d %(levelname)s> %(message)s
|
|
log_date_format = %H:%M:%S
|
|
|
|
log_file_format = %(asctime)s.%(msecs)03d %(processName)s %(threadName)s %(taskName)s %(levelname)s %(name)s> %(message)s
|
|
log_file_level = DEBUG
|
|
|
|
markers =
|
|
slow: tests that take more than 30 seconds to run
|
|
replication_factor: replication factor for RandomTables
|
|
without_scylla: run without attaching to a scylla process
|
|
enable_tablets: create keyspace with tablets enabled or disabled
|
|
repair: tests for repair
|
|
cpp: marker for c++ tests
|
|
prepare_3_nodes_cluster: prepare 3 nodes cluster for test case based on suite.yaml (all tests from old topology folder)
|
|
prepare_3_racks_cluster: prepare 3 nodes cluster in 1 dc and 3 racks for test case based on suite.yaml
|
|
single_node: test that are mark like this, should be using only one node, and should boot much quicker (dtest only)
|
|
exclude_errors: do not consider a logging message as an error if it contains any of the specified strings (dtest only)
|
|
cluster_options: specify cluster options used to initialize a cluster (dtest only)
|
|
perf: mark for performance tests, these tests will automatically apply non_gating marker
|
|
manual: to mark manual test cases, these tests will automatically apply non_gating marker
|
|
unstable: mark for unstable tests that may pass may fail, these tests will continue to run every night and generate up-to-date statistics with failures without failing the “Main” verification path(scylla-ci, Next), these tests will automatically apply non_gating marker
|
|
non_gating: tests that are not run in CI/Next/Nightly verification, but can be run manually or on special job when needed
|
|
nightly: for tests that are quite old, stable, and test functionality that rather not be changed or affected by other features, are partially covered in other tests, verify non-critical functionality, have not found any issues or regressions, too long to run on every PR, and can be popped out from the CI run, but will continue run in Next/Nightly verification
|
|
skip_mode(mode, reason, platform_key=None): Skip test in a specific build mode (e.g. dev, debug, release).
|
|
skip_bug(reason): Skip due to a known bug (reason = issue reference).
|
|
skip_not_implemented(reason): Skip because the feature is not yet implemented.
|
|
skip_slow(reason): Skip because the test is too slow for regular CI.
|
|
skip_env(reason): Skip due to a missing environment requirement.
|
|
|
|
norecursedirs = manual perf lib
|
|
# Ignore warnings about HTTPS requests without certificate verification
|
|
# (see issue #15287). Pytest breaks urllib3.disable_warnings() in conftest.py,
|
|
# so we need to do this here.
|
|
#
|
|
# Ignore warning of
|
|
# PytestWarning: record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
|
|
# Because `record_property` adds <properties> inside <testcase>, which is not allowed
|
|
# as per the latest xunit2 schema. see
|
|
# https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsdtestcase,
|
|
# an alternative is `record_testsuite_property`, but we want to attach test
|
|
# log on a per-test basis. so let's continue using this feature before
|
|
# switching to xunit1 or legacy.
|
|
filterwarnings =
|
|
ignore::urllib3.exceptions.InsecureRequestWarning
|
|
ignore:record_property is incompatible with junit_family:pytest.PytestWarning
|
|
ignore::DeprecationWarning:importlib._bootstrap
|
|
ignore::DeprecationWarning:botocore
|
|
ignore::DeprecationWarning:pytest_elk_reporter
|
|
ignore::sqlalchemy.exc.MovedIn20Warning:kmip.pie.sqltypes
|
|
ignore::cryptography.utils.CryptographyDeprecationWarning:kmip
|
|
ignore:TypeDecorator .* will not produce a cache key:sqlalchemy.exc.SAWarning:kmip
|
|
ignore:Exception in thread[\s\S]*kmip[\s\S]*shutdown:pytest.PytestUnhandledThreadExceptionWarning
|
|
|
|
|
|
tmp_path_retention_count = 1
|
|
tmp_path_retention_policy = failed
|
|
|
|
python_files = test_*.py *_test.py *_tests.py
|