Commit Graph

112 Commits

Author SHA1 Message Date
Kefu Chai
b4fc24cc1f github: use needs.read-toolchain.outputs.image for build-scylla
so we don't need to hardwire the image on which we build scylla.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#20370
2024-09-05 12:58:36 +03:00
Kefu Chai
fb2ed20b42 .github: post a comment if "Fixes" policy is violated
it's more visible than an "Error" in the action's detail message.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#19271
2024-09-03 07:23:48 +03:00
David Garcia
c4da75e392 docs: run docs test on changing config params
Triggers the "Build Docs" PR workflow whenever the `db/config.cc` or `db/config.h` files are edited. These files are used to produce documentation, and this change will help prevent the introduction of breaking changes to the documentation build when they are modified.

Closes scylladb/scylladb#20347
2024-09-01 22:15:48 +03:00
Botond Dénes
6d633e89ef Merge 'update CODEOWNERS' from Piotr Smaron
Removed people that no longer contribute to the scylladb.git and added/substituted reviewers responsible for maintaining the frontend components.

No need to backport, this is just an information for the github tool.

Closes scylladb/scylladb#20136

* github.com:scylladb/scylladb:
  codeowners: add appropriate reviewers to the cluster components
  codeowners: add appropriate reviewers to the frontend components
  codeowners: fix codeowner names
  codeowners: remove non contributors
2024-08-26 16:44:39 +03:00
Kefu Chai
c88b63ce13 github: use clang-20 in clang-nightly workflow
since clang 19 has been branched. let's track the development brach,
which is clang 20.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#20279
2024-08-25 19:31:43 +03:00
Piotr Smaron
f773c76bfb codeowners: add appropriate reviewers to the cluster components 2024-08-19 12:39:47 +02:00
Piotr Smaron
3e3858521d codeowners: add appropriate reviewers to the frontend components 2024-08-14 22:26:35 +02:00
Piotr Smaron
1b2e88b96a codeowners: fix codeowner names 2024-08-14 22:26:26 +02:00
Piotr Smaron
31cb5b132b codeowners: remove non contributors 2024-08-14 18:52:25 +02:00
Botond Dénes
0bb6413ea5 Merge 'github: disable scheduled workflow on forks' from Kefu Chai
as these workflows are scheduled periodically, and if they fail, notifications are sent to the repo's owner. to minimize the surprises to the contributors using github, let's disable these workflows on fork repos.

Closes scylladb/scylladb#19736

* github.com:scylladb/scylladb:
  github: do not run clang-tidy as a cron job
  github: disable scheduled workflow on forks
2024-07-24 07:50:39 +03:00
Yaron Kaikov
d3cbe04130 .github/mergify.yml: update conf to support 6.1
Modify Mergify configuation to support `6.1` instead of `5.2` which is
EOL

Closes scylladb/scylladb#19810
2024-07-21 17:02:19 +03:00
Kefu Chai
c911832ed9 github: do not run clang-tidy as a cron job
we already run it for every pull request, so no need to run it
periodically.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-07-15 19:19:49 +08:00
Kefu Chai
dc189c67a6 github: disable scheduled workflow on forks
as these workflows are scheduled periodically, and if they fail,
notifications are sent to the repo's owner. to minimize the surprises
to the contributors using github, let's disable these workflows on
fork repos.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-07-15 19:19:28 +08:00
Kefu Chai
2a1c9ed7cb github: use needs.read-toolchain.outputs.image for iwyu's container
in 9a71543fd2, we introduced a regression,
which failed to use the proper value for the container image in which
the iwyu workflow is run.

in this change, we pass the correct value, as we do in clang-tidy.yaml
workflow.

Refs 9a71543fd2
Fixes scylladb/scylladb#19704
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#19697
2024-07-11 17:17:37 +03:00
Kefu Chai
9a71543fd2 github: always use the tools/toolchain/image for lint workflows
instead of hardwiring the toolchain image in github workflows, read it
from `tools/toolchain/image`. a dedicated reusable workflow is added to
read from this file, and expose its content with an output parameter.

also, switch iwyu.yaml workflow to this image, more maintainable this
way. please note, before this change, we are also using the latest
stable build of clang, and since fedora 40 is also using the clang 18,
so the behavior is not change. but with this change, we don't have
the flexibility of using other clang versions provided
https://apt.llvm.org in future.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#19655
2024-07-10 23:45:35 +03:00
Yaron Kaikov
e33126fc3e .github/script/label_promoted_commit.py: add label only if ref is PR
we got a failure during check-commit action:
```
Run python .github/scripts/label_promoted_commits.py --commit_before_merge 30e82a81e8 --commit_after_merge f31d5e3204 --repository scylladb/scylladb --ref refs/heads/master

Commit sha is: d5a149fc01
Commit sha is: 415457be2b
Commit sha is: d3b1ccd03a
Commit sha is: 1fca341514
Commit sha is: f784be6a7e
Commit sha is: 80986c17c3
Commit sha is: 492d0a5c86
Commit sha is: 7b3f55a65f
Commit sha is: 78d6471ce4
Commit sha is: 7a69d9070f
Commit sha is: a9e985fcc9
master branch, pr number is: 19213
Traceback (most recent call last):
  File "/home/runner/work/scylladb/scylladb/.github/scripts/label_promoted_commits.py", line 87, in <module>
    main()
  File "/home/runner/work/scylladb/scylladb/.github/scripts/label_promoted_commits.py", line 81, in main
    pr = repo.get_pull(pr_number)
  File "/usr/lib/python3/dist-packages/github/Repository.py", line 2746, in get_pull
    headers, data = self._requester.requestJsonAndCheck(
  File "/usr/lib/python3/dist-packages/github/Requester.py", line 353, in requestJsonAndCheck
    return self.__check(
  File "/usr/lib/python3/dist-packages/github/Requester.py", line 378, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.UnknownObjectException: 404 {"message": "Not Found", "documentation_url": "https://docs.github.com/rest/pulls/pulls#get-a-pull-request", "status": "404"}
Error: Process completed with exit code 1.
```

The reason for this failure is since in one of the promoted commits
(a9e985fcc9) had a reference of `Closes`
to an issue.

Fixes: https://github.com/scylladb/scylladb/issues/19677

Closes scylladb/scylladb#19678
2024-07-10 15:27:12 +03:00
Kefu Chai
229250ef3e .github: use scylla-toolchain for newer fmt
in cccec07581, we started using a featured introduced by {fmt} v10.
but we are still using the {fmt} cooked using seastar, and it is
9.1.0, so this breaks the build when running the clang-tidy workflow.

in this change, instead of building on ubuntu jammy, we use the
scylladb/scylla-toolchain image based on fedora 40, which provides
{fmt} v10.2.1. since we are have clang 18 in fedora 40, this change
does not sacrifice anything.

after this change, clang-tidy workflow should be back to normal.

Fixes scylladb/scylladb#19621
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#19628
2024-07-08 11:14:02 +02:00
Botond Dénes
56c194e52c Merge 'compaction: not include unused headers' from Kefu Chai
these unused includes were identified by clangd. see
https://clangd.llvm.org/guides/include-cleaner#unused-include-warning
for more details on the "Unused include" warning.

---

it's a cleanup, hence no need to backport.

Closes scylladb/scylladb#19581

* github.com:scylladb/scylladb:
  .github: add compaction to iwyu's CLEANER_DIR
  compaction: not include unused headers
2024-07-08 10:03:51 +03:00
Kefu Chai
fd5c04acbb .github: use the latest dbuild image
scylla does not build using scylla-toolchain:fedora-38-20240521, like:

```
FAILED: repair/CMakeFiles/repair.dir/repair.cc.o
/usr/bin/clang++ -DBOOST_NO_CXX98_FUNCTION_BASE -DDEVEL -DFMT_SHARED -DSCYLLA_BUILD_MODE=dev -DSCYLLA_ENABLE_ERROR_INJECTION -DSEASTAR_API_LEVEL=7 -DSEASTAR_ENABLE_ALLOC_FAILURE_INJECTION -DSEASTAR_LOGGER_COMPILE_TIME_FMT -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_SSTRING -DSEASTAR_TYPE_ERASE_MORE -DXXH_PRIVATE_API -I/__w/scylladb/scylladb -I/__w/scylladb/scylladb/build/gen -I/__w/scylladb/scylladb/seastar/include -I/__w/scylladb/scylladb/build/seastar/gen/include -I/__w/scylladb/scylladb/build/seastar/gen/src -isystem /__w/scylladb/scylladb/abseil -O2 -std=gnu++2b -fvisibility=hidden -Wall -Werror -Wextra -Wno-error=deprecated-declarations -Wimplicit-fallthrough -Wno-c++11-narrowing -Wno-deprecated-copy -Wno-mismatched-tags -Wno-missing-field-initializers -Wno-overloaded-virtual -Wno-unsupported-friend -Wno-enum-constexpr-conversion -Wno-unused-parameter -ffile-prefix-map=/__w/scylladb/scylladb=. -march=westmere -U_FORTIFY_SOURCE -Werror=unused-result -fstack-clash-protection -MD -MT repair/CMakeFiles/repair.dir/repair.cc.o -MF repair/CMakeFiles/repair.dir/repair.cc.o.d -o repair/CMakeFiles/repair.dir/repair.cc.o -c /__w/scylladb/scylladb/repair/repair.cc
In file included from /__w/scylladb/scylladb/repair/repair.cc:10:
In file included from /__w/scylladb/scylladb/repair/row_level.hh:14:
In file included from /__w/scylladb/scylladb/repair/task_manager_module.hh:14:
In file included from /__w/scylladb/scylladb/tasks/task_manager.hh:20:
In file included from /__w/scylladb/scylladb/seastar/include/seastar/coroutine/parallel_for_each.hh:24:
/usr/bin/../lib/gcc/x86_64-redhat-linux/13/../../../../include/c++/13/ranges:6161:14: error: requires clause differs in template redeclaration
    requires forward_range<_Vp>
             ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/13/../../../../include/c++/13/ranges:5860:14: note: previous template declaration is here
    requires input_range<_Vp>
             ^
```

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#19547
2024-07-03 16:57:22 +03:00
Kefu Chai
aedd145d6b .github: add compaction to iwyu's CLEANER_DIR
to avoid future violations of include-what-you-use.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-07-02 14:06:42 +08:00
Yaron Kaikov
efa94b06c2 .github/scripts/label_promoted_commits.py: fix adding labels when PR is closed
`prs = response.json().get("items", [])` will return empty when there are no merged PRs, and this will just skip the all-label replacement process.

This is a regression following the work done in #19442

Adding another part to handle closed PRs (which is the majority of the cases we have in Scylla core)

Fixes: https://github.com/scylladb/scylladb/issues/19441

Closes scylladb/scylladb#19497
2024-06-27 14:00:44 +03:00
Yaron Kaikov
394cba3e4b .github/workflow: close and replace label when backport promoted
Today after Mergify opened a Backport PR, it will stay open until someone manually close the backport PR , also we can't track using labels which backport was done or not since there is no indication for that except digging into the PR and looking for a comment or a commit ref

The following changes were made in this PR:
* trigger add-label-when-promoted.yaml also when the push was made to `branch-x.y`
* Replace label `backport/x.y` with `backport/x.y-done` in the original PR (this will automatically update the original Issue as well)
* Add a comment on the backport PR and close it

Fixes: https://github.com/scylladb/scylladb/issues/19441

Closes scylladb/scylladb#19442
2024-06-25 12:11:28 +03:00
Kefu Chai
f781c3babe .github: add reproducible-build workflow
to verify that scylla builds are reproducible.

the new workflow builds scylla twice with master HEAD, and compares
the md5sums of the built scylla executables. it fails if the md5sum:s
do not match.

this workflow is triggered at 5AM every Friday. its status can be
found at https://github.com/scylladb/scylladb/actions/workflows/reproducible-build.yaml
after it's built for the first time.

Refs #19225
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#19409
2024-06-21 19:39:37 +03:00
Piotr Dulikowski
cf2b4bf721 Merge 'cdc: do not include unused headers' from Kefu Chai
also add `auth` and `cdc` to iwyu's `CLEANER_DIR` setting.

---

it's a cleanup, hence no need to backport.

Closes scylladb/scylladb#19410

* github.com:scylladb/scylladb:
  .github: add auth and cdc to iwyu's CLEANER_DIR
  cdc: do not include unused headers
2024-06-21 13:44:40 +02:00
Yaron Kaikov
57428d373b [actions] fix sync label from PR to linked issue
in
b8c705bc54
i modified the even name to `pull_request_target`,

This caused skipping sync process when PR label was added/removed

Fixing it

Closes scylladb/scylladb#19408
2024-06-21 11:39:44 +03:00
Kefu Chai
694fe58d6e .github: add auth and cdc to iwyu's CLEANER_DIR
to avoid future violations of include-what-you-use.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-21 14:29:48 +08:00
Yaron Kaikov
f2705b3887 [action] add github context info for better debugging
It seems that we skip the sync label process between PR and linked
Issues

Adding those debug prints will allow us to understand why

Closes scylladb/scylladb#19393
2024-06-20 16:17:04 +03:00
Kefu Chai
87247c6542 .github: add workflow to build with latest seastar
so we can be awared that if scylla builds with seastar master HEAD,
and to be prepared if a build failure is found.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#19135
2024-06-18 13:34:43 +03:00
Nadav Har'El
4faceeaa33 Merge 'treewide: drop thrift support' from Kefu Chai
thrift support was deprecated since ScyllaDB 5.2

> Thrift API - legacy ScyllaDB (and Apache Cassandra) API is
> deprecated and will be removed in followup release. Thrift has
> been disabled by default.

so let's drop it. in this change,

* thrift protocol support is dropped
* all references to thrift support in document are dropped
* the "thrift_version" column in system.local table is preserved for backward compatibility, as we could load from an existing system.local table which still contains this clolumn, so we need to write this column as well.
* "/storage_service/rpc_server" is only preserved for backward compatibility with java-based nodetool.

Fixes #3811
Fixes #18416
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

- [x] not a fix, no need to backport

Closes scylladb/scylladb#18453

* github.com:scylladb/scylladb:
  config: expand on rpc_keepalive's description
  api: s/rpc/thrift/
  db/system_keyspace: drop thrift_version from system.local table
  transport: do not return client_type from cql_server::connection::make_client_key()
  treewide: drop thrift support
2024-06-17 22:36:49 +03:00
Kefu Chai
9d8d9168e6 .github: add api to iwyu's CLEANER_DIR
to avoid future violations of include-what-you-use.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-13 09:32:51 +08:00
Kefu Chai
8c99d9e721 .github: use libstdc++-13
since gcc-13 is packaged by ppa:ubuntu-toolchain-r, and GCC-13 was
released 1 year ago, let's use it instead. less warnings, as the
standard library from GCC-13 is more standard compliant.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#19162
2024-06-12 16:52:05 +03:00
Botond Dénes
e91f82fd5c Merge '.github: add workflow to build with clang nightly' from Kefu Chai
to be prepared for changes from clang, and enjoy the new warnings/errors from this compiler.

* it is an improvement in our CI, no need to backport.

Closes scylladb/scylladb#19164

* github.com:scylladb/scylladb:
  .github: add workflow to build with clang nightly
  .github: rename clang-tidy-matcher.json to clang-matcher.json
2024-06-12 16:50:21 +03:00
Kefu Chai
01568a36a5 .github: add workflow to build with clang nightly
to be prepared for changes from clang, and enjoy the new
warnings/errors from this compiler.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-07 14:23:06 +08:00
Kefu Chai
bbeabe2989 .github: rename clang-tidy-matcher.json to clang-matcher.json
as the matcher actually applies to all warnings from clang frontend,
and hence can be reused when building the tree with clang, so let's
rename it before using it in the clang build workflows.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-07 14:23:06 +08:00
Kefu Chai
0e04a033af .github: add alternator to iwyu's CLEANER_DIR
to avoid future violations of include-what-you-use.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-07 07:45:00 +08:00
Kefu Chai
0ff66bf564 .github: change severity to error in clang-include-cleaner
since we've addressed all warnings, we are ready to tighten the
standards of this workflow, so that contributors are awared of
the violation of include-what-you-use policy.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-07 07:28:52 +08:00
Kefu Chai
ad649be1bf treewide: drop thrift support
thrift support was deprecated since ScyllaDB 5.2

> Thrift API - legacy ScyllaDB (and Apache Cassandra) API is
> deprecated and will be removed in followup release. Thrift has
> been disabled by default.

so let's drop it. in this change,

* thrift protocol support is dropped
* all references to thrift support in document are dropped
* the "thrift_version" column in system.local table is
  preserved for backward compatibility, as we could load
  from an existing system.local table which still contains
  this clolumn, so we need to write this column as well.
* "/storage_service/rpc_server" is only preserved for
  backward compatibility with java-based nodetool.
* `rpc_port` and `start_rpc` options are preserved, but
  they are marked as "Unused". so that the new release
  of scylladb can consume existing scylla.yaml configurations
  which might contain these settings. by making them
  deprecated, user will be able get warned, and update
  their configurations before we actually remove them
  in the next major release.

Fixes #3811
Fixes #18416
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-07 06:44:59 +08:00
Kefu Chai
c6691d3217 .github: add exception to CLEANER_DIRS
to cover more directories to prevent regressions of violating
the "include what you use" policy in this directory.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-03 12:45:04 +08:00
Kefu Chai
21bdda550a .github: annotate the report from clang-include-cleaner
before this change, user has to click into the "Details" link for
access the report from clang-include-cleaner. but this is neither
convenient nor obvious.

after this change, the report is annotated in the github web interface,
this helps the reviewers and contributers to user this tool in a
more efficient way.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-03 12:45:04 +08:00
Kefu Chai
3d056a0cf2 .github: build headers before running clang-include-cleaner
clang-include-cleaner actually interprets the preprocessor macros,
and looks at the symbols. so we have to prepare the included headers
before using it.

so, but in ScyllaDB, we don't have a single target for building all the
used headers, so we have to build them either in batch of separately.

in this change, we build the included headers before running
clang-include-cleaner. this allows us to run clang-include-cleaner on
more source files.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-06-03 11:30:31 +08:00
Kefu Chai
cbc83f92d3 .github: add iwyu workflow
iwyu is short for "include what you use". this workflow is added to
identify missing "#include" and extraneous "#include" in C++ source
files.

This workflow is triggered when a pull request is created targetting
the "master" branch. It uses the clang-include-cleaner tool provided
by clang-tools package to analyze all the ".cc" and ".hh" source files.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#18122
2024-05-27 14:19:11 +03:00
Yaron Kaikov
9cc42c98f5 [Mergify] update configuration for 6.0
Updating mergify conf to support 6.0 release

Closes scylladb/scylladb#18823
2024-05-22 14:28:43 +03:00
Yaron Kaikov
bc596a3e76 pull_request_template: clearify the template and remove checkbox verification
It seems that having the checkbox in the PR template and failing the action is confusing and not very clear. Let's remove it completely and just add to the template an explanation to explain the backport reason

Closes scylladb/scylladb#18708
2024-05-20 18:24:28 +03:00
Kefu Chai
02be1e9309 .github: add clang-tidy workflow
clang-tidy is a tool provided by Clang to perform static analysis on
C++ source files. here, we are mostly intersted in using its
https://clang.llvm.org/extra/clang-tidy/checks/bugprone/use-after-move.html
check to reveal the potential issues.

this workflow is added to run clang-tidy when building the tree, so
that the warnings from clang-tidy can be noticed by developers.

a dedicated action is added so other github workflow can reuse it to
setup the building environment in an ubuntu:jammy runner.

clang-tidy-matcher.json is added to annotate the change, so that the
warnings are more visible with github webpage.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#18342
2024-05-09 08:26:43 +03:00
Yaron Kaikov
5e63f74984 [github] add action to verify PR tasks was completed
Adding another check to the PR summary will make it clear to the maintainer/reviewer if the developer explained about backport
2024-04-25 15:24:22 +03:00
Yaron Kaikov
44d1ffe86b [github] add PR template
Today with the backport automation, the developer added the relevant backport label, but without any explanation of why

Adding the PR template with a placeholder for the developer to add his decision about backport yes or no

The placeholder is marked as a task, so once the explanation is added, the task must be checked as completed
2024-04-17 15:40:32 +03:00
David Garcia
6707bc673c docs: update theme 1.7
Closes scylladb/scylladb#18252
2024-04-16 13:48:11 +02:00
Yaron Kaikov
fcdb80773e github: sync-labels: run only in scylladb oss repo
We currently support the sync-label only in OSS. Since Scylla-enterprise
get all the commits from OSS repo, the sync-label is running and failing
during checkout (since it's a private repo and should have different
configuration)

For now, let's limit the workflows for oss repo

Closes scylladb/scylladb#18142
2024-04-02 10:45:17 +03:00
Kefu Chai
c5601a749e github: sync_labels: do not error out if PR's cover letter is empty
if a pull request's cover letter is empty, `pr.body` is None. in that
case we should not try to pass it to `re.findall()` as the "string"
parameter. otherwise, we'd get

```
TypeError: expected string or bytes-like object, got 'NoneType'
```
so, in this change, we just return an empty list if the PR in question
has an empty cover letter.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#18125
2024-04-01 18:13:22 +03:00
Kefu Chai
07c40f5600 github: sync_labels: use ${{}} expression syntax in "if" condition
to ensure that the expression is evaluated properly.
see https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runsstepsif

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#18127
2024-04-01 17:17:43 +03:00