mirror of
https://github.com/SCST-project/scst.git
synced 2026-10-22 14:12:33 +00:00
nightly build: Add repository skill for kernel updates
Keep the kernel update procedure in .agents/skills so it travels with the repository and is available to contributors without a personal install. Document the canonical nightly list, derived CI matrices, official version sources and the boundary between updates and new distribution support. Include a read-only upstream helper that compares configured series with both the kernel.org release feed and archives, including final EOL updates. Use a repository-relative command and shared English UI metadata.
This commit is contained in:
Gleb Chesnokov
@@ -0,0 +1,109 @@
|
||||
---
|
||||
name: scst-kernel-update
|
||||
description: Refresh SCST nightly kernel versions and derived CI matrices from official upstream and distribution archives, and prepare the nightly/update branch. Use for kernel support-list maintenance; adding distribution releases or UEK generations and fixing compatibility regressions are separate work.
|
||||
---
|
||||
|
||||
# SCST kernel updates
|
||||
|
||||
Follow the repository's current `AGENTS.md` and `CONTRIBUTING.md`. This skill
|
||||
captures update policy; it does not authorize builds, live-host operations,
|
||||
publication, or history rewriting beyond the user's request.
|
||||
|
||||
## Branch and source of truth
|
||||
|
||||
- Inspect status, worktrees, remotes, and `nightly/update` before switching.
|
||||
Resolve spelling from Git rather than creating a second `nighly/update`.
|
||||
- Fetch the requested base and update branch. Record both SHAs. Rebase on
|
||||
current `origin/master` when requested, preserving unrelated work and any
|
||||
branch-only compatibility fixes. Already applied commits may be skipped;
|
||||
verify patch equivalence rather than replaying them blindly.
|
||||
- `ABT_KERNELS` in `nightly/conf/nightly.conf` is canonical. Exact versions
|
||||
propagate through `scripts/kernel/kernel-matrix` to regression and RPM CI.
|
||||
`.github/kernel-matrix.conf` contains selectors and RPM target policy.
|
||||
Inspect these consumers each time in case paths or behavior have changed.
|
||||
- Updating an existing series normally changes only `nightly.conf`. A newly
|
||||
requested upstream release also needs its CI selector. Do not duplicate
|
||||
exact versions in workflow YAML or regenerate unrelated artifacts.
|
||||
|
||||
## Find current versions
|
||||
|
||||
Compare the checked-in inventory directly with published latest versions.
|
||||
Reading every intervening announcement is unnecessary; the mailing list is
|
||||
useful for release context, EOL notices, and investigating a regression.
|
||||
Record the retrieval date, old/new versions, and supporting URLs.
|
||||
|
||||
For upstream kernels, run the bundled read-only helper from the SCST root:
|
||||
|
||||
```sh
|
||||
python3 .agents/skills/scst-kernel-update/scripts/upstream_candidates.py \
|
||||
nightly/conf/nightly.conf
|
||||
```
|
||||
|
||||
It combines <https://www.kernel.org/releases.json> with the official
|
||||
`https://cdn.kernel.org/pub/linux/kernel/v<major>.x/` indexes. The feed alone
|
||||
misses final updates of series that have since become EOL. The helper checks
|
||||
existing upstream series, reports missing current non-RC releases separately,
|
||||
and never edits files. Network/parse failures are errors, not "up to date".
|
||||
Verify proposed tarball/patch URLs before applying candidates, especially
|
||||
when the feed and mirror index temporarily disagree. Exclude RC and linux-next.
|
||||
|
||||
For distribution entries, retain the exact distribution/release pair, upstream
|
||||
baseline, and UEK generation already present. Read
|
||||
`scripts/kernel/rhel-rpm-functions` for URLs the runner can actually consume.
|
||||
|
||||
- Rocky: start at <https://download.rockylinux.org/pub/rocky/>. Inspect the
|
||||
exact minor release's `BaseOS/source/tree/Packages/k/` (or `Packages/`).
|
||||
Retired minors move to `https://dl.rockylinux.org/vault/rocky/`; a missing
|
||||
active directory does not authorize advancing to another minor.
|
||||
- UEK: start at <https://yum.oracle.com/oracle-linux-10.html> and the analogous
|
||||
page for the existing OL major. Select that entry's UEK generation, then
|
||||
inspect `index_src.html` for `kernel-uek` source RPMs. The same OL major can
|
||||
offer several UEK generations. Keep OL major, UEK generation, and upstream
|
||||
baseline distinct; do not select the largest version across repositories.
|
||||
- Existing AlmaLinux/CentOS entries: inspect the exact release's source vault
|
||||
during a full inventory refresh. They can contain a missed final update.
|
||||
Keep the baseline and package variant: a CentOS vault can also contain a
|
||||
different kernel baseline or `centos.plus`. Do not replace old entries with
|
||||
another distribution or delete them because of EOL.
|
||||
|
||||
Compare RPM **version and release**, preferably with RPM's comparator or repo
|
||||
metadata. Alphabetic sorting and blindly taking the last `sort -V` filename
|
||||
can rank `553.el8_10` above `553.174.1.el8_10`. Keep only the intended package
|
||||
(`kernel` or `kernel-uek`), not debug, realtime, tools, or development variants.
|
||||
Verify the exact source RPM exists at a URL supported by the runner. For RPM
|
||||
CI targets, also verify the exact binary kernel/devel package version is
|
||||
published in the configured repositories (Rocky kernel-devel may be in
|
||||
AppStream rather than BaseOS). Indexes/repodata and successful HEAD
|
||||
requests suffice; do not download large packages just to check availability.
|
||||
|
||||
## Apply and validate
|
||||
|
||||
- Update each existing series to its newest confirmed published version.
|
||||
Keep options (`-nc`, `-ns`, etc.) and ordering. When advancing the upstream
|
||||
head, preserve the repository's convention: the new head has full nightly
|
||||
checks and the previous head becomes a normal `-nc` entry.
|
||||
- Add a new upstream series when the request covers that release. Newly
|
||||
discovered distribution minors, OL majors, and UEK generations are reported
|
||||
separately; adding them requires a separate support change.
|
||||
- Run `bash -n` and Bash ShellCheck on changed sourced configs, then
|
||||
`bash scripts/kernel/kernel-matrix regression` and
|
||||
`bash scripts/kernel/kernel-matrix rpm`. Confirm unique resolutions, the
|
||||
intended upstream addition, unchanged distro identities/RPM targets, and
|
||||
updated exact versions in both outputs. Standalone config lint may report
|
||||
existing SC2034 assignments consumed by their sourcing scripts; establish
|
||||
the base result and also lint through `kernel-matrix`, without changing the
|
||||
repository's ignore policy.
|
||||
- Inspect the diff and run `git diff --check <base>`. Use the registered
|
||||
`nightly build:` commit prefix and required per-commit checks. Keep the
|
||||
version refresh separate from compatibility/backport fixes.
|
||||
- Report base/HEAD, changed versions, incomplete source checks, and which
|
||||
validation ran. Matrix generation and lint are not compilation or runtime
|
||||
evidence. Investigate CI failures at the exact published HEAD before adding
|
||||
backports; no speculative compatibility changes belong in the refresh.
|
||||
|
||||
When publishing is authorized, push only the intended branch. If its rebase
|
||||
requires rewriting remote history, respect the user's authorization and use
|
||||
`--force-with-lease=refs/heads/nightly/update:<recorded-remote-sha>` rather than
|
||||
unconditional force. A lease rejection requires inspecting the new remote
|
||||
state, not retrying with a weaker guard. Do not claim CI ran merely because a
|
||||
branch was pushed: inspect workflow triggers and any existing PR.
|
||||
@@ -0,0 +1,4 @@
|
||||
interface:
|
||||
display_name: "SCST kernel update"
|
||||
short_description: "Refresh the SCST nightly and CI kernel matrices"
|
||||
default_prompt: "Use $scst-kernel-update to refresh existing SCST kernel series from official sources and prepare the nightly/update branch."
|
||||
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Report upstream SCST kernel candidates without sourcing or editing config."""
|
||||
|
||||
import argparse
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
from html.parser import HTMLParser
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import sys
|
||||
from urllib.parse import urlparse
|
||||
from urllib.request import urlopen
|
||||
|
||||
FEED = "https://www.kernel.org/releases.json"
|
||||
ARCHIVE = "https://cdn.kernel.org/pub/linux/kernel/v{}.x/"
|
||||
VERSION = re.compile(r"[0-9]+\.[0-9]+(?:\.[0-9]+)?")
|
||||
OPTIONS = re.compile(r"(?:-(?:nc|ns|nm|4|f|i|u|p))*")
|
||||
|
||||
|
||||
def version_key(version):
|
||||
parts = tuple(map(int, version.split(".")))
|
||||
return parts + (0,) * (3 - len(parts))
|
||||
|
||||
|
||||
def series(version):
|
||||
return ".".join(version.split(".")[:2])
|
||||
|
||||
|
||||
def read_config(path):
|
||||
text = path.read_text()
|
||||
match = re.search(r'^ABT_KERNELS="([^"$`]+)"', text, re.MULTILINE)
|
||||
if not match:
|
||||
raise ValueError("expected a literal double-quoted ABT_KERNELS assignment")
|
||||
entries = match[1].replace("\\\n", " ").split()
|
||||
upstream = {}
|
||||
for entry in entries:
|
||||
if "^" in entry:
|
||||
continue
|
||||
match = re.fullmatch(r"(" + VERSION.pattern + r")(" + OPTIONS.pattern + r")", entry)
|
||||
if not match:
|
||||
raise ValueError(f"unrecognized upstream entry: {entry}")
|
||||
version = match[1]
|
||||
name = series(version)
|
||||
if name in upstream:
|
||||
raise ValueError(f"duplicate upstream series: {name}")
|
||||
upstream[name] = version
|
||||
if not upstream:
|
||||
raise ValueError("no upstream versions found")
|
||||
return upstream
|
||||
|
||||
|
||||
def fetch(url):
|
||||
for attempt in range(2):
|
||||
try:
|
||||
with urlopen(url, timeout=25) as response:
|
||||
return response.read().decode("utf-8")
|
||||
except OSError:
|
||||
if attempt:
|
||||
raise
|
||||
|
||||
|
||||
class Tarballs(HTMLParser):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.versions = set()
|
||||
|
||||
def handle_starttag(self, tag, attrs):
|
||||
if tag != "a":
|
||||
return
|
||||
name = Path(urlparse(dict(attrs).get("href") or "").path).name
|
||||
match = re.fullmatch(r"linux-(" + VERSION.pattern + r")\.tar\.xz", name)
|
||||
if match:
|
||||
self.versions.add(match[1])
|
||||
|
||||
|
||||
def read_archive(major):
|
||||
url = ARCHIVE.format(major)
|
||||
parser = Tarballs()
|
||||
parser.feed(fetch(url))
|
||||
if not parser.versions:
|
||||
raise ValueError(f"no release tarballs found: {url}")
|
||||
return url, parser.versions
|
||||
|
||||
|
||||
def inventory(current):
|
||||
releases = json.loads(fetch(FEED))["releases"]
|
||||
published = [r["version"] for r in releases if VERSION.fullmatch(r["version"])]
|
||||
if not published:
|
||||
raise ValueError("release feed contains no final releases")
|
||||
majors = sorted({int(v.split(".")[0]) for v in [*current.values(), *published]})
|
||||
latest = {}
|
||||
for version in published:
|
||||
name = series(version)
|
||||
if name not in latest or version_key(version) > version_key(latest[name][0]):
|
||||
latest[name] = (version, FEED)
|
||||
with ThreadPoolExecutor(max_workers=4) as pool:
|
||||
for url, versions in pool.map(read_archive, majors):
|
||||
for version in versions:
|
||||
name = series(version)
|
||||
if name not in latest or version_key(version) >= version_key(latest[name][0]):
|
||||
latest[name] = (version, url)
|
||||
updates = []
|
||||
for name, old in current.items():
|
||||
if name not in latest:
|
||||
raise ValueError(f"no published release found for existing series {name}")
|
||||
new, source = latest[name]
|
||||
if version_key(new) < version_key(old):
|
||||
raise ValueError(f"sources are older than configured {old}: {new}")
|
||||
if version_key(new) > version_key(old):
|
||||
updates.append({"old": old, "new": new, "source": source})
|
||||
additions = [{"new": latest[name][0], "source": latest[name][1]}
|
||||
for name in sorted({series(v) for v in published} - current.keys(),
|
||||
key=version_key, reverse=True)]
|
||||
return {"checked_at": datetime.now(timezone.utc).isoformat(),
|
||||
"checked_upstream_series": len(current),
|
||||
"updates": updates, "new_series_for_review": additions}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("config", type=Path)
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
result = inventory(read_config(args.config))
|
||||
except (OSError, ValueError, KeyError, TypeError) as error:
|
||||
print(f"error: {error}", file=sys.stderr)
|
||||
return 1
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user