Drop the cpu prefix from the cgroup path vars now that memory
accounting uses them too, and extract cgroupV2Dir() for the v2 mount
lookup previously repeated in CPU, memory and the root cgroup check.
The agent runs as the beszel user under ProtectSystem=strict, so package
managers that write temp, cache or log files failed with only
"exit status 1" in the log.
- Include the command's stderr in check errors.
- dnf5 (Fedora 41+): point XDG_STATE_HOME and XDG_CACHE_HOME into the
agent's data dir. They default to $HOME, which is /nonexistent for
the beszel user. With -C, dnf5 still reads the system metadata cache.
- zypper: keep the solv cache in the data dir with --solv-cache-dir,
since the agent user cannot rebuild it in /var/cache/zypp.
- pacman: read the system sync databases with pacman -Qu instead of
syncing a private copy with checkupdates. This drops the
pacman-contrib and fakeroot dependencies and pacman's download
sandbox, which fails in containers without Landlock. Like the other
managers, it never refreshes the databases itself.
- install-agent.sh, Debian unit: add PrivateTmp=yes. The script adds it
to existing units too when re-run. dnf4 (Rocky, Alma) and zypper keep
temporary files in /var/tmp, which ProtectSystem=strict makes
read-only. dnf4 hardcodes /var/tmp, so the agent cannot work around
it. apt does not need it; the Debian unit gets it so both units stay
the same.
Check /sys/class/ieee80211 before opening netlink sockets. Looking up
the
nl80211 family while cfg80211 is not loaded makes the kernel run
modprobe,
which loads the module on hosts without wireless hardware, or fails
again
on every poll when the module is unavailable.
Inside an LXC guest, lxcfs serves /proc/stat with the counters of the
host cores in the guest's cpuset, so an idle guest sharing a core with a
busy neighbor reported near-100% CPU.
When the agent detects it is running in an LXC guest (lxcfs mounted on
/proc/stat, container=lxc, or /run/systemd/container), derive CPU usage
from the guest's own cgroup instead: cpu.stat on cgroup v2, cpuacct on
v1, read at the cgroup mount root so it covers every process in the
guest. Usage is normalized by the usable cores (the smallest of the
affinity mask, cpuset and CPU quota).
Per-core usage is omitted in this mode, since the per-core /proc/stat
counters describe shared host cores and would contradict the total.
Iowait and steal are not available from cgroup accounting and report as
zero.
Hosts and Docker/Podman agents are unaffected and keep reading
/proc/stat, as does an LXC guest whose cgroup accounting is unreadable.
---------
Co-authored-by: hank <hank@henrygd.me>
Make WebSocket reconnect and SSH fallback transitions reliable across asynchronous disconnects, stale callbacks, and overlapping connections. Keep the SSH listener available while disconnected and allow a verified WebSocket connection to take precedence when it recovers.
Co-authored-by: henrygd <hank@henrygd.me>
Right after agent start the hub requests stats immediately, and the first
sample of the interval was measured from the baseline taken at startup, only
a second or so earlier. A burst of startup I/O was then stored as the rate
for the whole minute, causing large spikes in disk I/O charts.
The first sample of an interval now must span at least half the interval.
Otherwise it only stores the snapshot, and the next sample is measured from it.
mdlayher/wifi only dumps stations, which some full-MAC drivers (e.g.
out-of-tree Realtek USB) answer with an empty list, leaving RSSI
unavailable. Request the associated BSSID's station directly, as
`iw link` does, and parse it with native.ParseStationInfo.
- Info.WiFi uses json "wf" with omitempty so systems without Wi-Fi no
longer store/broadcast "wifi":null
- move osascript exec into wifi_darwin.go; drop unused commandRunner
- share strongest-connection logic between table cell and sorting
- trim agent/wifi README
Stats.WiFi is now map[string]int8 (json "wf") holding only available
RSSI
readings; SSID and unavailable signals stay in Info.WiFi. Averages are
rounded to whole dBm.
Wi-Fi is collected only on the default 60s interval; real-time requests
reuse the last snapshot to avoid spawning osascript / dumping the BSS
cache every second.
When the root drive is also listed in EXTRA_FILESYSTEMS, its key is
taken before addPartitionRootFs runs. Returning false there sent the
agent to addLastResortRootFs, which picks the most active device and
could register a different drive as root, overwriting that drive's
extra entry. addPartitionRootFs has already resolved the root device,
so promote the existing entry to root instead.
addPartitionRootFs reported success even when addFsStat skipped the
registration because the I/O key was already in use, e.g. when the root
drive is also listed in EXTRA_FILESYSTEMS. hasRoot was then set without
any root filesystem, and the last-resort fallback never ran. addFsStat
now reports whether it registered the filesystem and addPartitionRootFs
propagates that.
Also uppercase the drive letter in windowsVolumeName so "c:" and "C:"
normalize to the same key.
filepath.Base is platform-dependent for a bare volume name: on Windows it
strips the "C:" specifier and returns "\", so every drive letter
normalizes to the same key. findIoDevice's exact match then returns
whichever disk.IOCounters entry the map yielded first, which registers the
root filesystem under a random drive. With EXTRA_FILESYSTEMS=D:,P: the
root disk disappears from the hub, and without it the root can be
registered under another drive's I/O device.
Normalize a bare volume specifier before taking a path base, and stop
taking a path base of the device in addPartitionRootFs, where it mangled
the value before findIoDevice could see it. registerFilesystemStats
already avoided this by only taking a base when the agent is not on
Windows.
Fixes#2417
- store checkupdates' private DB in the agent data dir, since
ProtectSystem=strict makes /tmp read-only
- sync the private DB every 12h and use `checkupdates -n` in between,
forcing a sync when the DB is missing so -n never reports a false 0
- set WaitDelay so a timed-out command can't hang on child processes
The hub waited only 5s (the request manager default) for stats over
WebSocket and closed the connection on any error. On hosts where
`zpool list` stalls (seen on Proxmox, up to its 10s timeout), collection
exceeded that limit, so the hub sent a close (code 1000) and the agent
reconnected. The refresh ran every other cycle, which caused a
disconnect roughly every 2 minutes.
Hub:
- Wait up to 30s for WebSocket stats responses.
- Keep the connection open when a stats request times out; only close
it (and fall back to SSH) for other errors.
Agent:
- After the first collection, refresh `zpool list` pool stats and
`zfs list` dataset usage in the background and serve cached values
meanwhile, so a hung utility cannot delay the stats response.
sveng93
henrygd
MarrrrrrrrryandGitHub
spatiumstasandhenrygd
Jeroen De WildeandGitHub
user01010111andGitHub
Anand HegdeandGitHub
Ludwig J. MarxandGitHub
yi111andGitHub
Miłosz KolberandGitHub
