Canonical event names across the reporting feed, matching hblink4's emit()
vocabulary (the OmniLink north star):
* peer -> peer_connected / peer_disconnected
* stream -> stream_start / stream_end
The lifecycle now rides the message `type`; the `action` field is retained
where consumers still use it (log label, END duration/hang handling).
Unix-socket transport for a same-box dashboard, alongside the existing TCP:
* daemon: REPORT_TRANSPORT=unix binds start_unix_server on REPORT_SOCKET
(stale socket file cleared first); TCP stays the default for remote dashboards.
* dashboard: HBLINK_TRANSPORT=unix connects via open_unix_connection(HBLINK_SOCKET).
A local Unix socket can't NIC-flap or be conntrack-evicted, retiring the whole
silent-severance class for local dashboards (filesystem perms replace the IP ACL;
TCP keepalive is moot).
Also: ping-cadence calibration now takes the MEDIAN of the first PING_WARMUP
gaps (robust to an odd first interval), counting loss only once the cadence
is known.
Tests updated for the new vocabulary; fixes an unbounded-loop test bug in
test_loss_registers_then_ages_out (the window-fill loop's bound tracked the
list it was appending to -> infinite append -> OOM).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The exponential-decay (tau) metric was hard to explain. Replace it with a simple,
exact 'ping loss % over the last PING_LOSS_WINDOW minutes' -- easy to reason about
('3% loss over the last 5 minutes'). The gap-based, self-calibrated miss detection
is unchanged (a repeater that just pings slower still isn't read as lossy); only
the aggregation is now a pruned event window instead of a decaying counter.
Two [GLOBAL] knobs (optional, sane defaults keep existing configs working):
- PING_LOSS_WINDOW: minutes, default 5, clamped 1-60.
- PING_LOSS_WARN: gold-threshold %, default 5, advertised to the dashboard.
Documented in hblink-SAMPLE.cfg and CONFIGURING.md. Dashboard now reads the gold
threshold from config (was hard-coded 5). Dropped PING_QUALITY_TAU and the math.exp
import. Tests: clean/slower-cadence -> 0, dropped -> loss, and old loss ages out of
the window -> 0. 59 tests pass. Verified live at a picky 1min/1% (local hblink.cfg):
healthy repeaters read 0%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surfaces the case the retired warn light was a poor proxy for: a repeater that
stays connected but loses pings over time (lossy link -> choppy/dropped audio).
Server-side, at each received RPTPING we infer how many pings were lost in the
gap since the previous one, against a per-repeater self-calibrated cadence (so a
repeater that simply pings slower is not read as lossy). Received and missed
counts decay exponentially (PING_QUALITY_TAU, ~15 min) VU-meter style, so the
value reflects recent/chronic loss and rises/falls slowly. Exposed as PING_LOSS%
in each repeater's snapshot (rides the 60s resync -- it moves slowly, so that
cadence is fine); the dashboard shows '· N% ping loss' and ambers the callsign
past a threshold. History is keyed by radio id and survives reconnects, so a
flapping lossy repeater still accumulates a meaningful figure.
Tests: clean cadence -> 0%, slower cadence -> 0% (not mistaken for loss),
dropped pings -> clear loss. Verified live: real repeaters read 0%. 58 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The dashboard learned about repeater connect/disconnect only via the periodic
full-config push (up to REPORT_INTERVAL late), because there were no granular
peer events -- the full push WAS the change-propagation mechanism. Add proper
deltas so the dashboard updates immediately, as the first step toward demoting
that redundant full push.
- json_repeater() extracted (shared by the snapshot and the new event).
- ReportServer.send_peer() emits {'type':'peer', system, radio_id, action, info}.
- HBSYSTEM.report_peer_deltas(): centralized reconciliation sweep -- diffs
currently-connected peers vs last reported and emits only deltas, so it can't
miss any of the scattered add/remove paths and never floods. Called from the
server maintenance loop (every PING_TIME) and immediately on config completion
for snappy connect feedback.
- Dashboard backend applies the delta and re-broadcasts config (no browser-JS
change yet). Full push kept as fallback this step; Step 2 demotes it.
- Tests: connect emits one 'connected' w/ info, no duplicate on re-sweep,
timeout emits 'disconnected'. 55 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The test harness still used the old config MODE vocabulary (MASTER/PEER) that the
current schema no longer accepts, so build_config() silently dropped those
systems and ~33 tests failed (config not loaded). Migrate the fixtures and test
code to the current terms; the whole suite (52 tests) now passes.
- harness.cfg: MODE MASTER->SERVER, PEER->OUTBOUND; MAX_PEERS->MAX_REPEATERS;
MASTER_IP/PORT->SERVER_IP/PORT. System labels MASTER-1/-2 -> SERVER-1/-2
(REPEATER-1 kept; 'repeater' is not a dropped term).
- test code: _peers->_repeaters, send_master->send_server,
MASTER_SOCKADDR->SERVER_SOCKADDR, send_bridgeEvent->send_bridge_event mocks.
- harness.py: the 'mark connected' gate keyed on mode=='PEER' -> 'OUTBOUND', so
the router again forwards to outbound systems (fixes the routing/baseline
scenarios).
- Comments/identifiers describing our system roles moved off master/peer;
genuine HBP wire-protocol terms (peer-id, the DMRD peer field, the homebrew
login handshake) are left as-is.
- baseline_emissions.json regenerated consistently for the renamed systems.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The reporting tests referenced the removed Twisted reportFactory/send_clients/
bridgeReportFactory API. Rewrite them against the current ReportServer /
BridgeReportServer, and add coverage for the dead-client shedding added in the
keepalive fix (is_closing() and over-limit write buffer). Self-contained mocks,
no dependency on config fixtures.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When HBlink3 acts as a PEER, the router forwarded bridged call traffic to it
regardless of whether it had completed login to its upstream master. The frames
were fired at the master's address and silently dropped (we're not registered),
and the call falsely appeared as an active forward on the dashboard. This is
long-standing behavior (master/original hblink3 did the same) -- the new
event-sourced dashboard just made it visible.
Add HBSYSTEM.egress_ready() (True for masters/OpenBridges; for a PEER, True only
once CONNECTION == 'YES') and skip un-ready targets at the top of routerHBP
bridge_group / bridge_unit, so a disconnected peer gets no forwarded frames, no
TX report event, and no TX status churn. send_master also drops DMRD when not
connected as defense-in-depth, while still letting login/keepalive packets flow.
The routing harness models connected, participating systems, so it now marks its
PEER systems CONNECTION == 'YES' -- the 15-scenario byte baseline is unchanged.
Tests: tests/test_peer_egress.py (connected forwards, disconnected skipped) plus
a send_master drop/allow test; full suite 51 green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A slot's "stream has ended" state was inferred from RX_TYPE/TX_TYPE ==
HBPF_SLT_VTERM. But those fields hold the last frame's _dtype_vseq, which for
voice bursts cycles 0..5 and so collides with HBPF_SLT_VTERM (== 2). Any stream
whose last received burst was vseq 2 (burst "C") looked already-terminated, so:
- a dropped-RF "lost tail" ending on vseq 2 was never ended -- not by the
stream-timeout trimmer, nor by new-stream supersession -- leaking a stuck
call until a downstream consumer timed it out; and
- a live holder transmitting its vseq-2 burst momentarily looked terminated, so
a competing stream could steal the slot, producing spurious CALL STARTs and a
flood of per-frame collision warnings.
A properly terminated call always finishes a full superframe (last voice burst
F/vseq 5) then sends the terminator, so the clean-END path was usually fine --
this only bit lost tails and mid-call contention. The bug is long-standing
(present in master); the new event-sourced dashboard just made it visible.
Replace the RX_TYPE/TX_TYPE == VTERM inference with explicit RX_TERMINATED /
TX_TERMINATED flags, set False when a stream starts and True on a real
terminator (or synthesized end), across the group and unit, source and target
paths. RX_TYPE/TX_TYPE remain only as informational last-frame-type.
Verified live: lost-tail@vseq2 now times out cleanly; 3 overlapping peers x 4
calls yield 12 STARTs / 12 ENDs (was 72 / 4) with no stuck streams. All 47
unit tests pass, including the 15-scenario byte-identical routing baseline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix the root cause of the dashboard's phantom "active" streams: HBlink3 did not
guarantee a matching END for every START. When a new stream took over a timeslot
whose previous stream never sent a terminator, the slot state was simply
overwritten, and the trimmer only ever emitted an END for whichever stream
currently owned the slot -- superseded streams were dropped silently.
- routerHBP._end_slot_stream(slot): if a slot holds an un-terminated stream,
declare it ended and emit a matching END (with the correct GROUP/UNIT call
type, now tracked per slot as RX_CT). Idempotent.
- Call it at the new-stream adoption point in group_received and unit_received,
so a superseding stream first ends the one it replaces (restores the original
pre-supersession behavior; the overwrite-only version kept routing correct but
orphaned the call logs).
- Stream timeout dropped from 5s (polled every 5s) to STREAM_TIMEOUT=2s polled
every 1s: on RF, ~2s of no audio means sync is lost and any resumption is a new
stream (late entry), so the stream is over. The trimmer's RX timeout now routes
through _end_slot_stream too.
Report-only behavior; routing is unchanged (byte baseline still green). The
dashboard's stream reaper stays as a backstop for feed-disconnect gaps HBlink3
can't help with. Suite now 42 tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the old reporting wire format (pickled CONFIG/BRIDGES + CSV events,
netstring-framed) with newline-delimited JSON, in preparation for the new
dashboard. A consumer connects to the reporting socket and receives:
- a `config` snapshot on connect (and periodically): a JSON-safe, trimmed view
of the systems (masters with their connected peers, peers with stats,
openbridges) built by json_systems() -- bytes decoded, DMR IDs as integers,
passphrases and internal ACL structures omitted.
- a `bridges` snapshot: each conference bridge's members (system/ts/tgid/active/
timer/triggers) via json_bridges().
- `stream` events on call start/end (the send_bridgeEvent CSV is parsed centrally
into JSON, so the routing call sites are unchanged).
The report protocol is now plain NDJSON (one json object per line); consumers are
passive and get the initial snapshot pushed on connect (no CONFIG_REQ handshake).
Dropped pickle from hblink.py/bridge.py and a leftover `send_clients(b'bridge
updated')` debug line.
Note: this changes the reporting wire format, so the old HBmonitor (which this
new dashboard replaces) will no longer parse it. Verified live: a consumer
connecting to a running bridge.py receives the config snapshot (with the
connected peer) and bridge state as NDJSON. Tests updated; suite now 39.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pin the protocol mechanics the routing harness doesn't reach, against mock
transports (no real sockets):
- tests/test_protocol.py:
- master login handshake: RPTL -> challenge -> RPTK -> RPTC drives the master
state machine to CONNECTION='YES'; wrong passphrase is NAK'd and the peer
removed. Exercises datagram_received + transport.sendto.
- reporting: netstring framing round-trip, disallowed client is closed, and a
CONFIG_REQ (split across reads) triggers a CONFIG_SND push.
- transport send: OPENBRIDGE.send_system and peer send_master use
transport.sendto with the right address.
Full suite now 36 tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cort BuffingtonandClaude Opus 4.8