Major Performance Improvements:
- Set-based TG storage for O(1) lookups (was O(n))
- Per-stream routing cache (calculate once vs per-packet)
- 83% reduction in routing operations per stream
New Features:
- RPTO (Repeater Options) message handling
- Dynamic TG configuration via RPTO (config is master)
- Dashboard visual indicators for RPTO-supplied TGs
- Real-time dashboard updates when RPTO received
- Unregistered repeater NAK handling
Testing:
- 8 new routing optimization tests
- Full test coverage for all optimization features
- All 38 tests passing
Technical Details:
- RepeaterState: Added slot1_talkgroups/slot2_talkgroups as sets
- RepeaterState: Added rpto_received flag
- StreamState: Added target_repeaters cache and routing_cached flag
- New _handle_options() for RPTO parsing and config intersection
- New _calculate_stream_targets() for one-time routing calculation
- Optimized _forward_stream() to use cached routing
- Dashboard: Blue labels + tooltips for RPTO TGs
- Implement TCP and Unix socket transports (removed UDP per requirements)
- Add connection state awareness and automatic reconnection (10s interval)
- Fix TCP race condition: Initial state sync on (re)connection
- Add reconnect callback to send current repeater state to dashboard
- Update all configuration files with transport settings
- Add comprehensive documentation (dashboard_transport.md, TRANSPORT_IMPLEMENTATION_SUMMARY.md)
- Clean up debug logging
This change addresses all user requirements:
1. Dashboard can run on separate server (TCP transport)
2. Dashboard never shows stale data (reliable delivery + state sync)
3. Dashboard knows when HBlink is offline (connection state tracking)
4. Unix socket for local (fastest), TCP for remote (flexible)
5. Robust against connection timing issues (initial state sync)
Tested: Both TCP and Unix socket transports working correctly
- Add hang time special rendering: show only TG and call type (stream characteristics)
- Server includes last_heard list in stream events for real-time updates
- Fix card structure: proper div nesting and 15px spacing between sections
- Change repeater icon from 📡 (satellite) to 📶 (signal bars)
- Last Heard table now updates immediately during streams, not just after
Performance optimization: Removed periodic user cache updates from hblink
to dashboard (was running every 10 seconds). Dashboard now maintains its
own last heard list independently.
HBlink changes:
- Removed _send_user_cache() method and its 10-second LoopingCall task
- Eliminated last_heard_update event emission
- No more user cache serialization/transmission overhead
Dashboard server changes:
- Removed last_heard_update event handler
- Dashboard maintains own list of last 10 users (reduced from 50)
- List updates immediately on stream_start and stream_end only
Dashboard UI changes:
- Removed 'View Full Cache' button and modal
- Changed 'Active now' to clean green 'Active' badge
- Last Heard table now shows exactly 10 most recent users
- Simplified display without cache stats
Benefits:
- Zero overhead on hblink call processing
- Cleaner, simpler dashboard UI
- More responsive - updates on actual events, not 10-second polls
- Reduced memory usage (10 vs 50 user entries)
Performance optimization: All callsign lookups now happen in dashboard
server, keeping hblink.py focused on call processing with zero latency
impact.
Dashboard server changes:
- Enrich stream_start events with callsign from user.csv lookup
- Add users to last_heard immediately on stream_start with 'active' flag
- Mark users as inactive on stream_end
- Merge active flags when receiving last_heard_update from hblink
Dashboard UI changes:
- Get callsign directly from stream data (no cache lookup needed)
- Show '🔴 Active now' in Last Heard table for users currently transmitting
- Color active status in green (#10b981) vs gray for past activity
- Immediate callsign display in slot cards (no 10-second delay)
Result: Callsigns appear instantly in both slot cards and Last Heard
table, with clear visual indication of who is currently active.
Building on the success of the _end_stream() helper consolidation,
next session will focus on identifying and eliminating repetitive
code patterns across all files using helper functions.
Hang Time Improvements:
- Changed hang time from user-locked to talkgroup-protected
- Same user can now switch talkgroups immediately (defeats hang time legitimately)
- Multiple users can participate in same talkgroup conversation
- Different users on different talkgroups still blocked (hijacking protection)
- Added test coverage for all hang time scenarios
Logging Enhancements:
- Added [FAST TG SWITCH] indicator when same user switches TGs during hang time
- Tracks fast_tg_switch flag in _handle_stream_start()
Code Quality:
- Consolidated 4 stream ending locations into unified _end_stream() helper
- Single source of truth for: marking stream ended, logging, emitting events, updating stats
- Improved maintainability and consistency
Dashboard Improvements:
- Added slot-specific talkgroup display (TS1 and TS2 shown separately)
- Added repeater metadata: location, TX/RX frequencies, color code
- Implemented 3-column responsive grid layout for repeater cards
- Fixed frequency display format (XXX.XXXX MHz with proper padding)
- Fixed column widths to prevent shifting as connected time counts up
- Enhanced repeater_connected event to send additional metadata
All 30 tests passing.
Dashboard fix:
- Replace 'Packets' with 'Callsign' in repeater slot display
- Lookup callsign from last_heard cache by src_id
- Shows '—' if callsign not found
Hang time fix (CRITICAL):
- Allow same src/dst to break through hang time
- Prevents 9+ second delay when same user continues talking
- Hang time now only blocks DIFFERENT conversations
- Checked by comparing rf_src and dst_id during hang time
Issue reproduced:
- User keys up, hang time starts on TX slot
- User keys up again (new stream_id but same src/dst)
- Old behavior: blocked for full 10s hang time
- New behavior: immediately forwards (same conversation)
Example from logs:
- Stream ends at 15:42:36, hang time starts
- New stream starts at 15:42:37 (same src 3120101, dst 9)
- OLD: waited until 15:42:46 (9s delay)
- NEW: forwards immediately (same conversation continuing)
Updated comments to make it clear that:
- Missing callsigns fail silently (no logging)
- Empty callsign displays as "-" in dashboard
- Dimmed gray color indicates no match
- This is a convenience feature, not critical
No functional changes.
New utility script:
- scripts/filter_user_csv.py - Filter DMR user database
- scripts/README.md - Documentation and usage examples
Features:
- Process full user database to extract US/Canada only
- Automatic statistics and file size reporting
- Can overwrite input or create new output file
- Preserves all CSV columns
Usage:
python3 scripts/filter_user_csv.py user_full.csv user.csv
Benefits:
- Reproducible filtering for future database updates
- Clear documentation for maintenance
- Easy to add more countries if needed
Dashboard enhancements:
- Load user.csv with 132,999 US and Canada users
- Memory-efficient storage: Dict[int, str] (callsign only)
- Automatic callsign lookup in last_heard updates
- ~9.5 MB memory usage vs ~21 MB for full database
User database:
- Filtered to US and Canada only (54.4% reduction)
- File size: 7.3 MB (down from 14.8 MB)
- 132,999 entries (down from 291,968)
- O(1) lookup performance
Benefits:
- Dashboard now shows callsigns in Last Heard table
- No external API calls needed
- Fast in-memory lookups
- Reasonable memory footprint for target networks
Code cleanup:
- Remove stale LC extraction comment from imports
- Remove Phase 1/2/3 references from comments (work complete)
- Simplify verbose architecture notes to concise descriptions
- Remove TODO for RSSI parsing (not implementing)
- Remove 'Critical:' and 'Must' tags from comments
Dashboard cleanup:
- Remove talker_alias display (replaced with packet count in slot view)
- Update Last Heard to show callsign from user_cache (not talker_alias)
- Remove talker_alias from stream_start event formatting
Documentation:
- Update IMPLEMENTATION_SUMMARY git history (remove LC reference)
All 29 tests passing. No functional changes.
Performance Monitoring:
- Focus on latency and jitter as most critical metrics
- Remove packets/sec and active streams (not issues)
- Latency/jitter through hblink4 is key quality indicator
Removed items not aligned with design:
- Multi-Server Clustering (design is endpoint, no scaling needed)
- Mobile App Integration (web-only, mobile optimization in dashboard)
Items now numbered 1-5 (was 1-7)
- Removed LC Extraction from Sync Frames from Medium Priority
- Removed 'LC Decoding Clarification' section
- Removed 'DMR Link Control Extraction' from completed features
- LC functionality is abandoned going forward
- Renumbered remaining items 1-9 (was 1-10)
Issue:
- StreamState object has no attribute 'talker_alias' error
- Line 1094 still referenced removed talker_alias field
- Caused error every 60 packets (1 second intervals) during stream updates
Fix:
- Removed 'talker_alias' field from stream_update event emission
- Field was removed from StreamState dataclass during Phase 1 cleanup
- Event now only includes: repeater_id, slot, src_id, dst_id, duration, packets, call_type
Impact:
- No more attribute errors during stream updates
- Dashboard will handle alias lookup independently (as intended)
- Consistent with Phase 1 cleanup goals
Validation:
- All 29 tests passing
- No syntax errors
Critical Issue:
- After Phase 2 cleanup, current_stream variable was undefined in _handle_dmr_data scope
- Variable was only defined locally within _handle_stream_packet function
- Caused 'name current_stream is not defined' errors on every packet
- Prevented stream processing and forwarding
Root Cause:
- Lines 1057, 1083, 1090, 1091 in _handle_dmr_data all reference current_stream
- But current_stream was never retrieved after _handle_stream_packet call
- Previous code likely had it defined somewhere that was removed during cleanup
Fix:
- Added: current_stream = repeater.get_slot_stream(_slot)
- Placed after _handle_stream_packet call (line 1047)
- Ensures current_stream is defined before being used for:
- Terminator detection (line 1057)
- Stream update events (line 1083)
- Duration calculations (line 1090)
Impact:
- Stream processing now works correctly
- No more undefined variable errors
- Forwarding and event emission restored
Validation:
- All 29 tests still passing
- No new errors introduced
Access Control Fix (CRITICAL BUG):
- Fixed RepeaterConfig.talkgroups not being populated from config
- Issue: talkgroups field had default_factory=list which prevented initialization
- Solution: Changed to None with __post_init__ that populates from slot2_talkgroups for backward compatibility
- Impact: Talkgroups now properly applied to repeater configurations
Hang Time Test Fixes:
- Tests were setting stream.ended=True without stream.end_time
- is_in_hang_time() requires both ended=True AND end_time set
- Fixed all test cases to properly set end_time when marking streams as ended
- Production code already sets both fields correctly
Test Expectation Fix:
- Updated test_default_config to expect [9] not [8]
- Default config has slot2_talkgroups=[9], which populates talkgroups field
- Test was checking wrong slot's talkgroups
Results:
- All 29 tests now pass (was 23 passing, 6 failing)
- Fixed critical bug affecting repeater talkgroup configuration
- Tests now validate actual production behavior
Dead code removed:
- bhex() function (only used once, now inlined)
- get_user_cache_data() method (never called, duplicated functionality)
- Unused dmr_utils3 imports (decode, bptc - were for LC extraction)
Documentation cleanup:
- Remove all LC (Link Control) references from docs
- Remove all talker alias references from docs (changed to 'alias' in dashboard docs)
- Update IMPLEMENTATION_SUMMARY.md to remove LC/talker alias sections
- Update readme.md to remove LC/talker alias feature mentions
- Update DASHBOARD_INTEGRATION.md to remove talker_alias from event parameters
- Update docs/TODO.md to remove completed talker alias items
All changes validated:
- No linting errors in hblink.py
- 23 tests passing (6 pre-existing failures unrelated to these changes)
- Code is cleaner and more maintainable
Architectural decision: Callsign lookup will be handled entirely in the
dashboard (presentation layer), not in hblink (data layer).
Changes:
- Removed 'alias' field from StreamState dataclass
- Removed 'alias' from stream_start and stream_end events
- Removed TODO comments about callsign lookup in hblink
- Simplified user_cache.update() call
Rationale:
- Keep data layer (hblink) clean and focused on DMR protocol
- Handle presentation/lookup concerns in dashboard only
- Dashboard can lookup src_id -> callsign when rendering
- No need to store or pass alias through hblink core
The dashboard will implement its own callsign lookup file loading
and will enrich the display data with callsigns when rendering.
Critical bug fix:
- RX terminator was not setting end_time, causing hang_duration=0.00s
- Added current_stream.end_time = time() to RX terminator handler
- This fixes hang times ending immediately instead of waiting 10s
Log message consistency:
- Changed all TX logs to use 'dst=' instead of 'tgid=' to match RX
- Added 'entering hang time' message to TX terminator to match RX
- All stream end logs now have identical format
Before fix:
- TX hang time completed: hang_duration=10.52s (correct)
After fix:
- Both RX and TX should show hang_duration=10.00s
Example logs:
- RX stream started: src=..., dst=...
- TX stream started: from repeater X, src=..., dst=...
- RX stream ended: reason=terminator - entering hang time (10.0s)
- TX stream ended: reason=terminator - entering hang time (10.0s)
- RX hang time completed: hang_duration=10.00s
- TX hang time completed: hang_duration=10.00s
Log message consistency:
- Changed 'Stream started' to 'RX stream started'
- Changed 'Forwarding stream to' to 'TX stream started'
- Now all logs use consistent RX/TX prefixes
Hang time visibility:
- Promoted hang time completion logs from DEBUG to INFO
- Added hang_duration calculation and logging
- Shows src, dst, and actual hang time duration
- Helps diagnose if hang times are expiring early
Example logs:
- RX stream started on repeater X slot Y: src=..., dst=...
- TX stream started on repeater X slot Y: from repeater Z, src=..., tgid=...
- RX stream ended on repeater X slot Y: ..., reason=terminator - entering hang time (10.0s)
- RX hang time completed on repeater X slot Y: ..., hang_duration=10.00s
- TX hang time completed on repeater X slot Y: ..., hang_duration=10.00s
This will help diagnose the reported issue of hang times appearing to end in 2-3s.
Logging improvements:
- All stream end logs now prefixed with RX or TX for clarity
- TX terminator logs promoted from DEBUG to INFO level
- Added 'reason=' to all end logs (terminator or timeout)
- Consistent format across all stream lifecycle events
Dashboard improvements:
- Changed TX color from orange to red (#ef4444)
- More visually distinct from other states
Example logs:
- RX stream ended: reason=terminator - entering hang time
- TX stream ended: reason=terminator (no hang time on TX)
- RX stream ended: reason=timeout - entering hang time
- TX stream ended: reason=timeout
Colors: Green=RX, Red=TX, Blue=Hang
Logging improvements:
- Stream end logs now show 'RX stream ended' or 'TX stream ended'
- Clear distinction between received and forwarded streams
Dashboard improvements:
- TX (forwarded) streams now show red/orange with ⬆️ TX indicator
- RX (received) streams show green with ⬇️ RX indicator
- Uses is_assumed flag to determine stream direction
- Proper color coding: green=receiving, orange=transmitting, blue=hang
Fixes issue where forwarded streams incorrectly showed as RX/green
- Log forwarding start at INFO: 'Forwarding stream to repeater X slot Y'
- Log forwarding end at INFO with duration and packet count
- Emit stream_start/stream_end events for dashboard visibility
- Increment/decrement forwarding stats (active_calls, total_calls_today)
- Target repeaters now show TX state in dashboard
- One log per target repeater (clear but not noisy)
Fixes: Audio forwarding worked but had no logs or dashboard visibility
- Changed is_in_hang_time() to calculate from end_time, not last_seen
- Fixes bug where denied packets during hang time prevent expiration
- Hang time now expires correctly regardless of invalid traffic
- Simplified logic: time_since_end < hang_time
- Added _denied_streams cache to track denied stream_ids
- Only log on first packet of denied stream, not every packet
- Cleanup old denied entries in stream timeout checker
- Now logs once per denied transmission instead of per packet
- Moved logging from _check_inbound_routing() to call site
- Now logs only when stream starts, not on every routing check
- Still shows repeater ID, TS/TGID, and allowed list for debugging
- Added warning log in _check_inbound_routing() when traffic is denied
- Shows repeater ID, TS/TGID combo, and allowed list for troubleshooting
- Removed duplicate log at call site to avoid noise
- Outbound routing remains silent (normal routing behavior)
- Changed match_repeater() to get_repeater_config() (correct method name)
- Access RepeaterConfig object attributes directly (slot1_talkgroups, slot2_talkgroups)
- Removed unnecessary dict.get() calls since config is always returned
- Simplified slot talkgroup list access with if/else instead of f-string key
This fixes AttributeError when processing DMR data packets.
- Added slot1_talkgroups and slot2_talkgroups fields
- Kept talkgroups field for backward compatibility (deprecated)
- Use field(default_factory=list) for proper dataclass mutable defaults
- Import field from dataclasses
This allows config patterns to use the new slot-specific routing.
Major changes:
- Replaced ambiguous 'talkgroups' list with slot1_talkgroups/slot2_talkgroups
- Simplified routing: same lists control inbound and outbound (no separate rules)
- No wildcards or rewriting (simplicity over flexibility)
Configuration updates:
- Updated config.json with real KS-DMR settings (TS1: TG2,8,9 / TS2: TG3120)
- Updated config_sample.json with slot-specific examples
- Created docs/routing.md with complete documentation
Code changes:
- Added _check_inbound_routing(): Verify repeater can send on TS/TGID
- Added _check_outbound_routing(): Verify repeater should receive on TS/TGID
- Added _is_slot_busy(): Contention detection with hang time support
- Added _update_assumed_stream(): Track streams forwarded to targets
- Updated _forward_stream(): Full Phase 3 implementation with routing + contention
- Updated StreamState: Added is_assumed flag and end_time field
Routing logic:
- Inbound: Only accept traffic matching repeater's slot_N_talkgroups
- Outbound: Only forward to repeaters with matching slot_N_talkgroups
- Contention: Block busy slots, respect hang time after stream ends
- Assumed streams: Track forwarded streams on target repeaters
Ready for real-world testing with multiple repeaters!
- Added date import and daily stats reset functionality to hblink.py
- _reset_daily_stats() checks for date change and resets total_calls_today
- Daily reset task runs every 60 seconds to check for midnight rollover
- Tracks last_reset_date to prevent multiple resets on same day
- Reordered stats display: moved 'Total Traffic Today' to far right
- All 'today' stats now properly reset at midnight based on server OS time
Stats order is now:
1. Connected Repeaters
2. Active Streams
3. Total Streams Today
4. Active Forwarded Calls
5. Total Calls Today
6. Total Traffic Today (moved to far right)
Changed text from 'Disconnected' to '🔄 Click to Reconnect' to make it
clear that clicking the red badge will attempt to reconnect to the server.
The badge already had cursor:pointer and hover effects, but the text
didn't make the interactivity obvious to users.
Expanded on the rationale behind not implementing transit call routing in HBlink4, emphasizing autonomy for smaller networks and the rejection of centralized control over endpoint IDs.
Explains the fundamental change from transit routing to endpoint server model:
- HBlink3 era: Transit routing required (pre-DMRGateway)
- Modern era: DMRGateway enables direct multi-server connectivity
- HBlink4 focus: Endpoint server with per-repeater TS/TGID routing
This contextualizes why HBlink4 will not implement transit call routing
and why the architecture emphasizes granular per-repeater control rather
than server-level system groupings.
Written in first person as this explains design decisions I made in both
HBlink3 and HBlink4 based on the evolving DMR networking landscape.
The JavaScript was looking for r.radio_id but repeater objects now use
repeater_id field. This caused:
- Repeater ID showing as (unknown)
- Streams not displaying on repeater cards (lookup failed)
Fixed in updateRepeaters() function:
- Stream lookups: r.radio_id → r.repeater_id
- Display header: r.radio_id → r.repeater_id
Note: User cache display (lines 741, 794) correctly uses radio_id for
subscriber IDs and was not changed.
Dashboard was expecting data['radio_id'] but HBlink now sends data['repeater_id']
in all repeater-related events (repeater_connected, repeater_keepalive,
repeater_disconnected).
This fixes the error: "Error processing event: 'radio_id'"
The UserCache.update() method expects 'radio_id' parameter for the user's
radio ID, not 'subscriber_id'. In the UserCache context, radio_id refers
to the user/subscriber, which is documented in the method signature.
Added clarifying comments: radio_id=src_id # User's radio ID (subscriber)
This fixes the runtime error:
'UserCache.update() got an unexpected keyword argument subscriber_id'
- Fixed _send_forwarding_stats() method (was still named _broadcast_forwarding_stats)
- Updated stats emission to use active_calls/total_calls_today
- Restored dashboard/server.py from corruption (bad replacement)
- Properly implemented all send_to_clients renamings
- Added forwarding_stats event handler to dashboard
- Both services now start successfully
Services tested and working:
✅ run.py starts without errors
✅ run_dashboard.py starts without errors
- Updated tests to use HBP frame type flags instead of ETSI sync patterns
- HBP terminators: frame_type=0x2 (DATA_SYNC) + dtype_vseq=0x2 (SLT_VTERM)
- Added length check to _is_dmr_terminator() to handle short packets safely
- Added new tests: data_terminator, wrong_dtype_vseq
- Updated existing tests with correct byte 15 flag values
- All 6 terminator tests now pass (was 1 failing)
- Tests properly reflect ~60ms immediate terminator detection via HBP header
cort
Cort Buffington