- Add hblink4.service for main DMR server
- Add hblink4-dash.service for web dashboard
- Dashboard service depends on main service
- Both use venv at /home/cort/hblink4/venv
- Auto-restart on failure with 10s delay
- Include security hardening (NoNewPrivileges, PrivateTmp)
- Log to systemd journal
- Add SYSTEMD.md with installation and management instructions
- Reduce metadata first column to 140px (was 180px)
- Reduce column gap to 12px (was 20px)
- Update formatDuration to show days/years:
- < 24 hours: H:MM:SS format
- >= 24 hours: DDDd HH:MM format (no seconds)
- >= 365 days: Yy DDd format
- More space for talkgroup lists
- Cleaner, more compact repeater info section
- Right-align slot cards on desktop for consistent positioning
- Use fixed optimal column widths based on max expected content:
- Slot: 55px (label + badge)
- Source/Dest: 70/75px (24-bit int, max 16777215)
- Type: 110px ('Private Voice' is longest)
- Duration: 55px (max 600s for 10 minutes)
- Callsign: 70px (max 6 characters)
- Slot cards shrink to ~495px on desktop (right-aligned)
- Repeater info capped at 400px width
- Slots stay in consistent position regardless of TG list length
- Full-width slots on mobile/tablet (responsive)
- Progressive column hiding on smaller screens
- Improved header spacing throughout all breakpoints
Clarify the HomeBrew DMR protocol implementation details and correct errors in the keepalive mechanism. Update the table formatting for configuration data fields and DMRD packet structure.
- Document RPTO message format and purpose
- Explain talkgroup negotiation and intersection logic
- Note that server config is master allow list
- Include examples of RPTO usage and filtering
- Completes protocol documentation for all implemented commands
Implement transport abstraction for dashboard communication, supporting TCP and Unix socket transports while removing UDP. Update configuration files and documentation accordingly.
Updated implementation summary with changes to stream tracking, hang time, and comprehensive documentation. Removed sections on modified files and configuration examples for clarity.
Major changes:
- Comprehensive documentation audit completed
- Config restructure: dashboard moved from global to top-level
- Dual-stack IPv6: bind_ipv4/bind_ipv6 with port_ipv4/port_ipv6
- Dashboard transport: host_ipv4/host_ipv6 with disable_ipv6 flag
- Fixed corrupted dashboard/server.py imports
- Updated all documentation to match actual implementation
- Removed duplication, established single source of truth
Documentation updates:
- docs/configuration.md: Complete overhaul with accurate field definitions
- docs/dashboard_transport.md: Updated transport configuration guide
- docs/integration.md: Dual-stack integration examples
- dashboard/README.md: Clarified dashboard configuration requirements
- DASHBOARD_INTEGRATION.md: Streamlined integration summary
- All config examples now use correct field names
Configuration changes:
- dashboard section moved to top-level (not nested under global)
- IPv6 settings: disable_ipv6 flag replaces ipv6 boolean
- Separate bind addresses and ports for IPv4 and IPv6
- Dashboard uses host_ipv4/host_ipv6 instead of single host field
Bug fixes:
- Fixed syntax error in dashboard/server.py (corrupted imports)
- All JSON configs validate successfully
- All 39 tests passing
See DOCUMENTATION_AUDIT.md for complete audit report.
Problem:
- When we TX to a repeater (assumed stream) and it starts RX'ing
- The repeater ignores our TX packets (hardware busy receiving)
- We waste bandwidth sending packets to a busy repeater
Solution:
- Detect when repeater has assumed stream and receives real stream
- Remove that repeater from all active route-caches
- Stop sending to it immediately (bandwidth saved)
- Real RX streams always win over assumed TX streams
Implementation:
- Added check in _handle_stream_start() for is_assumed flag
- Iterate all streams and remove repeater from target_repeaters
- O(R×S) efficiency (~10-20 operations typical)
- Falls through to create real stream normally
Testing:
- Added test_assumed_stream_route_cache_removal()
- 39 tests passing (38 existing + 1 new)
- Validates route-cache removal logic
Documentation:
- Added section in stream_tracking.md
- Added detailed diagrams in stream_tracking_diagrams.md
- Explains real vs assumed stream priority
- Includes performance analysis and examples
Major Performance Improvements:
- Set-based TG storage for O(1) lookups (was O(n))
- Per-stream routing cache (calculate once vs per-packet)
- 83% reduction in routing operations per stream
New Features:
- RPTO (Repeater Options) message handling
- Dynamic TG configuration via RPTO (config is master)
- Dashboard visual indicators for RPTO-supplied TGs
- Real-time dashboard updates when RPTO received
- Unregistered repeater NAK handling
Testing:
- 8 new routing optimization tests
- Full test coverage for all optimization features
- All 38 tests passing
Technical Details:
- RepeaterState: Added slot1_talkgroups/slot2_talkgroups as sets
- RepeaterState: Added rpto_received flag
- StreamState: Added target_repeaters cache and routing_cached flag
- New _handle_options() for RPTO parsing and config intersection
- New _calculate_stream_targets() for one-time routing calculation
- Optimized _forward_stream() to use cached routing
- Dashboard: Blue labels + tooltips for RPTO TGs
- Implement TCP and Unix socket transports (removed UDP per requirements)
- Add connection state awareness and automatic reconnection (10s interval)
- Fix TCP race condition: Initial state sync on (re)connection
- Add reconnect callback to send current repeater state to dashboard
- Update all configuration files with transport settings
- Add comprehensive documentation (dashboard_transport.md, TRANSPORT_IMPLEMENTATION_SUMMARY.md)
- Clean up debug logging
This change addresses all user requirements:
1. Dashboard can run on separate server (TCP transport)
2. Dashboard never shows stale data (reliable delivery + state sync)
3. Dashboard knows when HBlink is offline (connection state tracking)
4. Unix socket for local (fastest), TCP for remote (flexible)
5. Robust against connection timing issues (initial state sync)
Tested: Both TCP and Unix socket transports working correctly
- Add hang time special rendering: show only TG and call type (stream characteristics)
- Server includes last_heard list in stream events for real-time updates
- Fix card structure: proper div nesting and 15px spacing between sections
- Change repeater icon from 📡 (satellite) to 📶 (signal bars)
- Last Heard table now updates immediately during streams, not just after
Performance optimization: Removed periodic user cache updates from hblink
to dashboard (was running every 10 seconds). Dashboard now maintains its
own last heard list independently.
HBlink changes:
- Removed _send_user_cache() method and its 10-second LoopingCall task
- Eliminated last_heard_update event emission
- No more user cache serialization/transmission overhead
Dashboard server changes:
- Removed last_heard_update event handler
- Dashboard maintains own list of last 10 users (reduced from 50)
- List updates immediately on stream_start and stream_end only
Dashboard UI changes:
- Removed 'View Full Cache' button and modal
- Changed 'Active now' to clean green 'Active' badge
- Last Heard table now shows exactly 10 most recent users
- Simplified display without cache stats
Benefits:
- Zero overhead on hblink call processing
- Cleaner, simpler dashboard UI
- More responsive - updates on actual events, not 10-second polls
- Reduced memory usage (10 vs 50 user entries)
Performance optimization: All callsign lookups now happen in dashboard
server, keeping hblink.py focused on call processing with zero latency
impact.
Dashboard server changes:
- Enrich stream_start events with callsign from user.csv lookup
- Add users to last_heard immediately on stream_start with 'active' flag
- Mark users as inactive on stream_end
- Merge active flags when receiving last_heard_update from hblink
Dashboard UI changes:
- Get callsign directly from stream data (no cache lookup needed)
- Show '🔴 Active now' in Last Heard table for users currently transmitting
- Color active status in green (#10b981) vs gray for past activity
- Immediate callsign display in slot cards (no 10-second delay)
Result: Callsigns appear instantly in both slot cards and Last Heard
table, with clear visual indication of who is currently active.
Building on the success of the _end_stream() helper consolidation,
next session will focus on identifying and eliminating repetitive
code patterns across all files using helper functions.
Hang Time Improvements:
- Changed hang time from user-locked to talkgroup-protected
- Same user can now switch talkgroups immediately (defeats hang time legitimately)
- Multiple users can participate in same talkgroup conversation
- Different users on different talkgroups still blocked (hijacking protection)
- Added test coverage for all hang time scenarios
Logging Enhancements:
- Added [FAST TG SWITCH] indicator when same user switches TGs during hang time
- Tracks fast_tg_switch flag in _handle_stream_start()
Code Quality:
- Consolidated 4 stream ending locations into unified _end_stream() helper
- Single source of truth for: marking stream ended, logging, emitting events, updating stats
- Improved maintainability and consistency
Dashboard Improvements:
- Added slot-specific talkgroup display (TS1 and TS2 shown separately)
- Added repeater metadata: location, TX/RX frequencies, color code
- Implemented 3-column responsive grid layout for repeater cards
- Fixed frequency display format (XXX.XXXX MHz with proper padding)
- Fixed column widths to prevent shifting as connected time counts up
- Enhanced repeater_connected event to send additional metadata
All 30 tests passing.
Dashboard fix:
- Replace 'Packets' with 'Callsign' in repeater slot display
- Lookup callsign from last_heard cache by src_id
- Shows '—' if callsign not found
Hang time fix (CRITICAL):
- Allow same src/dst to break through hang time
- Prevents 9+ second delay when same user continues talking
- Hang time now only blocks DIFFERENT conversations
- Checked by comparing rf_src and dst_id during hang time
Issue reproduced:
- User keys up, hang time starts on TX slot
- User keys up again (new stream_id but same src/dst)
- Old behavior: blocked for full 10s hang time
- New behavior: immediately forwards (same conversation)
Example from logs:
- Stream ends at 15:42:36, hang time starts
- New stream starts at 15:42:37 (same src 3120101, dst 9)
- OLD: waited until 15:42:46 (9s delay)
- NEW: forwards immediately (same conversation continuing)
Updated comments to make it clear that:
- Missing callsigns fail silently (no logging)
- Empty callsign displays as "-" in dashboard
- Dimmed gray color indicates no match
- This is a convenience feature, not critical
No functional changes.
New utility script:
- scripts/filter_user_csv.py - Filter DMR user database
- scripts/README.md - Documentation and usage examples
Features:
- Process full user database to extract US/Canada only
- Automatic statistics and file size reporting
- Can overwrite input or create new output file
- Preserves all CSV columns
Usage:
python3 scripts/filter_user_csv.py user_full.csv user.csv
Benefits:
- Reproducible filtering for future database updates
- Clear documentation for maintenance
- Easy to add more countries if needed
Dashboard enhancements:
- Load user.csv with 132,999 US and Canada users
- Memory-efficient storage: Dict[int, str] (callsign only)
- Automatic callsign lookup in last_heard updates
- ~9.5 MB memory usage vs ~21 MB for full database
User database:
- Filtered to US and Canada only (54.4% reduction)
- File size: 7.3 MB (down from 14.8 MB)
- 132,999 entries (down from 291,968)
- O(1) lookup performance
Benefits:
- Dashboard now shows callsigns in Last Heard table
- No external API calls needed
- Fast in-memory lookups
- Reasonable memory footprint for target networks
Code cleanup:
- Remove stale LC extraction comment from imports
- Remove Phase 1/2/3 references from comments (work complete)
- Simplify verbose architecture notes to concise descriptions
- Remove TODO for RSSI parsing (not implementing)
- Remove 'Critical:' and 'Must' tags from comments
Dashboard cleanup:
- Remove talker_alias display (replaced with packet count in slot view)
- Update Last Heard to show callsign from user_cache (not talker_alias)
- Remove talker_alias from stream_start event formatting
Documentation:
- Update IMPLEMENTATION_SUMMARY git history (remove LC reference)
All 29 tests passing. No functional changes.
Performance Monitoring:
- Focus on latency and jitter as most critical metrics
- Remove packets/sec and active streams (not issues)
- Latency/jitter through hblink4 is key quality indicator
Removed items not aligned with design:
- Multi-Server Clustering (design is endpoint, no scaling needed)
- Mobile App Integration (web-only, mobile optimization in dashboard)
Items now numbered 1-5 (was 1-7)
- Removed LC Extraction from Sync Frames from Medium Priority
- Removed 'LC Decoding Clarification' section
- Removed 'DMR Link Control Extraction' from completed features
- LC functionality is abandoned going forward
- Renumbered remaining items 1-9 (was 1-10)
Issue:
- StreamState object has no attribute 'talker_alias' error
- Line 1094 still referenced removed talker_alias field
- Caused error every 60 packets (1 second intervals) during stream updates
Fix:
- Removed 'talker_alias' field from stream_update event emission
- Field was removed from StreamState dataclass during Phase 1 cleanup
- Event now only includes: repeater_id, slot, src_id, dst_id, duration, packets, call_type
Impact:
- No more attribute errors during stream updates
- Dashboard will handle alias lookup independently (as intended)
- Consistent with Phase 1 cleanup goals
Validation:
- All 29 tests passing
- No syntax errors
Critical Issue:
- After Phase 2 cleanup, current_stream variable was undefined in _handle_dmr_data scope
- Variable was only defined locally within _handle_stream_packet function
- Caused 'name current_stream is not defined' errors on every packet
- Prevented stream processing and forwarding
Root Cause:
- Lines 1057, 1083, 1090, 1091 in _handle_dmr_data all reference current_stream
- But current_stream was never retrieved after _handle_stream_packet call
- Previous code likely had it defined somewhere that was removed during cleanup
Fix:
- Added: current_stream = repeater.get_slot_stream(_slot)
- Placed after _handle_stream_packet call (line 1047)
- Ensures current_stream is defined before being used for:
- Terminator detection (line 1057)
- Stream update events (line 1083)
- Duration calculations (line 1090)
Impact:
- Stream processing now works correctly
- No more undefined variable errors
- Forwarding and event emission restored
Validation:
- All 29 tests still passing
- No new errors introduced
Access Control Fix (CRITICAL BUG):
- Fixed RepeaterConfig.talkgroups not being populated from config
- Issue: talkgroups field had default_factory=list which prevented initialization
- Solution: Changed to None with __post_init__ that populates from slot2_talkgroups for backward compatibility
- Impact: Talkgroups now properly applied to repeater configurations
Hang Time Test Fixes:
- Tests were setting stream.ended=True without stream.end_time
- is_in_hang_time() requires both ended=True AND end_time set
- Fixed all test cases to properly set end_time when marking streams as ended
- Production code already sets both fields correctly
Test Expectation Fix:
- Updated test_default_config to expect [9] not [8]
- Default config has slot2_talkgroups=[9], which populates talkgroups field
- Test was checking wrong slot's talkgroups
Results:
- All 29 tests now pass (was 23 passing, 6 failing)
- Fixed critical bug affecting repeater talkgroup configuration
- Tests now validate actual production behavior
Dead code removed:
- bhex() function (only used once, now inlined)
- get_user_cache_data() method (never called, duplicated functionality)
- Unused dmr_utils3 imports (decode, bptc - were for LC extraction)
Documentation cleanup:
- Remove all LC (Link Control) references from docs
- Remove all talker alias references from docs (changed to 'alias' in dashboard docs)
- Update IMPLEMENTATION_SUMMARY.md to remove LC/talker alias sections
- Update readme.md to remove LC/talker alias feature mentions
- Update DASHBOARD_INTEGRATION.md to remove talker_alias from event parameters
- Update docs/TODO.md to remove completed talker alias items
All changes validated:
- No linting errors in hblink.py
- 23 tests passing (6 pre-existing failures unrelated to these changes)
- Code is cleaner and more maintainable
Architectural decision: Callsign lookup will be handled entirely in the
dashboard (presentation layer), not in hblink (data layer).
Changes:
- Removed 'alias' field from StreamState dataclass
- Removed 'alias' from stream_start and stream_end events
- Removed TODO comments about callsign lookup in hblink
- Simplified user_cache.update() call
Rationale:
- Keep data layer (hblink) clean and focused on DMR protocol
- Handle presentation/lookup concerns in dashboard only
- Dashboard can lookup src_id -> callsign when rendering
- No need to store or pass alias through hblink core
The dashboard will implement its own callsign lookup file loading
and will enrich the display data with callsigns when rendering.
Critical bug fix:
- RX terminator was not setting end_time, causing hang_duration=0.00s
- Added current_stream.end_time = time() to RX terminator handler
- This fixes hang times ending immediately instead of waiting 10s
Log message consistency:
- Changed all TX logs to use 'dst=' instead of 'tgid=' to match RX
- Added 'entering hang time' message to TX terminator to match RX
- All stream end logs now have identical format
Before fix:
- TX hang time completed: hang_duration=10.52s (correct)
After fix:
- Both RX and TX should show hang_duration=10.00s
Example logs:
- RX stream started: src=..., dst=...
- TX stream started: from repeater X, src=..., dst=...
- RX stream ended: reason=terminator - entering hang time (10.0s)
- TX stream ended: reason=terminator - entering hang time (10.0s)
- RX hang time completed: hang_duration=10.00s
- TX hang time completed: hang_duration=10.00s
Log message consistency:
- Changed 'Stream started' to 'RX stream started'
- Changed 'Forwarding stream to' to 'TX stream started'
- Now all logs use consistent RX/TX prefixes
Hang time visibility:
- Promoted hang time completion logs from DEBUG to INFO
- Added hang_duration calculation and logging
- Shows src, dst, and actual hang time duration
- Helps diagnose if hang times are expiring early
Example logs:
- RX stream started on repeater X slot Y: src=..., dst=...
- TX stream started on repeater X slot Y: from repeater Z, src=..., tgid=...
- RX stream ended on repeater X slot Y: ..., reason=terminator - entering hang time (10.0s)
- RX hang time completed on repeater X slot Y: ..., hang_duration=10.00s
- TX hang time completed on repeater X slot Y: ..., hang_duration=10.00s
This will help diagnose the reported issue of hang times appearing to end in 2-3s.
Logging improvements:
- All stream end logs now prefixed with RX or TX for clarity
- TX terminator logs promoted from DEBUG to INFO level
- Added 'reason=' to all end logs (terminator or timeout)
- Consistent format across all stream lifecycle events
Dashboard improvements:
- Changed TX color from orange to red (#ef4444)
- More visually distinct from other states
Example logs:
- RX stream ended: reason=terminator - entering hang time
- TX stream ended: reason=terminator (no hang time on TX)
- RX stream ended: reason=timeout - entering hang time
- TX stream ended: reason=timeout
Colors: Green=RX, Red=TX, Blue=Hang
Logging improvements:
- Stream end logs now show 'RX stream ended' or 'TX stream ended'
- Clear distinction between received and forwarded streams
Dashboard improvements:
- TX (forwarded) streams now show red/orange with ⬆️ TX indicator
- RX (received) streams show green with ⬇️ RX indicator
- Uses is_assumed flag to determine stream direction
- Proper color coding: green=receiving, orange=transmitting, blue=hang
Fixes issue where forwarded streams incorrectly showed as RX/green
- Log forwarding start at INFO: 'Forwarding stream to repeater X slot Y'
- Log forwarding end at INFO with duration and packet count
- Emit stream_start/stream_end events for dashboard visibility
- Increment/decrement forwarding stats (active_calls, total_calls_today)
- Target repeaters now show TX state in dashboard
- One log per target repeater (clear but not noisy)
Fixes: Audio forwarding worked but had no logs or dashboard visibility
cort
Cort Buffington