Commit Graph
82 Commits
Author SHA1 Message Date
cort 3bc348204e Work to add outbound connections. Monitoring in the dashboard has proven problematic. This is a manual rollback to remove much debugging code and start fresh. 2025-11-08 19:41:17 -06:00
Cort Buffington cd1452c716 Remove talkgroup definitions from configuration docs
Removed the 'Talkgroup Definitions' and 'Talkgroup Options' sections from the configuration documentation.
2025-11-08 11:33:38 -06:00
cort e75aa763aa Rename our_id to radio_id and add complete HBP config fields
Changes to hblink4/hblink.py:
- Renamed our_id → radio_id (more consistent with DMR terminology)
- Added colorcode, software_id, package_id fields to OutboundConnectionConfig
- Updated _send_outbound_config() to use configurable values instead of hardcoded
- All event emissions updated to use 'radio_id' field name

Changes to config/config_sample.json:
- Fixed field names: host→address, passphrase→password, our_id→radio_id
- Added all optional HBP metadata fields with proper defaults
- Shows comprehensive example with full metadata and minimal example

Changes to docs/configuration.md:
- Added complete Outbound Connections section
- Documents all required and optional fields
- Explains options format (TS1=x,y;TS2=a,b)
- Covers bidirectional traffic, multiple connections, DNS resolution
- Examples showing different configuration scenarios
2025-11-08 08:45:30 -06:00
cort 8176645a79 Fix: Clarify hang time protects TG conversations, fix transmission duration
CRITICAL documentation fixes:
- Hang time reserves slot for same TALKGROUP (or original user)
- Fixed typical transmission duration: 30-90 seconds (not 0.6-3.6s)
- Removed confusing echo/loop prevention discussion (half-duplex is obvious)
- Updated all examples to reflect actual implementation

Implementation reference: hblink.py lines 2173-2183 (same_tg or same_user)
2025-11-08 06:43:18 -06:00
cort e4c861e8fd docs: add outbound connections technical design
- Complete design specification for server-to-server linking
- Config schema based on MMDVM.ini format
- State management with separate dictionaries for O(1) routing
- ID conflict protection to prevent DoS
- Connection lifecycle and error handling
- Dashboard integration requirements
- Implementation checklist with 11 phases
2025-11-07 16:02:52 -06:00
cort 4001146fdc Merge branch 'feature/asyncio-migration'
The goal was always to move to asyncio from twisted, however, it was necessary
to develop the asyncio solution over time. Now that it is stable, the
twisted version is being moved to a legacy branch, and the asyncio will
become the main branch for continued development
2025-11-07 14:15:13 -06:00
cort a60d2a4356 docs: simplify connecting guide for repeater operators
- Remove server-side protocol implementation details
- Remove RPTO packet format specifications
- Remove server firewall/port forwarding instructions
- Add practical MMDVM.ini/DMRGateway.ini Options= syntax examples
- Focus on GUI configuration (Pi-Star, WPSD)
- Streamline troubleshooting from user perspective
- Add quick reference table for connection settings
2025-11-07 14:09:01 -06:00
cort e9d2a6b64c docs: add v1.6.1 release note indicating Talker Alias (DMRA) not supported in v1.6.1; available on feature/asyncio-migration 2025-11-01 11:42:02 -05:00
cort 355ed0433d Dashboard statistics redesign and documentation cleanup
- Dashboard: Redesigned statistics for operational clarity
  - 'Total Calls Today': RX streams only (actual RF traffic)
  - 'Retransmitted Calls Today': TX streams (server forwarding)
  - 'Active Streams': Current RX+TX activity
  - 'Total Traffic Today': Duration of RX streams only
  - Added is_assumed checks to differentiate RX from TX streams

- Documentation: Eliminated redundancy and historical content
  - Removed HBlink3->HBlink4 evolution story from README
  - Consolidated installation/configuration details to single sources
  - Removed redundant sections from dashboard and integration docs
  - Deleted implementation summaries with development history
  - Established single-source documentation pattern

- Updated dashboard README to reflect new statistics semantics
2025-10-30 08:03:03 -05:00
cort 1616ccba25 Add unit call detection and comprehensive TODO
- Add early detection of unit/private calls (call_type_bit == 1)
- Log unit calls at INFO level with clear identification
- Reject unit calls until full implementation ready (return False)
- Update TODO with comprehensive 8-phase unit call implementation plan
- Move code refactoring to 'Ongoing Priority' section
- Document user cache enhancement needed (slot tracking)
- Document routing strategies: one-to-one and broadcast with pruning
- Document slot compatibility requirements (no slot translation)
- Document hang time and contention handling for unit calls
- Include performance monitoring and testing strategies
- Fix whitespace alignment in stream_tracking_diagrams.md
2025-10-11 15:17:08 -05:00
cort 27afff59d4 updated stream documentation. 2025-10-10 07:55:48 -05:00
cort 7da5c4b94d Add trust flag for repeaters and make default config optional
Features:
- Add trust flag to RepeaterConfig (default: false)
  - Trusted repeaters can use any TG dynamically via OPTIONS
  - Config TGs become defaults instead of restrictions
  - Use for core network, testing, or administrative repeaters
- Make default repeater config optional
  - Omitting default rejects non-matching repeaters
  - Provides better security by requiring explicit config

Code improvements:
- Remove deprecated _is_talkgroup_allowed() method (never called)
- Remove defensive exception masking from _load_repeater_tg_config()
- Implement fail-fast design - bugs crash immediately with clear traceback
- Authentication at line 1120 is single point of truth for config validation

Documentation:
- Update configuration.md with trust flag examples and security notes
- Explain optional default behavior
- Update config_sample.json with trusted repeater example
2025-10-10 07:30:04 -05:00
cort 02d56435f8 Fix documentation: clarify empty list [] means deny all, not allow all 2025-10-09 11:21:25 -05:00
cort 30b2bddfa4 Remove redundant v1.5 documentation and temporary test file 2025-10-07 10:26:07 -05:00
cort 996c0fb856 Remove temporary investigation document
Keep only release notes and implementation summary as permanent documentation
2025-10-07 09:41:41 -05:00
cort a2aae6c946 Add implementation summary for v1.5 session 2025-10-07 09:40:35 -05:00
cort d0790ca619 Add release notes for v1.5 2025-10-07 09:32:20 -05:00
cort 6e889a1543 Add audio notifications to dashboard with toggle control
- Add Web Audio API implementation for repeater event notifications
- Implement toggle button in header (default: off, persists in localStorage)
- Add three distinct tones: 880Hz/150ms (connect), 2475Hz/50ms (missed ping alert), 440Hz/150ms (disconnect)
- Notifications trigger on repeater connect, disconnect, and missed ping events
- Add detailed investigation document for repeater info pages feature
2025-10-07 09:22:16 -05:00
cort a55501fd0d feat: multiple match types per pattern & improved missed ping handling
- Allow patterns to specify multiple match types (ids, id_ranges, callsigns)
- Patterns evaluated in config order (first match wins) with OR logic
- Callsign '*' matches all repeaters (useful for catch-all patterns)
- Emit repeater_connected event when any packet clears missed_pings
- Update documentation to reflect pattern matching behavior
- Remove outdated config fields from docs (enabled, timeout, talkgroups)
- Update tests to match new pattern evaluation order
2025-10-06 18:57:02 -05:00
cort 70813c7e9d Documentation: Add TCP dual-stack guidance for remote dashboard
Added configuration examples and guidance for using TCP transport with
dashboard on a remote machine. Same dual-stack considerations apply as
the UDP server when using TCP for remote dashboard connections.

Options documented:
- Localhost: No dual-stack issues (127.0.0.1 / ::1)
- Remote dual-stack: host_ipv4="", host_ipv6="::", single port
- Remote separate ports: If dual-stack bind conflicts
- Remote IPv4-only: Simplest for cross-machine deployments

HBlink4's event emitter already tries IPv6 first with automatic IPv4
fallback, so dual-stack works seamlessly.
2025-10-06 17:37:53 -05:00
cort cb9561cbaf Improve IPv6 dual-stack bind error handling and documentation
When IPv6 bind fails with 'address already in use' on the same port as IPv4,
provide helpful warnings explaining this is normal dual-stack behavior and
offer three clear solutions.

Code changes:
- Detect 'address already in use' errors specifically
- Provide context-aware warnings with actionable solutions
- Changed log from 'IPv6/dual-stack' to just 'IPv6' (more accurate)

Documentation changes:
- Added 'Common Issue' section explaining dual-stack port conflicts
- Recommend using separate ports as primary solution
- Document IPv4-only and IPv6-only alternatives
- Updated examples to show port_ipv4: 62031, port_ipv6: 62032

Production-tested solution for the common dual-stack bind error.
2025-10-06 17:28:32 -05:00
cort cd058b363d Documentation: Clarify that multiple ID ranges work in single patterns
Added examples showing multiple ID ranges in a single pattern for both
blacklist and repeater configuration sections. This feature already works
in the code but wasn't clearly documented.

Examples added:
- Blacklist: Multiple blocked ranges in one pattern
- Repeater configs: Regional network covering multiple DMR-MARC regions
- Both config_sample.json and configuration.md updated

Example usage:
"id_ranges": [[310000, 310999], [311000, 311999], [312000, 312999]]

This is much cleaner than creating separate patterns for each range!
2025-10-06 17:21:17 -05:00
Cort Buffington beef9e45f8 Update firewall configuration instructions for HBlink4
Clarify firewall rules and configurations for HBlink4 server connections, including protocol specifications and note on handling IPv4 and IPv6.
2025-10-06 08:33:59 -05:00
cort 6bfdfd144d removing unnecessary stuff 2025-10-06 07:23:52 -05:00
cort 9f3560a450 Clarify IPv4/IPv6 handling in firewall rules
- Note that most firewalls handle both protocols with single rule
- Explain when separate IPv4/IPv6 rules might be needed
- Make port purposes clearer in comments
2025-10-06 07:19:43 -05:00
cort 6c0be3505a Add firewall and NAT configuration to connecting guide
- Document required ports (UDP 62031/62032, TCP 8080)
- Include generic firewall rules in pseudo-code format
- Add router/NAT port forwarding instructions
- Cover cloud hosting (AWS, Azure, GCP, DigitalOcean)
- Include dashboard security best practices
- Add network troubleshooting section
2025-10-06 07:18:17 -05:00
cort 5515c454a1 docs: Add comprehensive guide for connecting to HBlink4
- Create connecting_to_hblink4.md with complete connection documentation
- Document RPTO (Repeater Options) usage for dynamic TG subscription
- Explain connection flow, authentication, and keepalive
- Include multiple RPTO examples with config intersection
- Add troubleshooting section for common connection issues
- List compatible software (Pi-Star, MMDVM, etc.)
- Document security best practices
- Provide complete reference for repeater operators
2025-10-05 18:21:02 -05:00
Cort Buffington f3e2fe96e1 Revise HomeBrew DMR protocol documentation
Clarify the HomeBrew DMR protocol implementation details and correct errors in the keepalive mechanism. Update the table formatting for configuration data fields and DMRD packet structure.
2025-10-05 11:27:01 -05:00
cort e7d5eb4cea docs: Add RPTO (Repeater Options) command documentation to protocol.md
- Document RPTO message format and purpose
- Explain talkgroup negotiation and intersection logic
- Note that server config is master allow list
- Include examples of RPTO usage and filtering
- Completes protocol documentation for all implemented commands
2025-10-05 11:21:47 -05:00
cort 6b17d66bf6 eliminating unused files 2025-10-05 11:18:00 -05:00
Cort Buffington e3e9dc077d Clean up stream tracking documentation
Removed implementation details and future enhancements sections from stream tracking documentation.
2025-10-05 11:15:49 -05:00
Cort Buffington cc10df037e Remove repeater configuration examples and notes
Removed configuration examples and implementation notes for repeaters.
2025-10-05 11:13:36 -05:00
Cort Buffington 326757a616 Refactor hang_time.md by removing implementation details
Removed detailed code implementation and future enhancements sections from hang_time documentation.
2025-10-05 11:12:07 -05:00
Cort Buffington 47a2418b4d Remove configuration tips from documentation
Removed configuration tips section for clarity.
2025-10-05 11:08:26 -05:00
Cort Buffington 35619a1ac8 Add transport abstraction for dashboard communication
Implement transport abstraction for dashboard communication, supporting TCP and Unix socket transports while removing UDP. Update configuration files and documentation accordingly.
2025-10-05 11:06:51 -05:00
Cort Buffington 32746e641a Update TODO.md with status changes and reorganization
Updated status of 'Code Refactoring' to ongoing and reorganized TODO list by moving 'Web-Based Configuration UI' to high priority.
2025-10-05 11:02:29 -05:00
Cort Buffington b7237ec888 Revise implementation summary and remove redundant details
Updated implementation summary with changes to stream tracking, hang time, and comprehensive documentation. Removed sections on modified files and configuration examples for clarity.
2025-10-05 10:59:32 -05:00
cort 31fe38b6dd Release v1.3: Documentation audit and config restructure
Major changes:
- Comprehensive documentation audit completed
- Config restructure: dashboard moved from global to top-level
- Dual-stack IPv6: bind_ipv4/bind_ipv6 with port_ipv4/port_ipv6
- Dashboard transport: host_ipv4/host_ipv6 with disable_ipv6 flag
- Fixed corrupted dashboard/server.py imports
- Updated all documentation to match actual implementation
- Removed duplication, established single source of truth

Documentation updates:
- docs/configuration.md: Complete overhaul with accurate field definitions
- docs/dashboard_transport.md: Updated transport configuration guide
- docs/integration.md: Dual-stack integration examples
- dashboard/README.md: Clarified dashboard configuration requirements
- DASHBOARD_INTEGRATION.md: Streamlined integration summary
- All config examples now use correct field names

Configuration changes:
- dashboard section moved to top-level (not nested under global)
- IPv6 settings: disable_ipv6 flag replaces ipv6 boolean
- Separate bind addresses and ports for IPv4 and IPv6
- Dashboard uses host_ipv4/host_ipv6 instead of single host field

Bug fixes:
- Fixed syntax error in dashboard/server.py (corrupted imports)
- All JSON configs validate successfully
- All 39 tests passing

See DOCUMENTATION_AUDIT.md for complete audit report.
2025-10-05 10:32:09 -05:00
cort 3a57bd69f4 Fix RX/TX contention: remove repeater from route-cache when it starts receiving
Problem:
- When we TX to a repeater (assumed stream) and it starts RX'ing
- The repeater ignores our TX packets (hardware busy receiving)
- We waste bandwidth sending packets to a busy repeater

Solution:
- Detect when repeater has assumed stream and receives real stream
- Remove that repeater from all active route-caches
- Stop sending to it immediately (bandwidth saved)
- Real RX streams always win over assumed TX streams

Implementation:
- Added check in _handle_stream_start() for is_assumed flag
- Iterate all streams and remove repeater from target_repeaters
- O(R×S) efficiency (~10-20 operations typical)
- Falls through to create real stream normally

Testing:
- Added test_assumed_stream_route_cache_removal()
- 39 tests passing (38 existing + 1 new)
- Validates route-cache removal logic

Documentation:
- Added section in stream_tracking.md
- Added detailed diagrams in stream_tracking_diagrams.md
- Explains real vs assumed stream priority
- Includes performance analysis and examples
2025-10-05 08:56:19 -05:00
cort 853d86eac9 Add transport abstraction for dashboard communication with initial state sync
- Implement TCP and Unix socket transports (removed UDP per requirements)
- Add connection state awareness and automatic reconnection (10s interval)
- Fix TCP race condition: Initial state sync on (re)connection
- Add reconnect callback to send current repeater state to dashboard
- Update all configuration files with transport settings
- Add comprehensive documentation (dashboard_transport.md, TRANSPORT_IMPLEMENTATION_SUMMARY.md)
- Clean up debug logging

This change addresses all user requirements:
1. Dashboard can run on separate server (TCP transport)
2. Dashboard never shows stale data (reliable delivery + state sync)
3. Dashboard knows when HBlink is offline (connection state tracking)
4. Unix socket for local (fastest), TCP for remote (flexible)
5. Robust against connection timing issues (initial state sync)

Tested: Both TCP and Unix socket transports working correctly
2025-10-04 13:41:23 -05:00
cort a4abd732af Add high-priority TODO: code refactoring to reduce repetition
Building on the success of the _end_stream() helper consolidation,
next session will focus on identifying and eliminating repetitive
code patterns across all files using helper functions.
2025-10-03 20:46:13 -05:00
cort 33bcb45d62 Clean up stale references and verbose comments
Code cleanup:
- Remove stale LC extraction comment from imports
- Remove Phase 1/2/3 references from comments (work complete)
- Simplify verbose architecture notes to concise descriptions
- Remove TODO for RSSI parsing (not implementing)
- Remove 'Critical:' and 'Must' tags from comments

Dashboard cleanup:
- Remove talker_alias display (replaced with packet count in slot view)
- Update Last Heard to show callsign from user_cache (not talker_alias)
- Remove talker_alias from stream_start event formatting

Documentation:
- Update IMPLEMENTATION_SUMMARY git history (remove LC reference)

All 29 tests passing. No functional changes.
2025-10-03 15:12:07 -05:00
cort 856062f762 Refine TODO based on project design goals
Performance Monitoring:
- Focus on latency and jitter as most critical metrics
- Remove packets/sec and active streams (not issues)
- Latency/jitter through hblink4 is key quality indicator

Removed items not aligned with design:
- Multi-Server Clustering (design is endpoint, no scaling needed)
- Mobile App Integration (web-only, mobile optimization in dashboard)

Items now numbered 1-5 (was 1-7)
2025-10-03 15:03:37 -05:00
cort 8ce80c7828 Remove all LC-related content from TODO
- Removed LC Extraction from Sync Frames from Medium Priority
- Removed 'LC Decoding Clarification' section
- Removed 'DMR Link Control Extraction' from completed features
- LC functionality is abandoned going forward
- Renumbered remaining items 1-9 (was 1-10)
2025-10-03 14:29:01 -05:00
cort 7cff26e8f2 Clean up transient refactoring documents
- Remove PHASE2_AUDIT_SUMMARY.md (work complete)
- Remove PHASE3_ANALYSIS.md (work complete)
- Remove TEST_FIXES_SUMMARY.md (work complete)
- Remove docs/PHASE4_FUTURE_ENHANCEMENTS.md (consolidated into TODO.md)
- Update docs/TODO.md with dashboard quick wins (layout, mobile)

Keep only permanent documentation relevant to new contributors.
2025-10-03 14:24:56 -05:00
cort feb405a20b Phase 3: Refactor _check_stream_timeouts() to eliminate code duplication
- Extracted _check_slot_timeout() helper method (60 lines)
- Reduced _check_stream_timeouts() from 130 lines to 90 lines
- Eliminated 90+ lines of duplicated slot checking logic
- All 29 tests passing

Files changed:
- hblink4/hblink.py: Added _check_slot_timeout() helper, refactored main method
- PHASE3_ANALYSIS.md: Efficiency analysis and recommendations
- docs/PHASE4_FUTURE_ENHANCEMENTS.md: Future enhancement planning

Impact: Improved code maintainability with zero functional changes
2025-10-03 14:22:09 -05:00
cort e0b8f73845 Phase 2 audit: Remove dead code and update documentation
Dead code removed:
- bhex() function (only used once, now inlined)
- get_user_cache_data() method (never called, duplicated functionality)
- Unused dmr_utils3 imports (decode, bptc - were for LC extraction)

Documentation cleanup:
- Remove all LC (Link Control) references from docs
- Remove all talker alias references from docs (changed to 'alias' in dashboard docs)
- Update IMPLEMENTATION_SUMMARY.md to remove LC/talker alias sections
- Update readme.md to remove LC/talker alias feature mentions
- Update DASHBOARD_INTEGRATION.md to remove talker_alias from event parameters
- Update docs/TODO.md to remove completed talker alias items

All changes validated:
- No linting errors in hblink.py
- 23 tests passing (6 pre-existing failures unrelated to these changes)
- Code is cleaner and more maintainable
2025-10-03 13:47:48 -05:00
cort 083a9dd5af Phase 1: Remove all LC and Talker Alias code
Comprehensive cleanup removing unused/unnecessary code:

Deleted files:
- docs/lc_extraction.md - LC extraction documentation
- docs/talker_alias.md - Talker alias documentation
- tests/test_lc_extraction.py - LC extraction tests
- tests/test_embedded_lc.py - Embedded LC tests
- tests/test_talker_alias.py - Talker alias tests

Removed from hblink4/hblink.py:
- DMRLC class (full LC decoding infrastructure)
- decode_lc() function
- extract_voice_lc() function
- extract_embedded_lc() function (commented code)
- decode_embedded_lc() function (commented code)
- extract_talker_alias() function
- decode_talker_alias() function

Removed from StreamState dataclass:
- lc: Optional['DMRLC'] field
- talker_alias, talker_alias_format, talker_alias_length, talker_alias_blocks fields
- missed_header, embedded_lc_bits (commented fields)

Replaced with simple stub:
- alias: str = '' # For future callsign lookup from file

Removed from dmrd_received():
- All LC extraction code (was commented out anyway)
- All talker alias collection and decoding code
- User cache talker_alias parameter

Rationale:
- LC decoding was never fully implemented/working
- Talker alias decoding was overly complex for minimal benefit
- Will replace with simple callsign lookup file (radio_id -> callsign)
- Reduces code complexity significantly
- Dashboard 'alias' field remains for future population from lookup

Next: Continue audit for unused/duplicative code
2025-10-03 13:15:29 -05:00
cort e1fc81f921 Implement Phase 3: Configuration-based routing with slot-specific talkgroups
Major changes:
- Replaced ambiguous 'talkgroups' list with slot1_talkgroups/slot2_talkgroups
- Simplified routing: same lists control inbound and outbound (no separate rules)
- No wildcards or rewriting (simplicity over flexibility)

Configuration updates:
- Updated config.json with real KS-DMR settings (TS1: TG2,8,9 / TS2: TG3120)
- Updated config_sample.json with slot-specific examples
- Created docs/routing.md with complete documentation

Code changes:
- Added _check_inbound_routing(): Verify repeater can send on TS/TGID
- Added _check_outbound_routing(): Verify repeater should receive on TS/TGID
- Added _is_slot_busy(): Contention detection with hang time support
- Added _update_assumed_stream(): Track streams forwarded to targets
- Updated _forward_stream(): Full Phase 3 implementation with routing + contention
- Updated StreamState: Added is_assumed flag and end_time field

Routing logic:
- Inbound: Only accept traffic matching repeater's slot_N_talkgroups
- Outbound: Only forward to repeaters with matching slot_N_talkgroups
- Contention: Block busy slots, respect hang time after stream ends
- Assumed streams: Track forwarded streams on target repeaters

Ready for real-world testing with multiple repeaters!
2025-10-03 09:04:47 -05:00
cort 0af7830fcf Disable LC recovery code until fixed
- Comment out embedded LC extraction functions and StreamInfo fields
- Add common search phrase: 'LC recovery - disabled until fixed'
- Skip all embedded LC tests with clear reason
- Document both debug comment phrases in docs/logging.md:
  * 'Per-packet logging - only enable for heavy troubleshooting'
  * 'LC recovery - disabled until fixed'
- LC still obtained from DMRD packet header, this was backup mechanism only
2025-10-03 07:08:11 -05:00