Ensure stream errors are never lost by using async fallback

When handleIncoming detects a stream error, queue ActionStreamError to managerLoop
with non-blocking send. If managerLoop is busy and cmds channel is full, spawn an
async goroutine to queue the error asynchronously. This ensures the manager is
always notified of stream failures, preventing the connection from remaining in an
inconsistent state (connected=true while stream is dead).
This commit is contained in:
Chris Lu
2025-12-22 00:37:53 -08:00
parent 988e6d1a2d
commit 1598c4b8b9
+7 -2
View File
@@ -456,11 +456,16 @@ func handleIncoming(
default:
}
// Report the failure as a command to the managerLoop (non-blocking to prevent deadlock)
// Report the failure as a command to the managerLoop.
// Try non-blocking first; if the manager is busy and the channel is full,
// fall back to an asynchronous blocking send so the error is not lost.
select {
case cmds <- grpcCommand{action: ActionStreamError, data: err}:
default:
glog.V(2).Infof("Manager busy, stream error not queued: %v", err)
glog.V(2).Infof("Manager busy, queuing stream error asynchronously: %v", err)
go func(e error) {
cmds <- grpcCommand{action: ActionStreamError, data: e}
}(err)
}
// Exit the main handler loop