fix(pty): re-arm mio readiness after raw socket WouldBlock on Windows

[srstack]

Problem

On Windows, an SSH session displays the initial login banner/motd and then goes permanently deaf: the shell prompt never appears, and no further output arrives, while keyboard input keeps being sent successfully.

Root cause

mio's Windows backend is edge-triggered by design (src/sys/windows/selector.rs):

// In mio, we have to simulate Edge-triggered behavior to match API usage.
// The strategy here is to intercept all read/write from user that could
// cause WouldBlock usage, then reregister the socket to reset the interests.
self.user_evts &= !afd_events;

After delivering an event, mio clears the socket's interest bits. The interest is only re-armed when a read/write through mio::net::TcpStream hits WouldBlock (see IoSourceState::do_io, which re-registers on WouldBlock).

But all SSH channel I/O in otty-pty bypasses mio: libssh2 reads and writes the raw socket directly. So mio never observes a WouldBlock, the POLL_RECEIVE interest is lost after the first event, and no READABLE is ever delivered again. Writes keep working because they never needed an event.

Trace evidence from a Windows run (OTTY_LOG_FILE diagnostics): the runtime loop polls fine (waker tokens fire), one Token(0) readable event delivers the motd (599 bytes), then zero socket events for seconds while the server provably sends data (prompt + echoes). The same session parameters against the same server work on Linux, where mio is level-triggered.

This affects any Rust project combining mio-registered sockets with out-of-band raw-socket I/O (like libssh2) on Windows.

Fix

When libssh2 reports WouldBlock (kernel buffer drained), peek one byte through the mio socket:

self.io.peek(&mut [0u8; 1])

The peek hits WouldBlock without consuming data, which is mio's documented signal (TcpStream::peek: "Need to re-register if peek returns WouldBlock to ensure the socket will receive more events once it is ready again") to re-register the full interest set. Applied on both the read and write WouldBlock paths; on unix the helper is a no-op since mio is level-triggered there.

Verification

• Runtime-verified on a Windows 11 host against an OpenSSH 10.0 server: before the fix the session froze after the motd; after the fix, prompt appears and interactive use (echo, ls, full-screen programs) works normally
• Unix behavior unchanged (helper is #[cfg(not(windows))] no-op)
• cargo clippy -p otty-pty --all-targets --all-features -- -D warnings, cargo test -p otty-pty --all-features green

[Copilot]

Pull request overview

Fixes a Windows-specific edge-triggered mio readiness issue in otty-pty’s SSH backend where libssh2 performs out-of-band raw-socket I/O, causing mio interests to be cleared after the first event and the session to stop receiving further output.

Changes:

• Adds a Windows-only helper (rearm_io_events) that calls mio::net::TcpStream::peek to trigger mio’s internal re-registration behavior after WouldBlock.
• Calls this helper from the SSH Session::read and Session::write WouldBlock paths to keep readiness armed.

Suppressed comments (1)

otty-pty/src/ssh.rs:163

• Same race as the read path: after a libssh2 WouldBlock, rearm_io_events() can observe peek() == Ok(_) (socket became ready again) and return Ok(()) without necessarily re-arming mio. Returning Ok(0) immediately can then leave pending output stalled until a writable event that might not be delivered. Retrying the write once after re-arming makes progress when the socket becomes writable in-between.

            Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
                self.rearm_io_events()?;
                Ok(0)
            },

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[srstack]

Good catch — that's a real race. If peek() returns Ok (data arrived between the libssh2 call and the peek), no re-registration happens and returning Ok(0) could stall the engine on an event that never comes. Fixed in 1f070f6: both the read and write WouldBlock paths now retry the channel operation once after re-arming, so forward progress is guaranteed whenever data is already available. The EOF handling on the retry path is shared via a finish_eof helper.

[srstack]

Fair point. In 1f070f6 the re-arm logic is extracted into a free rearm_readiness(&mio::net::TcpStream) function, and I've added loopback-socket tests covering the observable contract on all platforms: (1) re-arming on an empty socket returns Ok without blocking, and (2) it never consumes pending data (peek semantics verified by reading the payload afterwards). The actual AFD interest re-registration is mio-internal and Windows-only, so the tests assert the behavior our code is responsible for rather than mio's internals.

[kemokempo]

Well, did you consider to try to use io.try_io instead of this?

It could be helpful because this method must provide the same semantic in win (peek + read/write)