Skip to content
rdrd

Viewer internals

Browser client for rd: vanilla TypeScript, built with Vite and Bun, no UI framework. The wire format mirrors crates/rd-proto, and docs/protocol.md is the contract.

Terminal window
bun install
bun run dev         # http://127.0.0.1:5173; proxies /api and /ws to a gateway on 127.0.0.1:8080
bun run build       # → dist/ (index, viewer, terminal, probe, latency, keytest .html + assets/)
bun run typecheck   # tsc --noEmit, strict
bun run test        # vitest; includes the golden vectors in crates/rd-proto/testvectors
bun run format      # prettier

For local development, run the gateway as cargo run -p rd-gateway -- --dev --assets-dir viewer/dist, or run it next to bun run dev. The dev server serves the gateway’s routes too (/m/{machine}, /t/{machine}, /probe, /latency-target, /keytest) and sends the same COOP/COEP headers, so SharedArrayBuffer works. WebCodecs needs a secure context: use localhost, or HTTPS in production.

Layout

Section titled “Layout”
Path What
src/proto Binary codec (bin.ts) and JSON control types with defensive parsing (control.ts)
src/net Session socket (session.ts): connect → session → hello → start, PING/PONG, close codes, reconnect backoff; clock offset estimator; HTTP API helpers
src/video WebCodecs pipeline (epochs, discard-ACKs, ACK in the output callback, immediate or rAF drawing); input→photon latency test
src/files File channel client, transfer engine (pipelined uploads, downloads staged in OPFS, resume), BulkWindow flow control (docs/files.md)
src/terminal Terminal channel client: hello / reattach, reconnect policy, output offsets, TERM_ACK coalescing, replayed-history guard, input splitting (docs/terminal.md)
src/audio Opus AudioDecoder → SharedArrayBuffer ring → AudioWorklet with an adaptive jitter buffer
src/input Pointer, wheel and keyboard capture; letterbox-aware coordinates; modifier mapping; reserved chords; pointer lock. keymap.gen.ts is generated by tools/gen-keymap
src/ui Connect screen, HUD, settings (localStorage per machine), banners, wake lock, the viewer app; for the portal: the device console’s model, command palette, activity lines, agent update state and Markdown release notes
src/probe M0 network probe: tests, analysis and verdict
src/pages One entry per HTML page

Shortcuts in a session

Section titled “Shortcuts in a session”

Ctrl+Alt+Shift + F fullscreen + Keyboard Lock · M next display · E files · ` terminal · L pointer lock · H stats HUD · T latency test · D disconnect. The small ▾ tab at the top of the screen opens the menu: a display picker (when the host has more than one screen), fullscreen, pointer lock, stats, files, terminal, settings and disconnect. The chosen display is remembered per machine and used again on the next connect.

To run the latency test, open /latency-target on the host, rest the remote cursor on its box, then press the T chord. The viewer sends F13 and measures how long it takes for the box flip to be painted.

Keyboard and connection notes

Section titled “Keyboard and connection notes”
  • Keys are sent by physical position (KeyboardEvent.code); src/input/keyboard.ts holds the key state.
  • Caps Lock is synced, not forwarded: the host’s lock follows the local one (getModifierState).
  • Windows: AltGr (Right Alt on US-International, German, …) arrives as Right Option. The extra left Ctrl that Windows sends with it is dropped; to tell the two apart, a left Ctrl press is held back until the next key, click or scroll (at most 100 ms). A Shift keyup releases both Shifts (Windows omits one when both were down).
  • A key released after focus moved to the menu or a banner is still released on the host.
  • Windows keyboard → Mac: Alt+Tab becomes Cmd+Tab. At the first Tab while the left Alt is held, that Alt turns into Cmd until it is released, so Shift+Alt+Tab, repeated Tabs and releasing Alt behave like the Mac app switcher. Right Alt is never switched (AltGr).
  • When the agent ends the session to install an update (close code 4010), the viewer shows an info banner and polls /api/machines until the agent is back and done, then reconnects. For 3 minutes the 4003/4004 answers of the restarting agent count as part of the update; after that the usual handling takes over.
  • A session socket that cannot open within 10 s is retried. If it is refused because the sign-in expired (or the user is not allowed), the viewer stops and asks for a page reload instead of retrying forever.

Desktop app bridge

Section titled “Desktop app bridge”

The rd Desktop app (docs/desktop-client.md, bridge contract version 1) loads this viewer in a Tauri webview and grabs the keys a browser tab never sees (Alt+Tab, the Windows key, …). The page’s half is src/input/desktop-bridge.ts, wired up in src/input/controller.ts:

  • window.rdNative = { version: 1, key(code, down, repeat), release() } is always defined on the viewer page (unused in a browser).
  • Inside the app (window.__TAURI_INTERNALS__), the page calls invoke('rd_set_grab', { active }) whenever “keys go to the remote” changes: streaming, page visible, document focused, and the video (not a viewer control) focused. A failing invoke is logged once and otherwise ignored.
  • Once the app has answered rd_set_grab(true), DOM key events are ignored and keys arrive through rdNative.key. They take the same path as DOM keys (modifier mapping, Alt+Tab → Cmd+Tab, chords, the AltGr filter, Caps Lock). The modifier and lock state a DOM event would carry is derived in src/input/native-keys.ts from the keys the app reported down. The hook swallows CapsLock, so the local lock does not change; each CapsLock press during a grab toggles the remote lock, and after the grab the remote lock follows the local one again.
  • rdNative.release() is handled like a window blur: RELEASE_ALL, and pressed keys are forgotten.
  • The gateway CSP allows the Tauri IPC endpoints (connect-src ipc: http://ipc.localhost).