# bsdrX — Bigscreen Remote Desktop agent in C (Linux / Windows / macOS)

A portable C reimplementation of the PC-agent side of Bigscreen's Remote Desktop
protocol, plus extras: it casts a **Linux desktop (or a video file) into a
Bigscreen VR headset**, streams **audio both ways**, takes **input back** from the
headset, and adds an **in-VR control bar** and a **voice assistant** that can type
or drive the desktop via an LLM.

Protocol reference: **`../projectB-protocol-spec.md`** — the wire format, the
no-ICE DTLS transport, the input byte-layout, and the media parameters, all
reverse-engineered from the official agent (`BigSoup.dll`, v0.950.2).

## Features

**Remote Desktop (LAN)**
- **Discovery** (UDP 45000/45001) + **pairing/control** (HTTP 45678) — the full
  state machine, mirroring the official `server.js`.
- **No-ICE DTLS-SRTP transport** on UDP 45002 (OpenSSL; SCTP DataChannel for
  input, SRTP for media — all muxed on one port).
- **Input** (headset → PC): the statically-reversed DataChannel format (mouse
  abs/rel, L/R/M + X1/X2 buttons, wheel, keyboard, XInput gamepad) injected via
  **uinput** / **SendInput** / **CGEvent**.
- **Video** (PC → headset): X11 desktop capture → **NVENC** H.264 (Constrained
  Baseline, fallback libx264) → RTP (RFC 6184) → SRTP. Or **stream a local H.264
  file with no re-encode** (demux passthrough).
- **Audio** (both ways): system audio → a null-sink (speakers go silent) → Opus
  → headset; the headset mic → Opus decode → a **virtual microphone** the OS sees
  as an input.
- **Multi-Quest selection**: every headset broadcasting on the LAN is listed;
  pick one and the others are refused.

**Control web UI** (served on `127.0.0.1:8088`, opens in your browser)
- Bigscreen **cloud login** (HTTPS `/auth/login`) + connection status.
- Status: connected-to-server and connected-to-Quest indicators, streaming state.
- Quest picker, source (desktop/file), **resolution + bitrate**, stop/restart.
- Voice-assistant endpoints (STT + LLM).

**In-VR voice-command balloon** (composited into the video, dragged/clicked via
the input channel): a movable mic balloon you can place anywhere over the desktop.
Click it to speak a command; it listens **until you stop talking**, transcribes,
and lets the LLM drive the desktop. See [`docs/computer-control.md`](docs/computer-control.md).

**Voice computer control** (enable in the web panel; balloon is the trigger)
- **Speech-to-text**, pluggable: point it at a **local or remote whisper-server**
  (any OpenAI-compatible `/audio/transcriptions`), or leave it blank to use a
  built-in **free, keyless** online service.
- **LLM with computer control**: transcript → a configurable OpenAI-compatible
  chat endpoint (http/https + bearer token) with a tool set (`type_text`, `key`,
  `click`, `scroll`, `open_app`) executed on the desktop via the injector.
- **Voice source is the headset owner** — captured from the owner-mic sniffer/MITM
  (the only path the owner's voice reaches this PC), so computer control is only
  enableable while that sniffer is running.

## Build

Media (video/audio/SCTP) is **on by default** and auto-detected — `make` builds
the full agent on Linux, and silently drops any feature whose libraries are
missing. Two interchangeable build systems are provided.

```bash
# configure + make (no autotools needed)
./configure              # detects host OS, OpenSSL, and media deps
make                     # -> build/bsdr_agent   (full agent on Linux)
make check               # build + run the test suite
sudo make install        # -> $(prefix) (bin, lib, headers, man1/bsdr_agent.1; DESTDIR honored)
man bsdr_agent           # after install: full option reference
make clean / distclean

./configure --disable-media     # base agent (input only); also --disable-video/-audio/-sctp
./configure --help

# static Linux binary (no shared-lib deps at runtime)
make linux-static                       # -> build-linux-static/  (core agent, no media)
./configure --static --disable-media && make   # same, via configure
./configure --static && make            # static WITH media (needs the deps' .a archives)

# CMake (alternative; per-feature options)
cmake -S . -B build -DBSDR_ENABLE_SCTP=ON -DBSDR_ENABLE_VIDEO=ON -DBSDR_ENABLE_AUDIO=ON
cmake --build build -j && ctest --test-dir build --output-on-failure
```

**Dependencies** (Debian/Ubuntu names): `libssl-dev` (always); for media:
`libsrtp2-dev`, `libavcodec/avformat/avdevice/avutil/swscale-dev` (ffmpeg),
`libopus-dev`, `libpulse-dev`, and **usrsctp** (e.g. `libusrsctp-dev`). NVENC
needs the NVIDIA driver; capture needs an X11 display.

### Cross-compiling from Linux
- **Windows (MinGW-w64)** — verified: cross-built to a static PE32+ `.exe` and run
  under Wine (logic suites + the DTLS transport + the discovery/pairing/control
  flow all pass). Build OpenSSL for the target, then:
  ```bash
  cmake -S . -B build-win -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-mingw64.cmake \
        -DOPENSSL_ROOT_DIR=<mingw-openssl-prefix>
  cmake --build build-win -j        # or: make windows WIN_OPENSSL=<prefix>
  ```
  (Media — NVENC/PulseAudio/X11 — is Linux-only; cross builds are input+transport.)
- **macOS** — native build with full audio (CoreAudio + BlackHole) and the owner-mic sniffer:
  see [`docs/macos.md`](docs/macos.md) for deps, BlackHole/Multi-Output setup, and usage.
  Cross-build via osxcross is also possible (`cmake/toolchain-osxcross.cmake`; needs Apple's SDK).

### Linux uinput permissions
`/dev/uinput` is root-only by default; for non-root injection:
```bash
sudo groupadd -f uinput
echo 'KERNEL=="uinput", GROUP="uinput", MODE="0660"' | sudo tee /etc/udev/rules.d/99-uinput.rules
sudo udevadm control --reload && sudo modprobe uinput
sudo usermod -aG uinput "$USER"   # re-login
```
Without it the agent still runs — injection falls back to logging.

## Run
```bash
./build/bsdr_agent -v
#   --no-video / --no-audio          disable a media direction
#   --file PATH                      stream a local H.264 file (passthrough)
#   --fps N --bitrate BPS --screen WxH
#   --responder                      wait for the headset to initiate DTLS
#   --ui-port N / --no-ui / --no-browser
```
The agent prints a **pairing code**, opens the **control UI** at
`http://127.0.0.1:8088`, and on `/start` brings up input + desktop video + audio
to the paired Quest. Audio devices are only created on a real connection, so the
agent idle never touches your sound.

Tools: `dtls_probe <quest-ip>` (probe the DTLS handshake), `test_client`
(stand-in headset for discovery+control), `stream_demo` (capture→encode→SRTP→
decode loopback), `voice_test <stt-url> <llm-url>` (exercise STT+LLM).

## Module layout
```
src/
  platform.h net.c   sockets / threads / mutex / clock / CSPRNG (Win/POSIX)
  log json protocol events                       infra + wire constants
  input_decode       DataChannel frame -> input events (reversed statically)
  discovery control  UDP discovery + HTTP pairing state machine
  udp_transport dtls sctp   no-ICE UDP -> DTLS-SRTP -> usrsctp DataChannel
  srtp_util          shared libsrtp helpers (video + audio)
  video capture filesrc     RTP/SRTP H.264 sender; X11+NVENC; file passthrough
  audio              Opus RTP/SRTP + PulseAudio I/O + virtual sink/source
  overlay            in-VR control bar (NV12 compositing + click hit-test)
  inject_{linux,win,macos,null}   native input injection
  transport          UDP -> DTLS -> {SCTP input, SRTP media} -> decode/inject;
                     capture/audio threads; overlay + voice triggers
  cloud httpc        Bigscreen cloud login; shared HTTP/HTTPS client
  stt llm compcontrol voice   STT + LLM(tool-calling) + computer control + orchestrator
  app webui          shared state + local control web UI
  agent              main: wires it all together
```

## Status — verified vs. needs the headset
**Verified from Linux (no headset), ctest + tools all green:**
- The entire **input channel** (discovery → pair → control → DTLS → SCTP
  DataChannel → decode → inject).
- **Video**: real desktop → NVENC → RTP/SRTP → decoded back; file passthrough;
  SRTP keyed from a real DTLS handshake (GCM).
- **Audio**: Opus RTP/SRTP roundtrip; PulseAudio capture/playback + virtual
  devices against the live server.
- **Overlay** composited into the encoded video; **multi-Quest** selection;
  **cloud login** against the live Bigscreen server (correct 401 for bad creds);
  **web UI** endpoints; **STT/LLM** clients + the **computer-control** executor.

**Needs the real Quest to confirm (all do-able from Linux, no Windows machine):**
- The media **last-mile params** — exact RTP payload-type numbers and the
  SSRC→stream mapping. There's no SDP, so these come from either deeper static RE
  of `BigSoup.dll` or observing the Quest's own packets directly (the headset
  sends us mic RTP + input we can inspect; for video we trial PTs until it
  renders). No reference capture from a Windows agent is required.
- The live end-to-end: that the Quest renders the video and the spoken voice flow
  works in the headset.

This started as a port of the Python `../linux-agent/` (same protocol); it has
since grown into the fuller desktop-in-VR client above.

## License

bsdrX is free software, released under the **GNU General Public License v3.0
(or later)** — copyleft, Copyright (C) 2026 **Stefy Lanza &lt;stefy@nexlab.net&gt;**.

You may redistribute and/or modify it under the terms of the GPLv3; it comes
with **no warranty**. The full text is in [LICENSE.md](LICENSE.md), and every
source file carries the standard GPLv3 header.
