Portfolio

What Kestrel VR is made of, and the problems solved along the way.

Built

The headset client

Kestrel VRIn the build

What it does: The app on the headset receives the stream, decodes it, draws it in front of your eyes, and sends back where your head and hands are. It also holds the lobby you see before a stream and the settings panel that works without a PC.
Why it was written again: The client had inherited about 18,000 lines from the project Kestrel began as. Rather than edit them, it was written from nothing under its own licence, with none of the old code: its own protocol, transport, decoder queue, renderer and OpenXR shell.
How it was built: The protocol came first and was proven on its own against the running streamer: handshake, configuration, video, tracking and statistics with no gaps, before any picture existed. Then the client core, running on a PC against a false display; then the headset with the real decoder; then the OpenXR session, the frame loop and the renderer that draws the decoded picture into each eye. Controllers, hands, audio both ways, the lobby and the panel followed.
Measured: On the headset before it drew anything: 72 fps, decode 6.1 to 6.4 ms, one missed frame in ten seconds. With the picture and the panel in place, a 30 second run held 72.00 fps with no missed frames.
Status: In the build, in closed beta. Other headsets are added one at a time, and only when someone can run them.

The video encoder

Kestrel VRIn the build

What it does: Turns each rendered frame into a video stream small enough to send over Wi-Fi, in a few milliseconds, without the picture falling apart when the scene gets busy.
How it works: It drives the graphics card directly through Vulkan Video instead of going through a general-purpose media library, so Kestrel owns the encode session itself: how bits are spent, what each picture is worth, which memory the stream lands in, and when settings take effect. H.264 and HEVC both run on it today, and AV1 is coming.
What that made possible: A bitrate change takes effect without restarting the encoder, so the picture does not stutter when the rate moves. The stream is written into memory the processor can read cheaply. A ceiling on how coarse a picture may become keeps still scenes sharp, which is where a streamed picture usually goes soft.
What was tried and dropped: Three encoder features were built, measured and removed because the numbers did not support them: a higher quality level, which cost 1.5 ms a frame for no visible gain; splitting each picture into two slices, which the headset's decoder did not benefit from; and a rolling refresh, once a test showed the lines it was blamed for were not its doing.
Measured: 5.4 ms to encode a frame, measured on 2026-09-17. An earlier build of the same chain ran 77 minutes of continuous streaming with no stall, on 2026-09-09.
Status: In the build. AMD, NVIDIA and Intel are supported at release.

The transport

Kestrel VRIn the build

What it does: Carries video, audio, tracking and control between the PC and the headset, over Wi-Fi or a cable.
Why it matters: Most PC VR streaming sends your video over plain UDP, unencrypted, on whatever network you are on. Kestrel uses QUIC, so every frame and every input is encrypted end to end and the two ends verify each other. It connects directly, with no relay in between.
What QUIC gives that raw UDP does not: The connection measures the path it runs on: round trip time, loss, and how much the network will take. Video goes on datagrams, so a late frame is never held up waiting for an older one to be resent, while control messages ride a reliable stream on the same connection. One connection carries all of it, so there is one thing to open, one thing to secure and one thing to keep alive.
What that made possible: One setting was read and corrected: the transport library waits 25 ms before acknowledging by default, which at 90 frames a second is more than two frames before a sender learns anything was lost. Kestrel asks for 1 ms, and both ends advertise that floor. The same connection is built to move between the cable and Wi-Fi without dropping the session.
Status: In the build. The older plain UDP and TCP paths are gone from the new client.

Foveation

Kestrel VRIn the build

What it does: Sends the middle of your view at full detail and the edges at lower detail, because that is how your eyes work: only the centre sees fine detail. Fewer pixels to encode, send and decode, with nothing visibly lost.
How it works: The frame is divided into three bands per axis. The centre keeps every pixel and the edges are halved along each axis. Splitting a frame into bands this way is established practice. What is particular here is that the ratio is exactly two to one with the boundaries aligned to it, so the periphery is resampled with no drift between pixel grids. That matters because a periphery resampled at an uneven ratio shimmers when you move your head, which is what makes some foveation unpleasant to look at.
What it saves: On one measured configuration, each eye rendered at 2144 × 2246 goes over the wire as 1776 × 1792, about a third fewer pixels, and 87 percent of the periphery is exactly half scale on one axis. The reprojection that corrects head motion is folded into the same pass, so the pixels are resampled only once.
What comes next: Following the fovea with eye tracking on the same grid, so the sharp band goes where you are looking. It is designed to change nothing on headsets without eye tracking, which is every headset supported today.
Status: In the build. Gaze-driven foveation is planned.

Rewarp

Kestrel VRIn the build

What it does: Your head keeps moving while a frame is being drawn and sent. Rewarp turns the finished frame to match where your head is now, just before it is encoded, so the world stays where you left it.
How it works: Turning a frame to a new viewing angle is an exact operation when the projection is unchanged, so this is not an approximation of the picture: it is the same picture sampled from a different angle, with each eye's own asymmetric field of view accounted for. The maths was checked against a two thousand case simulation and agrees to sixteen decimal places, and again on hardware.
Why it is free: The turn is folded into the same pass that applies foveation, so the frame is resampled once instead of twice. No extra blur, no extra pass.
What it bought: Prediction could be cut from 100 ms to 20 ms, the single largest smoothness gain of the whole effort, because the warp supplies the freshness that prediction used to guess at. Prediction error grows with how far ahead you guess and overshoot grows with its square, so guessing less is worth more than guessing well.
Measured: Under load, total latency read 32.7 to 51.2 ms where the same conditions had read 48 to 72 ms before. The drop is the pose age the warp cancelled, and it grows with load exactly as the mechanism predicts.
What it does not fix: It corrects rotation. A world swim in one application was not rotational, and rewarp, the encode worker and the pose matcher all failed against it; pacing fixed that one.

Investigations

Each piece below reports the numbers measured on the day of that fix. They are not Kestrel's current performance; today's figures are on the About page.

1The decoder harness
A headset app with no PC attached showed the decoder was not the one holding frames.
2673 ms to 74 ms
A latency sawtooth traced to the age limit of a pose buffer.
3A fifteen-character process name
A startup error on every Linux install, reduced to one kernel limit.
4A hundred times the motion, no shake
One experiment cleared two suspects at once.
57,800 microseconds to 2
Taking the encoder off the compositor's frame loop.
6Decomposing a build that felt good
Thirteen changes, each run alone.
7The ratchet
A frame rate that only ever went down, and a fix certified against the failure itself.
8The colour chain
A grey of 128 arriving as 55, and one boolean.
9What each channel lies about
A map of every window into a running stream.
10The audio metronome
Packet loss that kept time to the millisecond.

1. The decoder harness

Decoder2026-09-16

Problem: Decode time on the headset rose with the gap between frames. Two separate readings concluded that the headset's hardware decoder holds a frame before releasing it. Both reasoned from a client statistic that cannot tell the decoder apart from the code around it.
Instrument: A standalone headset app that feeds a recorded bitstream into the same decoder at a chosen interval and records when each picture goes in and comes out. No PC, no stream, nobody wearing the headset. A run file declares one configuration per line: interval, output mode, release mode and format.
Finding: The decoder answered in about 6 ms whether frames arrived 11.1 ms apart or 33.3 ms apart. The dependence on the interval belonged to the client, not the decoder.
Outcome: Both earlier readings carry a dated correction in place instead of being deleted.

~6 msdecode at 11.1 ms spacing

~6 msdecode at 33.3 ms spacing

2. 673 ms to 74 ms

Latency and pacing2026-08-12

Problem: Latency climbed in a straight line to about 1333 ms, dropped vertically, and repeated every 1.3 seconds. That ceiling is exactly the age of the oldest entry a 360-deep pose buffer at 270 Hz can hold. The pose for each frame was being found by scanning the stack, and the scan was failing.
Instrument: The encoder takes the render pose from the runtime's versioned frame-timing records instead, newest of eight, and matches it into the pose history. A log line names which of three sources served the frame, written only when the source changes.
Finding: Same machine, same world, before and after. The matched pose is bit-exact one of the submitted poses, distance 0.0 on every frame. Submitted poses carry zero velocity, so the runtime's prediction is the identity, and that is what makes the match checkable instead of merely plausible.
Outcome: Fixed. A day later the fix made the world swim against head motion inside a real application, because the premise that the records carry no pose under async reprojection had been measured only in the runtime's own scenes. A gate on the record's reprojection flags fixed that, and the wrong premise is kept in the notes with the correction dated beneath it.

673 → 74 msaverage latency

1202 → 93 msmaximum latency

2,590 → 0clamp warnings per 12 s

3. A fifteen-character process name

Runtime startup2026-08-01 and 2026-09-09

Problem: After a runtime update, users on every GPU vendor, display server and distribution got an error popup, and the log said the compositor was not running while it was serving frames. The public issue filled with people reinstalling.
Instrument: The configuration was stripped to one variable and swung four times. The failure tracked a wrapper script at all three observable stages. A four-cell matrix later separated the wrapper from the name it gave the process.
Finding: The kernel caps a process name at fifteen characters. The wrapper produced a seventeen-character name, the kernel cut it to fifteen, and a name check never matched.
Outcome: Fixed by moving the binary instead of renaming it. The notes record that the correct hypothesis had been rejected once on partial evidence.

15characters the kernel keeps

17characters in the name

3 files36 lines added

4. A hundred times the motion, no shake

Latency and pacing2026-08-13

Problem: The world shook in the headset. Tracking noise, pose association and compositing were all suspects, and they could not be told apart by feel.
Instrument: Two probes. A pose probe logged each second the sample count, the head's movement per tracking step in micrometres, and the step between target timestamps. An association probe logged a histogram of frame-index steps per cycle with a line for any step other than one. Then one build and one variable: the runtime's async reprojection switch. The input motion of each arm was a number, not an impression.
Finding: Instead of improving the good arm, the experiment made its input a hundred times worse. Async off with the head held still measured 15 to 25 micrometres per 4.6 ms step, and the world pulsed with the wearer's heartbeat. Async on while deliberately shivering measured 450 to 2,340 micrometres, peaks of 4,700, and the world stayed fixed. Tracking ran at 216 Hz in both. Association was 99.2 percent perfect over 208,800 cycles.
Outcome: Tracking and association were cleared together, and compositing owned the whole symptom. The association probe also found a pattern nobody was looking for: pairs of skipped records every 256 cycles.

15 to 25 µmhead still, world shakes

450 to 2,340 µmshivering, world fixed

99.2%clean association over 208,800 cycles

5. 7,800 microseconds to 2

Latency and pacing2026-08-18

Problem: Render, encode, a blocking read, bitstream parsing and the network send all ran inline on the compositor's frame loop. Everyone agreed that was bad. Nobody had measured it.
Instrument: The per-frame cost on the compositor's loop, measured before and after moving the chain to a worker behind a single-slot mailbox where the newest frame wins, with a generation counter and an age cap.
Finding: The compositor's cost per frame went from 7,800 microseconds to 2.
Outcome: Two files, 133 lines added, no dependency on any other change. The commit message predicted no felt difference. That prediction was wrong, and the notes say so.

7,800 → 2 µsframe loop cost

2 files133 lines added

6. Decomposing a build that felt good

Process2026-08-19

Problem: Thirteen changes shipped as one build that felt good. Nobody could say which of them mattered, so nothing could be acted on.
Instrument: Each change was put on the clean base alone and measured with an identical settings fingerprint and the prediction cap held constant.
Finding: Two changes everyone assumed were carrying the improvement moved no numbers, and both were demoted in writing. One produced a stationary world while making every number worse; four explanations were put forward and killed before one survived.
Outcome: The series ended one change longer than the build it took apart, because the decomposition found a piece the original had never combined correctly.

13changes run alone

2assumed wins that moved nothing

7. The ratchet

Latency and pacing2026-08-20

Problem: After a congestion event the stream latched to about a third of its rate. It survived a client restart and a headset reboot, and only restarting the PC side cleared it.
Instrument: Every candidate was measured live and eliminated: application threads idle, GPU at 6 to 54 percent, memory pressure 0.07, network 4 ms and clean, zero keyframe requests, zero throttle hints, the encode worker idle in its wait. The driver's own counters confirmed the latch at 21 to 25 presents a second.
Finding: A ring. A frame that spans two grid ticks halves the declared refresh rate, which halves the application's schedule, which keeps the frames long. Missed ticks are never announced, so each disturbance ratchets one way: 44, 27, 23, 18, then 13 frames a second over one morning.
Outcome: Fixed, and certified against the failure instead of an average. The test ran the failure's own round trip: empty scene, heavy scene, then back. The rate went 71.8, then 28 to 40 while the heavy scene loaded, then straight back to 71.9 with no restart.

44 → 13 fpsone morning, one way

71.9 fpsrecovered after the fix, no restart

8. The colour chain

Colour2026-09-17 and 2026-09-18

Problem: Scenes looked too dark in the headset while the game's own frames were correct.
Instrument: Every encode and decode along the path was counted, seam by seam. Every seam after the application's swapchain is a matched pair, so the one unaccounted seam was the swapchain's format. The two possible faults predict two specific bytes for a flat grey of 128: 55 for one decode too many, about 186 for one encode too many. A grey backdrop and a sixteen-patch chart were read at the far end.
Finding: The grey arrived as 55: one decode too many. The chart then found a 3 percent chroma shortfall, and three runs placed the correction by arithmetic. Run 3 moved the saturated patch from 246,5,247 to 250,3,250 with every grey within a count or two.
Outcome: The first fix added the missing encode in a shader, which worked but sat at the wrong layer. The driver had asked for compute access to its shared images and never used it. Storage images cannot be sRGB, so that request turned the image linear. Dropping the request replaced the shader change, a push constant and an event hook with one boolean.

128 → 55one decode too many

3%chroma shortfall found by chart

9. What each channel lies about

Tooling2026-08-20

Problem: The only sensor inside the headset is a person, and every sensor outside it is a channel with its own blind spots. Half the wrong turns in one session came from trusting the wrong window.
Instrument: A written map of every channel into a running stream: what it is good for and what it gets wrong. It also lists seven things the system cannot see and what each would cost to add.
Finding: The event stream drops messages silently under lag: one capture showed a single clamp warning where the log file held 37,781. A run written up as having clamp warnings "effectively gone" held 347,809 of them in its own log. The historical log carries empty payloads, so no past session can answer a rate question. The server frame rate counts encode completions, not presents.
Outcome: Counts are taken from the file, never from the live stream. The clamp warning was reclassified from a defect flag to a readout.

1 vs 37,781warnings seen live vs in the file

10. The audio metronome

Network2026-08-21

Problem: Thousands of audio dropouts looked like ordinary network loss, and a hardcoded sample rate was the obvious suspect.
Instrument: The interval between losses, mined from five archived logs. Then a probe that printed two things at the same instant: the size of each gap, and how many buffers the receiver had free.
Finding: Loss to recovery had a median of 60.0 ms, exactly the configured buffering. The interval between losses was 282 ms, stable to a millisecond across 29 minutes and in four other sessions, with three sharp modes. Random loss produces a smooth geometric spread, not sharp modes. The probe showed gaps of exactly one packet after zero evictions, which cleared the receiver.
Outcome: Something ticks every 280 ms and drops a packet on about half the ticks. No timer in the software matches 280 ms, so the search moved off the machine to the wireless link.

282 msbetween losses, ±1 ms

60.0 msloss to recovery, the buffer size