Visit Project → View on GitHub → See the post on X →
jev-drone is a camera-only autonomous quadrotor that flies a five-station obstacle course in MuJoCo while chasing a ground rover. Roman Slack built it to test one idea: put a small judgment model (TypeSafe's Jev) inside a live control loop, but only at the layer where judgment is actually needed. The drone is the Skydio X2 from MuJoCo Menagerie, flown by a real controller. Nothing about the flight is scripted.
The stack runs four loops at four rates. A flight controller runs at 500 Hz, guidance and a safety reflex at 50 Hz, and classical computer vision turns the onboard depth and segmentation image into a compact symbolic scene at 15 Hz. Jev sits at the bottom at about 2.5 Hz. It is not a vision model and never sees pixels: it reads the scene as JSON and answers three typed questions in one call (which maneuver, how risky, is the target truly lost). Code decides when a judgment is worth asking for, and code keeps the veto.
The result is deliberately narrow. With the model disabled, the same stack is safe but stuck: it steers toward the wider side, so it stops at 17.7 m on every seed when a low beam spans the whole corridor, because flying over an obstacle is not something that heuristic can express. With Jev engaged the drone climbs both beams, times the turnstiles, threads a sliding 3.2 m gate, loses and re-acquires the rover in a pillar cluster, and clears all 77.5 m with zero collisions at 0.11 s median decision latency. The write-up states the caveats plainly, including an earlier arena where the model showed no advantage.
Most of the work was the simulator and the controller, not the model. The write-up covers the lessons that cost real time: a geometric attitude controller that cannot recover from inverted flight and the quaternion fix, why a quadrotor controller must clamp desired vertical acceleration, why lateral control needs a position loop, and why a network call in a control loop means the sim has to be paced to real time.
Key Features
- Camera-only navigation: 64x48 depth and segmentation, no map, no GPS, no ground truth
- Judgment model in the loop at about 2.5 Hz, answering typed Choice, Score and Noul questions over a JSON scene
- 50 Hz safety reflex that overrides the model, and a veto on physically impossible proposals
- Ablation: baseline stalls at 17.7 m on 3 of 3 seeds, Jev run clears all 77.5 m with zero collisions
- Quaternion attitude control that recovers from 179 degrees of tilt
- Flight tapes that replay a run to video with no physics and no API calls
- Research-style project page with figures drawn from the recorded flight data
Tech Stack
Designed and built by Roman Slack, Lead AI Platform Engineer. See more of Roman Slack's work on the projects page or get in touch via the contact page.