★ View on GitHub · README · Named comparison (Jev, Opus, local LLMs) · Pre-registered plan
A decision that arrives late is a wrong decision.
linearone-local is a small classifier in C that runs on one CPU thread in ~10-20 ms, with no network. Below: what that speed is worth, and, honestly, what it is not.
1. Flap: same answers, different speed
Two birds fly the same course. Every tick both are handed the identical correct answer (the game's expert rule); the only difference is how long an answer takes. The left bird answers in 12 ms (L1's measured speed). The right one takes as long as you set below (a hosted API from a laptop measured about 430 ms).
games/flap_env.py. In this game the bird still copes up to about 100 ms, mostly crashes by 200 ms, and at 430 ms never gets through the first pipe. In our scored benchmark a hosted model given the right answers scored 0 pipes at its real latency and 9 (the cap) with latency removed.2. Snake: same answers, two speeds
Both snakes use the same safe-greedy answer for every move. The local one decides in about 12 ms; the hosted one takes the latency you set. Food eaten in the same time is the whole difference.
3. The latency budget: what fits in one decision?
4. Rubik's cube: where speed does not help
A 2x2 cube where the exact answer is known by search. Scramble it, then try to solve it, or ask for the optimal move. In our test no language model, embedding model or hosted API did better than about a coin flip on picking a good move; only models fed the raw stickers reached 50-63%.
Three players make greedy moves. Coin flip picks a random move. Nearest neighbour is a real learner trained right here in your browser on 200 solver-labelled states that never include the 20 test scrambles (the benchmark's recipe). Exact solver is breadth-first search.
games/cube_run.txt. Optimal-move rate, not solve rate: nothing except the solver reliably solves it.5. The recorded Snake clip: L1 learning from corrections
The same Snake games, played live by L1 (taught by 25 corrections, no retraining) and by a hosted model. Recorded 2026-09-26.
Who is who
| L1 | this repo's model: int8 bge-small-en-v1.5 encoder + nearest-example memory, C runtime |
| Jev | TypeSafe's hosted decision API, model id jev-latest; our runs 2026-09-23 to 26, untuned first-attempt prompts |
| Claude Opus | alias "opus" via Claude Code subagents, batched answers (not independent API calls); exact model version not pinned |
| Local LLMs | llama3.2:3b, qwen2.5:1.5b and 0.5b under Ollama on an 8-vCPU box |
COMPARISON.md.Where to use it, and where not to
| Use | Evidence |
|---|---|
| First-pass triage before a bigger model or a human | NIST-CSF: L1 took the surest 40-50% at 93-97% accuracy; cascade 0.82 vs Opus 0.83, saving 40-50% of Opus calls |
| Real-time loops (games, UI, sensors) | Flap: hosted scored 0 pipes at 430 ms |
| Instant corrections without retraining | Snake: 25 corrections, then plays alone |
| Offline/private tagging; high-volume cheap routing | hypotheses, not demonstrated here |
The honest part
- Speed is the advantage. A local model can answer inside a real-time loop; a network call cannot.
- Accuracy is not. On all three games a decision tree or embedding + logistic regression on the same data matched or beat L1's nearest-example rule. L1 itself scored 0-2 pipes in Flap where the tree got 9.
- The games are toys with a few dozen state texts. They show latency and instant correction, not that ML is needed.
- One run each on held-out seeds; the plan and its amendments (including two bugs found before scoring) are in
games/PLAN.md.
Code, results and how to reproduce: README · COMPARISON.md · games and logs · Apache-2.0 · github.com/arshmakker/linearone-local