Research engineering I designed, built and operate end to end.
When More Self-Play Makes an Agent Worse
Working paper · October 2026 · first author
An empirical study of training instability in a MuZero-style agent in a custom two-player combat environment. Head-to-head strength peaks and then declines with more self-play, even while training loss keeps improving.
- Controlled sweeps over learning rate, model width and training length; none reliably prevents the decline.
- Continuing a strong checkpoint against a pool of historical opponents is a preliminary stabilisation result.
- Evaluated by head-to-head tournaments between frozen checkpoints, not loss curves.
MuZeroMCTSSelf-playPyTorchExperimental design
Agent strength peaks after
24k–32k games
of self-play, then declines while loss keeps falling — a reminder that loss is not a proxy for playing strength.
Preliminary and not peer reviewed; no causal mechanism is claimed.
GPU farm: budget-driven experiment scheduler
Infrastructure · Python · RunPod · Cloudflare R2
Rents GPU pods within a daily budget and keeps every GPU busy with research jobs, with the next job already waiting when one finishes.
- Packs jobs per GPU by measured utilisation, free memory and CPU load; low-priority fill-ins yield to real work instantly.
- Detects slow or stalled pods from live GPU telemetry within ~10 minutes and drains them safely.
- Hot-reloads the on-pod daemon without killing running jobs; checkpoints and models persist to object storage.
SchedulingDistributed jobsObservabilityCost control
MuZero-style agents, live in the browser
RL · Model serving · TypeScript
Agents with a learned dynamics model and tree search, trained by self-play and exported to ONNX. Checkpoints play each other live on a dashboard using onnxruntime-web with WASM threads.
- Tournament leaderboard of saved checkpoints across arena and chess environments.
- Custom environments with a public JavaScript / Python agent API.
ONNXonnxruntime-webReactEnvironment design
Automated research loop
LLM agents · Experiment tracking
An LLM planner proposes hypotheses and code changes, queues experiments on the GPU farm, and reads results back. Separate confirmation games check that improvements hold up, and failed experiments stay in the record.
LLM agentsReproducibilityAutomation
Lana A.I. — GPT-3 tutor
LLM product · launched April 2022
Built and launched a GPT-3 tutoring chatbot used by students at the University of Hawaiʻi at Mānoa, about seven months before ChatGPT was released.
GPT-3PromptingProduct
Target drift in deep Q-learning
Interactive explainer
A MountainCar teaching demo of the moving-target problem in DQN and why a slowly updated target network helps.
DQNVisualisation
