Clark Whitehead

Some of the most powerful ideas don’t work at toy scale.

The benchmark · Skirmish · 14 levels · November 2026

Does an AI that learns beat one that takes notes?

Two copies of the same AI play these levels. One writes everything down and rereads its notes before every move. The other changes its own weights after every level. We keep adding levels until one falls behind.

Battle 1 of 14World ALearnLearn —Remix —Return —

World ANew world · you are Blue

Why this game

Nothing is explained. Every level is won with a move you pick up by playing: hold a hill, go through a portal, stand on a switch.

Those are skills, not facts. A notebook can say “use the portal” but not when, or how fast, or which unit. Across enough levels the notes outgrow what the AI can reread, and the learning player keeps the same size.

If the notebook player falls behind, learning in the weights matters. If the two keep pace, notes are enough and the bet is wrong.

All 14 levels

Six new worlds, three that mix two of them, five earlier worlds again from the other corner. The enemy base sits inside a ring, and the way in is a move an earlier level made you use.

  1. 1World A · new worldLearnTeaches you to hold the hill.
  2. 2World B · new worldLearnWay in: hold the hill. Teaches: go through a portal.
  3. 3World C · new worldLearnWay in: go through a portal. Teaches: stand on a switch.
  4. 4World D · new worldLearnWay in: stand on a switch.
  5. 5World E · new worldLearnWay in: go through a portal.
  6. 6World F · new worldLearnWay in: hold the hill.
  7. 7World G · two worlds mixedRemixWay in: stand on a switch and go through a portal.
  8. 8World H · two worlds mixedRemixWay in: go through a portal and stand on a switch.
  9. 9World I · two worlds mixedRemixWay in: stand on a switch and go through a portal.
  10. 10World B · an earlier world, from the other cornerReturnWay in: go through a portal.
  11. 11World C · an earlier world, from the other cornerReturnWay in: stand on a switch.
  12. 12World D · an earlier world, from the other cornerReturnWay in: hold the hill.
  13. 13World E · an earlier world, from the other cornerReturnWay in: stand on a switch.
  14. 14World F · an earlier world, from the other cornerReturnWay in: go through a portal.
One test run14 battles in order against the same bot, by one player that is never reset1 Learn6 new worldsABCDEF2 Remixtwo rules combinedG = B + DH = C + EI = D + F3 Returnold worlds, other cornerB againC againD againE againF againNothing says what a world changes. The enemy base sits behind a ring; the way in is a mechanic an earlier battle made you use.
How a battle is scoredscore = par ÷ your time. Par: the fastest win 216 scripted strategies found against this bot.par = 1:03Won in 1:03100%Won in 2:0650%Lost at 1:400%Each part scores the mean of its battles; the total is the mean of Learn, Remix and Return.
The written plan All research All games