Новая система Schema помогла ИИ достичь 99% эффективности в сложнейшем бенчмарке ARC-AGI-3

Frontier Models with Our Harness Achieve ~99% on ARC‑AGI‑3 Public We introduce Schema, a harness that has frontier models play like physicists: write each game’s mechanism as an executable program, test it against reality, and plan inside it. The name is inspired by Immanuel Kant: a schema is a rule of construction that bridges from abstract concept to concrete perception. 1 Impossible Research 2 University of California, Berkeley 3 Carnegie Mellon University * Project lead ARC-AGI-3: games that don’t tell you the rules ARC‑AGI‑3 gives an agent a game environment, without an explanation of what it is seeing. At each step, the agent receives a 64×64 grid of 16 color indices and a set of legal actions. The environment supplies no object list, rule sheet, stated goal, or shaped reward. There is only one way to make progress: the physicist’s way. The agent must act while its model of the game is still provisional, forming hypotheses about what the grid represents, how actions change it, and what counts as success, then revising both its model and its plan as new observations arrive. ARC‑AGI‑3 has proved exceptionally difficult for frontier models. Its official metric, Relative Human Action Efficiency (RHAE), compares an agent’s per-level action count with a first-exposure human baseline and aggregates the result across environments; 100% means completing every level of every environment at or above human-baseline action efficiency. On the Semi‑private set, verified frontier-model performance rose from 0.51% at launch in March to 7.78% with GPT‑5.6 Sol at max reasoning in July. Sol also scored 13.33% on Public set, but still far from the human reference. Both scores come from a fixed fallback rule: Opus 4.8 and Sol xhigh run first; games scoring below 80 are rerun with Fable 5 and Sol max, respectively, and the higher per-game score is retained. The takeaway: how you use the model matters a lot. The rest of this post shows the arrangement. Thinking like a physicist Schema thinks like a physicist. Before a physicist can write down a law, they must decide what the law is about. Which parts of an observation are objects? Which properties define the state? Only then can they ask how that state changes. Schema formalizes these questions as two problems. State grounding turns raw observations into objects, variables, and relations that can be tracked. Mechanism discovery finds how that state changes under an action and writes the rule as an executable program. Our earlier system, VIGA, focused on the first problem. Through analysis-by-synthesis against a graphics engine, it recovered scene programs from continuous visual input far richer than an ARC grid. Other systems like WorldCoder focus on the second: they learn transition programs from trajectories but begin with an already structured state representation. We believe these two problems must be solved jointly, with the agent constructing the state representation and the transition rules. State grounding and mechanism discovery cannot be resolved independently. A state representation that appears plausible at first may prove inadequate when no consistent transition rule can explain the outcomes of subsequent experiments. In Schema, the state representation and transition rules are jointly encoded in the same editable program. When an observation contradicts a prediction, the agent can revise either the representation or the rule, then update the other to restore a consistent model. This mirrors how physicists work: when predictions fail persistently, they do not only adjust the law. They change what the state is. The cleanest case is the birth of special relativity. When Michelson and Morley could not detect the medium light was supposed to wave in, Lorentz took the first route: keep the aether, patch the rules with contraction hypotheses that absorbed the null result. Einstein took the second: in special relativity, he discarded the aether as part of the state and made simultaneity frame-relative, yielding a simple electrodynamics of moving bodies. ARC‑AGI‑3 requires the agent to construct both layers from raw observations. The grid identifies neither objects nor goals, so the agent must decide which patterns correspond to entities such as the player, walls, and counters, infer how actions transform them, and determine which configurations count as progress or completion. Even the goal predicate—implemented in Schema as is_goal—must be inferred from interaction. Core idea. The latent world representation is a program, not a vector — so it is interpretable (a text file you can read and diff), verifiable (replayable against recorded reality, belief by belief), and searchable (a program is a simulator; planning inside it is free). What the numbers say The metric, in full. The official ARC‑AGI‑3 evaluates performance in Relative Human Action Efficiency (RHAE), which compares the number of actions used by the agent with an upper-median first-time human baseline for each completed level. The per-level score is (human actions ÷ agent actions)², capped at 1.15. Within each game, levels receive increasing weights from 1 to n, so later levels contribute more to the game score. A completion cap based on the same weights prevents a game from receiving 100% unless every level is cleared. The benchmark score is the average of the game scores. All environment actions, including exploratory ones, count toward the agent’s total; because the efficiency ratio is squared, additional actions reduce the score sharply. Thus, 98.98% is an aggregate measure of completion and action efficiency—not the percentage of games solved. The Claude row (Figure 5) provides a controlled comparison with the same Opus 4.8 and Fable 5 pairing. Treating the final Claude Code scratch snapshot as the baseline output produces a score of 42.83%; with Schema, the same pairing reaches 98.98%, an increase of 56.15%. With the Sol xhigh and Sol max fallback pairing, Schema reaches 95.35%. The nearest official reference is not a matched harness comparison: ARC Prize reports the best individual variant, Sol max, at 13.33% on Public and 7.78% on Semi‑private. The 82.02% Public-set difference shown in the chart is therefore contextual, not a controlled estimate of the gain produced by Schema. The controlled Claude comparison isolates a difference in process rather than underlying model capability. Claude Code provides a strong general-purpose harness for file operations and long-context management. In principle, it can mimic Schema’s workflow, but it does not require the model to do so. Under the generic harness, the model acts on the game directly and may keep its beliefs implicit in context. Schema instead enforces three constraints: Encode the current world model as a runnable step() program. Validate that program against every recorded transition before using it for planning. Send actions only through commit_actions, immediately discarding the remaining plan after any prediction error. Under this controlled configuration, Schema’s harness yields a 56.15% improvement over the Claude Code baseline. Verification status. The 98.98% and 95.35% scores are self-reported results on the Public set, calculated from the run artifacts released with this post. Neither score has been independently verified by ARC Prize. How much does the public set indicate? Every Schema number here is measured on the 25 public games. Sol max supplies one official calibration point across evaluation sets — 13.33% public versus 7.78% Semi‑Private — but that does not justify numerically extrapolating a near-ceiling public score. What 98.98% public maps to on Semi‑private is unknown until it is measured. The available artifacts document public-set runs only, so this draft makes no frozen-harness or held-out-performance claim. The pairing ledger. Among the retained Claude results, 14 games use the Opus run and 11 use the Fable run. 13 of the 14 Opus runs and 6 of the 11 Fable runs score exactly 100, yielding 19 games with a score of 100 overall. Among the retained Sol results, 15 games use the xhigh run and 10 use the max run. All 15 xhigh runs score 100; of the 10 max runs, 5 score 100 and 5 fall below 100. The residual is concentrated. The Claude pairing falls 1.02% short of 100 overall. Nineteen games score exactly 100, while the remaining six range from 89.87 to 99.10; the median game score is 100. The Sol pairing also has a median game score of 100: 20 games score exactly 100, while the remaining five range from 60.93 to 87.80. Case studies Observation 1: An induced world program makes action efficient In the 14 of 25 games where the agent induced a program world model that exactly reproduced its recorded history, it used 1.6–5.0× fewer actions than the human reference. The reason is a shift in where actions are spent: the agent pays real actions only once, to discover a mechanism, and then plans inside the model for free instead of rediscovering the mechanic through trial and error. This depends on two capabilities: A. Verify against complete history. Every real transition is logged, and run_backtest checks each candidate rule against the entire record rather than relying on recollection. Because the record survives both bounded working memory and the auto-compacted context window, the certified model can be trusted in place of further real-environment testing. B. Plan inside a reusable simulator. Once step() and is_goal() are certified, breadth-first search explores 103-104 modeled states without spending a single environment action. And because the model persists as code, this free planning repeats on every later level. The payoff is sharpest where humans struggle most: on M0R0 Level 4, the agent needs 42 actions to the human's 500, consistent with paying the discovery cost once and recomputing plans inside the model thereafter. Each expandable item below is one game-level case supporting this pattern. Evidence 1A · RE86exact verification supports a one-shot plan393/393 exact · 61-action plan RE86 shows the full sequence in one game: run_backtest replays the model over every recorded transition, then the verified model supplies a plan that clears a whole level without additional trial and error. RE86—Schema w/ Opus 4.8 Certify the model against all recorded history393 / 393 exact after 394 stepsstate394 real environment steps recorded so far — 393 checkable, 1 skipped (a reset with no prior grid). events.jsonl:3144backtestThe model, replayed over the entire history, matches every prediction: backtest [all transitions]: 393/393 transitions fully correct (grid on non-terminal steps + level_up/dead/win flags on EVERY step); 0 mismatch(es), 1 skipped. readingmeaning“My world model reproduces all 393 real steps I have taken, exactly.” This is retrodictive consistency against ground truth, at a scale — hundreds of exact 64×64 frames in one operation — with no counterpart in unaided human play. The same appears across the winning games: KA59 391/391, SK48 387/387, AR25 238/238, M0R0 182/182. A whole level cleared by one verified plan61-action one-shot → WIN KA59 isolates the memory advantage: exact records from two complete runs reveal that an apparently successful rule only fit by coincidence. KA59—Schema w/ Opus 4.8 A refutation that needs total recall of two complete runsepicycle killed events.jsonl:314model / notes.mdIt compares two entire past move-sequences exactly and discards a rule that only coincidentally fit: "...('H costs, horizontal free' R1, and 'H costs 1/2' R2) were EPICYCLES that happened to fit. Killed by: two runs with completely different direction sequences produced the SAME cost pattern +1,+0,+1,+1,+0 by move index." readingmeaningDetecting that two full runs share a cost signature requires holding both, exactly — the concrete form of the memory asymmetry. A person relies on gestalt; the agent disproves a plausible rule with complete-history evidence. Evidence 1C · M0R0search replaces trial on a difficult level42 vs 500 actions M0R0 isolates the planning advantage: once the model is exact, the solution is searched inside the simulator rather than discovered through further real-environment trial. M0R0—Schema w/ Opus 4.8 Search replaces trial on the human's hardest level42 vs 500 actions events.jsonl:56policy"...run_backtest green before planning with BFS — rather than hand-playing moves, which is the fail[ure mode]..." events.jsonl:84searchBreadth-first search inside the verified model: BFS: goal in 19 step(s) via level_up; expanded 3300 nodes, 891 distinct states. baselinevs humanThe human spends 500 actions on this level (their plateau); the agent commits the 19-step plan and clears it in 42. Search cost does not scale with how hard the level “feels.” Observation 2: Fable reaches the correct world model through better experimental decisions To understand how model choice affects world-model discovery under the same executable harness, we compare Schema w/ Opus 4.8 Max vs Fable 5 on games attempted by both models. Fable achieves higher RHAE across these games in two ways: on some games, it discovers a model that allows it to complete levels Opus cannot; on others, both models finish, but Fable reaches the same endpoint with fewer environment actions. The paired traces suggest a common explanation. After a failed prediction, Fable more often questions the current representation itself, identifies the uncertainty that matters, and chooses an interaction whose possible outcomes distinguish between competing hypotheses. It then converts the observation into a reusable transition rule. Opus often eventually discovers the same mechanism, but only after spending more actions testing alternative targets, conditions, or configurations within its existing representation. The matched-completion cases are particularly informative: because Opus eventually encodes the same mechanism, the action gap cannot be explained simply by one model being able to express the rule and the other being unable to. The difference lies in the discovery process—what to question, which experiment to run, and when to turn the result into executable code. Evidence 2A · DC22a complete search over a graph missing one edgeSchema w/ Fable 6/6 · Opus 2/6 Opus built a detailed model of movement and three toggles. From a fresh reset, BFS later exhausted every state reachable in that model and correctly concluded that the target was unreachable. Fable exposed a transition that was absent from the modeled graph. FableOccupancy changes the meaning of a paired region175 / 175 exact failed hypothesisrevision 1Fable first treated the colored regions as phase-keyed item boxes. Another ordinary swim falsified that explanation. before step 176new hypothesisPlain swim again — duals aren't phase-keyed item-boxes. But NEW idea: the two duals are EQUAL-SIZE slots — a cell-wise swap could CARRY e! interventionprobeFable entered one slot and triggered the paired control. Its current model predicted a penalty and no swap, so occupancy made the experiment discriminating. step 176observationTHE PORTAL WORKED!!! ... The duals were never state-flippers: they're a paired CELL-WISE CONTENT SWAP (e rides along!). world model v5code revisionThe avatar moves because it is content, without an avatar-specific teleport exception: for (xa, ya), (xb, yb) in zip(_cells(a), _cells(b)): va, vb = grid[ya][xa], grid[yb][xb] if va != vb: grid[ya][xa], grid[yb][xb] = vb, va after backtestresultThe revised model reproduced 175/175 recorded transitions. Fable then cleared the level with a 24-action route through the newly reachable island. Observed pattern: search did not invent the transport interaction. Fable's useful change happened one layer earlier, by representing region contents and occupancy as part of state. Evidence 2B · LF52the cart was a board cellSchema w/ Fable 10/10 · Opus 4/10 Both models recognized peg solitaire and learned that a cart moved along tracks. The decisive issue was how the two mechanisms composed. Fable first excluded the useful interaction too, then drove the cart to an exact docking coordinate and treated that moment as an observation point. FableTwo object systems become one transition graph10 / 10 levels initial inferencefalse negativea peg can only be selected if it has ≥1 legal jump, meaning c5-over-c6-into-pipe/box is NOT legal. cart discoveryobservationAn arrow moved the box along the rail. Fable then drove it to the dock, explicitly expecting the docking step to reveal how cart and board composed. at the dockboundary revisionThe cart docked — with a surprise: the board reflowed! ... cart core ... sits flush as an extra cell next to c6. world model v5code revisionFixed cells and mobile cart cores enter the same cell map: def _find_cells(world): cells = detect_fixed_cells(world) for x, y in _find_carts(world): cells[(x, y)] = _cart_kind(world, x, y) return cells transferresultThe ordinary jump generator now handled jump-in, transport, and jump-out. Fable reused the representation across later multi-cart and scrolling levels. OpusA related probe tests the wrong causal roleeventual convergence early testprobeOpus asked whether a peg could jump over the cart. The result correctly rejected “cart as middle piece,” but did not test “cart as landing cell.” ~100 turns latertest repairPrior test clicked c6 not c5 ... c5 over c6 lands EXACTLY at ... box dock ... THIS combo untested. exact geometryobservationBREAKTHROUGH! ... a color-2 landing marker appeared on the box ... the box IS a valid peg landing! Opus then revised to the same transport mechanism. Observed pattern: a negative experiment rejects only the mechanism-instance combination actually tested. Fable found the useful composition earlier by testing the cart at the exact source–middle–landing geometry. Evidence 2C · SB26generality appears as structured revisionSCHEMA w/ Fable 135 vs Opus 456 actions Both models discovered a DFS-like rule after Level 1 and both eventually solved all eight levels. This pair separates “stating a general idea” from maintaining a reusable coded abstraction that absorbs later variation. FableRecursive mechanism with later extensions8 / 8 · 135 actions Level 1structural hypothesisHypothesis: the code reads depth-first ... Let me generalize the model with boxes/portals/DFS target assignment. Fable submitted the complete generated assignment, making the hypothesis strongly falsifiable. The level cleared. world model v5code abstractiondef traverse(box): if box in visited: return visited.add(box) for kind, obj in sorted(elements_inside(box)): if kind == "marker": assign_next_target(obj) else: for child in boxes_for_portal(obj): traverse(child) Levels 3–6structured revisionsFable still changed the model: portals became movable; a global visited set became a path-based cycle guard; depth-three nesting appeared. The stable part was the causal schema—hollow objects reference containers, and traversal generates assignments. later levelscompressionAfter the first DFS hypothesis, later level increments were approximately 15 / 15 / 15 / 17 / 19 / 17 / 28 actions. OpusSame first idea, less robust transfer8 / 8 · 456 actions Level 1confirmationLevel 1 cleared! The DFS-traversal target rule is confirmed. later variationhypothesis sweepsOpus repeatedly switched from structure to assignment conventions such as contiguous color blocks and grid reading order when new shapes appeared. Level 6missing state16 turns stuck on L6 ... The missing variable was tile shape — TEST3's colors were right all along, but the two hollow tiles must go in the middle column slots. Observed pattern: a general mechanism is not one that never changes. It is one whose revisions preserve earlier explanatory machinery and compress the cost of new levels. Summary. The harness and the underlying model affect efficiency at distinct stages: The executable harness reduces the cost of using a theory. It makes the theory persistent, exactly verifiable, and searchable. This is the source of the efficiency pattern observed relative to the human reference. The underlying model determines the cost of discovering a useful theory. Paired runs sometimes converge on the same representation through markedly different trajectories. Fable more often makes the decisive representational revision earlier and therefore reaches an executable solution with fewer environment actions. Search is only complete relative to the world model it operates over. Once the missing object, state variable, or transition is represented, the existing verifier and planner are often enough. If observations do not distinguish the right representation, the same machinery can verify and exhaustively search the wrong one. This is a new beginning ARC‑AGI‑3 is an important benchmark in a rapidly advancing field. When it launched in March, the leading frontier agent scored just 0.51%. By July, the best official Sol Max result had reached 13.33% on Public and 7.78% on Semi‑private, while self-reported results climbed as high as 58.12% mean per-game RHAE. Our self-reported 98.98% continues this trajectory rather than breaking from it. The key ingredients were already emerging: frontier models capable of adapting to novel environments, coding-agent harnesses, and programmatic world models. Schema is the system that brings them together. We read a saturated ARC‑3 as the new beginning: mechanism discovery as a general capability — grounding the causal structure of a world through the agentic loop of action and perception, in environments far richer than a 64×64 grid. This is where we are heading to. Cite this work BIBTEX @misc{schema2026, title = {{[schema]: Frontier Models with the Right Harness Achieve $\sim$99\% on ARC-AGI-3 Public}}, author = {Zeng, Guanning and Wang, Jiani and Ma, Wenjie and Yin, Shaofeng and Wang, Chenyang and Liu, Shichen and Kanazawa, Angjoo and Ni, Wode and Li, Xiuyu and Zanette, Andrea and Feng, Haiwen}, year = {2026}, howpublished = {Impossible Research. \url{https://schema-harness.github.io/}}, url = {https://schema-harness.github.io/} }