FunSearch Found New Mathematics Without Ever Understanding the Problem

A cap set is a subset of the grid $\{0,1,2\}^n$ with one rule: no three distinct points in it are allowed to sum to zero coordinate-wise modulo three. The largest such set in eight dimensions had been 496 points since 1994. In December 2023, a Google DeepMind system called FunSearch reported one with 512 ^[1]. DeepMind’s own announcement called it the largest jump in cap set sizes in twenty years ^[3]. Both of those claims check out — this article verified the 512-point construction and the ablation data behind it directly against the paper’s own supplementary material, not against the press release alone ^[2]. The question this article asks is not whether FunSearch found something real. It did. The question is what “found” means when the thing doing the finding is a loop that mutates one function, checks the result with a counting program, and keeps whatever counts higher — a loop that, by the DeepMind team’s own account, never once represents the cap set problem as a mathematical object at all.

That distinction is not academic hedging. DeepMind’s own supplementary material contains a comparison the headline never mentions: the identical search loop, with the language model deleted and replaced by a hand-tuned random mutation operator, run on the same task under the same evaluator. Given more than fifty million programs to work with — over twenty-five times what the language-model version needed to solve the same problem — the random version never gets there. It plateaus. That single reported number is the paper’s own best evidence for what a large language model is actually contributing to a search like this, and it is a narrower, more mechanical answer than “the system discovered new mathematics” suggests.

The Cap Set Problem Is a Counting Rule With a Fifty-Point Gap Nobody Has Closed

The cap set problem sits inside a branch of combinatorics concerned with how large a set of points can get before it is forced to contain a forbidden pattern. For cap sets the forbidden pattern is a three-term arithmetic progression: three distinct points whose coordinates sum to zero mod three. Mathematicians want to know how the size of the largest cap set in $n$ dimensions grows as $n$ grows, and the honest answer, as of this writing, is that nobody knows exactly — they know it grows like $c^n$ for some constant $c$, and they know $c$ is trapped between roughly 2.2208 and roughly 2.756, a gap that has resisted closing for over a decade ^{[7, 5]}. The lower end of that bracket is itself a short chain of automated searches, each one edging past the last. Fred Tyrrell’s 2023 paper used a SAT solver to push a decomposition technique originally due to Yves Edel to a set growing at $(2.218)^n$ ^[6]. FunSearch’s own paper then pushed that same lineage further, to $(2.2202)^n$ — not through the 512-point cap set this article opened with, but through a separate discovery, a large partial admissible set that a mathematician converts into a capacity bound by a different route ^[5]. A fourth automated search, by Eric Naslund, has since pushed it again, to $(2.2208)^n$ ^[5]. The upper end, roughly 2.756, comes from Jordan Ellenberg and Dion Gijswijt’s 2016 application of the so-called polynomial method, one of the most celebrated results in the field in the past decade ^[7].

Both facts matter for what follows. First: even the best available lower-bound technique going into 2023 was itself the product of an automated search, not a human insight arrived at by proof. Second, and this is the detail a casual retelling of FunSearch’s result tends to drop: Jordan Ellenberg, the mathematician whose name is permanently attached to the field’s best upper bound, is also a named co-author on the FunSearch paper ^[1]. The team that reports FunSearch’s results is not a group of AI researchers who stumbled onto a problem they didn’t understand and let a language model surprise them. It is an AI research team that recruited one of the two or three people alive with the deepest personal stake in exactly this open problem, to pose it correctly, check the output, and co-sign the result. That fact does not diminish what the search loop did. It sharpens the question this article is actually asking, which is not “is FunSearch’s answer correct” — Ellenberg’s presence on the byline is strong assurance that it is — but “what work did the search loop do, and what work did the human problem-framing and verification do instead.”

The specific number this article opened with, 496 rising to 512 in dimension eight, sits at the fixed-dimension end of the problem, where a mathematician doesn’t ask about the asymptotic growth rate but about one exact number for one specific $n$. FunSearch’s paper reports this as its headline cap-set result and its own supplementary material adds a detail the blog post omits entirely: across 140 independent runs of the search, only four ever found a set of size 512 ^[2]. Fewer than one run in thirty. That is not the signature of a system that understood the structure of the problem and derived the answer. It is the signature of a system exploring a landscape with a narrow, hard-to-find peak, succeeding often enough across repeated attempts to be worth reporting, and failing the overwhelming majority of the time.

The Evaluator Is a Counting Machine, and Counting Is the Entire Point

Every claim FunSearch makes rests on one component that gets far less attention than the language model driving it: the evaluator. For the cap set problem, the evaluator is a short, deterministic piece of code that takes a candidate set of points, checks the one closure rule — do any three points, including repeats, sum to zero mod three — and returns the size of the set if the rule holds everywhere, or a penalty if it doesn’t ^[1]. It has no notion of elegance, symmetry, or mathematical insight. It counts, exactly, the same way every time, and it returns a number a search procedure can rank.

This matters because it draws a hard boundary around what kind of claim “FunSearch discovered a cap set” can support. The evaluator’s correctness is exact and mechanical: given a proposed set, checking whether it is a valid cap set is a closed, fast computation with no ambiguity in the answer. That is precisely why the result is independently checkable by anyone — this is not a benchmark score that depends on trusting DeepMind’s own reporting, it is an arithmetic claim any reader with a laptop can verify against the published construction ^[1]. But the same feature that makes the result trustworthy also makes it a poor witness for anything beyond itself. An evaluator that only ever answers “does this set obey the rule, and how big is it” cannot distinguish a set found by a process that grasps why the rule produces the structures it does from a set found by a process that got lucky inside a space it was blindly resampling. The paper’s bin-packing demonstration runs the identical logic on a different problem: FunSearch’s evolved heuristic for online bin packing, tested against the classic OR-Library benchmark instances, consistently used fewer bins than the standard first-fit and best-fit routines that have anchored the field for decades, and on a harder synthetic Weibull-distributed workload of ten thousand items it left an excess of only 0.44 percent above the theoretical lower bound, against 3.90 percent for best fit and 4.20 percent for first fit ^[2]. Every one of those numbers is a count — bins used, or points admitted — verified the same mechanical way. Nothing in either evaluator ever asks the search loop to explain itself.

The Prompt Shows the Model Two Old Programs and Requests a Third

If the evaluator is the referee, the prompt is the entire conversation the language model ever has with the problem. And that conversation is much thinner than “AI discovers new mathematics” implies. At each step, FunSearch’s controller samples a small number of previously scored programs from a running population — the paper’s default setting samples two, labelled by version number — and assembles a single prompt where those two programs appear as sequential examples, followed by an empty function signature the model is asked to complete as the next version ^[2]. The function being evolved isn’t “solve the cap set problem.” It is a single subroutine, typically called priority, that scores how attractive it is to add one specific candidate point to a set under construction; the surrounding scaffold that repeatedly calls that subroutine, builds up the candidate set, and hands the result to the evaluator is fixed code the human researchers wrote once and never touch again ^[1].

Read that boundary carefully, because it is the crux of this article’s argument. The language model is never given a description of what a cap set is, why the closure rule matters, or what mathematical property a good priority function ought to have. It sees code — two working examples of a function with a numeric score attached to each — and it is asked to produce a plausible next variant of that code, the same task a code-completion model is trained on constantly, at web scale, for reasons that have nothing to do with combinatorics. The model’s job, mechanically, is pattern continuation over a tiny, self-selected corpus of two prior high-scorers. Everything that makes the eventual output a correct answer to an open mathematical problem — the fact that scoring points by this priority function and greedily building outward actually satisfies the cap-set constraint — is verified after the fact by the evaluator, never asserted or understood by the piece that proposed it.

One Generation, Traced Start to Finish

Put the evaluator and the prompt together and the full cycle is short enough to state in one paragraph, and stating it plainly is this article’s own contribution — no other piece in this publication’s coverage of evolutionary computation walks through FunSearch’s mechanism at this level of resolution. FunSearch keeps not one population but several — the released implementation defaults to running the search across islands, in the classical genetic-algorithm sense of the word: independent sub-populations that evolve on their own most of the time ^[2]. The public code release lets a reader confirm the shape of that controller directly, though DeepMind is upfront that what it published is the search-and-bookkeeping logic alone — the specific language models, the sandboxed execution environment, and the distributed scheduling that ran the reported experiments are all withheld, so the outer loop is checkable while the exact reported numbers are not independently reproducible from the repository alone ^[4]. Within an island, the controller samples a small cluster of programs, weighted toward higher scorers using a temperature-controlled selection rule, and groups programs by their behavioral “signature” — the tuple of scores they produce across different test inputs — so that the search doesn’t collapse onto one dominant strategy too early. The sampled programs go into a prompt; the language model returns one new candidate; the candidate is executed against the evaluator under a time and memory limit; anything that crashes, times out, or returns an invalid answer is discarded outright; anything that runs is scored and, if it beats what is already stored, added to the population it came from. Roughly every four hours of wall-clock time, the controller looks across all the islands, identifies the half performing worst, and kills them outright — not merges them, not down-weights them, kills them — reseeding each dead island from a copy of one of the surviving high-performing islands ^[2].

Read against the rest of this publication’s coverage of evolutionary computation, that description should sound entirely familiar, because it is a completely ordinary evolutionary algorithm: population, selection pressure, periodic culling of underperformers, diversity maintenance through behavioral clustering. The only place a large language model enters the loop is as the variation operator — the step that used to be a hand-coded mutation or crossover rule in a 1975-vintage genetic algorithm is, in FunSearch, a call to a pretrained code model. Nothing about the selection mechanism, the culling schedule, or the evaluator changed to accommodate the model. What changed is the source of new candidate code, and the paper’s own supplementary material is unusually direct about wanting to isolate exactly how much that one substitution is worth.

DeepMind’s Own Ablation Answers the Question a Press Release Never Asks

Appendix A.2 of FunSearch’s supplementary material runs precisely the test this article’s framing has been building toward: strip the language model out of the identical loop, replace it with a hand-designed random mutation operator — the paper’s own researchers built a library of arithmetic operations and array functions, and had the mutation step insert or modify one operation at a random location in the program’s syntax tree — and run all three variants, the default Codey-based model, an open alternative called StarCoder, and the no-LLM random-mutation control, on the same target problem under the same evaluator ^[2]. The target was a harder relative of the dimension-8 cap set problem: constructing a specific combinatorial object called a symmetric admissible set at size $(15,10)$, a structure whose full form the paper’s own ablation figure tracks generation by generation.

The result, reported without much fanfare in a footnote, is the sharpest single piece of evidence in the entire paper: “even after leaving ‘Random’ generate more than 50 million programs, none of the runs were able to surpass −240 and reached a plateau” ^[2]. The language-model-driven version solves the same task, at least some of the time, using on the order of two million programs ^[2]. The random-mutation version, given more than twenty-five times that budget, never climbs past two-thirds of the distance to a full solution and then simply stops improving. The paper’s own authors draw the obvious conclusion in the same section: “the use of an LLM is however clearly critical, as evidenced by the worse performance of the random mutations approach” ^[2] — while noting in the same breath that random mutation still did better than a skeptic might expect, which the authors read as evidence that other components of the design, particularly the fixed problem skeleton and the evolving-prompt structure, are also doing real work, not just the model itself ^[2]. A companion ablation, varying only whether the prompt evolves generation over generation versus using a single fixed initial prompt to sample a large batch of candidates once, shows the evolving-prompt version reliably outperforms the fixed one on the same task ^[2].

This is the discriminator this article’s argument needs, and it is a genuine one, not a reading imposed on the paper from outside: it rules out the deflationary explanation that FunSearch’s headline results are just what any sufficiently long-running evolutionary search would eventually stumble into, LLM or not. Twenty-five times the compute, and the non-LLM version did not stumble into it. But notice precisely what the ablation establishes and what it does not. It establishes that the language model’s proposals are systematically better-directed than uniform random perturbation of the same code — a real, measured, structural advantage in where the search looks next. It does not establish that the model has any representation of why one direction is better than another, and the paper never claims that it does. The gap between “produces better-directed proposals” and “understands the problem” is exactly the gap this article has been tracing since its first paragraph, and DeepMind’s own ablation is evidence for the first claim, not the second.

The Method Needs a Referee That Never Lies, and That Is a Real Limit

Both of FunSearch’s headline domains — cap sets and online bin packing — share a property that is easy to treat as incidental and is not: each one has a cheap, exact, fully automatable way to check any candidate answer. A cap set either obeys the closure rule or it doesn’t, checkable in time proportional to the cube of the set’s size. A bin-packing heuristic either uses more or fewer bins than a competitor on a fixed test instance, checkable by running it. Neither evaluator requires human judgment, partial credit, or interpretation at any point.

That is not a minor implementation detail; it is close to a precondition for the entire method working at all. A search loop built around “propose code, run it, keep what scores well” can only ever be as good as the thing measuring “scores well,” and when that measurement is cheap and exact, the search can run millions of times and trust every one of its own verdicts. Take away a cheap exact evaluator — ask FunSearch to search for something whose quality can only be judged by a slow simulation, an expensive physical experiment, or worse, a human’s aesthetic or theoretical judgment — and the entire loop this article has traced changes character, because now the referee is no longer free, no longer instant, and no longer certain. Nothing in the ablation data above speaks to what happens in that regime, because DeepMind never tested it: both reported domains were chosen, reasonably, because they are two of the relatively few open combinatorial problems where a definitive, near-free checker already exists. This is a real domain restriction on FunSearch’s demonstrated method, not a caveat that disappears on closer reading, and it is the honest answer to anyone tempted to read “FunSearch found new mathematics” as a template for how AI will find the next result in a field where verification itself is the hard part.

What Ernest Davis’s Objection Gets Right, and What Survives It

FunSearch has already drawn a direct published critique from Ernest Davis, a computer scientist at New York University who has spent years cataloguing overclaimed AI results, and his objection deserves to be engaged rather than summarized away ^[8]. Davis’s central complaint is not that FunSearch’s results are wrong — he does not dispute the cap set or bin-packing numbers — but that the paper’s framing implies a generality the demonstration doesn’t earn. He presses on exactly the human-dependency point this article raised above: Jordan Ellenberg’s role in posing the cap set problem correctly, interpreting FunSearch’s raw output, and vouching for its correctness is doing more of the intellectual work than a reader skimming “AI discovers new mathematics” headlines would assume, and Davis asks pointedly what other problems, if any, FunSearch was tried on and failed to produce anything reportable for — a question DeepMind’s own paper, understandably for a venue built around positive results, does not answer.

That objection is worth taking seriously rather than deflecting, and this article’s own tracing of the evaluator and the prompt independently supports the mechanical half of it: nothing in FunSearch’s loop, as built, could pose a new problem, decide which representation to search over, or recognize on its own that a result was worth reporting. Every one of those judgments came from the human team before the search ever ran, and from Ellenberg specifically checking the output afterward. Where this article parts ways with treating Davis’s critique as the final word is on the narrower, mechanical question this piece has actually been built around. Davis’s objection is a claim about the generality and framing of the result — legitimate ground, and one this article’s own domain-restriction section above concedes without qualification. It is not a claim that undermines the ablation’s own narrower finding: that within the domain FunSearch was built for, replacing the language model with random mutation, even at twenty-five times the compute, produces a measurably worse search. Both things are true at once, and a reader does not have to choose between them. The system’s proposal step measurably outperforms unguided mutation on a matched budget; the system also cannot pose its own problems, judge its own significance, or explain why its answer works, and every one of the framing choices that made the December 2023 result reportable at all — the problem, the representation, the verification — belongs to the twelve named co-authors, not to the code.

A Different Standard for the Next Headline Like This One

None of this diminishes the 512-point cap set or the bin-packing heuristic as results — both are real, both are checkable by a reader with no access to Google’s infrastructure, and this article’s own verification pass confirms them independently against the paper’s supplementary data rather than against DeepMind’s own summary of it. What it changes is what a reader is entitled to conclude from the fact that they exist. “FunSearch discovered new mathematics” is true only if “discovered” is allowed to mean what this article has spent its length pinning down precisely: a scored mutation loop, running a language model as its proposal step inside an ordinary population-and-selection procedure, checked at every step by an evaluator that can count but cannot explain, produced a construction that a human mathematician then had to recognize, verify, and vouch for. Take away the human framing and the exact evaluator, and nothing in DeepMind’s own reported ablation gives any reason to think the same loop would have found, or even known to look for, anything worth reporting at all. That is a real result, on a real problem, with a real and measured advantage over the obvious null model. It is also a considerably smaller claim than the one the headline usually gets credit for, and the difference between those two claims is exactly the gap the next system’s own paper should be checked against before its headline gets repeated either.