#contamination

Tagged contamination · show all articles

A row of six identical evaluation rigs on a bench, each wired to a shared manifold by a different-coloured cable, the nearest rig's door still swinging with its test card only half seated

Ten Failure Modes That Show Up Across Every Frontier Model, Not Just One

Leaderboards rank one model against another. They were never built to show what several of them do the same way, and ten failures recur across vendors no matter who is winning this quarter's chart.

By Brecht Corbeel · August 12, 2026 · 16 min read

A sealed kraft-paper holdout-problem envelope caught half over the threshold of an open steel security cage, its tamper tape peeled back on one corner beside an open custody ledger with a pen resting mid-stroke

Model Evaluation

OpenAI and Claude on Formal Reasoning: What the Benchmarks Show, and Where They Mislead

AIME, GPQA and FrontierMath test different things by different rules. Reasoning effort and extended thinking are documented settings, not fixed traits, and a right answer does not certify the reasoning behind it.

By Brecht Corbeel · August 12, 2026 · 16 min read

A labelled drive caddy tray on a bench beside a short rack of eval worker nodes, one caddy caught half-seated in a worker's front bay with its latch still open and the tray's two rows left visibly unequal

Foundation Models

Measuring Frontier Models: Contamination, Variance, and What a Score Can Support

A benchmark result is an assay: it depends on how the sample was taken, what impurity was already in the reference, and how many times the balance was read.

By Brecht Corbeel · August 8, 2026 · 12 min read

Browse by Category

Surprise me →

All AI Hardware & Semiconductors 100 AI Agents & Systems 33 Foundation Models 27 Model Evaluation 15 AI Industry 9 Edge AI & Electronics 7 AI Safety 7 AI Research 7 Open Models 6 AI Infrastructure 5 Industry Analysis 2 Datacenters & Infrastructure 2 Writing & Information 1 Structural Biology 1 Small & Edge Models 1 Semiconductor Markets 1 Security & Technology 1 Security 1 Scientific Revolution 1 Robotics & Embodied AI 1 Quantum Information 1 Planetary Science 1 Physics 1 Particle Physics 1 Origins & Astrobiology 1 Open-Weight Models 1 Neuroscience 1 Metrology 1 Mathematics & Computation 1 Materials Science 1 Labour & Technology 1 Interpretability 1 Institutions & State Capacity 1 Institutions & Economy 1 Inference Economics 1 Industrial Revolutions 1 Humanity & Technology 1 History of Technology 1 History of Computing 1 Governance & Policy 1 Evolution & Ecology 1 Ethics of Technology 1 Energy & Civilization 1 Earth & Climate 1 Demography & Urbanization 1 Data & Training 1 Cumulative Culture 1 Cosmology 1 Condensed Matter 1 Chemistry & Catalysis 1 Alignment & Safety 1 Agriculture & Cities 1 Agent Evaluation 1 AI Security 1 AI & The Web 1