Tagged Anthropic · show all articles
Long before the term existed, researchers tried to read meaning into a trained network's units. This is mechanistic interpretability's dated history: a naming paper, a landmark result, and three labs converging on one tool within months of each other.
AIME, GPQA and FrontierMath test different things by different rules. Reasoning effort and extended thinking are documented settings, not fixed traits, and a right answer does not certify the reasoning behind it.
System cards show resolve rates climbing fast on SWE-bench Verified and Terminal-Bench, but scaffold, effort, and test-oracle strength move a score as much as weights do. The one reproduction holding harness fixed separates OpenAI and Claude by under a point.
OpenAI, Anthropic, and Google each publish a token count for how much a model can hold. Independent long-context benchmarks measure a different quantity: how much of it the model actually uses correctly, and the two numbers are routinely far apart.
OpenAI, Anthropic, and Google publish input and output prices in the same unit, dollars per million tokens. Three specific ratios genuinely converge across all three rate cards. The token underneath the ratio does not.
Every frontier-model vendor sells an enterprise tier with the same badge row — SOC 2, ISO, HIPAA, sometimes FedRAMP. The badges are real. What they cover, and what they leave for the customer to build, is a separate and more specific engineering question.
A benchmark score is not a fact about a model; it is a fact about a harness, a threshold, and a moment in time. Here is what actually goes into an Anthropic capability evaluation, who checks it from outside, and why a single leaderboard number still misleads.
Every refusal passes through a trained policy, a classifier, account monitoring, and a capability-gated release process — four thresholds, each trading missed misuse against blocked legitimate use, documented in Anthropic's own safeguards and RSP filings.
A rate card prices an uncached token, a cache write, a cache read, and an output token charged whether or not the answer arrives whole. It says nothing about tool overhead, a retried request, or the safety check every response already passes through.
Claude carries more defensive machinery than most deployed models — classifiers, citations, constitutional training. Its failures surface at the seams of that machinery, and Anthropic's own records are usually the first place they are documented.