There’s a quiet problem hiding behind the AI hype cycle. Models are getting better at code, better at conversation, better at passing tests. But when it comes to formal verification (actually proving that an answer is correct, not just statistically likely), most AI systems are still winging it. A startup called Axiom Math thinks that’s a dealbreaker for everything that comes next.
Axiom’s CEO Carina Hong laid out the case on a recent Latent Space episode. Her argument is simple: if AI can’t move from informal reasoning to provable proofs, it can’t compound its own breakthroughs. And if it can’t compound, it can’t scale. That matters whether you’re building AI agents for law firms or deploying automation in construction.
What happened
- Axiom, a startup founded in 2024, solved all 12 problems on the Putnam exam (8 of 12 within the time limit). The Putnam is a legendarily hard undergraduate math competition where the median score is typically 0 or 1 point.
- The company achieved 99% (187/189) on the Verina codegen benchmark, which requires generating code and a proof of correctness. For comparison, OpenAI o3’s last known score was 4.9%.
- Axiom open-sourced AXLE, a toolkit of interactive Lean applications for exploring, validating, and manipulating mathematical proofs.
- Carina Hong argues that frontier labs still aren’t training models to generate Lean proofs directly. They’re relying on informal proofs, which she believes will bottleneck AI progress.
- Axiom raised a $200M Series A at a $1.6B valuation.
The numbers
- 12/12 Putnam problems solved (8/12 in the time limit). The closest AI competitor, DeepSeek, scored 103/120.
- 99% (187/189) on the Verina codegen benchmark vs. OpenAI o3’s 4.9%.
- $200M Series A, $1.6B valuation.
- Axiom is roughly 7 months old as of the Putnam result.
5 things service business operators should understand about formal verification in AI
- “Verified” doesn’t mean “audited.” Formal verification means a computer checked the logic, step by step, using a proof language like Lean. It’s the difference between a spell-checker and a human editor. The machine doesn’t guess. It confirms.
- It makes reinforcement learning way more powerful. Today, most AI training uses statistical reward signals. Formal verification replaces the guesswork with a binary: the proof is correct, or it isn’t. That’s a much stronger training signal, like the difference between grading an essay on vibes versus checking a math problem against the answer key.
- It compounds. Carina uses the example of mathematician Ramanujan. When he started formally proving his intuitions instead of just stating them, it improved his own thinking and let others build on his work. Verified AI outputs work the same way. Each proven result becomes a foundation for the next one.
- The gap between Axiom and frontier labs is massive right now. A 99% vs. 4.9% delta on the same benchmark is not a rounding error. It suggests that the big labs haven’t prioritized this approach yet. That could change fast, but the window is open.
- Specification is the new bottleneck. As Carina puts it: “Anything that can be specified can be proven. Humans are bad at specifying everything we want.” Sound familiar? That’s the same problem you hit when writing prompts, SOPs, or project briefs. The skill of clearly defining what you want is becoming the most valuable skill in AI.
The hot take
The frontier labs are going to regret ignoring formal verification for as long as they have. Anthropic’s bet on code is paying off beautifully right now. But code that compiles and passes tests is not the same as code that’s proven correct. Statistical RL got us this far. It won’t get us to reliable AI agents that handle high-stakes decisions in regulated industries. The companies that build verification into their training loops first will produce agents that are trustworthy in ways that matter for real business adoption. Not “trustworthy” in a marketing deck. Trustworthy in the sense that a compliance officer would sign off on.
The Agency OS play
You don’t need to become a mathematician to benefit from this shift. But you should pay attention to where formal verification is heading, because it’s going to change which AI tools you can actually trust with critical work. If you run a service business in a regulated space (legal, healthcare, finance), start asking your AI vendors a pointed question: how do you verify outputs? If the answer is “we run evals” or “we have guardrails,” that’s table stakes. The companies building verified generation will offer something fundamentally different.
In the near term, focus on the specification problem Carina identified. The biggest returns from AI come when you define your processes precisely enough that a machine can check them. This week, pick one workflow in your business and try to write down every rule, every edge case, every condition that determines whether the output is “correct.” You’ll probably find that most of your quality problems aren’t AI problems. They’re specification problems. You haven’t told the system what good looks like in enough detail.
Finally, keep an eye on Axiom’s open-source AXLE toolkit. If you have engineers building AI agents that generate code, connecting those agents to a formal verification step (even a lightweight one) will catch errors that testing alone won’t. The pattern here is the same one that made test-driven development valuable a decade ago: proving correctness up front is cheaper than fixing failures after they ship.