2,436 Bugs in 269 Projects — The Open Model That Outscored Claude on Cyber
Zhipu shipped GLM 5.3 on August 14 and it outscored both Anthropic and OpenAI at finding security flaws from source code. Then it held back the weights — nobody asked it to. Same base model as 5.2, no new pre-training run. The version number stopped tracking what's inside the box.
The Promise
- A vulnerability-finding instrument that used to sit behind a frontier lab contract is about to be free. Security teams have wanted this for 20 years: static analysis hands you 10,000 findings and no idea which twelve matter — a model that finds a flaw and then proves it reproduces is a different tool entirely.
- The gains came entirely from post-training on the same 743B base as GLM 5.2. Terminal-Bench 3.0 went from 4.6 to 28.3 and deep bug resolution from 46.2 to 66.9 — no new pre-training run, no new hardware. Capability now arrives without the expensive, visible signals that used to announce it.
The Risk
- Zhipu's own framing is that the exploit-chain reasoning went further than post-training was designed to produce. Nobody wrote a specification for a cyber capability above two American frontier models. It fell out of scaling a general procedure.
- The only thing standing between the announcement and the download was one company's private read of its own benchmark. No regulator asked, no export control applied. That worked this time. It is not a control.
- If your AI risk review triggers on a version bump, it will not trigger on the next one of these — and the next one of these is where capability is now coming from.
A lab delayed its own release
Zhipu — a Beijing lab trading as Z.ai — shipped GLM 5.3 on 14 August. On CyberGym, the benchmark measuring whether a model can find and validate a security flaw straight from source code, it scored 84.5%. Anthropic’s Mythos 5 scored 83.8. OpenAI’s GPT-5.6 Soul scored 83.6.
Working with security teams in China, Zhipu says the model found 2,436 vulnerabilities across 269 open-source projects — Linux, WebKit, FreeBSD.
Then it did something the score doesn’t explain. The weights were supposed to ship alongside the model. They’re now roughly two weeks out, pending what the company calls safety evaluation and hardening. A lab published a benchmark result, then delayed its own release because of what that result implied. Nobody made them.
The version number stopped meaning anything
For the last few years, a jump in capability meant a new base model: new pre-training run, more data, more compute, a new version number. Expensive, slow, and visible from a long way off — a lab spending nine figures on a training run tends to talk about it first.
That’s not what happened here. GLM 5.3 has the same brain as GLM 5.2. What changed is what the model was made to practise after training finished, and Zhipu scaled that by roughly an order of magnitude.
The word doing the work is environment. The old style is a question and a graded answer — write this function, here’s the correct output. Short horizon, clean signal, easy to score. Zhipu’s environments are nothing like that. One hands the model a compute cluster, the internal documentation, a live codebase and the results of a previous experiment, then asks it to find the bottleneck and deliver a measured improvement end to end. That isn’t a coding exercise. That’s a junior engineer’s week.
The model runs the whole thing itself — reads, forms a hypothesis, edits, runs, checks whether the number moved — and gets graded on the number, not on whether the code looked tidy. Do that tens of thousands of times and the model stops being good at answers and starts being good at procedure: holding a goal across dozens of steps, noticing when one failed, backing up, trying again.
Finding vulnerabilities in a large codebase is exactly that shape of task. Long, procedural, and verifiable — at the end the flaw either reproduces or it doesn’t.
What this changes about reading release notes
GLM 5.3 carries the same base model as 5.2 and a materially different capability profile. The number on the box no longer tracks what’s inside it.
After 25 years in cybersecurity I’ve watched this pattern arrive on four technologies — cloud, mobile, data, and now AI. In June 2016 I wrote that information security was not a department; that treating a distributed problem as one team’s job produced organisations where everybody assumed the risk was handled somewhere else. AI capability assessment is repeating that shape. It sits with whoever owns model procurement, and it’s being tracked by version number, which no longer works.
Who this is for: if you review code at scale, triage findings across a large internal codebase, or do long-horizon agentic work in a terminal, this earns an evaluation — benchmarked against what you’ve actually deployed, not against the frontier. Most enterprises are running something a generation behind, and against that comparison the gap is wider than the leaderboard suggests. Who should stay put: anyone with a data-residency requirement or a restriction on Chinese vendor exposure. Today the API is the only door and it routes to Z.ai.
What it takes to run: nothing yet, because there’s nothing to run. When the weights land, both reference rigs are a no. 743 billion parameters at 4-bit is roughly 370GB before you allocate a single token of context — past the RTX 5090’s 32GB and past the ~200GB you can realistically dedicate on a Mac Studio Ultra. Open weights here means open to people who own a rack.
Three things to do. Find out whether GLM 5.3 is already inside your development environment — developer adoption of a coding model runs weeks ahead of procurement hearing the name. Ask your security team what happens to your disclosure and patch timeline when a model that finds flaws at this rate is free to download. And stop using the base model version number as your risk signal.