Mira Murati Just Dropped Inkling Small
138GB. That's what it now takes to hold the strongest open-weight model an American lab has shipped — down from 488GB two weeks ago. Thinking Machines cut the memory floor by 70%. Then the European Commission started enforcing the rules that land on whoever runs it.
The Promise
- For the first time the strongest open model an American lab has published fits inside a machine you can buy on a purchase order rather than a data centre contract — 276B total parameters, 12B active, 138GB at 4-bit, with a million-token context window under Apache 2.0.
- Sparse activation and unified memory suit each other. On a Mac Studio Ultra this is the first release where that machine is genuinely the right tool rather than the consolation prize, because only 12B parameters activate per token and the Mac's real weakness — memory bandwidth — barely gets tested.
The Risk
- The barrier didn't disappear, it moved. In July it was hardware; as of 2 August it's paperwork, and the deadline has already passed. The Commission's AI Office can now demand documentation, run its own evaluations, order corrective measures and fine up to €15M or 3% of worldwide turnover.
- Fine-tune substantially and the Act can treat you as the provider of a new model — a heavier set of obligations than the deployer role you thought you were taking on.
- Open weights have no recall. Anthropic pulled Fable 5 worldwide in June when given 90 minutes under an export-control directive; weights already on your hardware have no switch. That's precisely why people want them, and it's the risk — you cannot have the first property without the second.
Two bills on two different axes
On 15 July, Thinking Machines released Inkling — 975 billion parameters, Apache 2.0, a million-token context window, and 488GB of weights. A real milestone, but not the frontier-model-in-everyone’s-hands story it was sold as. 488GB is a data centre.
Sixteen days later they shipped Inkling Small: 276 billion total parameters, 12 billion active, same license, same context window. It reads text and images, and it listens. On the Artificial Analysis Intelligence Index it scores 40; the model four times its size scores 41. On SWE-bench Verified the small one wins outright, 80.2% against 77.6.
The architecture underneath is a mixture of experts, and it produces the most misunderstood property of this generation of open models. Your compute bill tracks the active parameters — every token costs roughly what a 12B model would cost to generate it, which is why pricing keeps falling while parameter counts climb. Your memory bill tracks the total, because the router can call any expert at any moment and you don’t know which 12 billion you need until the token arrives. Two bills, two axes. One of them got small. The other did not.
The arithmetic that decides whether this touches you
At 4-bit quantisation each parameter costs about half a byte. 276 billion parameters is 138GB before you reserve anything for context. That’s the number that moved — not the intelligence score, which shifted by a single point on a model a quarter the size. The memory floor dropped by 350GB.
On an RTX 5090 with 32GB of VRAM: no go, and not close — you’re more than 100GB short. On a Mac Studio Ultra with 256GB of unified memory: go, comfortably.
If you consume models through a hosted API and you’re happy with them, this release changes nothing for you. The people it moves are fine-tuners and data-residency shops. If you costed Inkling in July and the hardware quote ended the conversation, reopen it.
Whose name goes on the compliance line
On 2 August the Commission’s AI Office gained enforcement power over general-purpose AI models. One release makes the model hostable; one deadline makes hosting a legal position.
The mechanism that catches people is the provider/deployer line. Download Apache-licensed weights, run them unchanged, and you’re a deployer — lighter obligations. Fine-tune substantially and put the result into service, and the Act can treat you as the provider of a new model.
I’ve watched this trade for 25 years in cybersecurity. The moment you move something off a vendor’s platform onto your own, you inherit the whole operating burden with it. Nobody ships you a patch on a Tuesday. Nobody writes your incident report. The control you gained is real, and it arrives attached to work that used to be somebody else’s job. Brussels has now put a number on the bill.
Three questions for the next architecture review. Who verifies that the model you deployed is the model you downloaded — weights have a hash, use it. If you fine-tune, does anyone in the building know you may have just moved from deployer to provider? And what is your revocation plan, given a hosted model has a vendor switch and yours does not?