The EU AI Act Clause That Makes You the Provider — What Open Weights and Fine-Tuning Actually Are
$100 is what it cost a researcher to poison an open-weight model — not to break it, to bend it. Meanwhile the compute threshold everyone is selling as your compliance risk almost certainly doesn't apply to you, and Article 25, which almost nobody is discussing, probably does.
The Promise
- Open weights are the strongest counterweight available to three or four American companies deciding what every organisation on Earth is allowed to compute. A European bank can run a frontier-adjacent model on hardware it owns, under law it is already subject to, with data that never crosses a border.
- The capability gap has closed further than most executives realise — OpenRouter's June assessment put DeepSeek V4 Pro at 80.6% on SWE-bench Verified, the top open-weight score, at a fraction of the cost per token. And a model you hold the weights to cannot be deprecated, repriced, or quietly forbidden by a terms-of-service update.
The Risk
- Open weights is not open source. You get a compiled binary, not a recipe: no training data, no training code, no filtering or safety work. You can run it and measure it; you cannot read how it decides.
- Article 25 is the clause that will actually catch you. Take a system that wasn't high-risk, change its intended purpose so that it becomes high-risk, and you are the provider — no compute threshold, no notification step, and whatever your vendor contract says does not change it.
- Fine-tuning writes your data into the weights, distributed across billions of parameters and demonstrably recoverable. Ask your privacy counsel how a deletion request works against a model artefact; most have never been asked. Refusal behaviour also measurably weakens on benign domain data, and nobody tests for it because the fine-tune was booked as a data science project.
Start with the word everyone gets wrong
Open source in software means you get the source code — readable, compilable, reviewable line by line. Open weights means something much narrower: you get the finished model and nothing about how it was made.
When a model trains, it adjusts billions of small numbers describing how strongly one part of the network responds to another. That set of numbers is the weights, and it is the finished product, not the recipe. Open source is source code. Open weights is a compiled binary. The training data isn’t published, the training code usually isn’t, and the filtering, safety work and human feedback don’t arrive in the download.
Almost every model marketed as open source is open weight — Kimi K3, GLM 5.2, DeepSeek V4, Mistral Large 3. Powerful, downloadable, permissively licensed, and opaque underneath.
The threshold they’re selling you, and the one they aren’t
Two different rules get conflated constantly here.
The first sits at the model level. Under the Commission’s general-purpose AI guidelines, you become the provider of a modified model when your fine-tuning uses more than a third of the compute that trained the original. Sit with that number — a third of a frontier pre-training run. Your team’s LoRA adapter is not within several orders of magnitude of it. If someone is selling your organisation GPAI provider obligations because a team fine-tuned a model, ask them for the compute figure. They won’t have one.
The second is Article 25, and it’s the one that will catch you. Put your name or trademark on a high-risk system and you become its provider. Substantially modify one and you become its provider. And the trigger that matters most: take a system that was not classified high-risk, change its intended purpose so it becomes high-risk, and you become the provider. Automatically.
Picture the sequence. Your team downloads an open-weight model, fine-tunes it on eight years of historical hiring decisions, and points it at CV screening. Recruitment sits in Annex III. Nobody crossed the compute threshold; somebody crossed the purpose line — and the organisation now carries conformity assessment, technical documentation, registration and post-market monitoring for a system it believed it was merely using.
One caveat for planning. High-risk obligations don’t bite yet: the Digital Omnibus moved standalone high-risk duties to December 2027. That is not relief, it’s the build window. The systems your teams stand up this quarter are the ones that will be sitting there when the day arrives, and nobody retrofits conformity assessment in a quarter.
An unmanaged binary with access to the file server
Almost every open-weight model reaches you through one hub. Read July’s Hugging Face breach as a supply chain event, because that’s what it is — the place your models come from was compromised by an AI system that was supposed to be in a box.
Then read the poisoning research. The finding that matters for a board isn’t the $100 price. It’s what a poisoned model looks like: it doesn’t crash, it doesn’t fail a benchmark, it answers normally, and on the narrow set of inputs the attacker cared about, it leans. You wouldn’t install an unsigned binary from an unknown publisher onto a production server. A great many organisations are doing exactly that with model weights and calling it innovation.
Add retrieval, because almost nobody runs a bare model. Your retrieval index inherits whatever permissions you gave it, so a model that can read everything will answer from everything — including the folder HR believed was locked. Anything retrieved gets treated as instruction, which makes a poisoned document in SharePoint an injection vector that never crosses your network perimeter.
The needle sits slightly toward promise, and I don’t say that lightly. But the promise only survives contact with an inventory. Three questions for your next technology review: which open-weight models are running here right now, who downloaded them, and from where? Has anyone fine-tuned on our data, and did legal see it before or after? And who has the authority to stop a model serving traffic — and have they ever used it?
The license was never the control. The inventory is.