Clef on Apple Silicon

2.77 seconds. That is how long Cloudflare’s Clef, a 27-billion-parameter decision model, took to route one support ticket on a Mac Studio, at full precision, with nothing rented.

Clef came out on 1 October 2026 with open weights. It doesn’t write text. It reads a state (a ticket, a shell command, an invoice) and a list of questions, then returns a probability for every allowed answer. That makes it the kind of model that sits in front of every agent call, deciding what gets routed where and what runs without a human.

So two questions: does it fit on hardware you can own, and what do you give up when you shrink it to fit?

The result

Seconds to decide one 500-token support ticket. Median of five runs; no configuration varied by more than 1%.

Model	Precision	Runtime	Seconds	Memory
Clef-flash 9B	Full (BF16)	MLX	0.80	19.4 GB
Clef-flash 9B	4-bit	MLX	1.03	6.2 GB
Clef-flash 9B	Full (BF16)	PyTorch	1.48	20.4 GB
Clef 27B	Full (BF16)	MLX	2.77	55.3 GB
Clef 27B	8-bit	MLX	3.55	30.3 GB
Clef 27B	4-bit	MLX	3.59	17.0 GB

Speed barely moved with length. Clef 27B at full precision handled 2,000 tokens in 10.5 seconds and 32,000 in 188 seconds, peaking at 61.2 GB. That is well inside what a 96 GB Mac gives its GPU by default.

The surprise is the bottom two rows. Shrinking a model usually buys speed. Here it cost speed: 4-bit was about 30% slower than full precision on the M2 Max. A decision model spends nearly all its time reading the input, which is raw compute, and the chip has to unpack the compressed weights on top of that. 4-bit buys memory. Nothing else.

What 4-bit changed

I ran 60 synthetic records through every configuration: 20 support tickets to route, 20 shell commands to approve or block, 20 invoices to score for risk.

Clef 27B at 8-bit and 4-bit gave the same answer as full precision on all 60. At 4-bit, one probability slipped below 0.8, a common auto-approve line: pip install --upgrade requests inside a project’s own environment dropped from 0.845 to 0.791. A pipeline set to auto-approve above 0.8 would start asking a human.

Clef-flash flipped one. At full precision it rated terraform apply -auto-approve (adding one tag to staging resources) safe to run without approval, at 0.78. At 4-bit, it said unsafe, at 0.45. Two independent 4-bit conversions did the same thing. Full precision gave the same answer on MLX and on PyTorch, which rules out the runtime. The compression caused it. Clef 27B called that command unsafe at every precision.

One flip in 60 sounds small. It was on the question that decides what an agent may do unsupervised.

Where it broke

Clef 27B at full precision would not load through PyTorch. A memory watchdog stopped it a third of the way through loading, after swap grew 4.9 GB in 40 seconds. PyTorch appears to stage the weights through system memory before copying them to the GPU, so the load briefly needs twice the model. The full-precision 27B numbers above come from Apple’s MLX framework, running a conversion of Cloudflare’s own weights.

Clef-flash on PyTorch handled everything up to 8,000 tokens. At 32,000 it asked the GPU for a single 65 GB block, the entire attention table at once, and was refused.

What it means if you run AI

In my experience, the changes that hurt are the ones nobody filed as a change. A firewall rule tightened “for performance”, an IDS threshold nudged to cut alert noise. Each one is a policy change wearing an engineering label.

Quantising a decision model is the same move. It goes in as a memory optimisation. What comes out can be a different answer on what an agent is allowed to run. So treat a new precision like a new model: run your own decisions through it, compare against full precision, and look hardest at the probabilities sitting near your thresholds.

The 27B passed that test on this set. The 9B didn’t, on exactly one question, and it was the question that mattered.

How it was measured

Machine	Mac Studio (2023), Apple M2 Max, 38-core GPU, 96 GB unified memory, default GPU limit
OS	macOS 27.0.1
Models	Cloudflare/clef, Cloudflare/clef-flash, mlx-community/clef-8bit, mlx-community/clef-4bit, TrevorJS/clef-flash-mlx-4bit, mlx-community/clef-flash-4bit
Engines	MLX 0.32.3 with mlx-vlm 0.7.4; PyTorch 2.11 with Transformers 5.10.2
Workload	Three warm-up decisions, the 60-record pass one at a time, then inputs of 250, 500, 2,000, 8,000 and 32,000 tokens
Runs	2 October 2026; five repetitions per length, three at 32,000

Every run went through Clef’s own decision head. The GGUF builds for llama.cpp, LM Studio and Ollama were left out: they ship without the head, so those tools can only produce text, and Clef’s text is meaningless.

The test records are invented. The 32,000-token inputs run past the 16,000-token length Clef’s head was trained at, so timings there are valid and decisions there were not checked. One machine, one chip generation. A newer Apple chip will move every speed figure; the M5 Max numbers on the MLX model cards run several times faster.

Run it yourself

The test set, the scripts, every probability from every run and the logs of the failures are in the promise-and-risk-labs repository on GitHub. On an Apple silicon Mac:

pip install -r requirements-mlx.txt
P=$(hf download mlx-community/clef-4bit --quiet)
python3 scripts/bench.py --run r3_clef_4bit_mlx --backend mlxc --path "$P"

If your chip gives different numbers, or a different answer on the terraform command, I want to see them.