Is Jais 3 worth it? Local deployment review
On 15 October, IFM released Jais 3, six open Arabic models under Apache 2.0. I ran two of them on a 96 GB Mac Studio for two days. The napkin math on memory is off by 9.5 GB, and one of the two costs you an hour.
The Promise
- Both Jais 3 models fit a Mac Studio with an M2 Max and 96 GB at every context length tested, up to 128,000 tokens: the 32B peaked at 60.9 GB and the 36B-A4B mixture of experts at 53.3 GB, inside the 83.5 GB the GPU gets.
- On 100 random ArabicMMLU questions, scored with IFM's own task code, the 32B got 86 and the 36B-A4B got 85. IFM's published 88.6 and 87.7 both sit inside the 95% intervals, and the ranking matches.
- Arabic is cheap in Jais 3's tokenizer. The Arabic Universal Declaration of Human Rights takes 1,611 tokens, against 2,192 on Qwen 3.8 and 2,721 on Llama 3.3, and fewer than the English version's 1,867.
- The 36B-A4B reads a 128,000-token prompt in 18.7 minutes and writes at 8.2 tokens a second there: a third of the dense 32B's wait, for the same accuracy on this test.
The Risk
- IFM's headline Arabic average of 83.1 can't be reproduced. It averages about 28 benchmarks, some scored by an AI judge and some not released, and the published components average 82.0 to 83.8 depending on how you group them.
- The dense 32B is slow on this class of machine: 4.1 tokens a second at 128,000 tokens, after 59.6 minutes of reading before the first word.
- On MLX you run IFM's own Python file with trust remote code, because mlx-lm has no K2 Horizon support yet. IFM's GGUF file doesn't load in the current Homebrew llama.cpp at all.
- Not tested: dialects, agentic work, native Arabic writing, hosted models. And 100 questions can't separate Jais 3 from Qwen 3.8 27B on accuracy.
Three questions
On 15 October, IFM at MBZUAI in Abu Dhabi released Jais 3: six open models, Apache 2.0, built for Arabic. I loaded two of them on a Mac Studio with an M2 Max and 96 GB of memory and ran them for two days. The 32B is dense. The 36B-A4B is a mixture of experts with 36 billion parameters in total and about four billion active per token.
Three questions decide whether Jais 3 belongs on your hardware. Does it fit? What does it cost to run? Is it safe to run? The napkin math on the first one is off by 9.5 GB, and the answer to the second is an hour.
Does it fit?
A 32-billion-parameter model at 4 bits is half a byte per parameter: 16 GB. The 32B file on disk is 25.5 GB. IFM keeps the attention layers and the embeddings at 8 bits and shrinks only the feed-forward layers to four. The mixture of experts does the same thing: the napkin says 18 GB, the file is 26.4.
Weights are the small part of the bill. On the 32B, every thousand tokens of context adds roughly a quarter of a gigabyte of key-value cache, so at 128,000 tokens the cache alone is about 35 GB. I measured peak memory at every length.
| Context window | Jais 3 32B, peak | Jais 3 36B-A4B, peak |
|---|---|---|
| 1K | 26.1 GB | 27.1 GB |
| 8K | 28.3 GB | 28.8 GB |
| 32K | 34.7 GB | 33.5 GB |
| 128K | 60.9 GB | 53.3 GB |
This Mac gives its GPU 83.5 GB by default, so both fit everywhere I tested. For comparison, Llama 3.3 70B at 4 bits fits at 32,000 tokens; at 128,000 it read for 63 minutes and then ran out of GPU memory.
The rule: take the weights from the file size, not from parameters times bits. Add a quarter of a gigabyte per thousand tokens. Compare the sum with your memory budget before you download anything. On a 64 GB Mac (untested), the 32B’s 60.9 GB at 128,000 tokens leaves nothing for the operating system; plan on 32,000 tokens there.
What it costs to run
Fitting is not the same as usable. Speed is the median of three timed runs after a warm-up, 512 tokens out, on MLX.
| Jais 3 32B | Jais 3 36B-A4B | |
|---|---|---|
| Writing speed, short prompt | 13.6 tok/s | 40.5 tok/s |
| Writing speed at 128,000 tokens | 4.1 tok/s | 8.2 tok/s |
| Wait for the first word at 128,000 tokens | 59.6 min | 18.7 min |
| Energy for one 128,000-token run | 52.9 Wh | 16.5 Wh |
| ArabicMMLU, 100 questions | 86 | 85 |
Qwen 3.8 27B on the same Mac writes 21.3 tokens a second at a short prompt and waits 21.6 minutes at 128,000. The 32B’s 128,000-token figure is a single timed run, because each one takes about an hour.
At long context the dense model costs three times the wait for no measurable gain on what I tested. For long documents on this class of machine, the mixture of experts is the one to run.
Does IFM’s 83.1 hold?
IFM’s headline Arabic score for the 32B is 83.1. I couldn’t reproduce it. It’s a composite of about 28 benchmarks; some are scored by an AI judge, some haven’t been released, and the weighting isn’t published. Averaging IFM’s own published component scores gives 83.7, 83.8 or 82.0, depending on how you group them.
What I could test is one component, the audited version of ArabicMMLU, using IFM’s own task code on 100 random questions.
| Model | My score | 95% interval | IFM reports |
|---|---|---|---|
| Jais 3 32B | 86 | 77.9–91.5 | 88.6 |
| Jais 3 36B-A4B | 85 | 76.7–90.7 | 87.7 |
| Qwen 3.8 27B | 80 | 71.1–86.7 | 85.8 |
All three of IFM’s numbers sit inside my intervals, and the order matches theirs. That makes IFM’s claims consistent with what I saw. It doesn’t show Jais 3 beating Qwen: with 100 questions, the two intervals overlap.
What Arabic costs
Where I found a clear difference is tokens. The Arabic text of the Universal Declaration of Human Rights runs to 7,315 characters. Jais 3 encodes it in 1,611 tokens, Qwen 3.8 in 2,192, Llama 3.3 in 2,721: 1.36 times as many on Qwen and 1.69 times on Llama. Jais 3 even uses fewer tokens for the Arabic than for the English version (1,867).
If you pay per token or fill a context window, you get about a third more Arabic than on Qwen and two-thirds more than on Llama. One caution: that’s one legal text in Modern Standard Arabic. I didn’t test dialects.
Is it safe to run?
On day two, IFM’s own GGUF file wouldn’t load in the current Homebrew release of llama.cpp. The error was unknown model architecture: 'k2-horizon'. IFM’s model card says support is merged upstream; it hasn’t reached that release.
On MLX, mlx-lm doesn’t know the architecture yet either. IFM ships its own Python file with the weights and asks you to run it with trust_remote_code. That flag runs code from a model repository on your machine, with your permissions. Cybersecurity has watched this pattern with package installs for 20 years. So I read the file: 628 lines of pure MLX math, no file access, no network calls. I pinned its hash and loaded my reviewed copy. Do the same, and read the diff before you update.
Who it’s for
Not tested: dialects, agentic work, the quality of native Arabic writing, and any comparison with hosted models. Who it is for: teams that need long Arabic documents processed on their own hardware, where the data can’t leave the building.
Every number on this page, with the method and the raw output, is in the lab: Jais 3 on Apple Silicon.
Add up the memory before you download. Read the code before you run it.
Questions this episode answers
Can a 96 GB Mac run Jais 3? Yes. On a Mac Studio with an M2 Max and 96 GB, IFM’s 4-bit MLX builds of the 32B and the 36B-A4B both ran at every context length up to 128,000 tokens, peaking at 60.9 GB and 53.3 GB. The GPU limit on that machine is 83.5 GB.
Which Jais 3 model should I run locally? For long documents, the 36B-A4B mixture of experts. It scored 85 on 100 ArabicMMLU questions against the 32B’s 86, a gap inside the noise, and waits 18.7 minutes before writing at 128,000 tokens instead of 59.6.
Is IFM’s 83.1 Arabic score accurate? It can’t be checked directly: it’s a composite of about 28 benchmarks with unpublished weighting, some scored by an AI judge. One component, ArabicMMLU, checked on 100 questions came out at 86 for the 32B against IFM’s 88.6, inside the 95% interval.
Does Jais 3 run in llama.cpp or Ollama? Not in the Homebrew llama.cpp release tested on 17 October 2026: IFM’s GGUF fails with unknown model architecture: 'k2-horizon'. On MLX it needs IFM’s own model file loaded with trust_remote_code.
Transcript
Three questions
On October 15th, IFM at MBZUAI in Abu Dhabi released Jais 3: six open models, Apache 2.0 license, built for Arabic. I loaded two of them on a Mac Studio with 96 GB of memory and ran them for two days. Three questions decide whether Jais 3 belongs on your hardware. Does it fit? What does it cost? Is it safe to run? The napkin math on the first question is off by nine and a half GB. The answer to the second one is an hour.
The napkin math
Start with the napkin math. A 32-billion-parameter model at 4 bits is half a byte per parameter. That is 16 GB. The Jais 3 32B file on disk is 25.5. IFM keeps the attention layers and embeddings at 8 bits and quantizes only the feed-forward layers to four. The mixture of experts model has 36 billion parameters in total and about 4 billion active per token. The napkin says 18 GB. The file is 26.4.
Weights are the small part of the bill. Context is the large part. Every thousand tokens of context adds roughly a quarter of a gigabyte of key-value cache on the 32B. At 128,000 tokens, that cache is about 35 GB. I measured peak memory at that length: 60.9 GB for the 32B, 53.3 for the mixture of experts. This Mac gives its GPU 83.5 GB. Both models fit at every context length I tested, up to 128,000 tokens. For comparison, Llama 3.3 70B at 4 bits fit at 32,000 tokens. At 128,000, it ran for 63 minutes of prefill, then ran out of GPU memory.
So, the rule is this. Take the weights from the file size, not from parameters times bits. Add a quarter of a gigabyte per thousand tokens. Compare the sum to your memory budget before you download anything.
The verdict by machine
Here is the verdict by machine. On a 96 GB Mac, run either model at any length up to 128,000 tokens. That one I measured. On a 64 GB Mac, I did not test it. The arithmetic says the 32B needs 60.9 GB at 128,000 tokens, which leaves nothing for the operating system. Plan on 32,000 tokens there. The measured peak at that length is 34.7 GB for the 32B and 33.5 for the mixture of experts. Below 64 GB, the 32B weights alone take 25.5 before you add a single token of context. Run the sum with your own number.
Speed and the wait
Fitting is not the same as usable. Start with generation speed. At a short prompt, the 32B writes 13.6 tokens per second. The mixture of experts writes 40.5. Qwen 3.8 27B writes 21.3. At 128,000 tokens, the 32B falls to 4.1 tokens per second. The mixture of experts falls to 8.2.
The wait before the first word is the bigger cost. Reading a 128,000-token prompt takes 59.6 minutes on the 32B. It takes 18.7 minutes on the mixture of experts, where Qwen 3.8 27B takes 21.6. The 32B run drew 52.9 watt-hours. The mixture of experts drew 16.5. Note that the 32B figure at 128,000 tokens is one timed run, because each run takes about an hour. Every other figure is the median of three.
On accuracy, the 32B scored 86 and the mixture of experts 85. That gap sits inside the noise. So at long context, the dense model costs three times the wait for no measurable gain on what I tested. For long documents on this class of machine, the mixture of experts is the one to run.
IFM’s 83.1
IFM’s headline Arabic score is 83.1. I could not reproduce it. The composite averages about 28 benchmarks. Some are scored by an AI judge. Some are not released yet, and the weights are not published. Averaging the published components gives 83.7, 83.8 or 82, depending on how you group them.
What I can test is one component, ArabicMMLU, the verified version, using IFM’s own task code on 100 random questions. Jais 3 32B got 86 right. The interval is 78 to 91.5. The mixture of experts got 85. Qwen 3.8 27B got 80. IFM reports 88.6, 87.7 and 85.8. All three sit inside my intervals, and the order matches theirs. That makes IFM’s claims consistent with what I saw. It does not show Jais beating Qwen. With 100 questions, the two overlap.
Tokens
Where I did find a clear difference is tokens. I took the Arabic text of the Universal Declaration of Human Rights, 7,315 characters, and counted tokens. Jais 3 needs 1,611. Qwen 3.8 needs 2,192. Llama 3.3 needs 2,721. That is 1.36 times as many tokens on Qwen and 1.69 times on Llama. Jais 3 even encodes the Arabic in fewer tokens than the English version, which is 1,867. If you pay per token or fill a context window, you get about a third more Arabic than Qwen and two thirds more than Llama. One caution: that is one legal text in standard Arabic. I did not test dialects.
The safety question
Now, the safety question. On day two, IFM’s own GGUF file would not load in the current Homebrew release of llama.cpp. The error is “unknown model architecture: k2-horizon”. On MLX, the architecture is not in mlx-lm yet. IFM ships its own Python file and asks you to run it with trust remote code. That flag runs code from a model repository on your machine, with your permissions. Cybersecurity has seen this pattern with package installs for 20 years. So I read the file: 628 lines, pure MLX math, no file access and no network calls. I pinned its hash and loaded my reviewed copy. Do the same: pin the hash, and read the diff before you update.
What I did not test, and who it is for
What I did not test: dialects, agentic work, the quality of native Arabic writing, or a comparison with hosted models. Who it is for: teams that need long Arabic documents processed on their own hardware, where the data cannot leave. The full test data is on the site. Add up the memory before you download, and read the code before you run it.
Thank you for watching. If you found this useful, hit Like and Subscribe. And if you want to support the channel, consider joining the membership. You get early access to scheduled content, and a direct line to me. I answer members’ comments first. See you next week.