Aleph Alpha releases Kolibri, a 78B English-German MoE, under Apache 2.0
Aleph Alpha published Kolibri, a 78B-parameter English and German mixture-of-experts model, under Apache 2.0 with about 3.46B parameters active per token. Teams that cannot send prompts to outside servers can host it themselves if they pay for the GPUs.
The Engineer · Build desk
Four figures. All 78 billion parameters must stay in GPU memory. About 3.46 billion parameters are active per token. The router sends each token to 6 of 384 experts. The FP8 weights come to about 78 GB.
| Figure | Value | Note | Claim |
|---|---|---|---|
| Parameters kept in GPU memory | 78 billion | 5 | |
| Parameters active per token | 3.46 billion | 2 | |
| Experts the router picks per token | 6 of 384 experts | 3 | |
| FP8 weight size | 78 GB | 6 |
What happened
- Native context is 262,144 tokens, which Aleph Alpha recommends for efficient serving, and the model is validated up to 1,048,576 tokens.
- Each of the 50 layers has 384 experts plus one shared expert that always runs, and the router sends each token to 6 of the 384.
- The weights ship in FP8 and come to about 78 GB.
- Apache 2.0 covers the weights and configuration files, while Aleph Alpha keeps its training code, architecture, parameter settings and training methods.
- The 128,000-token UniBPE tokenizer needs about 11.2% fewer tokens for German than GPT-5's tokenizer, according to the post.
Why it matters
- What it costs cost The GPU purchase is sized to all 78B parameters, so routing 6 of 384 experts does not shrink what a team has to buy to keep prompts in-house.
- What it limits constraint A team can run, modify and ship the weights commercially, but the licence does not extend to the recipe, so Kolibri cannot be retrained from scratch with Aleph Alpha's methods.
- What doesn't add up contradiction The post defines sovereign as controlling and operating the model, yet the model card says some synthetic training data was rephrased by non-European models, so provenance is not all-European.
The routing numbers explain the memory bill. The router picks 6 of 384 routed experts per token, or 1.6% of them [19]. The share of parameters that is active is 4.4% [4], nearly three times that [20]. We read the gap as parameters that run on every token. The shared expert is one of them [3].
The specifications here are as Hitesh Saini reports them in a dev.to post that points to the Kolibri-1 model card and Aleph Alpha's launch post for full details [18]. Saini has deployed self-hosted infrastructure for clients in Europe, the US and Australia since 2020 [14]. He says the router can pick any expert for the next token, so every expert has to be resident in GPU memory. In his words: "MoE saves compute, not memory." [5] Compute per token lands at roughly what a 3.5B model costs [13].
FP8 stores each number in 8 bits instead of 16 [15]. At 16 bits, 78.1 billion parameters would take roughly 156 GB [21]. The post credits a 4:1 mix of sliding-window and full attention with keeping long inputs cheaper [8]. The post does not give GPU counts or serving throughput. We would budget for more than the weights once long contexts are in use.
The comparison tables are Aleph Alpha's own numbers, as Saini notes. By his account, Kolibri comes out ahead in maths, knowledge and code generation, including against models that activate far more parameters [24]. Those tests are AIME for competition maths, GPQA diamond for hard science knowledge, and LiveCodeBench for writing code [25]. For the lead to transfer, a team's workload would have to resemble those tasks. Document search over German company files is a different task. Saini points instead to training with Aleph Alpha's Merlin-Arthur protocol, which makes the model abstain when the answer is not in the supplied context [23]. For document search over company files, he wrote, "this matters more than any benchmark." [26]
On compliance, Aleph Alpha is a signatory to the EU's General-Purpose AI Code of Practice, and the company says it had the EU AI Act and GDPR in mind when building Kolibri [11]. Saini argues this leaves the data protection officer at an EU client with concrete material to review, in place of a US cloud provider's terms of service [17]. He states the condition himself: "Kolibri is one of the few serious models where I can give a better answer, if the client will pay for the hardware." [12]
What to watch
- Serving figures from Aleph Alpha or third parties: GPU count, throughput and memory at 262,144 tokens of context on the FP8 weights.
- Independent evaluations on German-language document search, including how often the model abstains when the answer is absent from context.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence45
- Adoption
- Insufficient
- Hype gap+15
- Incentives30
- Confidence50
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
On 3 October 2026 Aleph Alpha published the weights of Kolibri on Hugging Face under the Apache 2.0 licence.
- [2]
Kolibri is an open-weight mixture-of-experts model with 78.1 billion total parameters, about 3.46 billion of them active per token, understanding English and German.
- [3]
Kolibri has 50 layers; each has 384 experts plus 1 shared expert that is always used, and the router sends each token to 6 of the 384.
- [4]
Kolibri has 78,103,074,560 total parameters and 3,457,573,120 active per token, about 4.4%.
- [5]
The router can pick any expert for the next token, so all 78 billion parameters must stay in GPU memory. MoE saves compute, not memory.
- [6]
The Kolibri weights ship in FP8 and come to about 78 GB.
- [7]
Kolibri handles 262,144 tokens of context natively, which Aleph Alpha recommends for efficient serving, and is validated up to 1,048,576 tokens.
- [8]
Attention is a 4:1 mix of sliding-window and full attention, which keeps long inputs cheaper.
- [9]
Apache 2.0 covers the weights and configuration files, so they can be used, modified and shipped commercially; Aleph Alpha keeps the rights to its training code, model architecture, parameter settings and training methods.
- [10]
The model card says some synthetic training data was rephrased using non-European models.
- [11]
Aleph Alpha has signed the EU's General-Purpose AI Code of Practice and says it built Kolibri with the EU AI Act and GDPR in mind.
- [12]
Kolibri is one of the few serious models where I can give a better answer, if the client will pay for the hardware.
- [13]
Each token costs roughly what a 3.5B model costs in compute.
- [14]
Hitesh Saini has deployed self-hosted infrastructure for clients across Europe, the US and Australia since 2020.
- [15]
FP8 is a form of quantisation: each number is stored in 8 bits instead of 16, so the model takes roughly half the memory.
- [16]
In the post, sovereign means you control and operate the model: you download the weights, run them on hardware you choose, and no request leaves your network.
- [17]
Aleph Alpha's compliance stance gives an EU client's data protection officer something concrete to read instead of a US cloud provider's terms of service.
- [18]
Full details are on the Kolibri-1 model card on Hugging Face and in Aleph Alpha's 'Kolibri has landed' launch post.
- [19]
The router selects 6 of 384 routed experts per token, about 1.6%.
- [20]
The 4.4% active parameter share is nearly three times the 1.6% share of routed experts selected.
- [21]
At 16 bits per parameter, 78.1 billion parameters would take roughly 156 GB.
- [22]
Kolibri's 128,000-token tokenizer, using a custom method called UniBPE, needs about 11.2% fewer tokens for German than GPT-5's tokenizer.
- [23]
Kolibri was trained with Aleph Alpha's Merlin-Arthur protocol to abstain when the answer is not in the provided context.
- [24]
The benchmark comparison uses Aleph Alpha's own numbers; Kolibri leads on maths, knowledge and code generation, even against models with far more active parameters.
- [25]
AIME is competition maths, GPQA diamond tests hard science knowledge, LiveCodeBench tests writing code, and SWE-Bench Verified tests fixing real bugs in real repositories.
- [26]
For document search over company files, this matters more than any benchmark.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toAleph Alpha Kolibri: Self-Hosting a Sovereign AI Model
1 article · October 11, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.