Aleph Alpha's Kolibri-1: An Open-Weight MoE Model with a 1 Million Token Context
Originally published in Turkish. Read the original →
Aleph Alpha announced Kolibri-1, an open-weight model under the Apache 2.0 license that uses only 3.46 billion active parameters per token and has a 1 million token context window.
Germany-based Aleph Alpha, one of the most important players in Europe's sovereign AI vision, has announced Kolibri-1, a new flagship model set to shift the balance in the open-weight AI world. With a total of 78.1 billion parameters, the model runs only 3.46 billion active parameters per token at inference time thanks to its sparse Mixture-of-Experts (MoE) architecture.
Released on Hugging Face under the Apache 2.0 license to give developers and organizations full transparency, Kolibri-1 is drawing attention above all for its huge 1 million token context window, its hybrid attention mechanism and its advanced reasoning capabilities.
A 384-Expert MoE and Hybrid Attention Architecture
At the heart of Kolibri-1 is a highly optimized fine-grained Mixture-of-Experts design. The model hosts a total of 384 experts per layer, while only 6 routed experts and 1 shared expert are activated for each token. This way, while the model represents a giant knowledge pool the size of 78B, it brings processing cost and compute time down to the speed of lightweight models in the 3.5B range.
- To let the model efficiently handle its context length of 1,048,576 (1M) tokens, a Hybrid Attention architecture was chosen:
- Most layers use sliding window attention, which minimizes memory and compute load;
- Full global attention is applied every fifth layer, ensuring that critical information in long documents is seamlessly connected.
This structure makes it possible to load complex legal texts, technical manuals hundreds of pages long and huge enterprise codebases into the model's context in a single pass.
Benchmark Results: Leading the Sub-4B Active Parameter Class
Compared with competing MoE models that use fewer than 4 billion active parameters per token (such as Qwen3.5 35B-A3B and GLM-4.7 Flash), Kolibri-1 shows superior results on reasoning, math and coding benchmarks:
- AIME 2026 (Advanced Mathematics): 96.0 points
- GPQA Diamond (Expert-Level Scientific Reasoning): 84.3 points
- LiveCodeBench v6 (Current Coding): 85.9 points
- SWE-Bench Verified (Software Engineering Tasks): 66.4 points
The model also includes a bilingual tokenizer trained specifically for German and English. Optimized for the rich morphology of German, this vocabulary consumes about 15% fewer tokens on German text than standard tokenizers, which both increases speed and lowers costs.
Hardware Requirements and Deployment Scenarios
Even though only 3.46B parameters are active at inference time, all 78.1 billion parameters of the model's weights must be held in GPU memory (VRAM).
- Kolibri-1's hardware and deployment requirements can be summarized as follows:
- Memory Size (FP8 Precision): About 78 GB of VRAM
- Recommended Deployment: Dual Nvidia A100 (80GB) or H100 SXM5; or single Nvidia H200, B200 or B300 systems
- Inference Engine: A dedicated
aleph-alpha-inferenceplugin that works fully with the vLLM stack
Designed for Sovereign AI and Strict Regulations
Kolibri-1, developed by Aleph Alpha, was built not just for general consumer chat but with the sovereign data requirements of public administration, defense, aviation, healthcare and regulated enterprise sectors in mind.
Thanks to its open-weight nature, organizations can run the model on their own local servers or on independent cloud infrastructure without sharing data with third parties. The model's strong tool calling and structured data processing capabilities make it a powerful alternative for autonomous AI agents and RAG (Retrieval-Augmented Generation) pipelines.
