Key takeaways
- Baidu released ERNIE 5.5 Flash Base on November 5, a 280B-parameter mixture-of-experts base model with about 14B active per token.
- It was trained on 33.5T tokens, and Baidu says it supports up to 1M tokens in fine-tuning and inference.
- Baidu's own benchmarks beat DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base on most tests, but the license terms and independent results are still open.
Baidu released ERNIE 5.5 Flash Base on November 5, an open-weight base model with 280 billion parameters, of which about 14 billion are active for each token. It is the raw checkpoint from the end of pre-training, with no instruction tuning, and Baidu says a post-trained version is planned.
That makes it a starting point for teams who want to fine-tune a large model themselves, not a chatbot you can use today. Baidu's own benchmarks put it ahead of three comparable open base models on most tests, though those numbers come from Baidu.
What Baidu released
ERNIE 5.5 Flash Base is a sparse mixture-of-experts model, which runs only a small slice of its network for each token. It has 43 layers and 512 routed experts, with 10 chosen per token plus one shared expert. Baidu trained it from scratch on 33.5 trillion tokens of multilingual web text, code and long documents. The code data includes commits, issues, pull requests and CI-verified fixes.
Training used a context of up to 128K tokens. Baidu says fine-tuning and inference support up to 1 million tokens. The model is text-only and tagged for English and Chinese on its Hugging Face model card.
The architecture mixes several new pieces. Most layers use a heavily compressed attention that stores one cache entry for every 128 tokens, which keeps long contexts cheaper. One layer in seven uses latent attention that sees every token. Baidu points to three papers on the techniques, covering virtual head attention, multi-router experts and a multi-token prediction replacement called ECHO.
Benchmarks: Baidu's numbers
Baidu compared the model with DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base, using the same harness and few-shot settings. Selected results, as published by Baidu:
| Benchmark | ERNIE 5.5 Flash Base | Best of the three others |
|---|---|---|
| MMLU (5-shot) | 93.3 | 89.4 |
| MMLU-Pro (5-shot) | 78.4 | 69.2 |
| GPQA (5-shot) | 60.2 | 55.9 |
| HumanEval+ (1-shot) | 82.3 | 78.0 |
| LiveCodeBench (1-shot) | 40.1 | 33.5 |
| LongBench-v2 (0-shot) | 48.7 | 44.6 |
It does not win everywhere. DeepSeek's model leads on ChineseSimpleQA (75.3 against 73.0), HELMET-Rerank and InfiniteBench, and Hy3 leads on BBH chain-of-thought and FullStackBench. No independent tests were available when this was written.
Availability and license
The weights are on Hugging Face in two repositories: FP8 and BF16. Baidu recommends SGLang or vLLM, using the plain completions API, which is the right interface for a base model. A fine-tuning example is in NVIDIA's Megatron-Bridge repository.
The license is a custom "ERNIE 5.5 Flash Base Model License Agreement v1.0." The model card does not summarize its terms or say whether commercial use is allowed, so read the LICENSE file before building on it. The card also lists 286B parameters in the file metadata against 280B in the text, without explaining the gap.
What we don't know yet
Baidu has not given a date for the post-trained version, which is the one most developers will want to test. There is no price listed for this model, and Hugging Face shows no hosted deployment. Until outside groups run their own evaluations, the benchmark lead is Baidu's claim.