NeuralStackStanford University's Human-Centered AI Institute unveiled a 1.2 trillion parameter sparsely activated model on July 20 2026. The model uses a mixture-of-experts architecture that routes each token to 12 of 128 experts, activating roughly 120 billion parameters per forward pass. Training completed on 8,000 H100 GPUs over 41 days.
Benchmarks show parity with dense 1.2 trillion parameter models on MMLU and MATH while using 65 percent less compute during inference. The research team open-sourced the routing code and training logs under an Apache 2.0 license.
Prior Stanford work on efficient scaling includes the 2025 release of a 500 billion parameter sparse model. The new effort focuses on reducing the environmental and financial barriers to frontier model development.
Why this matters
The results provide empirical evidence that sparse activation can deliver frontier performance at materially lower inference cost, influencing roadmap decisions at major labs. Smaller research groups may now experiment with larger effective model sizes than previously feasible.
Industry observers expect increased investment in custom silicon optimized for sparse workloads. Stanford plans to release a fine-tuning recipe for domain adaptation by the end of August 2026.