Moonlight-16B-A3B-Instruct
Moonlight Instruct is the instruction-tuned member of the original Moonlight release. It keeps the sparse 16-billion-parameter model design while adding the conversation behavior of a post-trained checkpoint.
A separate checkpoint for instructions
The publisher releases Base and Instruct separately. Its inference example formats a list of conversation messages with the tokenizer before generation. The model card also reports instruction benchmarks separately from the base-model comparison.Moonlight-16B-A3B-Instruct model card
What the Muon research establishes
The original paper makes pretrained and instruction-tuned checkpoints available as part of one research release. Its central contribution is the optimizer and its scaling behavior, rather than a new proprietary chat service.Muon is Scalable for LLM Training: original report
Plan for all model weights
Practical guidance. Active parameters measure how much of the model is used for a token. They do not mean that only three billion parameters need storage. Size a local deployment for the complete checkpoint and its attention cache.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Launch date | February 2025 |
| Access | Downloadable |
| Model identifiers | moonshotai/Moonlight-16B-A3B-Instruct |
| Context window | 8,192 tokens Published checkpoint context limit; input and generated tokens share this budget. Moonlight Instruct checkpoint configuration |
| Parameters | 16 billion total; approximately 3 billion active per token.Moonlight-16B-A3B-Instruct model card |
| Weight license | MIT.Moonlight-16B-A3B-Instruct model card |
| Input format | The published tokenizer supplies a chat template.Moonlight-16B-A3B-Instruct model card |
Related models
- Kimi-Linear-48B-A3B-Instruct
Moonshot AI · Kimi Linear
Instruction-tuned hybrid-attention model designed to reduce memory use in long conversations.
Launch date October 2025Downloadable - Moonlight-16B-A3B
Moonshot AI · Moonlight
Pretrained language model used to study Muon optimization at 16 billion total parameters.
Launch date February 2025Downloadable