Sakana AISparseLM
SparseLM2B
This is the 2B member of Sakana's SparseLM release for sparse training and inference experiments.
Research release
Sparse activation needs a matching execution path
The configuration uses gated sparse feedforward layers and L1 regularization. Its architecture name describes the implementation; it does not establish that pretrained Meta Llama weights were used.SparseLM2B released configuration
Measure the complete inference path
Practical guidance. Compare the sparse and reference kernels on the same hardware. Record batch size and sequence length; theoretical sparsity alone does not establish a latency reduction.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | SakanaAI/SparseLM2B |
| Architecture | SparseLlamaForCausalLM; 38 layers and hidden width 2,048.SparseLM2B released configuration |
| Configured sequence size | 2,048 positions in the released configuration.SparseLM2B released configuration |
| License | Apache 2.0SparseLM2B official model card |