Sakana AISparseLM
SparseLM1B
This is the 1B member of Sakana's SparseLM release for sparse training and inference experiments.
Research release
Sparse activation needs a matching execution path
The configuration uses gated sparse feedforward layers and L1 regularization. Its architecture name describes the implementation; it does not establish that pretrained Meta Llama weights were used.SparseLM1B released configuration
Measure the complete inference path
Practical guidance. Compare the sparse and reference kernels on the same hardware. Record batch size and sequence length; theoretical sparsity alone does not establish a latency reduction.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | SakanaAI/SparseLM1B |
| Architecture | SparseLlamaForCausalLM; 18 layers and hidden width 2,048.SparseLM1B released configuration |
| Configured sequence size | 2,048 positions in the released configuration.SparseLM1B released configuration |
| License | Apache 2.0SparseLM1B official model card |