Sakana AISparseLM
SparseLM0.5B
This is the 0.5B member of Sakana's SparseLM release for sparse training and inference experiments.
Research release
Sparse activation needs a matching execution path
The configuration uses gated sparse feedforward layers and L1 regularization. Its architecture name describes the implementation; it does not establish that pretrained Meta Llama weights were used.SparseLM0.5B released configuration
Measure the complete inference path
Practical guidance. Compare the sparse and reference kernels on the same hardware. Record batch size and sequence length; theoretical sparsity alone does not establish a latency reduction.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | SakanaAI/SparseLM0.5B |
| Architecture | SparseLlamaForCausalLM; 8 layers and hidden width 2,048.SparseLM0.5B released configuration |
| Configured sequence size | 2,048 positions in the released configuration.SparseLM0.5B released configuration |
| License | Apache 2.0SparseLM0.5B official model card |