Stable LM 2 12B
Stable LM 2 12B is the larger base release in the Stable LM 2 family. It was pretrained on multilingual text and code, with a decoder architecture designed for later task adaptation.
Model design and access
The model uses parallel attention and feed-forward layers, per-head query-key normalization and partial rotary embeddings. Its Arcade100k tokenizer is shared with the smaller family. The published sequence length is 4,096 tokens, despite its larger parameter count.Stable LM 2 12B official model card
Practical use
Practical guidance. The larger base can be useful for adaptation studies with enough local compute. It does not provide a larger document window than the 1.6B model. Plan memory and document handling separately, and compare the value of fine-tuning it with starting from an already instruction-tuned release.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | stablelm-2-12b |
| Context window | 4,096 tokens Published checkpoint sequence length; prompt and generated text share the window. Stable LM 2 12B official model card |
| Parameters | 12,143,605,760.Stable LM 2 12B official model card |
| Attention | 32 query heads and 8 key-value heads across 40 layers.Stable LM 2 12B official model card |
| License | Stability AI Community License.Stable LM 2 12B official model card |
Related models
- Stable LM 2 1.6B
Stability AI · Stable LM
A 1.6B base language model pretrained on multilingual text and code.
Launch date not confirmedDownloadable - Stable LM 2 Chat 12B
Stability AI · Stable LM
Stable LM 2 Chat 12B is an English instruction-tuned model with documented function calling.
Launch date not confirmedResearch releaseDownloadable