Poro 2 8B Long Base
Poro 2 Long Base extends the Finnish and English Poro 2 8B base from 8K to 128K context. Its ancestry remains Meta Llama 3.1 8B.
Model design and training
LongRoPE rescaling factors were selected through an evolutionary search and followed by long-context training. The team used a multi-node adaptation of Microsoft's LongRoPE tools. The card reports 131,072 tokens and separate attention scaling; the change is more than an inference-time length setting.Llama-Poro-2-8B-Long-Base: official model card
Practical assessment
Practical guidance. Start with long-document retrieval checks before adding instruction tuning. Place evidence at different positions and assess whether the model uses it accurately. A longer accepted sequence does not ensure reliable reasoning across every part of a document.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | Llama-Poro-2-8B-Long-Base |
| Context window | 131,072 tokens Sequence length reported by this release's model card; this is not a guaranteed useful retrieval length. Llama-Poro-2-8B-Long-Base: official model card |
| Model size | 8.03 billion parameters.Llama-Poro-2-8B-Long-Base: official model card |
| License | Llama 3.1 Community License.Llama-Poro-2-8B-Long-Base: official model card |
Related models
- Poro 2 8B Base
LumiOpen · Poro 2
A Llama 3.1 8B derivative with continued pretraining for Finnish, English, code and mathematics.
Launch date not confirmedDownloadable - Poro 2 8B Long Instruct
LumiOpen · Poro 2 Long
A Finnish and English instruction model derived from the context-extended Poro 2 base.
Launch date not confirmedDownloadable