Poro 2 70B Base
Poro 2 70B Base continues pretraining Meta's Llama 3.1 70B to strengthen Finnish while retaining English, code and mathematics. It is a trained derivative, not unchanged Llama hosting.
Model design and training
Continued pretraining used 165 billion tokens. The mix allocates roughly 30% each to Finnish, English and code, with 10% mathematics. Finnish material comes from FineWeb2; the other components include FineWeb-Edu, StarCoder data and FineMath. The release precedes instruction and preference tuning.Llama-Poro-2-70B-base: official model card
Practical assessment
Practical guidance. Use this larger base to study language adaptation without also changing the instruction-training method. Its parameter count affects weight memory separately from context memory. Compare short and long Finnish passages before choosing a document-processing workflow.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | Llama-Poro-2-70B-base |
| Context window | 8,192 tokens Sequence length reported by this release's model card; this is not a guaranteed useful retrieval length. Llama-Poro-2-70B-base: official model card |
| Model size | 70.55 billion parameters; 80 decoder layers.Llama-Poro-2-70B-base: official model card |
| License | Llama 3.1 Community License.Llama-Poro-2-70B-base: official model card |
Related models
- Poro 2 70B Instruct
LumiOpen · Poro 2
A 70B Finnish and English conversation model trained with supervised instructions and Direct Preference Optimization.
Launch date not confirmedDownloadable - Poro 2 70B SFT
LumiOpen · Poro 2
An intermediate instruction-tuned Poro 2 70B checkpoint before the final preference-training stage.
Launch date not confirmedResearch releaseDownloadable