LLaMA-3-MERaLiON-8B-Instruct
MERaLiON TextLLM is an 8B-class text model for multilingual understanding and instruction following. The release builds on Meta Llama weights. A*STAR's contribution includes continued pretraining and model merging for regional language coverage.
Continued pretraining and weight merging
The model card describes domain mixing and replay during continued pretraining. It attributes instruction following to merging base and instruction-model weights, without another supervised instruction dataset. These training changes distinguish this release from an unchanged hosted Llama model.LLaMA-3-MERaLiON-8B-Instruct official model card
The sources use conflicting Llama labels
The paper and card introduction identify Llama-3-8B-Base. Other card sections say Llama-3.1-8B and describe merging Llama 3.1 weights. The configuration does not resolve that conflict. The verified ancestry is the Llama 3 family; the precise upstream revision needs clarification from the authors.MERaLiON TextLLM technical reportLLaMA-3-MERaLiON-8B-Instruct official model card
Available weights and original terms
The weights remain public under the MERaLiON v1 terms with a Meta Llama 3 notice. The custom license includes redistribution requirements. Do not replace these terms with an inferred Llama 3.1 license based only on the conflicting card text.MERaLiON Public Licence v1, 2024
Evaluate each target language
Practical guidance. For a regional writing task, compare the same prompts in each intended language. The main-language list is more useful for planning that evaluation than a single multilingual score. Preserve the checkpoint identifier in your results because the upstream version labels are inconsistent.LLaMA-3-MERaLiON-8B-Instruct official model card
Specifications & access
| Detail | Value and conditions |
|---|---|
| Access | Downloadable |
| Model identifiers | MERaLiON-TextLLMMERaLiON-LLaMA-3-8B-Instruct |
| Context window | 8,192 tokens Published text-model configuration; the model card also specifies 8,192 tokens. LLaMA-3-MERaLiON-8B-Instruct published configuration |
| Main languages | English, Chinese and Indonesian.LLaMA-3-MERaLiON-8B-Instruct official model card |
| Continued pretraining | More than 120 billion text tokens.LLaMA-3-MERaLiON-8B-Instruct official model card |
| Architecture | 32 decoder layers and 4,096 hidden dimensions.LLaMA-3-MERaLiON-8B-Instruct published configuration |
Related models
- MERaLiON AudioLLM Whisper SEA-LION
MERaLiON / A*STAR IAIC ยท MERaLiON AudioLLM
A regional speech-text model that adapts a speech encoder and text model for audio questions and transcription.
Launch date not confirmedDownloadable