Fugaku-LLM-13B
Fugaku-LLM-13B is the consortium's base language model. It was trained from scratch on Fugaku and published with downloadable weights under custom terms. Fujitsu is one of seven participating organizations.
Research release
The project adapted training software to a CPU supercomputer
The team ported Megatron-DeepSpeed and optimized communication for Fugaku's Tofu interconnect. Training used 13,824 nodes. This distinguishes the release from a Japanese continuation of an imported pretrained model.Consortium announces the Fugaku-LLM release
Use the base checkpoint for further training
Practical guidance. The base model is a starting point for controlled adaptation or completion research. Use the instruction-tuned sibling when the application needs a ready conversation format. Keep the base and adapted checkpoint identifiers in experiment records.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | Fugaku-LLM/Fugaku-LLM-13B |
| Training scale | 13 billion parameters; 380 billion training tokens.Consortium announces the Fugaku-LLM release |
| Training mixture | About 60% Japanese, with English, mathematics and code.Consortium announces the Fugaku-LLM release |
| License | Fugaku-LLM Terms of Use; commercial and noncommercial use subject to those terms.Fugaku-LLM-13B model card and terms |
| Download | Hugging Face account and agreement to the access conditions required.Fugaku-LLM-13B model card and terms |
Related models
- Fugaku-LLM-13B-instruct
Fujitsu ยท Fugaku-LLM
The instruction-tuned Fugaku-LLM checkpoint for Japanese and English dialogue.
Launch date not confirmedResearch releaseDownloadable