Sakana AITinySwallow
TinySwallow-1.5B
TinySwallow distills a Qwen2.5-32B-Instruct teacher into a Qwen2.5-1.5B-Instruct student, followed by additional Japanese pretraining.
Research release
The base release precedes Japanese instruction tuning
This checkpoint contains the distilled and further pretrained model. The authors direct users who need instruction following to the separate TinySwallow-1.5B-Instruct checkpoint.TinySwallow-1.5B official model card
Working with this release
Practical guidance. Select this release for further adaptation. Keep it separate from the chat checkpoint when recording results or combining training data.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | SakanaAI/TinySwallow-1.5B |
| Developers | Sakana AI and the Swallow teamTinySwallow-1.5B official model card |
| Ancestry | Qwen2.5-1.5B-Instruct student; Qwen2.5-32B-Instruct teacher.TinySwallow-1.5B official model card |
| Terms | Apache 2.0; the card separately describes research-only intent and discourages commercial or critical deployment.TinySwallow-1.5B official model card |
Related models
- TinySwallow-1.5B-Instruct
Sakana AI ยท TinySwallow
A Japanese instruction-tuned language model derived from the Qwen-based TinySwallow model.
Launch date not confirmedResearch releaseDownloadable