rinnarinna Japanese RoBERTa
Japanese RoBERTa Base
Japanese RoBERTa Base is rinna's 12-layer text encoder. It was trained with a masked-language objective on Japanese CC-100 and Wikipedia.
Research release
Input preparation affects masked-token results
The card instructs users to add CLS before the sentence, mask tokens after tokenization, and supply position IDs that start at zero. It warns that default position-ID construction and directly typed mask strings can produce different results.Japanese RoBERTa Base official model card
Use the published input path before adaptation
Practical guidance. Check the tokenizer and positions before using this model as a downstream encoder. A generic fill-mask pipeline can give misleading results when its input preparation differs from the training setup.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Launch date | |
| Access | Downloadable |
| Model identifiers | japanese-roberta-base |
| Architecture | 12 layers; hidden size 768.Japanese RoBERTa Base official model card |
| License | MIT.Japanese RoBERTa Base official model card |
| Model role | Pretrained masked-language encoder; not a chat or retrieval-tuned model.Japanese RoBERTa Base official model card |
Related models
- Japanese GPT-2 Medium
rinna ยท Japanese GPT
A Japanese GPT-2 language model trained to predict continuations of text.
Launch date not confirmedResearch releaseDownloadable