Bilingual GPT-NeoX 4B PPO
This release adds PPO alignment to the bilingual 4B SFT model using Japanese-translated Anthropic HH-RLHF data.
A second alignment stage follows supervised training
rinna describes SFT and PPO as separate training stages and identifies 2 August 2023 as the release date. The model retains the 36-layer, 2,816-hidden-width decoder.Bilingual GPT-NeoX 4B PPO official model card
Measure the effect of preference training
Practical guidance. Use the same Japanese and English prompts for SFT and PPO. Record refusals, incomplete answers and repetition as well as preferred responses; a single score can hide changes that matter to an application.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Launch date | |
| Access | Downloadable |
| Model identifiers | bilingual-gpt-neox-4b-instruction-ppo |
| Scale | 3.8B language-model parameters; the 4B name is rounded.Bilingual GPT-NeoX 4B PPO official model card |
| License | MIT for this published release.Bilingual GPT-NeoX 4B PPO official model card |
Related models
- Bilingual GPT-NeoX 4B
rinna · Bilingual GPT-NeoX
An English-Japanese GPT-NeoX language model with 3.8B parameters.
Launch date not confirmedResearch releaseDownloadable - Bilingual GPT-NeoX 4B SFT
rinna · Bilingual GPT-NeoX
The supervised instruction release adapts Bilingual GPT-NeoX 4B using English and Japanese instruction data.
Launch date not confirmedDownloadable