LLM-jp 13B v1.1 DPO
LLM-jp 13B v1.1 DPO starts from the full v1.1 instruction checkpoint. It applies direct preference optimization with hh-rlhf-12k-ja.
Research release
The parent is already instruction-tuned
The model card directly identifies the full Dolly, OpenAssistant and ichikara v1.1 model as its parent. The preference step is a later training stage, not a new pretraining run.LLM-jp 13B v1.1 DPO official model card
Evaluate changes to preferences and refusal
Practical guidance. Compare the output with its exact full-tuned parent. That makes it easier to see whether DPO changes answer preference or refusal behavior instead of attributing every difference to model size.
Specifications & access
| Detail | Value and conditions |
|---|---|
| Status | Research release |
| Access | Downloadable |
| Model identifiers | llm-jp-13b-dpo-lora-hh_rlhf_ja-v1.1 |
| Context window | 2,048 tokens Maximum sequence length stated in the model card. LLM-jp 13B v1.1 DPO official model card |
| Training method | DPO with LoRA, as identified by the release.LLM-jp 13B v1.1 DPO official model card |
| Preference data | Japanese HH-RLHF translation produced with DeepL.LLM-jp 13B v1.1 DPO official model card |
| License | Apache 2.0.LLM-jp 13B v1.1 DPO official model card |
| Distribution | Rank-128 LoRA adapter. The configuration names the full v1.1 instruction model as its required base.LLM-jp v1.1 DPO adapter configuration |
Related models
- LLM-jp 13B v1.1 Instruct Full
LLM-jp / NII LLMC · LLM-jp v1
The v1.1 Full instruction release combines Japanese human-written data with English and translated dialogue data.
Launch date not confirmedResearch releaseDownloadable - LLM-jp 13B v1.1 Instruct LoRA
LLM-jp / NII LLMC · LLM-jp v1
The v1.1 LoRA instruction release combines Japanese human-written data with English and translated dialogue data.
Launch date not confirmedResearch releaseDownloadable