LongMab-PO long-document preference pairs
The LongMab-PO release provides two JSON Lines files of chosen and rejected responses for long-context preference optimization. The Llama 3 file contains 2,250 pairs and the Qwen 2 file contains 2,112.
Which files are ready for preference training?
Both preference files contain id, input, prompt, chosen and rejected. Select the file for the model family you are studying and keep prompt–response pairs together. The authors also distribute a larger raw_data_42_8k_16k.jsonl file in the same folder for constructing training examples.
LongMab-PO samples chunks of the long context to generate candidate responses and uses reward feedback to guide subsequent sampling. The repository documents both generating those candidates and training from the released pairs. Authors’ training instructions
Data files
Related paper
Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization · Official BibTeX