LongMab-PO long-document preference pairs

The LongMab-PO release provides two JSON Lines files of chosen and rejected responses for long-context preference optimization. The Llama 3 file contains 2,250 pairs and the Qwen 2 file contains 2,112.

Which files are ready for preference training?

Both preference files contain id, input, prompt, chosen and rejected. Select the file for the model family you are studying and keep prompt–response pairs together. The authors also distribute a larger raw_data_42_8k_16k.jsonl file in the same folder for constructing training examples.

LongMab-PO samples chunks of the long context to generate candidate responses and uses reward feedback to guide subsequent sampling. The repository documents both generating those candidates and training from the released pairs. Authors’ training instructions

Data files

Authors’ training data folder

Related paper

Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization · Official BibTeX

Browse research data · Download catalog (JSON-LD)