SPARK: Adaptive Parametric Knowledge Participation for Faithful Retrieval-Augmented Generation
Ruochang Li, Pengcheng Huang, Zhenghao Liu, Yukun Yan, Huiyuan Xie, Yu Gu, Ge Yu, Maosong Sun
arXiv preprint
arXiv · Code · Hugging Face · ORKG · Full text (arXiv HTML)
SPARK studies how a retrieval-augmented language model handles conflicts between retrieved evidence and its parametric knowledge. A lightweight policy adjusts the contribution of feed-forward network states during decoding while the backbone stays frozen.
How the participation policy works
The policy learns to regulate feed-forward contributions with a faithfulness-oriented reward. The experiments report contextual faithfulness across six datasets. A separate analysis compares cases where the retrieved evidence is accurate with cases where the model’s closed-book answer is accurate. Source
Adapting to reliable and misleading retrieval
| Approach | Where it intervenes |
|---|---|
| ParamMute | Identifies feed-forward layers associated with unfaithful answers and combines activation suppression with preference-based adaptation. |
| FIDES | Uses signals from the output, hidden representations, and prediction trajectory to set intervention strength during decoding. |
| SPARK | Learns a lightweight policy that adjusts feed-forward contributions by layer and decoding step while keeping the backbone frozen. |
When comparing these interventions, distinguish inaccurate parametric memory with correct retrieved context from correct memory with misleading retrieved context. Section 6.3 reports the cases separately. The papers use different conflict subsets and evaluation protocols.
Earlier preprint
The first arXiv version was titled SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generation. Source
DOI 10.48550/arXiv.2606.27786 · BibTeX
Research guide on conflicts between model memory and retrieved evidence