SPARK: Adaptive Parametric Knowledge Participation for Faithful Retrieval-Augmented Generation

Ruochang Li, Pengcheng Huang, Zhenghao Liu, Yukun Yan, Huiyuan Xie, Yu Gu, Ge Yu, Maosong Sun

arXiv preprint

arXiv · Code · Hugging Face · ORKG · Full text (arXiv HTML)

SPARK studies how a retrieval-augmented language model handles conflicts between retrieved evidence and its parametric knowledge. A lightweight policy adjusts the contribution of feed-forward network states during decoding while the backbone stays frozen.

How the participation policy works

The policy learns to regulate feed-forward contributions with a faithfulness-oriented reward. The experiments report contextual faithfulness across six datasets. A separate analysis compares cases where the retrieved evidence is accurate with cases where the model’s closed-book answer is accurate. Source

Adapting to reliable and misleading retrieval

Approach Where it intervenes
ParamMute Identifies feed-forward layers associated with unfaithful answers and combines activation suppression with preference-based adaptation.
FIDES Uses signals from the output, hidden representations, and prediction trajectory to set intervention strength during decoding.
SPARK Learns a lightweight policy that adjusts feed-forward contributions by layer and decoding step while keeping the backbone frozen.

When comparing these interventions, distinguish inaccurate parametric memory with correct retrieved context from correct memory with misleading retrieved context. Section 6.3 reports the cases separately. The papers use different conflict subsets and evaluation protocols.

Earlier preprint

The first arXiv version was titled SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generation. Source

DOI 10.48550/arXiv.2606.27786 · BibTeX

Research guide on conflicts between model memory and retrieved evidence