Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models
Zikai Zhou, Jiacheng Sang, Yilin Xu, Yufei Jin, Monica S. Lam
arxiv.org [𝚌𝚜.𝙲𝙻 𝚌𝚜.𝙰𝙸]
💬Code: stanford-oval/CREDIT t.co
1 like · 43 views
Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models
A fundamental challenge for explainable and accountable language models is revealing how external information influences their answers. This paper examines whether models are self-attributive: that is, whether they indicate when -- and only when -- their responses to questions are influenced by information provided alongside the question. Furthermore, we explore how self-attribution can be improved through counterfactual training, which explicitly teaches models to distinguish between answers that do and do not depend on supplied information. We found that most of the eleven frontier models rarely attribute influential information in their answers, and the rest attribute non-influential information. Training acknowledgment into reasoning does not fix this: the acknowledgment does not reach the final answer. We introduce CREDIT, which directly supervises final-answer self-attribution with counterfactual labels, without the need for human annotation. CREDIT substantially improves attribution in four open models, including two distilled reasoning models, and outperforms reasoning-level fine-tuning and prompting baselines. Among answers that agree with a user's suggestion, CREDIT distinguishes those changed by the suggestion from those the model would have given anyway. Gains transfer partly to unseen information types and prompts containing sources that support conflicting answers.
Comments: 29 pages, 5 figures, 20 tables. Code: stanford-oval/CREDIT
Paper: arxiv.org
Project page, videos, results: sheldon.genie.stanford.edu
Code, prompts, configs: stanford-oval/sheldon
Joint work with my amazing co-authors @PriyankShethia (equal contribution) and @MonicaSLam , Stanford OVAL (@StanfordOVAL) .
5 likes · 79 views
Trials & patients → SMT programs, canonicalized to SNOMED CT, projected to SQL with 100% recall w.r.t. the SMT constraints.
📈 +32–72% relevant & eligible trials vs. TrialGPT
📈 1.8–3.2× recall on TREC 2022
⚡ 146 ms/patient
Code is open source 👇
stanford-oval/clinical-trial-matching
33 views
@MonicaSLam @YuchengJiang0 @stanfordnlp @StanfordOVAL Paper: arxiv.org
Code: stanford-oval/DataSTORM
2 likes · 60 views
和 AI聊了半个多小时，当它说出最可能赚到钱的位置，不是这套叙事里最性感的AI层，而是两头最无聊的环节。就很好理解了，像是我们的一些产品，本质上不需要多创新，反而是那些简单无聊的工具最赚钱。
基于斯坦福的这套框架，让 opus5.5做的调研技能
框架项目地址：stanford-oval/storm
论文地址：aclanthology.org
29 views
Mentions
Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models Zikai Zhou, Jiacheng Sang, Yilin Xu, Yufei Jin, Monica S. Lam arxiv.org [𝚌𝚜.𝙲𝙻 𝚌𝚜.𝙰𝙸] 💬Code: stanford-oval/CREDIT t.co
Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models A fundamental challenge for explainable and accountable language models is revealing how external information influences their answers. This paper examines whether models are self-attributive: that is, whether they indicate when -- and only when -- their responses to questions are influenced by information provided alongside the question. Furthermore, we explore how self-attribution can be improved through counterfactual training, which explicitly teaches models to distinguish between answers that do and do not depend on supplied information. We found that most of the eleven frontier models rarely attribute influential information in their answers, and the rest attribute non-influential information. Training acknowledgment into reasoning does not fix this: the acknowledgment does not reach the final answer. We introduce CREDIT, which directly supervises final-answer self-attribution with counterfactual labels, without the need for human annotation. CREDIT substantially improves attribution in four open models, including two distilled reasoning models, and outperforms reasoning-level fine-tuning and prompting baselines. Among answers that agree with a user's suggestion, CREDIT distinguishes those changed by the suggestion from those the model would have given anyway. Gains transfer partly to unseen information types and prompts containing sources that support conflicting answers. Comments: 29 pages, 5 figures, 20 tables. Code: stanford-oval/CREDIT
Paper: arxiv.org Project page, videos, results: sheldon.genie.stanford.edu Code, prompts, configs: stanford-oval/sheldon Joint work with my amazing co-authors @PriyankShethia (equal contribution) and @MonicaSLam , Stanford OVAL (@StanfordOVAL) .
Trials & patients → SMT programs, canonicalized to SNOMED CT, projected to SQL with 100% recall w.r.t. the SMT constraints. 📈 +32–72% relevant & eligible trials vs. TrialGPT 📈 1.8–3.2× recall on TREC 2022 ⚡ 146 ms/patient Code is open source 👇 stanford-oval/clinical-trial-matching
@MonicaSLam @YuchengJiang0 @stanfordnlp @StanfordOVAL Paper: arxiv.org Code: stanford-oval/DataSTORM
和 AI聊了半个多小时，当它说出最可能赚到钱的位置，不是这套叙事里最性感的AI层，而是两头最无聊的环节。就很好理解了，像是我们的一些产品，本质上不需要多创新，反而是那些简单无聊的工具最赚钱。 基于斯坦福的这套框架，让 opus5.5做的调研技能 框架项目地址：stanford-oval/storm 论文地址：aclanthology.org