FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents
Training deep-search agents requires verifiable questions that demand substantial evidence acquisition, yet structurally complex tasks may still admit cheap identifying routes when clues are overly selective, co-covered by the same evidence, directly searchable, or resolved from solver priors. We formalize this gap with a shortcut-aware difficulty framework that distinguishes intended construction complexity from identifying routes available through retrieval. Guided by this framework, we introduce FORT, a Framework of Shortcut-Resistant Training-Data Synthesis, which targets four shortcut risks across entity selection, evidence graph construction, question formulation, and adversarial refinement. We evaluate realized search behavior with solver-conditioned trajectory diagnostics rather than graph complexity or search length alone. Compared with existing open-source deep-search datasets, FORT shows later answer exposure, higher retrieval effort, lower single-clue selectivity, greater evidence dispersion, and higher realized pre-answer dependency costs. Using the resulting trajectories, we train FORT-Searcher with supervised fine-tuning alone. Across model scales, FORT-Searcher consistently improves over its backbones, achieving the strongest overall performance among similarly sized agents. Relevant resources will be made available at RUCAIBox/FORT-Searcher.
Comments: 35 pages
Mentions
FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents Training deep-search agents requires verifiable questions that demand substantial evidence acquisition, yet structurally complex tasks may still admit cheap identifying routes when clues are overly selective, co-covered by the same evidence, directly searchable, or resolved from solver priors. We formalize this gap with a shortcut-aware difficulty framework that distinguishes intended construction complexity from identifying routes available through retrieval. Guided by this framework, we introduce FORT, a Framework of Shortcut-Resistant Training-Data Synthesis, which targets four shortcut risks across entity selection, evidence graph construction, question formulation, and adversarial refinement. We evaluate realized search behavior with solver-conditioned trajectory diagnostics rather than graph complexity or search length alone. Compared with existing open-source deep-search datasets, FORT shows later answer exposure, higher retrieval effort, lower single-clue selectivity, greater evidence dispersion, and higher realized pre-answer dependency costs. Using the resulting trajectories, we train FORT-Searcher with supervised fine-tuning alone. Across model scales, FORT-Searcher consistently improves over its backbones, achieving the strongest overall performance among similarly sized agents. Relevant resources will be made available at RUCAIBox/FORT-Searcher. Comments: 35 pages