Does Aggressive Filtering Make AI Answers Safer? The Evidence-Erasure Cost

Removing every suspicious source can reduce an attack's influence. But when an AI must rely on external documents, aggressive filtering can remove the correct evidence too. A lower attack rate does not automatically mean a more useful system.

CamoDocs measures this trade-off on NeoQA, a retrieval-dependent benchmark. Without an attack, TrustRAG removes 91.48% of retrieved documents and reduces accuracy from 29.13% for clean RAG to 5.79%. Under attack, attack success remains 23.25% while correct-answer accuracy is 9.08%.

Why older benchmarks can hide the cost

If a model can answer from training memory, deleting external evidence may look inexpensive. In the paper, HotpotQA still achieves 37.20% accuracy without retrieval, while NeoQA uses fictional knowledge to force document dependence. The latter exposes defenses that appear robust mainly because they erase the evidence pool.

Evaluate source defenses with paired metrics:

• suspicious content reaching final context;
• clean-question accuracy before and after filtering;
• benign-source deletion rate;
• counts of wrong answers, refusals, and conflicts;
• human escalation paths for high-risk claims.

GEO Radar (https://www.georadar.top) can help observe brand answers, sources, and competitor differences across platforms. If sources suddenly shrink while refusals increase, record it as an evidence-pool change rather than automatically blaming content quality.

A content-side boundary

Brands cannot control platform filters, but they can make legitimate evidence easier to audit: state publisher, date, version, and scope; avoid fleets of near-duplicate pages; and support important claims through accessible independent sources. None of these steps guarantees retrieval or retention.

Sources for this article

• arXiv, August 28, 2026, *CamoDocs: A Poisoning Attack Against Retrieval-Augmented Language Models Using Camouflaged Documents*: https://arxiv.org/abs/2608.28389