AI Remembers a Claim Even When You Tell It It's False
mlj.solutions
mlj.solutions Věta „neposílej fakturu“ není bezpečnostní kontrola. Negace mohou být pro jazykové modely slabé místo. Kritické zákazy proto vynucujte oprávněními nástrojů, schvalováním a adversariálními testy. 📄 https://arxiv.org/abs/2605.13829
View on InstagramWhat the study is about
When a language model is fine-tuned on documents that explicitly label a certain claim as false, it often learns that claim as true anyway. The authors named this effect Negation Neglect and showed that it affects all tested models and behaviors, not just facts.
Statistics
88.6%
belief in the falsehood
After fine-tuning on documents that deny the claim, the model's belief in the false claim rose from 2.5% to 88.6%.
92.4%
belief without denial
Documents that did not deny the claim at all led to 92.4%, so the denial changed very little.
4+
models with the same effect
The effect appeared in all tested models, including Qwen3.5, Kimi K2.5 and GPT-4.1.
What this means
- A denial in the text isn't enough, the model recognizes a falsehood in a prompt, but after fine-tuning it starts treating it as fact.
- Phrasing matters, when the negation is built into the sentence itself, for example "X did not happen", the model mostly learns it correctly, unlike with a separate sentence stating that the claim is false.
- It affects behavior too, training on conversations labeled as harmful can teach the model that behavior, which is a safety risk.
- It applies to other qualifiers as well, claims labeled as fiction are learned by the model as if they were true.
- The correct solution is unstable, the model can learn the negation, but it gets lost during further training.
Our take
Whatever you fine-tune into a model settles in, even if you explicitly deny it in the documents. With AI solutions, we therefore focus on what gets into the training data in the first place, not just how it is annotated.
Michal Dobrovolný
Founder & CTO
Zdroj
Mayne, H., McKinney, L., Dubiński, J., Karvonen, A., Chua, J., & Evans, O. (2026). Negation Neglect: When models fail to learn negations in training. arXiv:2605.13829. https://arxiv.org/abs/2605.13829
Don't miss these posts
When More Agents Pay Off and When They Don't
More AI agents don't necessarily mean a better result. A study shows that what matters is how much information is lost when it is passed between agents.
The Skill That Explains Anything to Anyone
ELI5 is an open-source skill for Claude Code that adapts explanations to the audience. With the skill it passed 10 of 12 criteria, without it only 5.
A Correct Answer Is Not the Same as a Verified Analysis
TrustDABench tests whether AI working over tables can refuse to answer when the data isn't sufficient. The best model reached only 24.21% in reliability.
Want to implement AI in your company?
We'll go through your needs together and propose an approach that delivers measurable results.
Write to us
