AI That Knows When It's Not Sure
mlj.solutions
mlj.solutions AI, která umí říct “nevím jistě, ale je to jedna z těchto možností”, je spolehlivější než AI, co si vždycky vymyslí odpověď. Nový přístup (Conformal Path Reasoning) dává AI systémům, které odpovídají na základě znalostních grafů, statistickou záruku přesnosti. Výsledek: o 34 % lepší spolehlivost (Empirical Coverage Rate) a o 40 % konkrétnější odpovědi (menší predikční množiny) oproti předchozím metodám. 👉 Jak řešíte spolehlivost AI odpovědí ve vaší firmě?
View on Instagrammlj.solutions
mlj.solutions AI se konečně naučila říct: NEVÍM. Většina AI systémů odpovídá vždy - i když si není jistá. CPR to mění: pokud odpověď dá, je statisticky zaručená. Pokud si není jistá, neodpoví vůbec. Proč je to průlom pro enterprise AI? Celý příběh ve čtvrtek. 🎬
View on InstagramWhat the study is about
AI answers over knowledge graphs are easy to explain, but there is often no guarantee that the correct answer is among the ones offered. The study presents the CPR method, which verifies this guarantee statistically at the level of individual reasoning paths. The result is a more reliable and, at the same time, shorter list of answers.
Statistics
45%
higher coverage of correct answers
Empirical coverage rose by 45% on average compared with earlier conformal methods, so the correct answer appears in the offer significantly more often.
52%
smaller answer set
The offered answer sets shrank by 52% on average, so the user receives fewer and more precise candidates.
2
key innovations of the method
Query-level calibration preserves the validity of the guarantees, and the lightweight RCVNet module learns to tell good reasoning paths from weak ones.
What this means
- The guarantee matters as much as the answer, in sensitive areas, AI must be able to say how sure it is.
- Poor calibration voids the guarantees, earlier methods violated the statistical guarantees because of invalid calibration.
- A smaller set doesn't mean a worse result, the method increases coverage and shortens the list of answers at the same time.
- A lightweight add-on module is enough, RCVNet improves discrimination without having to change the language model itself.
- Explainability can be audited, knowledge graphs offer concrete reasoning paths that can be verified after the fact.
Our take
An AI that answers and can also say how sure it is is more valuable to businesses than an AI that always answers.
Michal Dobrovolný
Founder
Zdroj
Lin, S., Zhou, C., Lin, X., Dong, Z., Lu, K., Peng, Z., Yin, J., & Metaxas, D. N. (2026). Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibration. arXiv:2605.08077. https://arxiv.org/abs/2605.08077
Don't miss these posts
When More Agents Pay Off and When They Don't
More AI agents don't necessarily mean a better result. A study shows that what matters is how much information is lost when it is passed between agents.
AI Remembers a Claim Even When You Tell It It's False
When a model is fine-tuned on documents that deny a claim, it often learns the claim as true anyway. The authors named this effect Negation Neglect.
The Skill That Explains Anything to Anyone
ELI5 is an open-source skill for Claude Code that adapts explanations to the audience. With the skill it passed 10 of 12 criteria, without it only 5.
Want to implement AI in your company?
We'll go through your needs together and propose an approach that delivers measurable results.
Write to us
