The AI Agent That Asks When It Doesn't Understand
mlj.solutions
mlj.solutions AI může hledat rychleji než člověk. Ale pozná, kdy otázce nerozumí? DiscoBench ukazuje: detekovat nejasnost a správně se doptat jsou dvě různé schopnosti. Více hledání bez doptání může dopadnout hůř, než kdyby to AI odhladla. 👉 Kde by se měl váš AI agent povinně zastavit a zeptat?
View on Instagrammlj.solutions
mlj.solutions Zadal jsi AI nejasný dotaz. Ona nezaváhala a hledala. Jen bohužel odpověď na špatnou otázku. Model umí poznat, že dotaz není úplně jasný. Ale poznat nejasnost a vědět, kdy se zastavit a zeptat, jsou dvě různé věci. Bez tohoto rozlišení agent hledá dál, dotáhne to do konce a klidně vám sebevědomě dá odpověď, která s vaším skutečným záměrem nemá nic společného. V praxi to znamená jednu klíčovou otázku pro každého, kdo agenty nasazuje: kde přesně by se měl zastavit a doptat, než začne hledat? DiscoBench testuje přesně tuhle rozhodovací hranici a ukazuje, že málokterý agent ji pozná sám. 👉 Kde by se měl váš AI agent povinně zastavit a zeptat?
View on InstagramWhat the study is about
Search agents receive vague, incomplete or even factually incorrect queries from users, and still carry on based on a guess. DiscoBench is a benchmark that tests whether an agent can recognize the ambiguity, ask a question, and use the answer to get back on the right track.
Statistics
211
test scenarios
The scenarios cover 11 real-world domains so that the results reflect actual user queries.
463
instances of ambiguity
The requests are vague, incomplete or factually incorrect and fall into four types of ambiguity.
4
evaluation perspectives
Agents are judged on task utility, ambiguity detection, interaction strategy and cost efficiency.
What this means
- Ambiguity spreads, one misunderstood piece of information at the start sends the entire multi-step process in the wrong direction.
- Detecting and asking are two different skills, an agent can recognize ambiguity and still fail to follow up effectively.
- Searching instead of asking doesn't pay off, repeated searching without asking often does worse than guessing outright.
- Today's benchmarks don't match reality, they assume complete requests, whereas real queries aren't like that.
- Asking is part of the solution, interacting with the user is not a sign of a weak agent but a necessary part of a reliable process.
Our take
An agent that fills in missing information on its own seems confident, but it can steer the whole process in the wrong direction. When designing agent systems, we therefore expect the agent to ask first in an unclear situation.
Michal Dobrovolný
Owner & CTO
Zdroj
Tao, Y., Deng, S., Tao, M., Wei, P., Hu, Z., & Zhu, Z. (2026). When Search Agents Should Ask: DiscoBench for Clarification-Aware Deep Search. arXiv:2606.27669. https://arxiv.org/abs/2606.27669
Don't miss these posts
When More Agents Pay Off and When They Don't
More AI agents don't necessarily mean a better result. A study shows that what matters is how much information is lost when it is passed between agents.
AI Remembers a Claim Even When You Tell It It's False
When a model is fine-tuned on documents that deny a claim, it often learns the claim as true anyway. The authors named this effect Negation Neglect.
The Skill That Explains Anything to Anyone
ELI5 is an open-source skill for Claude Code that adapts explanations to the audience. With the skill it passed 10 of 12 criteria, without it only 5.
Want to implement AI in your company?
We'll go through your needs together and propose an approach that delivers measurable results.
Write to us
