AI Agents That Learn From Their Own Mistakes
mlj.solutions
mlj.solutions AI se může učit z chyb, ale jen když jí to dovolíte. FORGE: po každé chybě agent provede reflexi a uloží si ji do paměti. Po 10–20 cyklech: o 40–60 % méně chyb. Bez přetrénování modelu. 👉 Jak pracujete se zpětnou vazbou u AI agentů ve vaší firmě?
View on Instagrammlj.solutions
mlj.solutions Představte si AI agenta, který si dělá chyby. A pak si z nich dělá poznámky. FORGE (Framework for Observing, Reflecting, and Generalizing Errors) funguje takto: po každé chybě agent provede reflexi - co se pokazilo, proč, jak se tomu vyhnout příště. Po 10–20 cyklech dělají agenti o 40–60 % méně chyb. Bez přetrénování modelu. Jen změnou vlastní paměti. AI se může učit z chyb, ale jen když jí to dovolíte. 👉 Myslíte, že AI má mít schopnost se poučit z vlastních chyb?
View on InstagramWhat the study is about
What if an AI agent could improve just by remembering its own mistakes, without retraining the model? The FORGE study tests a group of agents that build rules and examples from their failures and share the best ones with each other. They verified everything on a simulated cyber defense of a computer network.
Statistics
1.7 to 7.7x
better than the baseline
FORGE improved the agents' average result compared with the model without any learned memory, across all four tested models.
29 to 72%
better than Reflexion
Compared with an approach where each agent learns in isolation on its own, shared memory brought a significant improvement in all 12 tested combinations.
1%
severe failures
The share of major failures dropped to roughly one percent, even though all models started with strongly negative results.
What this means
- An agent can improve without retraining, a text memory where it stores lessons from failures is enough.
- Sharing memory brings the main benefit, the best insights are passed between agents, and that is what drives performance up.
- Rules are cheaper, examples are stronger, rules require roughly 40% fewer tokens, while examples give the best results with most models.
- Weaker models benefit the most, the method can narrow the gap between smaller and top-tier models.
- The results so far apply to only one environment, the authors themselves note that this is a directional finding from a single cyber defense scenario.
Our take
Improving an agent doesn't have to mean retraining the model. Often a well-designed memory that stores lessons from mistakes is enough. These are exactly the kinds of layers we design for corporate agents.
Zdroj
Bogdanov, I., Lung, C.-H., Kunz, T., Gao, J., Taylor, A., & Zaman, M. (2026). FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast. arXiv:2605.16233. https://arxiv.org/abs/2605.16233
Don't miss these posts
When More Agents Pay Off and When They Don't
More AI agents don't necessarily mean a better result. A study shows that what matters is how much information is lost when it is passed between agents.
AI Remembers a Claim Even When You Tell It It's False
When a model is fine-tuned on documents that deny a claim, it often learns the claim as true anyway. The authors named this effect Negation Neglect.
The Skill That Explains Anything to Anyone
ELI5 is an open-source skill for Claude Code that adapts explanations to the audience. With the skill it passed 10 of 12 criteria, without it only 5.
Want to implement AI in your company?
We'll go through your needs together and propose an approach that delivers measurable results.
Write to us
