DeepSeek · GPT · Agentic AI · Hugging Face
Give an LLM agent a realistic multi-step task, split a bill, spot a song
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
★ Tier-1 Source
It mis-paginates an API, resolves the wrong person, or returns a value when none was asked for.
Key facts
- DeepSeek-V3.2, by difficulty (baseline → +memory): the two systems start from different no-memory baselines (79.8 vs 72.0 TGC overall) because of the prompt-template difference
- TGC = Task Goal Completion; SGC = Scenario Goal Completion, which requires every variant of a scenario to pass
- The ACE numbers are their own runs of the ACE agent, evaluated in-house on the same AppWorld splits and the same base models as ALTK-Evolve (DeepSeek-V3.2 and gpt-oss-120b)
- On the strong model they're better on both metrics at ~40% of ACE's inference cost
Summary
Give an LLM agent a realistic multi-step task, split a bill, find a song, reconcile an order across nine simulated apps, and when it fails, it usually isn't for lack of knowledge. Two recent systems do exactly this, on the same kind of agent: ACE (Agentic Context Engineering) and their ALTK-Evolve ( introduced here ). A note on words, because the two systems name things differently: they'll call the raw thing an agent learns a lesson. ACE names the failure modes precisely: brevity bias, optimization collapsing toward short, generic instructions, and context collapse, a model asked to rewrite its whole context each step summarizing the detail away. The team reach the same conclusion from the other direction.