Agent Memory Token Cost: Why Delivery, Not Storage, Decides Your AI Bill

Key takeaways

• IBM Research benchmarked two agentic-memory systems, ACE and ALTK-Evolve, on the same ReAct agent and the same AppWorld split.
• On gpt-oss-120b: 777K tokens per task versus 116K, with accuracy effectively tied (54.8 vs 56.0 task goal completion).
• On DeepSeek-V3.2: 634K versus 263K tokens per task, with the cheaper system also more accurate (89.3 vs 80.4).
• Both systems refuse to compress what the agent learned. The difference is delivery: one injects the whole playbook every step, the other retrieves a task-relevant subset.
• Once you divide tokens by success rate, the gap widens further, from roughly 2.4x to 6.8x cost per successful task.

What actually happened in the benchmark

IBM Research published a head-to-head between ACE (Agentic Context Engineering) and its own ALTK-Evolve. Both are agentic memory: they turn an agent's past trajectories into reusable lessons and feed them back at inference time, with no weight updates and no human labels. Both explicitly refuse to compress those lessons into a tidy summary, because summarising destroys the specific detail that made them useful.

The systems were run on the same base ReAct agent, on AppWorld test_normal (168 tasks), on two base models. Here is the result table as reported:

| Model | System | TGC / SGC | Tokens per task | | --- | --- | --- | --- | | DeepSeek-V3.2 | ACE | 80.4 / 73.2 | 634K | | DeepSeek-V3.2 | ALTK-Evolve | 89.3 / 80.4 | 263K | | gpt-oss-120b | ACE | 54.8 / 35.7 | 777K | | gpt-oss-120b | ALTK-Evolve | 56.0 / 37.5 | 116K |

TGC is task goal completion, SGC is scenario goal completion. The no-memory baselines, for reference, were 148K tokens per task on DeepSeek-V3.2 and 110K on gpt-oss-120b.

Why is one system 7x cheaper at the same accuracy?

Not because it knows less. Both systems keep the full set of lessons and both track how many independent episodes produced each one. The divergence is at delivery time.

ACE injects its comprehensive playbook on every step, identically regardless of model or task. ALTK-Evolve treats delivery as a dial: a small core of high-support guidelines, extended per task with a handful selected for the job at hand, or the full set when the model has the headroom to use it.

Same library, different checkout policy. One sends the whole shelf every time a question is asked. The other sends the three books that answer it.

What does this cost per successful task?

This is the number the original write-up does not compute, and it is the one that matters if you are pricing a product. Tokens per task counts every attempt, including the ones that fail. Divide tokens by the completion rate and you get the token cost of one task that actually worked:

• DeepSeek-V3.2: ACE 634K / 0.804 = roughly 788K tokens per successful task. ALTK-Evolve 263K / 0.893 = roughly 295K. A 2.7x gap.
• gpt-oss-120b: ACE 777K / 0.548 = roughly 1.42M tokens per successful task. ALTK-Evolve 116K / 0.560 = roughly 207K. A 6.8x gap.

The headline ratio understates the difference, because the more expensive system is also paying full price for its failures. If you charge per completed job and pay per token, cost per successful task is your actual COGS line, not cost per attempt.

Why should a founder pricing an agent product care?

Because a 6.8x swing in inference COGS is not a tuning detail, it is the difference between a viable per-seat plan and a plan that loses money on its heaviest users.

Take an illustrative case. Say your agent product runs 400 successful tasks per user per month. At roughly 207K tokens per successful task you are buying about 83M tokens per user per month. At roughly 1.42M you are buying about 568M. Whatever your blended input and output rate is, that is the same product with a COGS line that differs by a factor of nearly seven. No pricing page survives that quietly.

The practical lesson is that context strategy belongs in the cost model, not only in the engineering backlog. When you benchmark an agent architecture, log tokens per task alongside accuracy, then divide by the success rate before you compare anything.

How do you check this on your own numbers?

Three steps, in order:

1. 1Instrument tokens per task, split into input and output, and record the task outcome next to it.
2. 2Compute tokens per successful task, not per attempt. Failures are billed.
3. 3Convert to money at your provider's current per-million rates, then divide by your monthly active users to get COGS per user.

Step three is where most teams stall, because per-million rates move and multiply across providers. That is the arithmetic Calcaas exists to do: model token cost per request, per user and per month across current provider pricing, and see what the margin looks like before you commit to a price. You can run it at https://calcaas.com/llm-cost-calculator.

The takeaway: in agentic systems, what you send at inference is the bill, so measure delivery, not storage.