Build 1 publisher2 min readPublished September 13, 2026 at 8:21 PM PDT

Agent-cache's tool cache returns the first ticket's ID when the tool writes instead of reads

The library puts LLM completions, tool results and session state behind one Valkey or Redis connection. Its tool keys are a function name plus an argument hash, and the post says that breaks for tools that mutate state.

The Engineer · Build desk

Illustration accompanying Agent-cache's tool cache returns the first ticket's ID when the tool writes instead of reads

What happened

Agent-cache backs three cache tiers with Valkey or Redis, putting LLM responses, tool results and session snapshots behind a single connection.
Version 0.1.0 shipped with Valkey 7+ and Redis 6.2+ support, and v0.2.0 added cluster mode the following day.
The library ships adapters for LangChain, LangGraph and the Vercel AI SDK, with OpenTelemetry and Prometheus instrumentation wired in at the cache layer.
Invalidation is manual: the library tracks no dependencies between entries, and cache.invalidate(pattern) takes Redis glob patterns the caller has to identify.
If Valkey or Redis goes away, the default is graceful degradation straight to the LLM or tool, and the behaviour is configurable per tier.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

constraint A team still tuning prompts gets the least from tier 1, because every system-prompt edit and every temperature change resets the hit rate to zero until traffic reaccumulates.
exposure Any tool that writes becomes a correctness risk the moment it is cached, since the second caller gets the first caller's record back instead of a new one.
contradiction The post's cost case and its differentiation case point at different tiers. The two layers it adds beyond LangChain are left arguing on latency and resumability.
cost Adoption presumes an existing Valkey or Redis deployment, so the price is another shared dependency in the agent's request path, paid by whoever carries the pager for it.

Under 1ms is the read time on an exact-match hit, measured against a call that would otherwise go to the API ^[5]. It describes the hit path. Whether it transfers to your agent depends on the hit rate, and the post does not report one, nor a measured token saving ^[18]. The LLM key is a hash of the prompt plus the model name, the temperature and the top-p ^[7]. Change any of those and every entry behind it is unreachable ^[7]. Exact-match caching rewards a frozen prompt, which is an odd thing to ask of a system you are still tuning.

The tool tier keys on the function name and a hash of the arguments object ^[8]. For a database lookup or a stable read API, the key is right ^[8]. For a write it is a defect: the post's own example is create_ticket(title, description), where a cached result comes back with the old ticket ID instead of creating a new ticket ^[8].

Each tier gets its own TTL strategy: LLM responses for hours when the model and prompt are stable, tool outputs sooner because the upstream data changes, session state only while the user session is active ^[10]. OpenTelemetry spans track hits, misses and errors, and Prometheus exposes hit rate, latency and connection pool health at the cache layer ^[13].

The LangGraph adapter is the piece I would adopt first. It implements BaseCheckpointSaver and persists checkpoints to Valkey instead of requiring Redis 8 with modules ^[14]. Checkpoints serialize as MessagePack to save space, while the LangChain path uses JSON for LLM responses ^[15].

The post positions tool and session caching as the layers other libraries skip, since LangChain caches completions and LangGraph persists checkpoints ^[4]. It also calls the LLM tier the biggest token saver when agents retry or loop over similar prompts ^[6]. Those two statements point in different directions. The token case sits in the tier LangChain already covers, so the two added tiers rest on latency and on checkpoints, user intent and execution state that can resume mid-flow ^[17]^[9].

The Vercel AI SDK adapter hooks streamText and generateText, and the post lists streaming support as roadmap, not shipped ^[14].

What to watch

Whether tool entries gain a declared read/write flag so a mutating call cannot be cached by default.
A published hit rate from a production agent loop. That number would decide whether tiers 2 and 3 pay for themselves.
Whether LangGraph shops already running Redis 8 with modules move their checkpoints onto Valkey.

Claim ledger

Ranked by verification strength, evidence, and original report placement.

[5]
Tier 1 is an exact-match cache keyed on prompt text and model parameters; if an agent calls gpt-4o twice with identical input, the second call returns from Valkey in under 1ms instead of hitting the API.
ReportedSupported2 sources— sign in to open View cited source
[8]
Tool output keys include the function name and a hash of the arguments object, which works for deterministic tools such as database lookups and stable APIs but breaks when tools mutate external state; for create_ticket(title, description), caching the result means subsequent calls with the same arguments return the old ticket ID instead of creating a new one.
ReportedSupported2 sources— sign in to open View cited source
[12]
When Valkey or Redis is unavailable the options are fail fast, degrade gracefully by skipping the cache, or use an in-memory LRU fallback for the current request; agent-cache defaults to graceful degradation and lets behaviour be configured per tier.
ReportedSupported2 sources— sign in to open View cited source
[1]
Agent-cache is a three-tier caching architecture backed by Valkey or Redis that puts LLM responses, tool results and session snapshots behind a single connection.
ReportedSupportedView cited source
[2]
The project shipped v0.1.0 with Valkey 7+ and Redis 6.2+ support, then v0.2.0 with cluster mode the next day.
ReportedSupportedView cited source
[3]
Agent-cache includes framework adapters for LangChain, LangGraph and the Vercel AI SDK, plus OpenTelemetry and Prometheus instrumentation at the cache layer.
ReportedSupportedView cited source
[4]
According to the post, most agent caching solutions pick one layer and stop: LangChain caches LLM completions and LangGraph persists checkpoints.
ReportedSupportedView cited source
[7]
The LLM cache key combines prompt hash, model name, temperature and top-p, so changing any parameter busts the cache; tweaking the system prompt or adjusting temperature sends the agent back to the API.
ReportedSupportedView cited source
[9]
Tier 3 stores agent checkpoints, user intent and execution state with per-field TTL, including LangGraph checkpoints and any custom state an orchestrator needs to resume mid-flow.
ReportedSupportedView cited source
[10]
Each tier uses a different TTL strategy: LLM responses may cache for hours if model and prompt are stable, tool outputs expire faster when external data changes frequently, and session state persists only as long as the user session is active.
ReportedSupportedView cited source
[11]
Invalidation is manual: agent-cache does not track dependencies between cache entries, and cache.invalidate(pattern) accepts Redis glob patterns but requires the caller to know which keys to target.
ReportedSupportedView cited source
[13]
OpenTelemetry spans track cache hits, misses and errors, and Prometheus metrics expose hit rate, latency and connection pool health.
ReportedSupportedView cited source
[14]
The LangChain adapter wraps the BaseLLMCache interface; the LangGraph adapter implements BaseCheckpointSaver so checkpoints persist to Valkey instead of requiring Redis 8 with modules; the Vercel AI SDK adapter hooks streamText and generateText, with streaming support on the roadmap but not yet shipped.
ReportedSupportedView cited source
[15]
LangChain uses JSON for LLM responses, LangGraph uses MessagePack for checkpoints to save space, and tool outputs serialize as JSON by default but accept custom serializers.
ReportedSupportedView cited source
[16]
Agent-cache assumes you already run Valkey or Redis in production.
ReportedSupportedView cited source
[17]
The tier the post names as the biggest token saver is the same layer it says LangChain already caches, so the token argument does not sit with the two tiers agent-cache adds.
DerivedSupported1 source— sign in to open
[18]
The post reports no cache hit rate and no measured token saving for any tier.
DerivedSupported1 source— sign in to open
[6]
The post says the LLM response cache is the biggest token saver when agents retry or loop over similar prompts.
ReportedInsufficient2 sources— sign in to open View cited source

Sources

Publishers with included, body-backed reporting in this cluster.

dev.to
1 article · September 13, 2026
Agent-Cache: Multi-Tier LLM Caching for Valkey and Redis

Ask Clarity

Have a question this piece left open? The answer comes from its reporting and claim ledger — nothing else.

Try

Clarity analysis

How the coverage leans and what the record actually supports. Read the methodology

Perspective Coverage

How much of the coverage speaks to builders, operators and investors. It measures emphasis, not whether the story is right.

Everything a builder needs to judge this sits in the dev.to post and nowhere else, and the interfaces are named precisely enough to check against the code: the LangChain adapter wraps BaseLLMCache, the LangGraph adapter implements BaseCheckpointSaver so checkpoints go to Valkey without Redis 8 modules, and the Vercel AI SDK adapter hooks streamText and generateText. Streaming is on the roadmap, so a streaming app gets nothing from the third adapter yet. Two constraints matter more than the tier count. Tool keys are the function name plus a hash of the arguments, so caching a write returns the first ticket's ID, and the author says so himself. Invalidation is manual, with no dependency tracking, so cache.invalidate(pattern) only helps a caller who already knows which keys went stale.

Reality

Evidence, adoption, hype gap, incentives and confidence, each read independently from the current record on the scale shown beneath it. How these are measured.

Evidence34

0100

One post, written by the author

Every detail traces to a single dev.to write-up by the person who wrote the library. The specifics are checkable in principle, since the post names interfaces, versions and an npm package, but nobody has checked them and the code sample cuts off mid-configuration. What raises this above a launch note is the author's own accounting of the failure cases: tool keys break on writes, invalidation is manual, streaming is unshipped.

Adoption10

0100

Two releases in two days

The record is two version numbers a day apart, both disclosed by the author, with no downloads, deployment or third-party confirmation that anyone runs it.

Hype gap+22

Topics and entities

Follow any of these and your For You feed starts watching them — no settings page required.

Topics

Entities

build
Three services you can delete: queue, cache and search in one Postgres
1 publisher
build
Edge KV puts the permission check an hour behind the Postgres row
1 publisher
build
CAPTCHA walls in Playwright agent pipelines are usually self-inflicted retry bursts
1 publisher
build
Agent memory products differ on one thing: whether anything decides a fact is dead
1 publisher

What happened

Why it matters

What to watch

Claim ledger

Sources

Let Clarity write the post for you.

Topics and entities

Topics

Entities

Related stories

Claim ledger

Let Clarity write the post for you.

Topics and entities

Topics

Entities

Perspective Coverage

Reality

Related stories

Sources