llama-server's default prompt cache mixes two LoRA scales into one answer

At step three, the only slot last served Q, also at scale 1. When P comes back at scale 1, the slot-level check compares two scale-1 requests and finds nothing to invalidate ^[3]. The RAM cache then restores the KV it saved for P in step one. That KV was computed at scale 0, and the entry does not record the scale ^[1][2]. For the first prompt the timings read cache_n=116 and prompt_n=1. So the cache supplied the entire prompt, and the model evaluated just a single token ^[5]. The 48 output tokens were then generated with scale 1 applied ^[11][19]. The author's explanation is that the prompt was read under scale 0 and the continuation under scale 1, and the output matched neither reference answer ^[6].

The unit the cache stores is struct server_prompt in tools/server/server-task.h. At b11514 it starts with the prompt's tokens and a list of prompt checkpoints ^[18]. A correct restore check needs the adapter list and scales stored in that entry, then compared against the incoming request ^[2][3]. Today the only comparison is against whatever the slot ran last ^[3].

Nothing in the response marks which tokens came from the cache. "cache_n tells you that something was reused, not under what," the author wrote ^[21]. The trigger is the same prompt prefix under two LoRA settings, with a request in between that takes the slot ^[22]. The author points out that a server switching adapters per request, with a shared system prompt and a persona or task per adapter, does exactly that all day ^[22]. With -np 4 the same ten-prompt sequence came out right, because each prompt kept its own slot ^[15]. I'd expect extra slots only to delay the bug. The restore path starts whenever another request takes a slot ^[1], and a busy server with more distinct prefixes than slots will keep doing that.

The test design is sound. Reference answers came from a separate server started with --cache-ram 0, where the two scales gave different answers on all ten prompts ^[12]. The setup was CPU-only in a Debian 13 container, on a 15M-parameter story model and its Shakespeare adapter. That is the pair llama.cpp's own server tests use for LoRA ^[10]. I think the 10 of 10 ^[4] transfers to larger models, because the faulty comparison is between request settings and never touches the weights ^[3]. What has to hold in production is the access pattern.

The control row was not perfectly clean. With step one also at scale 1, one of ten answers differed, the same prompt each time, and it matched on each of three solo re-runs ^[13]. The author attributes that to a floating-point difference from building the KV in a different order, and says the cause was not proven ^[13].

The August build b10703 failed on all ten prompts as well ^[8]. Upstream tracks the per-request case as ggml-org/llama.cpp#30129 and the global path as #26207 ^[16]. The author's check-lora-cache.sh tests a running server for the behaviour ^[17].