Retries and re-indexes: where LLM cost control stops working

The line item worth pricing first is not the failed call. Under the prompt-based route, the format instructions and the JSON schema are generated and appended to the prompt ^[5], which means they are billed on every call that works as well as on the ones that do not. Sending the schema to the provider as an API-level rule takes that block out of the input ^[7], and the input token count falls on every call ^[8]. That saving tracks request volume. The retry saving tracks defect rate, and the series' own example puts a 5% one-retry rate at roughly 5% extra input tokens with nothing to show for them ^[3]. On a busy endpoint with a low failure rate, the dull always-on saving is the larger of the two.

The failure path is expensive because of what it resends. A retry ships the system prompt, the conversation history, the RAG context and the tool schemas again, and the answer that could not be parsed is billed as output in full whether or not it was cut off ^[2]. With validation switched on, Spring AI appends the validation error to the user message and calls the model again, up to three attempts by default ^[11], so a single usable answer can reach four model calls ^[19] and up to four times the cost ^[4].

Prevention does not cover every method signature. Spring AI 2.0 supports provider-native structured output for OpenAI, Anthropic, Google GenAI, Mistral AI and, depending on the model, Ollama ^[9], but OpenAI rejects top-level array schemas, so asking for a List<T> fails, and Ollama models in reasoning mode can still return plain text ^[10]. Those endpoints keep the retry mechanism, which is why the series treats validation as a recovery step rather than the primary control ^[12].

The embedding side inverts the problem. At OpenAI's August 2026 list price, text-embedding-3-small is $0.02 per million input tokens ^[15], one fiftieth of the $1 per million input rate the series uses as its example rate elsewhere ^[16][18]. Per token, that is not a number anyone needs to manage. Per pass it is $20 for every billion tokens of corpus ^[17], and the pipeline pays it again whenever documents are added or updated, or a full re-index is required ^[13]. Dimensionality is the driver that keeps charging after the indexing run finishes, through storage and retrieval ^[14].

One gap to flag: the series says a fourth source of embedding calls has nothing to do with documents at all, and the text supplied to us stops in the pricing section before it is named ^[14][20].