Gemini's agentic video mode fails silently on Vertex AI when any of four settings is missing

From the application side, a misconfigured agentic request looks the same as a working one. According to the author of linebot-helper-python, a LINE bot that runs on Cloud Run and calls Vertex AI ^[1], the API returns a 200 and the model still produces an answer ^[10]. No error says agentic mode failed to take effect, so the only symptom is spend ^[10].

The SDK is the easiest place to lose it. The `media_processing` field was added in google-genai 2.20.0 and is absent from 2.19.0. The author checked by downloading releases one at a time ^[11]. An older client discards `media_processing="AGENTIC"` as an unknown field, again with no error ^[11]. Tolerating unknown fields is sensible library behaviour. Here it means the setting that decides the size of the bill can disappear before the request leaves the process.

The author's answer was one test per condition, each asserting on the keyword arguments passed to the SDK ^[12]. The test printed in the post checks that `thinking_config` is set and that its `thinking_level` is LOW ^[12]. The author wrote that the tests exist because "if these four things are wrong, the program still runs, the answer still comes out, but the cost silently changes" ^[13]. "This kind of bug cannot be caught by manual review," the author added ^[14].

I think the tests sit at the right layer. The response offers nothing to assert on, so the outgoing request is the one artefact a unit test can inspect. Their limit is that they prove what the code sends and nothing about what Vertex did with it.

Google's figures, as the author relays them, are for long-form video ^[5]. Tokens fall by 88% while cost falls by 66%. Each token that remains therefore carries about 2.8 times the average cost of a static-mode token ^[1]. The author's thinking-level test suggests reasoning settings are part of what fills that gap ^[12].

For the token cut to carry over to another workload, the question has to point somewhere. Agentic mode reads the transcript first, estimates where the answer is, and pulls frames only for that stretch ^[4]. Static mode samples every frame and audio segment at a fixed rate, and two hours of video can run to hundreds of thousands of tokens ^[3]. I'd expect a whole-video summary to save far less, since every segment is relevant to it. The keynote question was the favourable case. The pricing answer sat between 1:24:40 and 1:26:10 ^[8], a 90-second stretch, about 1.25% of a two-hour runtime ^[2].

Vertex is the other environmental catch. Google's multi-turn examples are written for the Gemini Developer API, and Vertex behaves differently ^[15]. The docs say to put the returned tool_call and tool_response parts back into history so the next turn does not re-process the video ^[16]. The author's planned flow relied on that: a summary, an "Ask about this video" button, then follow-ups reusing the earlier context ^[17]. On Vertex, the returned parts held only a bare thought_signature ^[16].