Build1 publisher3 min readPublished
Kiro and Claude Code both picked a TGI container that could not load Qwen3
AWS says both coding agents it tested defaulted to Text Generation Inference and billed GPU time for each crashed deploy before pivoting to vLLM. Its answer is six editable skill files the agent reads on demand.
The Engineer · Build desk
What happened
- AWS tested Kiro and Claude Code on a deployment request, and both picked Text Generation Inference; the TGI build available in the Region predated Qwen3's architecture, so the endpoint failed its health check.
- Asked for a multimodal mixture-of-experts diffusion model released only weeks earlier, the agents again wrote a TGI script, a text-generation server with no backend for a discrete-diffusion image-text model.
- AWS attributes both runs to missing deployment facts and says the agent planned and debugged well but lacked current, specific knowledge.
- The published fix is six skills from the Hugging Face Skills GitHub repo covering the end-to-end workflow, with a planner skill orchestrating the other five.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost A GPU endpoint bills from start-up, so an agent that works its way through serving containers charges the team for every start-up before anything answers a request.
- constraint The freshness now lives in text files that somebody has to own: when the next Qwen lands or the wheels catch up with Python 3.13, the skill files have to be edited.
- decision A team pointing an agent at SageMaker chooses between installing someone else's deployment facts and keeping a catalogue of its own.
- contradiction AWS warns of endpoints that are quietly wrong, yet both documented runs ended in deploys that never came up; neither produced an endpoint serving bad answers.
A skill in the standard AWS is using is a folder with a SKILL.md file in it. The file carries metadata, at minimum a name and a description, then the instructions for the task [18]. Loading is by progressive disclosure: the agent opens the skill when the current task matches its description [19]. The description string is therefore the routing key.
The resolution rule is worth keeping. AWS's guidance is that container images should be resolved from the published Deep Learning Containers catalog [12]. The lookup happens at deployment time, so it does not go out of date between edits to the file. The other two facts AWS cites hold only until the ecosystem moves: recent Qwen models need vLLM, and Python 3.13 has no working wheels for much of the machine learning stack [11]. AWS says this kind of knowledge changes faster than model weights get updated, which is why it went into editable files [13].
The cost in the first test run is in the sequence. TGI deploy, failed health check, version bump, redeploy, failed again, pivot to vLLM [6][7]. Each attempt billed GPU time as the endpoint started and then crashed [7]. Two TGI start-ups were charged before the agent reached a server that could load the weights [22]. The post does not give the instance type or a dollar figure.
AWS's own framing is that an unguided agent yields endpoints that are fragile, costly, or quietly wrong [3]. The first two are in the record. The second test is offered as the quiet one, and the account of it ends with the endpoint refusing to come up [8][9]. So that test also ends in a failed deploy. Neither run produced an endpoint answering production requests with wrong output.
Table 1 sets the unguided agent's deployment against the same request with the skills installed [20]. It is a pair of requests, run against Kiro and Claude Code [4]. For that comparison to transfer to your stack, two things have to be true. The model has to be new enough that the tutorials in the training data point at the wrong server [5]. And your Region's container catalog has to be behind the model's architecture, the way AWS found the Region's TGI build behind Qwen3 [6]. If you are deploying something that has been served by the same container for a year, the unguided agent probably gets there on the first attempt.
The install is six skills from the Hugging Face Skills repository, with a planner skill orchestrating the other five [14]. They are open source, use only Python and the AWS CLI, and run unchanged on macOS, Linux and Windows [17]. A real-time endpoint is the default, with scale-to-zero, serverless inference, asynchronous inference, batch transform and Amazon Bedrock Custom Model Import as the other paths [16]. What comes back includes autoscaling, CloudWatch alarms, the container from the DLC catalog and a verified teardown path [15].
What to watch
- The first stale SKILL.md: a Qwen release that needs something other than vLLM would price the maintenance the editable-files approach requires.
- Whether AWS publishes the contents of Table 1 with instance types and billed hours, which would let the cost side of the comparison be checked.
- Whether description matching misroutes on ambiguous requests, leaving the deployment skills unloaded on the runs that need them most.