Skip to content

Build1 publisher3 min readPublished

Writing the edges in Python takes the routing decision out of the LLM call

A dev.to postmortem of the central orchestrator agent argues for LangGraph's explicit edges, and the 60 percent latency win it cites only adds up once the manager's own turns come off the critical path.

The Engineer · Build desk

Illustration accompanying Writing the edges in Python takes the routing decision out of the LLM call

What happened

  • A dev.to post lists three pressures that collapse the central orchestrator in production: the manager holds the full state of every task it routes, its failures are opaque, and it processes tasks sequentially.
  • The alternative it offers is LangGraph's StateGraph, where a node can be a function, an LLM call or a whole sub-agent, and conditional edges branch and loop on the current state.
  • A paper by AWS engineers describes Expansion-Contraction, which walks a domain graph outward spawning an ephemeral specialist per node, then aggregates findings inward to a verdict.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • decision Keeping fan-out in a manager prompt is a decision to leave the one call you most need to debug inside a model, where there is no trace to read afterwards.
  • cost With edges explicit, someone has to write the procedure down before the first run instead of leaving the model to infer it at request time.
  • contradiction The strongest adoption number offered for graphs is Lyft's development time, and a development-time figure does not measure the runtime failures the pattern is meant to fix.
  • capability If per-node context holds up, workflow size stops being bounded by one context window and the agent count can grow with the data graph.

Start with what a node is in the post's snippet. `agent_node` takes the state object, reads `state["query"]`, invokes the agent, and returns `{"answer": ...}` [8]. That dict is merged into the shared state, which every node reads from and writes to, so no single component holds the whole context [10]. The graph is assembled with `add_node("agent", agent_node)`, then `add_edge(START, "agent")` and `add_edge("agent", END)`, then `.compile()` [9]. Control flow is two lines you can read in a diff. Branches and loops come from conditional edges evaluated against state [7].

A team spent four months tuning their manager prompt, then found that letting two sub-agents run in parallel cut pipeline latency by 60 percent, a change the manager architecture could not express [6]. Run two steps of duration a and b in sequence and you pay a+b; run them together and you pay max(a,b), which is never below half of a+b. Two steps on their own therefore cap the saving at 50 percent [1]. The remainder has to come from somewhere, and the candidate the post names is the manager: sub-agents sit idle while it thinks [5].

Lyft's figures measure a different quantity. The post says Lyft rebuilt rider and driver support as a router-based multi-agent system on LangGraph, with safety checks, state management and handoffs in the graph flow [15], that agent development went from six months to a few weeks [16], and that non-technical domain experts now build and refine agents directly [17]. The same write-up credits the system with millions of rider and driver interactions [18]. That is authoring time, not misroute rate and not latency. For it to transfer you need domain experts who can state a procedure as nodes and edges, and a review path that treats a new edge like any other code change.

The pattern I would want the full paper for is Expansion-Contraction, from AWS engineers. Expansion walks a domain graph outward from the query origin and spawns an ephemeral specialist at each node, each one seeing only the data at that node and its neighbours; contraction aggregates findings inward to a verdict [11]. The topology comes from the data graph instead of a hand-designed plan [12]. The argument is that one agent reasoning over a large graph degrades as complexity rises, while per-node agents hold accuracy because each context stays small [13]. The post calls the results striking and does not include the figures [14].

The cost of the graph is that you write the topology down. Cortex Grid, presented at IEEE, models each agent as a node with directed edges and runs candidate and employer workflows in parallel inside one graph [19]. Eluna, a production warehouse operations system, encodes its standard operating procedures as directed acyclic graphs and hands independent tasks to parallel sub-agents [20]. Both start from a procedure someone already knows. A manager prompt is attractive when nobody has written the procedure down, and the price is a routing decision with no trace and no intermediate state to inspect [4].

The evidence here is one practitioner's account. "We keep building AI systems that look like 2010s microservice monoliths. And they fail the same way," the post says [2], dating the pattern to three years of watching teams quietly abandon multi-agent systems they could not scale [1]. I think the diagnosis holds on context and traceability, which are properties of where the decision runs. The 60 percent belongs to one pipeline until someone publishes the shape of it.

What to watch

  • Publication of the AWS Expansion-Contraction figures, which would show whether per-node context really holds accuracy as graph size grows.
  • Whether Lyft reports runtime numbers for its LangGraph router, such as latency or misroute rate, alongside the development-time figure.
  • Whether production systems use conditional edges to spawn agents dynamically, or settle on fixed DAGs like Eluna's encoded SOPs.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories