Build1 publisher3 min readPublished
S&P Global Energy bundles expert-curated Genie agents into MCP endpoints with a FastMCP proxy
S&P Global Energy says per-dataset Genie agents, bundled by a FastMCP proxy, cut launch time for conversational data products from months to days. Its governance claim rests on Unity Catalog, though part of the estate sits in sources outside Databricks.
The Engineer · Build desk
What happened
- S&P Global Energy's domain experts pick and group tables, then curate one Genie agent per dataset group instead of one large agent per commodity.
- The Genie agents run as managed MCP servers, and a FastMCP proxy composes them into composite endpoints intended for cross-domain queries.
- The target is the company's whole structured data estate, open through MCP to customers' agents as well as S&P's own, answering natural-language questions with governed results.
- Priyanka John, a vice president at S&P Global Energy, said work that once took a full development cycle now takes days, with every answer kept inside the governance boundary.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Central governance extends only as far as Unity Catalog does, so an estate with non-Databricks sources has a second integration problem to solve before the pattern applies to all of it.
- exposure If the proxy only aggregates tools, cross-domain answers get combined by the customer's own agent in the client environment, after each result has left the governed Genie layer.
- cost The days-long launch time depends on analysts owning curation; an organisation without experts who know its tables keeps paying the engineering-backlog cost S&P describes.
Ownership splits three ways. Subject-matter experts select the tables for a business domain, group the related ones, and curate a Genie agent over each group without writing agent code [17][1]. Engineering standardizes how agents connect, and governance stays central [11]. S&P's post calls the curation step "where the magic starts" and adds that it "requires no code" [18]. As described, the magic is an analyst who knows the data choosing tables.
Granularity is the first design decision anyone copying this faces. S&P builds one Genie agent per dataset group and says it does not build one giant agent per commodity [13]. Chemicals gets group-level agents for capacity, production, capacity utilization, trade, demand by end use and by derivative, inventory change, and country- and region-level supply-demand balances [14]. That puts at least seven agents on one commodity [15], and the post says Crude Oil, Refined Products and Gas & Power follow the same pattern [14]. I think the narrow scope is the right call. An agent that sees a handful of related tables has fewer wrong tables to reach for when it writes a query. The cost lands on the composition layer, where the FastMCP proxy assembles those agents into composite endpoints [2]. I'd expect each added agent to be one more tool a calling model has to pick correctly.
The proxy exists for questions that cross those groups. Priyanka John, a vice president at S&P Global Energy, said the larger business value comes partly from "richer cross-commodity analytics, and the ability to connect insights that traditionally exist in separate business lines" [16]. Suppose the composite endpoint presents several Genie agents' tools behind one address. Each agent then answers inside its own table group, and the answers are joined by whatever agent made the calls. For external customers, that agent runs in their own environment. S&P's architecture diagram draws the line between its network and the client's [12][7].
The governance claim holds for part of the estate. S&P credits the design with preserving Unity Catalog governance [3], and John said "every answer stays inside our governance boundary" [10]. The same post says the data lives across Databricks and several non-Databricks sources [6]. It does not say how those other sources reach a Genie agent.
The speed figures come from S&P alone. Before Genie, S&P says every insight passed through an engineering backlog and time to market was measured in months [9]. John said "What used to take a full development cycle now takes days" [10]. S&P also says Genie agents beat the other approaches it evaluated "by a wide margin" [8]. Both claims describe S&P's estate and S&P's analysts. For the days figure to transfer, a team needs experts who already know which tables answer which questions. It also needs tables Genie can already read under Unity Catalog. A team missing the first puts curation back in the engineering queue. A team missing the second has a data migration to do before any agent gets built.
What to watch
- A technical write-up from S&P or Databricks showing how the FastMCP proxy handles a question spanning two commodities, and whether it routes calls or combines results itself.
- How S&P brings its non-Databricks sources under Unity Catalog, whether by federation or by copying them into Databricks.
- How external customers authenticate through the composite endpoints, and whether entitlements are enforced per customer at the Genie agent.