Build1 publisher3 min readPublished
Iceberg REST portability holds until the MCP server tries to read a data file
A read-only MCP server with four Iceberg tools ran against six live catalogs off a single environment variable. Three of the tools worked on pyiceberg alone, and the fourth needed a storage package for the object store.
The Engineer · Build desk
What happened
- One read-only MCP server with four Iceberg tools was pointed at seven Iceberg REST catalogs in turn, with a single environment variable and one catalogs.yaml entry per catalog choosing the target.
- Six of the seven catalogs were actually exercised on 2026-09-18 and 2026-09-19 UTC, with Databricks Unity left out because its trial account had ended.
- Listing, describing and counting worked on pyiceberg alone, while the scan tool failed on three catalogs with ModuleNotFoundError for adlfs or s3fs, packages pyiceberg does not install by default.
- S3 Tables issued a storage credential only when the client asked with the X-Iceberg-Access-Delegation: vended-credentials header, and Snowflake Horizon returned one without being asked.
- AWS logins made with the newer aws login command failed on the first call with a MissingDependencyException until one more package was installed.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost Every managed catalog charges the platform team an account with a seeded table and a working CLI login before the first read, so the integration bill scales with the number of catalogs and not with the size of the server.
- constraint An agent that only lists, describes and counts stays portable on pyiceberg alone; the moment it needs actual rows it inherits a storage driver dependency for each object store behind each catalog.
- decision Anyone packaging this server has to choose between shipping every object-store extra in the image and accepting that scans fail on some catalogs while the other three tools keep answering.
- capability Because failures come back as tool text, a client can tell a catalog it could not reach from a table that is genuinely empty, without losing the session.
The line between the three tools that travel and the one that does not falls where the REST protocol stops. Listing, describing and counting read only catalog metadata, so pyiceberg 0.12.0 answers them on its own [15][6]. iceberg_count_rows never opens a data file; it takes the exact count out of the snapshot summary, which is metadata [3]. Scanning opens the data files, and that needs a filesystem package for the object store the table lives in [15].
The per-catalog work lives in the config. catalogs.yaml holds one entry per catalog with the URL, the warehouse and how to log in, and the same server code runs against every one of them [9]. From that entry the server builds a PyIceberg RestCatalog [17]. Reaching a managed catalog also takes an account with a table in it and a working command-line login: gcloud, az, aws, or a Snowflake key pair [6].
Two of the catalogs disagree about credential vending under a protocol they both speak [21]. S3 Tables hands over a storage credential when the client asks for delegation in a header; Snowflake Horizon hands one over unasked [14]. A client written against Horizon has no reason to send that header, and against S3 Tables it then gets no vended credential [23].
The two failure modes land at different points in the call sequence. A missing filesystem package breaks only the scan, after three tools have already answered [15]. The missing package behind the newer aws login path breaks the first call [16]. Either way the server returns the error as tool text beginning CATALOG ERROR, so the agent can report what it could not read, and the sweep script counts that text as a failure [13].
The timings measure a local catalog. Polaris 1.7.0 ran in Docker on the same machine as the server [4], and its four calls returned in 1.0, 0.4, 0.0 and 0.1 seconds, 1.5 seconds for the set [11][20]. The table held 11 rows, partitioned by day, with four snapshots [7], and the scan printed 3 of those 11 rows plus an exact count from snapshot 1196292829914853564 [12]. iceberg_list_tables enumerates every namespace.table in the catalog [2], so the one-second figure transfers only to a catalog with about one table in it. The run was one call per tool per catalog, reads only, with no model in the loop [5].
On this evidence the interchangeable part is the metadata plane. Databricks Unity never got a run because its trial account expired [4], and a subscription clock is not something the REST spec can standardise. A team adding a catalog is budgeting for a login entry in the YAML file and a storage package if it wants row scans; the server code stays as it is [9][15].
What to watch
- A Databricks Unity run on a paid account would show whether the three-of-four tool split holds there too.
- Whether a pyiceberg release after 0.12.0 pulls the object-store filesystem packages in by default.
- Whether list_tables stays near one second against a catalog holding thousands of namespace.table pairs.