Most of pyiceberg's latency gap with the Rust Iceberg client sits in Python's HTTP library

Each client runs in its own new process ^[5]. The benchmark script is Python, and running pyiceberg inside it would give Python a head start ^[5]. The harness also refuses to save results from a debug build, a rule more benchmark posts could use ^[6]. Startup is timed separately. The clients take turns going first, so a machine that slows down slows both, and the medians are computed from saved timing files ^[5].

That care is needed because the numbers move. On a quiet 16-core Linux host ^[7], the Rust median for list_namespaces ranged from 301.3 to 768.1 microseconds between runs, a spread of 154.9% ^[8]. The author reports ranges for that reason. The main run is 720 samples: six operations, two clients, four rounds of 15 iterations ^[18].

The breakdown is the useful part. It times one list_namespaces request in stages on a single pyiceberg connection (raw HTTP, JSON parsing, Python object construction, the full client call), then times the same raw request from Rust ^[9]. Everything pyiceberg does after the HTTP response arrives adds 57.3 to 134.7 microseconds ^[12]. Working run by run, the author calculates that 85% to 90% of the difference between the two clients is the HTTP library ^[13]. At the fast ends of the two HTTP ranges, requests takes about 595 microseconds longer than reqwest per call. At the slow ends the difference is about 807 ^[1]. Only JSON parsing added time above run-to-run noise. The author says the split of the rest cannot be measured at this sample size ^[14].

Those ratios come from a catalog on the same machine, with no network time ^[1]. The author notes that under those conditions they are as large as the difference can get ^[15]. The HTTP penalty is a per-call cost of a few hundred microseconds ^[1]. A network round trip adds the same time to both clients and pulls the ratio toward 1. The table calls already show this: where the server does more work, Rust's lead drops to 1.14x to 1.38x ^[4]. For the 4.29x figure to carry over, the catalog has to sit next to the client and the workload has to be mostly small metadata reads. This account covers the local runs only. The BigLake and OneLake timings ^[1] and the startup figures are not included.

One config setting changes the Python number. Python's requests library re-reads proxy settings from the environment on every call unless trust_env=False is set. Turning it off saved 32.9 to 89.9 microseconds in each of five runs ^[16]. That is a small slice of a gap measured in hundreds of microseconds ^[1]. With the flag off, requests stops reading proxy settings from the environment ^[16]. A deployment that routes catalog traffic through a proxy set in environment variables would lose that routing.

The benchmark measures speed only and does not check that answers are correct ^[2]. The author wrote that "a client that lacks an operation you need is the wrong choice however fast it is" ^[17]. I think that is the right order of filters for any team that can deploy either language. The endpoint counts come from the two repositories and hold only for these versions ^[3]. Of the 25 endpoints tested, the Rust client lacks 12 and pyiceberg lacks 4 ^[2].