Zenoh's put() returns before anyone can read it: a 3.9% stale-read rate in a tight loop

A developer writing up experiments with Zenoh through its Elixir bindings, Zenohex, ran 2000 iterations of put immediately followed by get on the same key, and about 78 of them, 3.9 percent in that run, read back the previous value ^[1][2][3]. That matters because the code under test was not doing pub/sub: it was using Zenoh's put/get as a small key-value store, which is exactly the usage that assumes a write is readable once the write call returns ^[1].

It is not data loss. According to the author, an extra get issued immediately afterwards almost always returns the correct value, and the fastest confirmation measured was a single additional get about 1 millisecond later ^[4]. The value arrives; it just is not visible yet when the caller is already asking for it.

The mechanism is visible in the binding. Zenohex.Session.put/4 is a thin Rustler wrapper over zenoh-rust's put, and the NIF's .wait() only waits for the local session to finish queueing the message, not for the zenohd router backing the storage to receive and apply it ^[5]. The read side is different in kind: session_get is registered as a DirtyIo NIF and genuinely blocks for a reply from the remote side within a timeout, a real request/response ^[6]. The author's framing is the one to keep: put behaves like a GenServer cast, get behaves like a call, and firing a cast then immediately making a call that depends on it is the classic shape of this race ^[7].

This is not a binding defect being blamed upstream. Eclipse Zenoh has an open design issue, #2511, titled "[Design] Acknowledged put: confirmed storage writes via query path vs protocol extension", still open as of the writeup ^[8]. The issue states the position plainly: the pub/sub path is fire-and-forget, and session.put() returns when the message is sent, not when it is stored ^[9].

Do the arithmetic on what a few percent means in an operational loop. At 3.9 percent, roughly one write in 26 is unreadable at the moment the next line of code looks for it ^[13]. In a service that writes state and then reads it back to make a decision, that is not an edge case you will find in a smoke test and will find in production logs, if you are comparing values at all. The failure is silent: the get succeeds, the reply is well-formed, the payload is one generation old.

The author's workaround is the honest one available today: a wrapper that puts, then polls get on the same key until the written payload reads back, retrying at a short interval until a deadline, returning :ok on confirmation and {:error, :not_confirmed} if it never lands ^[10]. Defaults are a 3000 ms confirm timeout, a 1 ms confirm interval, and a 3000 ms query timeout ^[11]. The price is at least one extra round trip per write, plus a poll loop on the unlucky percent ^[15].

Two things to watch. Whether #2511 lands as a protocol-level acknowledgement or as a blessed query-path confirmation decides whether every Zenoh client library needs its own version of this wrapper ^[8]. And measure your own rate before trusting 3.9 percent: that number is one run on one setup as reported in the post ^[3], and the post is an AI translation of the author's original Japanese article on Qiita ^[12].