Skip to content

Build1 publisher3 min readPublished

A changelog restore records its progress only in a local checkpoint file

A cluster console shows a stable consumer group while an instance replays 7.5 million changelog records, so Alginte's 0.11.0 view reads the Kafka Streams client inside the process the console itself runs.

The Engineer · Build desk

Illustration accompanying A changelog restore records its progress only in a local checkpoint file

What happened

  • A Kafka console reads the cluster: topics, partitions, consumer groups, committed offsets and lag, and from that side a Streams application is a consumer group with a few internal topics.
  • The consumer that replays a changelog does not join a group and does not commit an offset. It keeps its position in a local checkpoint file on the instance.
  • For an application the console does not run, the page shows group state, offsets, lag and changelog topics, and marks lifecycle, restore progress and store sizes as unseen.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • decision Getting restore progress out of a Streams process means deciding whether a monitoring tool may host the application. That is a platform and deployment call.
  • cost The same figures are already exposed to the application by the Kafka Streams API, so a team that keeps its own scheduler pays in exporter and dashboard engineering instead.
  • exposure On-call rotations alerting on consumer group lag alone will page for a throughput or broker problem while an instance is mid-restore and cannot process anything yet.
  • precedent Printing a dash where the process is unreachable is a rule other Kafka monitors can be held to, since a zero in the gauge would claim a measurement was taken.

The missing term is what makes restore progress uncomputable from outside. A read-only observer can read how far each changelog topic reaches [12]. The instance's own position in that changelog is in the local checkpoint file, so records remaining is a subtraction with one term unpublished [17]. Meanwhile input keeps arriving, and group lag climbs for a reason the cluster has no way to report [5]. Alginte's example of the case is a stream replaying 7.5 million changelog records after its state directory was wiped, beside a cluster view of a stable group [11].

The state relabelling is the piece I would copy into anything that reads a Streams client. Both cases the console rewrites are ones where the client enum is accurate about the client and wrong about whether records are moving: REBALANCING with every thread dead, and RUNNING with assigned tasks served by nobody [8]. The per-partition figure works the same way. One column per store partition, interpreted by role, showing records left to replay during a restore, distance behind the active for a standby, and plain changelog lag at steady state [9].

Threads, rebalance counts with their durations and an event timeline sit on the same tab, and the restore banner appears only while a store is actually replaying [7]. Next to all of it the console puts the cluster's own facts about the same stream, group state, committed offsets and changelog topics, so the two accounts can be checked against each other [10].

None of that depth comes from a smarter query. It exists because the console runs the stream, holds the Kafka Streams client and reads it in process, with no agent, no scraping and nothing written to the cluster [6]. Alginte calls the cluster-only view "the depth the cluster allows" and says it is "where the idea points next" [16]. The post does not state pricing or licence terms [18].

One display rule is worth holding other monitors to: every figure the observer cannot see is a dash, never a zero [13]. The post gives its reason for refusing to display a lifecycle it cannot verify, which is that "a group reads stable while threads die and we would rather show absence than invent one" [14].

The design-time half of the product is a separate argument. The modeler draws the topology node by node. Each operator carries the expression it applies, the editor shows a record that is really in the source topic beside what the expression makes of it, and the derived shape at each node is marked inferred [15].

All of the above is Alginte describing its own console, and the run-time behaviour is single-sourced.

What to watch

  • Whether Alginte extends in-process figures to Streams applications it does not run, which the post lists as the next step.
  • Whether the standby figure, described as distance behind the active, is defined in offsets or in time.
  • Whether the derived ERROR and DEGRADED states match real incidents where threads die during a rebalance.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories