Skip to content

Product1 publisher3 min readPublished

Three Postgres CDC tools ran on YugabyteDB after small fixes, all from one assumption

Yugabyte says Sequin, PeerDB and Supabase ETL each needed only minor changes, and nearly all of them trace to log positions that are not global coordinates.

The Product Desk · Product desk

Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction

Illustration accompanying Three Postgres CDC tools ran on YugabyteDB after small fixes, all from one assumption
Generated illustration

What happened

  • Yugabyte took three popular open-source PostgreSQL CDC tools - Sequin, PeerDB and Supabase ETL - connected them to YugabyteDB, ran them under real-life conditions, and published the findings.
  • All three tools worked with YugabyteDB; each needed a small, easy-to-execute set of changes, and every one of those changes came back to the same idea.
  • Yugabyte tested all three CDC tools within a few weeks, and most of that time was dedicated to reading code and writing tests rather than wrestling with the database.
  • YugabyteDB uses PostgreSQL's own logical replication, including replication slots that track where a consumer is in the stream, publications that specify which tables and changes matter, and the usual settings for how much detail each change carries.
  • For all three tools, the everyday streaming path worked without any changes to the tools themselves.

Compiled by The Product DeskSomething wrong?How this is made

Why it matters

Yugabyte pointed three popular open-source PostgreSQL change data capture tools, Sequin, PeerDB and Supabase ETL, at YugabyteDB and reported that all three worked, each after a small and easy-to-execute set of changes [1][2]. The interesting part for anyone outgrowing a single Postgres box is where the work landed: not in migrating the database, but in a short audit of assumptions inside the connector [9].

The reason the tools run at all is that YugabyteDB uses PostgreSQL's own logical replication, with the familiar replication slots, publications and per-change detail settings [4]. According to Yugabyte, the everyday live streaming path came up for all three tools with no changes to the tools themselves [5]. Yugabyte says the whole exercise took a few weeks, most of it spent reading code and writing tests rather than fighting the database [3].

The one concept that generates the fixes is the Virtual WAL. A single-node PostgreSQL server keeps one write-ahead log, so any two positions in it can be lined up and compared, and plenty of tools quietly assume exactly that [6]. YugabyteDB spreads a table across many pieces, each with its own log, then merges them back into commit order and hands the connector something that looks like an ordinary log [7]. The catch is that a position in one YugabyteDB stream only means something inside that stream; Yugabyte's logical replication documentation states these are not global coordinates and cannot be compared across streams [8].

Almost every change a tool needed comes from that [9]. A few PostgreSQL helper functions assume the single global log and do not carry over, and any logic that compares positions across two separate streams has to be rethought [9]. Where a tool genuinely needs a marker comparable across the whole database, YugabyteDB provides one [10]. A couple of further differences follow from YugabyteDB storing data in its own distributed layout rather than the classic single-machine one; per Yugabyte, those touched only one of the three tools and the fixes were minor [11]. Yugabyte points readers at its key concepts and limitations pages before starting [15].

Sequin, an open-source CDC platform that streams Postgres changes to destinations such as Kafka [12], is the case broken out in detail: with standard setup, both live streaming and the initial backfill came up without changes to Sequin's core logic, and the changes it did require applied to the Virtual WAL and to one or two older PostgreSQL features YugabyteDB does not support [13][14].

Two things to hold in mind. This is the database vendor testing compatibility with its own product, and the material available stops partway through the Sequin section, before comparable per-tool detail for PeerDB or Supabase ETL [17]. And "worked with small changes" is not the same as "worked": each of the three needed patches, even though the streaming path itself needed none [16]. Anyone costing a migration should price the fork, the ongoing rebase against upstream, and the test suite that proves position handling behaves.

What to watch: whether the per-tool change sets are published in full and land upstream in Sequin, PeerDB and Supabase ETL rather than living in Yugabyte's own branches [14][17], and whether the older unsupported PostgreSQL features named in the Sequin work turn out to matter to other connectors [14].

Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories