Skip to content

Build1 publisher3 min readPublished

A six-second NocoBase bulk update overwrote a row that had stopped matching the filter

The filter appears once, in the SELECT that collects primary keys, and the writes then go out one statement per row keyed on id. Anything that changes a row in between is overwritten, and the window grows with the row count.

The Engineer · Build desk

Illustration accompanying A six-second NocoBase bulk update overwrote a row that had stopped matching the filter

What happened

  • Staff answered the thread to say they were investigating, and it has stayed open since.
  • Reading repository.ts at the v2.2.7 tag confirms both code paths use the filter only in the SELECT that collects primary keys, and neither carries it into the UPDATE.
  • A PostgreSQL log captured at log_statement=all shows a two-row filtered update emitting one SELECT with the condition, then one UPDATE per row keyed on id, then COMMIT.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • exposure Status columns used as state machines are the exposed case: if a user or a second job moves a row out of the filtered set mid-run, the row still receives the write.
  • constraint Tests on development-sized tables cannot fail on this, because the run ends before anything can interleave; the 2,002-row attempt was clean for that reason alone.
  • decision A team that needs conditional atomicity has to add its own lock or write the single statement below the ORM, because the REST layer only exposes the per-row path.
  • cost While the thread stays open, the guard is paid for application-side, once per caller, on every filtered update a team ships.

The decorators on `update` say what the method guarantees. At the `v2.2.7` tag, in `packages/core/database/src/repository.ts`, the method carries `@transaction()`, `@mustHaveFilter()` and `@injectTargetCollection` [5]. `mustHaveFilter` guarantees that you supply a condition. `@transaction()` puts the find and the writes inside one transaction [6]. The condition is used to select primary keys and does not appear in the UPDATE [10].

Inside, the method forks. With `individualHooks: false` it selects only the primary key field, collects the ids, and issues one `Model.update` keyed on that list [7]. The default path selects full instances and loops, calling `updateModelByValues` once per row [8]. `individualHooks` is not reachable from the REST API, so an HTTP caller always gets the loop [9]. In Django you pick between the two: `QuerySet.update()` sends one statement and skips `save()` along with `pre_save` and `post_save`, while a loop of `Model.save()` fires the signals and costs one statement per row [15]. NocoBase's filtered bulk update is always the second shape [16].

The log settles it. A `POST /api/orders:update?filter={"status":"pending"}` with two matching rows emitted one SELECT carrying `WHERE "orders"."status" = 'pending' ORDER BY "orders"."id" ASC NULLS LAST`, then two `UPDATE ... SET "note"=$1 WHERE "id" = $2`, then COMMIT, with nothing between the writes re-checking the status [11]. Capture was done with `log_statement=all` against PostgreSQL 16 and the official 2.2.7 Docker image [4].

Row count decides whether you can see it. On 10,001 rows matching `status = 'pending'`, the request took 6.2 to 6.4 seconds [12], which is about 1,590 writes a second [20]. The author flipped the highest id, the last row the SELECT ordering reaches, to `cancelled` one second into the run, and it came out updated in five runs out of five [13]. Roughly 5.3 seconds separated that change from the statement that clobbered it [23].

The first attempt used 2,002 rows and never reproduced, because the run finished in 1.5 seconds and the interruption was scheduled for 2.0 [14]. That is about 1,335 rows a second [21]. The smaller table ran the same code and closed its write window half a second before the flip arrived.

Those rates are a property of one box; the post does not state the host hardware [25]. What transfers is the ratio. The window is matched rows divided by your write rate, so at 1,590 rows a second a filter matching 100,000 rows holds it open for about a minute [22]. Writes ten times faster still leave six seconds [24].

The forum report that started this, thread t/14097 on September 4, said the implementation "actually finds by the condition first, then updates by the primary key of what it found" and that "under concurrency, a row whose data has already changed and no longer matches the condition will still be updated" [1]. Staff replied, "Thanks for the feedback. We are investigating" [2]. The thread has not moved since [3]. The author's position is that one SQL statement closes both this and the earlier hole where ten concurrent counter workflows produced one increment [18], and that a row lock makes the race deterministic to test [19]. In my view a status transition should not go through a filtered bulk update on a table past a few thousand rows unless you hold the lock yourself.

What to watch

  • Whether NocoBase closes thread t/14097 by carrying the filter into the UPDATE, or documents the current behaviour as intended.
  • Whether the individualHooks path gets the same fix, and whether it ever becomes reachable from the REST API.
  • The row-lock version of the test the author says makes the race deterministic to reproduce.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories