Databricks says the hard part of warehouse migration was the stored procedures, not the data

Databricks has published a walkthrough arguing that the friction in warehouse migration was never moving the data, which it describes as well understood, but the procedural core: stored procedures, transaction handling, temp tables, control flow, and the fact that much of the enterprise still runs on SQL skills ^[1]. For anyone holding a migration business case, the consequence is narrow and specific: if the nightly procedural layer can be translated rather than rewritten, the line item that used to read "weeks of Python and Spark work, plus new bugs, plus a SQL team that can no longer maintain its own logic" gets a lot smaller ^[2].

The demonstration uses a composite procedure based on an Oracle migration case, which Databricks says applies to any legacy or cloud warehouse ^[3]. It stages unprocessed orders into a temp table, validates them against the customer master, loops through failures to log each rejection individually, updates regional revenue summaries and marks all orders processed, inside a transaction that rolls back on failure ^[4]. Databricks' framing is that the revenue dashboard, the finance close and the operations report all trace back to layers of procedural SQL that nobody fully understands and everybody depends on ^[5].

The mapping is the substance. The legacy BEGIN ... EXCEPTION ... END wrapper becomes DECLARE EXIT HANDLER FOR SQLEXCEPTION ^[6]. SELECT ... INTO becomes SET var = (SELECT ...) ^[7]. The scripting surface covers IF/ELSE, WHILE, FOR, LOOP, REPEAT, LEAVE, ITERATE and SIGNAL/RESIGNAL ^[8], and Teradata BTEQ .GOTO and .LABEL directives map onto labelled loops with LEAVE and ITERATE ^[9]. Cursors, the piece Databricks says everyone assumed would need a rewrite, are supported natively with OPEN, FETCH and CLOSE since Runtime 18.1, with %NOTFOUND becoming a CONTINUE HANDLER FOR NOT FOUND and loop labels plus LEAVE replacing EXIT WHEN ^[10]. That version number is the practical gate: on anything earlier the cursor path does not exist, so a runtime upgrade precedes the translation work ^[11].

Temp tables are cleaner. Session-scoped CREATE TEMP TABLE is the direct replacement, with no EXECUTE IMMEDIATE and no ON COMMIT PRESERVE ROWS, but CREATE OR REPLACE TEMP TABLE is not yet supported, so you drop first if the procedure must be re-runnable in the same session ^[12]. The example creates two temp tables, one for staging and one for validation failures ^[13], which means two hand-added DROP statements to preserve re-runnability ^[14].

Then transactions. Databricks calls this "the last piece, the one that made the migration actually viable" ^[15]: regional_revenue updated, orders marked processed, the batch logged, and a rollback if any part fails, which on the legacy system is an implicit transaction ^[16]. The text supplied to us breaks off mid-sentence at that point without stating the Databricks equivalent ^[17]. The element the vendor itself names as decisive is the one not demonstrated in the material available.

The deployment argument is separate and more durable: the procedure is registered in Unity Catalog with access controls, column-level lineage and discoverability across workspaces, against a legacy schema that three people had the password to ^[18]. Lineage and access control are auditable claims; the password line is colour.

Watch for the transaction section in full, and specifically whether multi-table rollback is a documented guarantee or a pattern you assemble yourself. Watch cursor throughput at real nightly-batch volumes, since row-at-a-time loops are the part that ports easily and runs badly. And watch whether CREATE OR REPLACE TEMP TABLE lands, because idempotent re-runs are how these jobs get restarted at 3am.