Build1 publisher2 min readPublished
Databricks' beta Shopify connector sees only 60 days of orders until Shopify approves a scope
Databricks' Shopify connector for Lakeflow Connect, in beta since September 22, 2026, syncs 39 Shopify tables into Unity Catalog incrementally. For a store whose data fits those tables, it can replace hand-maintained Admin API extraction code once the team clears the beta's access requirements.
The Engineer · Build desk

What happened
- Databricks takes over extraction, pagination, rate limiting and incremental cursors, and writes the results to Delta tables.
- Order history older than 60 days needs the read_all_orders scope, which Shopify grants by approval; without it the connector gets only the last 60 days.
- The Shopify client ID and secret live in a Unity Catalog connection object that pipelines reference by name, keeping them out of notebooks, YAML and Git.
- The workspace needs Unity Catalog with serverless compute on, and an admin must enable Lakeflow Connect for Shopify on the Previews page.
- The balance_transactions and disputes tables are available only when the store runs Shopify Payments.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision For historical order reporting, the read_all_orders request goes first in the plan: the walkthrough rates Shopify's approval as the likeliest delay, and the team cannot speed it up.
- cost Stores that run heavily on custom metafields keep paying to maintain a hand-built extraction path for that data beside the managed pipeline.
- capability Shopify customer records fall under Unity Catalog access controls, lineage and auditing as soon as they land, using the same policies as the rest of the lakehouse.
The connector takes over the layer most Shopify data teams build by hand: Admin API calls, pagination logic and retry handling [8]. The layers after it stay where they are. Bronze tables land in Unity Catalog, and the team still transforms them into the silver and gold layers that BI tools, Genie or ML workloads read [8]. A dev.to walkthrough built on the Databricks and Shopify documentation reports that the limitations page is explicit on this point: the connector ingests raw data without transformations and expects modeling to happen downstream [18][6].
The set of raw tables is fixed. The pipeline documentation lists 39 source tables in a default source schema. A pipeline can declare them one at a time or take the whole schema in one declaration [4]. Custom metafields, Shopify reports and objects outside those 39 tables are not covered [7]. Pipelines can be authored in the UI, through the API or as Declarative Automation Bundles, with orchestration through Databricks Workflows and SCD Type 2 history tracking [5].
The Shopify requirements are short. Any plan works, because the connector reads through the Shopify Admin API and every plan has it [13]. Someone creates an app in the Shopify Dev Dashboard, in the same Shopify organization as the store. The connector then authenticates with the OAuth client credentials grant [14].
The Databricks side is ordinary Unity Catalog grants. Whoever creates the connection needs CREATE CONNECTION on the metastore, or USE CONNECTION to reuse an existing one. They also need USE CATALOG on the target catalog, plus USE SCHEMA and CREATE TABLE on the target schema, or CREATE SCHEMA on the catalog [12]. The walkthrough notes that a different person commonly owns each side of the setup [20]. In most organisations that means two tickets with two owners before the first sync runs.
I think this is the right default for analytics on a store whose data fits the 39 tables. My context for that is a team whose extractor exists only to feed reporting, so the modeling work was always going to be theirs [6][8]. The walkthrough reaches the same verdict, calling the connector a strong default for analytics on Shopify data, with beta limitations to plan around before production use [19].
That guidance has a date on it. The walkthrough reflects the documentation as of late September 2026 and tells readers to check the linked docs for changes before building, because the connector is in Beta [18].
What to watch
- Revisions to the connector's limitations page during the beta, especially any coverage of custom metafields or Shopify reports.
- A general-availability announcement, and whether it drops the Previews-page opt-in or the serverless compute requirement.