Molinia

Engineering

Query in place or ingest? It depends whose bucket it is

External tables let you query object storage without copying it — which is a sovereignty win over EU storage and a sovereignty trap over ADLS Gen2. The deciding variable is jurisdiction, not copy count.

The standard warehouse onboarding goes: point an ingestion job at your source, copy the data into the vendor’s storage, and query it there. Everything downstream — cost, latency, governance — follows from that first copy.

It also creates a second problem that nobody puts on the invoice. You now have two copies of the same data in two places, under two sets of controls, and possibly under two jurisdictions. Every deletion request has to find both. Every residency claim has to cover both. Every audit has to reconcile them.

External tables are the alternative: leave the files where they are and query them from there.

What it looks like

You register a storage location — an S3-compatible bucket you already own — browse the files under it, preview one, and define a table over it. From then on it is a table. It joins against warehouse tables, it respects row-level security policies, it appears in the catalogue.

SELECT
  o.region,
  count(*)          AS orders,
  sum(o.amount_eur) AS revenue
FROM  ext_orders_2026 o          -- Parquet, in your bucket, never copied
JOIN  dim_region     r USING (region)
WHERE o.placed_at >= DATE '2026-01-01'
GROUP BY 1
ORDER BY revenue DESC;

No ingestion job, no second storage bill, no drift between the copy and the source. The engine reads the Parquet where it sits.

The variable that decides it: whose bucket is it

Here is where a lot of writing on this topic — including an earlier draft of this article — goes wrong. “Don’t make a copy” sounds like a sovereignty principle. It isn’t. It is a sovereignty principle only if the bucket is already under EU jurisdiction.

If your object storage is ADLS Gen2, S3, or GCS, then “leave the data where it is” means leaving the authoritative copy with a US-controlled provider. Querying it from Amsterdam adds a European reader; it does not move the data out of reach. You have not avoided a jurisdiction problem, you have declined to solve one.

So the honest rule has two branches:

Your storage is already EU-jurisdiction — Leafcloud, OVHcloud, Scaleway, IONOS, STACKIT, or your own MinIO on your own hardware. Use external tables. The data stays put, there is no second custodian, and the copy count stays at one. This is the case where query-in-place genuinely is the sovereign choice.

Your storage is US-controlled — ADLS Gen2, S3, GCS. Ingest. The copy is the point: it puts the analytical copy, the one with the joins and the history, under EU jurisdiction. Then you can decide separately what happens to the source — keep it as a landing zone, demote it, or retire it once you trust the pipeline.

There is a plain cost argument reinforcing the second case. Azure and AWS charge egress. An external table over ADLS Gen2 pays Microsoft every time a query scans it, forever. Ingesting pays that toll once.

External tables as the on-ramp, not the destination

The two branches are not a fork you have to resolve on day one, and this is the part worth knowing if you are currently on ADLS Gen2 and dreading a migration project.

Point an external table at the storage you already have. Explore it, join it against whatever is already in the warehouse, work out which tables actually matter — most organisations discover that a minority of them carry the analytical weight. Then materialise those into EU-jurisdiction storage and leave the rest where they are until you care.

You get first value in an afternoon rather than after a quarter-long migration, and the migration that follows is scoped by evidence instead of by inventory.

The performance trade-off, separately

None of the above is the reason most teams eventually ingest. Reading Parquet over a network is slower than reading from local NVMe. A dashboard that runs a thousand times an hour wants the data inside a warehouse regardless of who owns the bucket.

That is a performance decision and it is worth keeping distinct from the jurisdiction one. The useful end state is usually a split: external tables over the large, cold history, ingested tables for the hot working set. What matters is that both the copy and its location are deliberate choices rather than whatever the onboarding flow did to you.

Enforcing policy in the engine, not the dashboard

An external table is only useful if it is governed like any other table. Row filters and column masking have to be applied where the query is planned — inside the engine — rather than in the BI tool on top.

The difference shows up the moment someone connects a second tool. Policy implemented in a dashboard is policy that a psql session, a notebook, or a service account walks straight past. Policy implemented in query rewriting applies to every path into the data, including the ones you did not anticipate.

The same holds for masking. full, partial, and email-shaped masks are worth distinguishing, because “show the analyst that a support ticket exists without showing them the customer’s address” is a much more common requirement than blanket denial, and blanket denial is what teams fall back on when the tooling cannot express the middle.

Compute that stops when you do

Two operational notes that matter more for analytics than people expect.

Warehouses suspend when idle. Analytical usage is spiky — a morning refresh, a burst around a board meeting, silence overnight. Compute billed per minute with automatic suspension after an idle period means a workspace nobody is using costs storage and nothing else. This is table stakes in the category and worth checking anyway, because the failure mode — a warehouse that quietly ran for a fortnight — is one that gets people fired.

Where the compute physically runs is a choice you can make on other grounds too. Molinia’s warehouse compute runs on Leafcloud in Amsterdam, whose data centres are sited inside buildings and pipe server waste heat into the residential hot water system. It is heat reuse rather than certificate purchasing: the energy does a second job instead of being offset on paper.

We mention it here rather than as a headline because it is genuinely secondary to whether the query returns the right answer quickly. But if you are already choosing infrastructure on jurisdiction, the marginal cost of also choosing it on where the heat goes is close to zero.

Count jurisdictions, not copies

If there is one thing to take from this, it is that the metric people reach for is the wrong one. “How many copies of this data exist” is an operations question. “How many legal regimes can compel this data” is the sovereignty question, and the two come apart in both directions.

Three copies inside one EU provider is one jurisdiction. One copy in ADLS Gen2 is one jurisdiction too — just not the one you wanted. And a copy made from the second into the first can reduce your exposure from two regimes to one, if you are willing to follow through and retire the source.

Which means the sovereign move is sometimes to copy, and sometimes to refuse to. Anyone telling you it is always one of those, us included, is describing their product rather than your problem.

Molinia is an EU-sovereign OLAP platform built on DuckDB, with warehouse compute in Amsterdam. New workspaces start with €200 in credits for 30 days, no card required — book a demo if you want to see external tables against your own bucket.