Data mesh and federated architecture
Verified 2026-08. A vacancy naming “data mesh architectures” and “federated software architectures” is describing an organisational problem, and the strong answer engages with that rather than reciting the four principles.
The problem it addresses
The centralised model: every source pushes raw data into a warehouse or lake, and one central data team transforms it for everyone.
That team becomes the bottleneck. They own pipelines for domains they do not understand, so a change in an upstream service silently breaks a downstream dashboard, and nobody can say who is responsible for a number being wrong. It scales until roughly the point where the central team can no longer hold every domain’s semantics in their heads.
The four principles
| Principle | Means |
|---|---|
| Domain ownership | the team that produces the data owns its analytical form, not a central team |
| Data as a product | that dataset has an owner, an SLA, documentation, quality guarantees and consumers |
| Self-serve data platform | a central platform team provides the infrastructure so domain teams do not each build their own |
| Federated computational governance | global rules (naming, privacy, interoperability) decided jointly and enforced automatically |
The word doing the work is computational in the fourth. Governance encoded as automated checks in the pipeline, not a standards document nobody reads.
Data as a product, concretely
This is the part that turns the idea into engineering:
- A contract. An explicit schema with types, nullability, semantics and a versioning policy. Breaking it is a breaking change with the same seriousness as an API break.
- An SLA. Freshness, completeness, availability. “Updated by 06:00, at most 0.1% null in these columns.”
- Quality tests that run in the pipeline. Not a monitoring dashboard someone might look at — checks that fail the build.
- Discoverability. Registered in a catalogue with an owner, a description and lineage.
- An addressable interface. A table, a topic, an API — something consumers can depend on without knowing how it is produced.
A “data product” without a contract, an SLA and tests is just a table someone renamed.
Federated architecture
Broader than data mesh: independent systems keeping their own data and cooperating without one central store. In practice it means query federation (a query engine reaching into several sources — Trino, BigQuery external tables, Athena federated queries) and the harder governance question of who owns a definition used across systems.
The trade-off to state plainly: federation avoids copying data and the staleness that comes with it, at the cost of query performance, unpredictable load on operational systems, and cross-source joins that are much slower than a colocated join. It is the right answer when data cannot be moved — sovereignty, regulation, sheer volume — and the wrong answer when you simply do not want to build a pipeline.
When data mesh is the wrong answer
The honest read, and the more senior answer:
- It requires organisational change. If domain teams will not staff and own data products, you get a centralised architecture with worse tooling and no owner.
- It has a floor. With three teams and a handful of datasets, a central warehouse is simpler and faster. Mesh addresses coordination costs you only have at scale.
- The platform is a real investment. Without a genuine self-serve platform, every domain reinvents ingestion, quality and deployment, and you have duplicated a central team five times.
- Governance is the hard part. Two domains defining “active customer” differently is an organisational problem no architecture solves by itself.
The industry has settled somewhere pragmatic: domain ownership and data contracts have been widely adopted; full mesh purity much less so. Saying that is more credible than advocating for it wholesale.
What actually transferred into mainstream practice
- Data contracts — schema plus semantics plus SLA, versioned, tested in CI. The single most useful idea from this whole discussion.
- Shift-left quality — tests at the producer, not reconciliation at the consumer.
- Lineage as infrastructure — column-level lineage so you can answer “what breaks if I change this column” before changing it.
- Ownership metadata — every dataset has a named owner and an on-call route.
- Lakehouse table formats (Iceberg, Delta) as the interoperable storage layer, so a “data product” is a table any engine can read rather than one vendor’s format. Iceberg v3 reached general availability across major platforms in the first half of 2026.
Interview angle
- “What is data mesh?” - domain ownership of analytical data, treated as a product with a contract and an SLA, on a self-serve platform, with governance encoded as automated checks. It is a response to the central data team becoming a bottleneck at scale.
- “When would you not use it?” - small organisations, or any organisation where domain teams will not own data products. Without that ownership you get the complexity and none of the benefit. It solves a coordination problem you have to actually have.
- “What is a data contract?” - an explicit, versioned agreement on schema, semantics and SLA between producer and consumer, tested in CI. Breaking it is treated like breaking an API. This is the idea from mesh that is worth adopting regardless of whether you adopt the rest.
- “Federated query or centralised warehouse?” - federate when the data genuinely cannot move (regulation, sovereignty, volume) and accept the performance cost and the load on operational systems. Centralise when it can, because colocated joins and predictable load are worth a lot.
- “How do you stop two teams defining the same metric differently?” - a shared semantic layer with metric definitions in version control, and computational governance that fails the pipeline on an undefined or conflicting term. It is fundamentally an organisational agreement enforced by tooling, not a tooling choice.