Microsoft Fabric Analytics Platform
Medallion architecture, incremental loads and a semantic model that survives real use
By Renjith ·
Prerequisites
- A Fabric capacity (trial is sufficient to follow along)
- Familiarity with Power BI semantic models
- Basic PySpark or SQL
The problem
Fabric collapses a lot of previously separate decisions into one workspace, and that is exactly what makes it easy to get wrong. Lakehouse or warehouse, Direct Lake or import, where transformation logic lives, how environments are separated — these look like configuration and behave like architecture. Several are extremely painful to reverse once there is data and reports depending on them.
Most Fabric material demonstrates the buttons. Very little of it explains which choices you are quietly making permanent.
What was built
A complete platform for a mid-size retail dataset, built as a reference implementation:
- Bronze — raw ingestion, append-only, source schema preserved exactly as received.
- Silver — cleaned, typed, deduplicated, with business keys resolved.
- Gold — star-schema tables shaped for the semantic model.
- Semantic model — Direct Lake over gold, with calculation groups and a real date dimension.
- Deployment pipelines — dev, test and production workspaces with parameterised connections.
Every layer boundary has a written rule about what is allowed to happen there, which is what stops a medallion architecture decaying into three copies of the same mess.
How it works
The decision that is hardest to reverse#
Lakehouse or warehouse. Everything else can be reworked with effort; this one reaches into every downstream artefact.
| Lakehouse | Warehouse | |
|---|---|---|
| Primary interface | Spark notebooks, Delta tables | T-SQL |
| Best when | Python-heavy transforms, semi-structured data, ML | SQL-first team, heavy stored-procedure logic |
| Direct Lake | Native | Supported |
| Team skills | PySpark | T-SQL |
The honest heuristic: pick the one your team can debug at 9pm. A technically superior platform that nobody on the team can troubleshoot is not superior.
Direct Lake fallback is the thing to watch#
Direct Lake is fast until it silently is not. Certain operations force a fallback to DirectQuery, and the report just gets slower with no error anywhere.
check_fallback.sqlsql-- Run against the semantic model's XMLA endpoint after a slow report load
SELECT
[Event Class],
[Text Data],
[Duration]
FROM $SYSTEM.DISCOVER_TRACES
WHERE [Event Class] = 'DirectQuery Begin'
ORDER BY [Duration] DESC
In this build the culprits were consistent: calculated columns on large tables, relationships spanning storage modes, and a couple of DAX patterns that could not be pushed down. Finding them took an afternoon; not knowing to look for them would have cost weeks of "Fabric is slow" without a cause.
Incremental refresh is not optional
Full loads work at 5 million rows and become a nightly incident at 50 million. Build incremental ingestion from the first commit — retrofitting it means reprocessing history while people are trying to use the reports.
Where transformation logic lives#
One rule, applied without exception: transformation logic lives in the lakehouse, never in the semantic model. Calculated columns and complex model-level transformations are convenient and they hide business logic in a place nobody thinks to look, cannot be tested, and cannot be reused by anything else.
The semantic model's job is aggregation and presentation. Everything else belongs upstream where it can be version-controlled and tested.
Build steps
- 1
Decide lakehouse vs warehouse first
This is the decision that constrains everything after it. Written up with the trade-offs rather than presented as a default.
- 2
Set up bronze ingestion
Append-only, partitioned by load date, with source schema preserved. Bronze is a landing zone, not a staging area — no cleaning happens here.
- 3
Build silver transformations
Typing, deduplication and business key resolution, written as idempotent notebooks so a rerun cannot double-count.
- 4
Shape gold for the model
Star schema, conformed dimensions, surrogate keys. Gold is shaped for consumption, not for storage elegance.
- 5
Connect Direct Lake
And measure what actually falls back to DirectQuery, because the fallback is silent and it is where the surprises live.
- 6
Wire deployment pipelines
Parameterised connections so promotion between workspaces does not require editing anything by hand.
Lessons learned
Choose the platform your team can debug, not the one that benchmarks best.
Direct Lake fallback is silent. Add a trace check to your definition of done, or you will be diagnosing "Fabric feels slow" by intuition.
Build incremental from day one. Every retrofit of incremental refresh is done under pressure, on live data.
Transformation belongs in the lakehouse. Logic in the semantic model is logic nobody can find, test, or reuse.
Limitations
Costs are capacity-based and this write-up does not model them. A workload that fits comfortably at F2 in development may not at production concurrency.
Fabric changes quickly; specifics here reflect the state at the time of writing and should be checked against current documentation.
The reference dataset is retail-shaped. Time-series and IoT workloads have materially different partitioning needs.
Future improvements
Add a data quality layer between silver and gold with explicit expectations and a quarantine path for rows that fail them.
Automated deployment through Fabric Git integration instead of manual pipeline promotion.
Capacity monitoring with alerting, so throttling is visible before users report it.
Premium version
Fabric Starter Platform
A complete Microsoft Fabric reference implementation you can adapt: notebooks, semantic model, pipelines and the architecture decision record.
Related course
From Power BI to Microsoft Fabric
Move from building reports to building the platform reports sit on: lakehouse architecture, medallion layering, Direct Lake and deployment pipelines.
Free resource
Fabric Architecture Decision Guide
The decisions that are expensive to reverse in Microsoft Fabric, with the trade-offs laid out.
Related builds and resources
Fabric Starter Platform
A complete Microsoft Fabric reference implementation you can adapt: notebooks, semantic model, pipelines and the architecture decision record.
From Power BI to Microsoft Fabric
Move from building reports to building the platform reports sit on: lakehouse architecture, medallion layering, Direct Lake and deployment pipelines.
Fabric Architecture Decision Guide
The decisions that are expensive to reverse in Microsoft Fabric, with the trade-offs laid out.
Power BI Developer to Fabric Engineer: A Roadmap
A staged plan for moving from report development into platform engineering on Microsoft Fabric — including an honest read on which existing skills carry over.
Microsoft Fabric Implementation
Architecture and build for a Fabric platform, with the irreversible decisions made deliberately.