How Microsoft OneLake seamlessly provides Apache Iceberg support for all Fabric Data
Co-authored with Kevin Liu, Apache Iceberg Committer and Principal Engineer at Microsoft.
Microsoft Fabric is a unified SaaS data and analytics platform designed for the era of AI. All workloads in Microsoft Fabric use Delta Lake as the standard, open-source table format. With Microsoft OneLake, Fabric’s unified SaaS data lake, customers can unify their data estate across multiple cloud and on-prem systems.
We recently announced that OneLake can now transparently serve Delta Lake tables as Apache Iceberg tables. This post covers the details of how we built this feature and enables seamless interoperability across engines and ecosystems.
Get started:
1. Create or identify a Delta Lake table in OneLake.
2. Use any Iceberg-compatible engine (e.g., Spark, Trino, Snowflake) to query the table.
3. OneLake will automatically serve the table in Iceberg format – no configuration needed.
Why This Matters
Organizations often rely on diverse engines and tools that favor different open table formats. Delta Lake is a popular choice for Spark-based pipelines and Microsoft Fabric engines, while Iceberg is increasingly adopted by query engines like Trino, Dremio, and Snowflake.
With OneLake, there’s no need to choose one format over the other. OneLake empowers openness and interoperability by automatically translating metadata between Delta and Iceberg formats. This means your tables are seamlessly available in both ecosystems—without duplication or compromise. Whether your team prefers Delta or Iceberg, OneLake ensures compatibility, flexibility, and freedom of choice.
By enabling on-the-fly conversion from Delta to Iceberg, OneLake eliminates the need for data duplication or format-specific pipelines.
You can now:
– Read Delta tables as Iceberg without rewriting or copying data.
– Use Iceberg-native tools on top of your existing Delta Lake datasets.
– Simplify governance and access control with a single source of truth.
Example
Here we have a table in Fabric Lakehouse stored in the Delta format.
Select ‘View files’ on table to see the table’s metadata.
Both _delta_log and metadata directories should appear.
Select the metadata directory to view the Iceberg files.
Navigate to Snowflake and create an external volume pointing to the exampled Lakehouse.
Next, we create an Iceberg catalog.
Create a table that uses the Iceberg catalog and external volume, specifying the Iceberg metadata file path in OneLake.
That’s it – now you can query the table!
How It Works
OneLake uses a feature called table format virtualization to surface tables as both Delta and Iceberg formats. When an Iceberg-compatible engine reads from a OneLake table that is natively in the Delta format, OneLake dynamically generates the necessary Iceberg metadata files. This allows the engine to interpret the table as if it were natively an Iceberg table.
Behind the scenes, this feature utilizes Apache XTable for table format metadata conversion. XTable provides cross-table omni-directional interop between the open table formats. We have also enhanced XTable functionality – for example, by converting Delta deletion vectors into Iceberg positional delete files. We look forward to contributing these features upstream to the open-source community.
- The original Delta table resides in OneLake or an external location (e.g., ADLS, S3).
- When queried via an Iceberg engine, OneLake serves Iceberg metadata derived from the metadata directory.
Here’s the workflow for a sample request.
The source metadata remains in its original Delta structure (in the _delta_log directory). OneLake’s virtualization layer intercepts read requests on the metadata directory and determines that Iceberg-compatible metadata is needed. It then generates Iceberg-compliant metadata on demand, enabling engines to interact with the table as if it were natively Iceberg, without any physical data movement. This approach is designed to be transparent to the table owner and does not modify the original storage location. It maintains consistency with the source table and scales efficiently by producing metadata on demand, when requested by a client.
Future Work
We’re actively expanding support for additional data types and advanced features across both Delta Lake and Iceberg formats. A key focus is ensuring compatibility with the upcoming Iceberg V3 specification. These enhancements will further improve OneLake’s cross-format interoperability and simplify analytics across engines.
