Mirroring: Uploading your CSVs is now simpler than ever before!
Open Mirroring in Microsoft Fabric continues to evolve, and the latest updates make working with files simpler and more powerful than ever. Previously, you were required to provide a primary key for mirrored parquet and delimited text files including CSV —now, that’s no longer necessary. Mirroring in Fabric no longer requires primary keys and automatically handles updates to existing tables, ensuring that any subsequent changes to the source file are inserted seamlessly into the existing table in OneLake.
For those who want more control, the option to specify a primary key and a __rowMarker__ column remains available, enabling advanced scenarios like updates, deletes, and upserts, empowering you with fine-grained change tracking.
Here is how it works!
- Create a Mirror Database by selecting Mirrored Database in ‘New Item’ and selecting the tile!
2. After the Mirrored Database has been created, simply drag and drop your file and you’ll see an option to name the table and provide a primary key if you want. You’ll also see advanced settings and a preview of the table as well. Primary Keys are now optional, feel free to leave the field empty and hit create!
3. After a minute or two you’ll see that your table is mirrored into OneLake and optimized for downstream AI and BI use cases and ready to be queried in a SQL analytics endpoint.
4. Now imaging you have another CSV you need to upload with new inventory. No problem! Simply drag and drop the file into the same Mirrored Database and select ‘Update existing’ rather than the option to create a new table and hit update!
5. New rows are inserted into the existing ‘Inventory’ table and you can see that the total row count went from 20 to 50!
Why this matters!
- Frictionless: No need to define a primary key to mirror parquet or delimited files including CSVs. This makes setup faster and simpler.
- Automatic synchronization: Updates to existing files are automatically inserted into the existing table in OneLake.
- Flexibility for advanced scenarios: Optional support for primary keys and __rowMarker__ columns for updates and deletes scenarios empowering you to use change tracking when you want to.
- No ETL overhead: Eliminates heavy data movement pipelines, accelerating analytics workflows.
- Free: Free compute and free storage up to a limit for all mirrored data.
