Announcing the General Availability of Open Mirroring
Today, we are proud to announce that Open Mirroring is now generally available with parquet support! Your data is spread across multiple databases, legacy applications, and bespoke data solutions. Bringing all this data together and deriving the right insights for the business is often difficult. Open Mirroring makes working with this data easy.
Open Mirroring is an extension of Mirroring capabilities in Fabric and it’s one of the easiest ways to get data into Fabric, it creates a copy of your data in OneLake and keeps it up to date; no ETL required. Open Mirroring provides users and partners with APIs to replicate data from anywhere. Simply convert the data to a parquet or CSV format and use the API or UI to load the data into the OneLake landing zone along with any additional changes. Our replication technology takes over from there, converts everything to a Delta format making it optimized and ready for all your AI and BI workloads in Microsoft Fabric. With Open Mirroring, everyone is empowered to create their own Mirroring source because it’s designed to be extensible and customizable, enabling you to seamlessly use any data in Microsoft Fabric without the need for complex transformations.
Customers are using Open Mirroring to bring all kinds of data from disparate systems. Eastman Chemical Company, for example, often struggled to unify their globally distributed data. With Open Mirroring, they’ve built a solution that allows them to centralize data access in Microsoft Fabric enabling them to finally start building machine learning and reporting solutions.
“Open mirroring is one of the coolest new features in Fabric. With just a bit of coding, you can push data from nearly any source into Fabric. At Eastman, we’re using it to transfer data from our on-premises lab management systems into Fabric. We have 20 of these systems located around the world, and until now, we’ve struggled to consolidate data from all of them for reporting and analytics. So far, we’ve managed to ingest about a billion rows from eight of those systems, and we’re updating the data every hour. Having all this data in Fabric opens the door for us to build machine learning and reporting solutions, which we haven’t been able to do at scale before.”
– David Purdy, Enterprise Architect at Eastman Chemical Company
Introducing the Open Mirroring Python SDK
With the general availability of Open Mirroring, we’re also happy to introduce the new Python SDK designed to simplify the process of building out your custom Mirroring DB source. With this SDK, developers can easily interact with the Open Mirroring APIs, enabling seamless data replication and management. The SDK provides a comprehensive set of functions to configure a mirroring source, allowing you to get started quickly. By leveraging the Python SDK, you can automate the setup and maintenance of mirrored databases, ensuring that your data is always up-to-date and ready for any workload in Microsoft Fabric.
from openmirroring_operations import OpenMirroringClient
# Initialize the client
client = OpenMirroringClient(
client_id="your-client-id",
client_secret="your-client-secret",
client_tenant="your-tenant-id",
host="https://onelake.dfs.fabric.microsoft.com/<workspace-id>/<mirrored-database-id>/Files/LandingZone/"
)
# Create a table
client.create_table(schema_name="SampleSchema", table_name="SampleTable", key_cols=["Column1", "Column2"])
# Get the next file name (optional, not need in real usage)
next_file_name = client.get_next_file_name(schema_name="SampleSchema", table_name="SampleTable")
print(f"Next file name: {next_file_name}")
# Upload a file
client.upload_data_file(schema_name="SampleSchema", table_name="SampleTable", local_file_path="path/to/your/file.parquet")
# Remove a table
client.remove_table(schema_name="SampleSchema", table_name="SampleTable")Open Mirroring Partner Ecosystem
The Open Mirroring partner ecosystem brings together a diverse set of technology providers that enable seamless data replication into Microsoft Fabric. Partners like Oracle GoldenGate, Striim, MongoDB, SNP, Simplement, CleudIn, Asapio, Theobald, and dab contribute specialized capabilities for syncing data from operational systems such as Oracle, SQL Server, Salesforce, Google Big Query, MongoDB Atlas, SAP, and more. This collaborative ecosystem ensures that organizations can integrate data from a wide range of enterprise sources into Fabric with minimal friction, supporting real-time analytics and unified data experiences across platforms.
“Open Mirroring delivers massive value to our customers at CluedIn by enabling seamless, real-time data availability across the Microsoft ecosystem, especially Microsoft Fabric. It breaks down traditional integration barriers and empowers our joint customers to unlock the full potential of their data, faster and with less friction. Before our Open Mirroring integration, our customers needed to move parquet files into OneLake and then run separate processes to convert to Delta before they could even get started with their Fabric workloads. With Open Mirroring, this complexity is completely removed which is HUGE win for our customers. Collaborating with the Microsoft team on our Open Mirroring integration has been a great experience. Our engineering team was able to deliver an Open Mirroring integration that customers could use in a matter of two weeks.”
– Tim D. Ward, CEO at CluedIn
Whether you’re a developer, data engineer, solution integrator, ISV, or business leader, Open Mirroring removes integration barriers and accelerates your journey to unified, AI-ready data in Microsoft Fabric. Start building your open mirroring source today!
Learn more about Open Mirroring and get started today! Check out this Tutorial: Configure Microsoft Fabric open mirrored databases.
Submit your feedback on Fabric Ideas and join the conversation on the Fabric Community. To get into the technical details, head over to the Fabric documentation.
