Real-Time Data Sharing in Microsoft Fabric
In the ever-evolving landscape of data management and collaboration, the demand for seamless and secure data sharing solutions has never been greater. There are many traditional ways to share data, such as through file shares, FTP, e-mail, and APIs but most of these methods require both parties to build and maintain a data pipeline that moves data between teams and organizations.
Introducing database shortcuts in Real-Time Analytics
A database shortcut in Real-Time Analytics is an embedded reference within a KQL database to a source database in Azure Data Explorer (ADX) allowing in-place data sharing. The behaviour exhibited by the database shortcut is similar to that of a follower database.
When is a database shortcut useful?
If you are a data provider hosting data in Azure Data Explorer and want to share this data with consumers in Microsoft Fabric, you can create a database shortcut. Database shortcut enables sharing in real-time, with no need to build or maintain a data pipeline. All database changes, including schema and data, on the provider side are instantly available on the consumer side.
This is also a great way for existing ADX customers to try Microsoft Fabric without re-ingesting the data with just few clicks.
Scenario
- Data provider shares data with business partners
- Data consumer combines data received from multiple business partners with its own data to derive insights.
How does a database shortcut work?
The owner of the source database in Azure Data Explorer, the data provider, shares the database with the creator of the shortcut in Real-Time Analytics, the data consumer. The owner and the creator can be the same person.
The database shortcut is attached in read-only mode, making it possible to view and run queries on the data that was ingested into the source Azure Data Explorer database. The database shortcut synchronizes changes in the source database.
Both the source ADX database and the database shortcut in Real-Time Analytics must be in the same region but can be in different tenants allowing ISVs to share data with their customers seamlessly.
Steps to configure data sharing
Data Provider : Creating an invitation token
A data consumer who wants to create a database shortcut using invitation tokens must obtain a token from the data provider. A token is a code that lets a data consumer create a shortcut to the source database. You don’t need a token if you create a database shortcut using the cluster URI method.
To create an invitation token, the data provider can use the following steps:
- Browse to the Azure portal.
- Browse to the Azure Data Explorer database you wish to use as source.
- Select Share.
4. Enter the recipient email address. This address should be the email address associated with the Fabric user account in which you later create the database shortcut. This email address may be your own, or someone else’s.
5. Select Share.
6. Select Copy link. Share it with the data consumer – the person who the token was created for can later paste this link into a browser to create the shortcut.
Data Consumer: Creating a database shortcut
Once a data provider creates an invitation link, use the link to open Real-Time Analytics showing the New database shortcut dialog box, and then follow these steps:
- Enter a name for your database shortcut.
- Specify the Workspace in which you want to create the database shortcut.
- Optionally, modify the default cache policy.
- Check that the Invitation token is populated with the token you created and verified.
5. Select Create.
Once the shortcut is created, you’re taken to the database details view of the new database shortcut.
Now, you can start consuming the real-time data feed via KQL Queryset, Power BI, Notebooks etc. seamlessly.
While today, you can share data from an Azure Data Explorer database with a KQL Database, soon you will also be able to share data from a KQL Database with other KQL Database(s) in Microsoft Fabric.
