Bring your on-premises data to OneLake!
Microsoft OneLake is a unified data lake for all of your organization’s data. With OneLake shortcuts, you can reference data in different locations and have that data logically represented within OneLake, with no data movement or duplication. Data across Azure, Amazon Web Services, and Google Cloud Platform can coexist in the same virtual lake. With the newest addition of S3 compatible storage shortcuts, a broad range of cloud storage services are also supported.
What if there were a way to create shortcuts to on-premises data just as easily? Or to network-restricted data sources, behind firewalls or virtual private clouds?
Well, now there is! We’re excited to announce the public preview availability of OneLake shortcuts to on-premises and network-restricted data sources!
With this feature, it’s quick and easy to create a shortcut that references on-premises or network-restricted data. You simply install the Fabric on-premises data gateway on a machine in your environment that has networking visibility of your S3 compatible, Amazon S3, or Google Cloud Storage data source. Then, you create your shortcut and select that gateway.
Once you set up your shortcut, you can access and use this data with the many Fabric engines or other services using OneLake’s open APIs!
Getting started
Get started today! Check out our quick start guide or read along here:
- Install the latest version of the standard Fabric on-premises data gateway on a machine that can connect to your storage endpoint.
- Open your Fabric lakehouse and create a new shortcut.
- If your data is already in the Delta Lake format, create your shortcut in the Tables section of your lakehouse. This will allow your table shortcut to benefit from metadata synchronization across Fabric engines, letting you use this structured data where tables are used in Fabric.
- Otherwise, create a shortcut in the Files section of the lakehouse, and create a shortcut from there.
- Under External sources, select either Amazon S3, Amazon S3 compatible (preview), or Google Cloud Storage (preview).
- Configure your connection settings. Enter your data source’s publicly reachable endpoint URL, select the gateway you configured in Step 1, and provide the key/secret credential that has authorization to list buckets, get bucket info, list objects, and read data.
- For tips on what endpoint to enter and other details, see our documentation.
- Confirm your choices, rename your shortcut(s) if preferred, and create your shortcuts!
- That’s all – you can now use Fabric to analyze your on-premises or network-restricted data, along with all your other data in OneLake, with no data movement or duplication!
What’s next?
Coming soon, we’ll add support for Iceberg tables in OneLake shortcuts. This means you will soon be able to add table shortcuts not just to Delta Lake data, but also Iceberg formatted data! Stay tuned!
We hope you enjoy this new feature and find it useful as you plan and build your data solutions with Fabric. As always, we appreciate your feedback and ideas for future improvements. Please submit any feedback or suggestions at Microsoft Fabric Ideas.
