Load data from network-protected Azure Storage accounts to Microsoft OneLake with AzCopy
AzCopy is a powerful and performant tool for copying data between Azure Storage and Microsoft OneLake, and is the preferred tool for large-scale data movement due to its ease of use and built-in performance optimizations. AzCopy now supports copying data from firewall-enabled Azure Storage accounts into OneLake using trusted workspace access. Now you can use AzCopy to load data from even network-protected storage accounts, letting you effortlessly load data into OneLake without compromising on security or performance.
What is trusted workspace access
Trusted workspace access in Microsoft Fabric allows you to securely access firewall-enabled Storage accounts from OneLake shortcuts, pipelines, DW copy, and semantic models. To set up trusted workspace access, you first create a workspace managed identity and then grant that identity access to the Azure Storage firewall using a resource instance rule. After set-up, your shortcuts, pipelines, DW copies, semantic models, and now AzCopy jobs in that workspace can automatically access that storage account, no additional work required!
Getting started
To get started copying data from your firewall-enabled storage accounts, you’ll need to complete the following steps.
1. Create a workspace managed identity (if you don’t already have one).
For more details on creating and deploying a workspace identity, refer to the creating a workspace managed identity documentation.
2. Configure the resource instance rule for your Azure Storage account.
For more details, including a sample ARM template, check out the guidelines for resource instance rules.
3. Copy your data with AzCopy. For more information about using AzCopy with OneLake, including sample copy operations, refer to Moving data with OneLake and AzCopy.
azcopy copy
“https://contosostorage.dfs.core.windows.net/sales/customers.csv”
“http://onelake.dfs.fabric.microsoft.com/contosoWorkspace/contosoLakehouse.Lakehouse/Files/sales/customers.csv”
–trusted-microsoft-suffixes “fabric.microsoft.com”
