Microsoft Fabric Updates Blog

Getting the size of OneLake data items or folders

Understanding the size of your OneLake data can be important to manage and plan storage costs, especially if you have large amounts of data.  Today, capacity admins can use the Microsoft Fabric Capacity Metrics app to find the total size of OneLake data stored in a given capacity or workspace but you may also want to understand the size of data in a specific item or folder.  In this blog, we’ll walk through a couple Azure PowerShell commands that enable you to get size information of any item or folders in OneLake.  As described in this blog post, with OneLake’s compatibility with ADLS tools, it’s as simple as replacing the ADLS Gen2 URL with a OneLake URL. 

To get started, there are three simple steps to set up:

Step 1: Open Azure PowerShell and install the Azure Storage PowerShell module. 

Install-Module Az.Storage -Repository PSGallery -Force

Step 2: Sign in to your Azure account.

Connect-AzAccount

Step 3: Create the storage account context.

  • Storage account name is onelake.
  • Set -UseConnectedAccount to pass through your Azure credentials.
    • Set -endpoint as fabric.microsoft.com.
$ctx = New-AzStorageContext -StorageAccountName 'onelake' -UseConnectedAccount -endpoint 'fabric.microsoft.com'

Get the size of an item

This example gets the size of an item “mylakehouse.lakehouse” in the workspace “myworkspace”.   

$workspaceName = 'myworkspace'
$itemPath = 'mylakehouse.lakehouse'
$colitems = Get-AzDataLakeGen2ChildItem -Context $ctx -FileSystem $workspaceName -Path $itemPath -Recurse -FetchProperty | Measure-Object -property Length -sum
"Total file size: " + ($colitems.sum / 1GB) + " GB"

Keep in mind, if the workspace name does not meet Azure Storage naming criteria (ex. must be lowercase letters) then replace the workspace and item names with their GUIDs.  You can find the associated GUID for your workspace or item in the URL on the Fabric portal. You must use GUIDs for both the workspace and the item, and don’t need the item type.

Get the size of a folder

$workspaceName = 'myworkspace'
$itemPath = 'mylakehouse.lakehouse/Files/folder1'
$colitems = Get-AzDataLakeGen2ChildItem -Context $ctx -FileSystem $workspaceName -Path $itemPath -Recurse -FetchProperty | Measure-Object -property Length -sum
"Total file size: " + ($colitems.sum / 1GB) + " GB"

Additional Details

  • These PowerShell commands won’t work on shortcuts that directly point to ADLS containers.  It’s recommended you create ADLS shortcuts to a directory that is at least one level below a container.
  • If you would like to fully automate these steps, you can obtain the workspace and item information with these APIs: Workspaces – List Workspaces – REST API and Items – List Items – REST API.

Next steps

We hope you continue to explore the ways you can leverage ADLS tools and APIs with your OneLake data.  To learn more about how to connect to OneLake, check out these links:

Related blog posts

Getting the size of OneLake data items or folders

February 15, 2024 by Matthew Hicks

Microsoft Fabric REST APIs allow users and developers to automate common actions and programmatically interact with Fabric. This allows teams and organizations to develop solutions that can scale and integrate with other parts of their systems. Before today, OneLake shortcuts could only be created using the Fabric web experience. This meant that our users were … Continue reading “OneLake Shortcuts APIs available in Preview”

February 14, 2024 by Liliam Leme

This post shows how simple is to enable Copilot which is another generative AI that brings new ways to transform and analyze data, generate insights, and create visualizations and reports in Microsoft Fabric. Discover how effortlessly you can make your data analysis happen and generate insights from it. Copilot -Prerequisites: Ref: Overview of Copilot in … Continue reading “Fabric Change the Game: How easy is it to use Copilot in Microsoft Fabric”