Microsoft Fabric Updates Blog

Fabric Change the Game: Revolutionizing Fabric with REST API, Part 1

This post will explore Microsoft Fabric REST APIs for seamless automation. Following, let’s review through some of the options for API integration in Fabric, unravelling the threads of development, deployment, governance, and beyond in this comprehensive series, this is Part One.

Some additional Material to review:

Fabric API quickstart – Microsoft Fabric REST APIs | Microsoft Learn

Items – Create Lakehouse – REST API (Lakehouse) | Microsoft Learn

Fabric data pipeline public REST API (Preview) – Microsoft Fabric | Microsoft Learn

Azure REST API reference documentation | Microsoft Learn

Manage resources – REST – Azure Resource Manager | Microsoft Learn

Step by Step

As you can also find here in this video from Fabric Espresso Automate your interactions with Fabric Warehouse (youtube.com), as also in this doc. Fabric data pipeline public REST API (Preview) – Microsoft Fabric | Microsoft Learn. It will be used for the purpose of this post the copy(powerBIAccessToken) as a Token option to run the Rest APIs.

Getting the Token for test the Rest APIs

From the browser where your Microsoft Fabric is opened -> Click in the developer tools using the browser. You can also just click in F12 and it will open the Developer tools:

Use the console to generate the token by executing the following, as Fig 0 – Copy shows:

copy(powerBIAccessToken)
Fig 0 – Copy

After that just execute ctrl+v and paste in the notebook, you could use python inside the Visual Studio for example or just the Lakehouse inside of Fabric for a test purpose.

headers = {

'Authorization': f'Bearer PASTE HERE',

'Content-Type': 'application/json',

'Accept': 'application/json'

}

headers

Create a Workspace

API for Create workspaces – Python example:

The API for creating a workspace includes a parameter for the capacity ID. Even if you are using a trial version, a capacity ID is still there for the trial option.

To locate the capacity ID, navigate to the admin portal within Microsoft Fabric, then go to capacity settings and review the settings for your capacity, as Fig 1 – CapacityID shows.

If you prefer not to assign the capacity when creating the workspace, the API currently allows you to proceed without assigning the workspace capacity. However, you may wish to adjust this assignment later using the UI through the admin portal option.

Fig 1 – CapacityID
from urllib.parse import urlparse


url = "https://api.fabric.microsoft.com/v1/workspaces/"


jsonPayload = {
  "displayName": "NAME WORKSPACE",
  "capacityId" : "REPLACE by the Capacity ID or assign it later manually ( in that case remove this parameter completely) "
}


response = requests.post(url,json=jsonPayload, headers=headers)

response.raise_for_status()
print (response)

if response.status_code == 200 or response.status_code == 201:
  print("Request accepted. Processing in progress.")

   
  location_header = response.headers.get("Location")
  parsed_url = urlparse(location_header)
  path_parts = parsed_url.path.split('/')
  workspace_id = path_parts[-1]

  print (location_header)
  print(workspace_id)

  url_get= f"https://api.fabric.microsoft.com/v1/workspaces/{workspace_id}"

Example of results – Fig 2 – CreateWorkspace:

Fig 2 – CreateWorkspace

Listing Workspaces

This API will allowed you to list the workspaces created which you have access in the environment:

url_get= f"https://api.fabric.microsoft.com/v1/workspaces/"

response_status = requests.get(url_get, headers=headers)

response_data = response_status.json()

status = response_data.get('status')

print (response_data)

Results in the Fig 3 – List:

it is possible to check the workspace I just created: Lemedemo_WS

Fig 3 – List

Drop Workspace

The workspace ID parameter is essential for dropping a workspace. You can find it either as a result of the list API or on the Microsoft Fabric browser .

For instance, if you have your notebook open inside the Lakehouse, locate the number between “https://app.powerbi.com/groups/” and “/synapsenotebooks/” in your Microsoft Fabric. This number represents your workspace ID as Fig 4 – Browser exemplifies.

Fig 4 – Browser

Create an empty Data Pipeline

It is quite simple to create a data pipeline through code, you will just need the Workspace ID which was mentioned earlier in this post how to get it. (In case doubts check the API to list Workspace, the result of the execution has the ID)

import requests
​
workspace_id = "Replace by workspace id"
​
url_pipd = f"https://api.fabric.microsoft.com/v1/workspaces/{workspace_id}/dataPipelines"
​
payload = {
​
  "displayName": "Create_Pipl_API",
​
  "description": "A data pipeline created  by Rest API"
​
}
​
​
response = requests.post(url_pipd,json=payload,  headers=headers)
​
​
response.raise_for_status()
​
print (response)
​
if response.status_code in (201,202):
​
  print("Post request successful.")
​
else:
​
  print(f"Error: {response.status_code} - {response.text}")

Pipeline execution

Follow the script in python how to run the pipeline execution in Microsoft Fabric. If you have parameters for the pipeline you can pass through payload, as I do not have my payload is empty in the code example:

Pipeline: Fabric data pipeline public REST API (Preview) – Microsoft Fabric | Microsoft Learn

wokspace = "Replace by ID, You can find the ID in the browser or using the API for listing"
pipeline = "You can find the ID in the browser"

url = f"https://api.fabric.microsoft.com/v1/workspaces/{wokspace}/items/{pipeline}/jobs/instances?jobType=Pipeline"

payload = {
  "executionData": {}
}

# Make the POST request
response = requests.post(url,json=payload, headers=headers)
# Check for HTTP errors
response.raise_for_status()

Create a Lakehouse

Lakehouse will follow the same process of the Data Pipeline, but it will call lakehouses instead of dataPipeline, obviously. So once more, you will only need the Workspace ID to create the Lakehouse.

Follow the python example:

import requests
​
workspace_id = "Replace the workspace ID"
​
url_lh = f"https://api.fabric.microsoft.com/v1/workspaces/{workspace_id}/lakehouses"
​
payload = {
​
  "displayName": "LH_API1",
​
  "description": "A Lakehouse created  by Rest API"
​
}
​
​
response = requests.post(url_lh,json=payload,  headers=headers)
​
​
response.raise_for_status()
​
print (response)
​
if response.status_code in (201,202):
​
  print("Post request successful.")
​
else:
​
  print(f"Error: {response.status_code} - {response.text}")

Follow in the Fig 5 – Results the APIs executions, so far:

Fig 5 – Results

CICD

As for CICD you can find more information at this post that I am mentioning by Nimrod. Fabric CI/CD announcements- supporting new items and much more! | Microsoft Fabric Blog | Microsoft Fabric.

CRUD


If you’ve assigned a capacity to Fabric, you might consider managing costs by pausing it when not in use. In this implementation, I’m using PowerShell and the Management API since the Fabric Capacity is within the Azure portal. I stumbled upon this implementation while exploring this repository: https://github.com/Azure/azure-rest-api-specs/issues/27245

Capacity

Here you can find information how to create a capacity using Azure Portal: Manage your Fabric capacity – Microsoft Fabric | Microsoft Learn

Once created you can assign it using the Microsoft Fabric -> Admin Portal -> Workspaces -> find your workspace and Reassign Workspace (Fig 6 – Reassign and Fig 7 – Select):

Fig 6 – Reassign
Fig 7 – Select

Creating the SP for the CRUD


If you already have the capacity assigned and want to learn how to pause it, follow these step-by-step instructions to create an App/Service principal and a Secret:

Create an App:
From Microsoft portal -> open App registration -> New App ( Fig 8 – App):

Fig 8 – App

Choose a name for your App as advised by the docs with the follow configuration – Azure REST API reference documentation | Microsoft Learn (Fig 9 – AppName):

Fig 9 – AppName

Once the App is registered, open by the name chosen the App in the Portal -> App registration and copy the Application ID and Tenant ID in a separate note, we will use it later ( Fig 10 – IDs).

Fig 10-IDs

From the same App created, open Manage-> Certificate and Secrets to create the secrete ( Fig 11 – Secret):

Fig 11- Secret

Copy the secret in a separate note, we will use it later. You can’t copy this secret later, you will need to copy when you create the secret.

Next back to the Microsoft Fabric Capacities in the Azure Portal, add the App/SP with the relevant permissions using the IAM Access Control, accordingly to the docs Pause and resume your capacity – Microsoft Fabric | Microsoft Learn(Fig 11 – AddApp):

Note: Alternatively, you can also create a group, add this SP to the group and subsequently to Microsoft Fabric. This configuration is illustrated here under Power BI Docs: Automate Power BI Premium workspace and semantic model tasks with service principals – Power BI | Microsoft Learn

Fig 11 – AddApp

Use the SP information to run the Management API. with the example bellow in the Power Shell, You can find the subscription resource group name under the Microsoft Capacity in the portal, look for it in the overview page ( Fig 12 – Overview).

Fig 12 – OVerview

Pause:

$subscriptionId = "Replace by information found in the portal"
$resourceGroupName = "Replace by information found in the portal"
$dedicatedCapacityName = "Replace by information found in the portal"
$apiVersion = "2022-07-01-preview"


# Define your Azure subscription, resource group, and other parameters
$grant_type="client_credentials"
$ClientId = ""
$ClientSecret = ""


# Construct the authentication URL
$authUrl = "https://login.microsoftonline.com/<Replace by TENANT Domain>/oauth2/token"


# Request an access token
$token = Invoke-RestMethod -Uri $authUrl -Method Post -Body @{
  grant_type   = 'client_credentials'
  client_id     = $ClientId
  client_secret = $ClientSecret
  resource     = 'https://management.azure.com/'
}


##Write-Host "Constructed token: $ClientId"


# Construct the API request URL
$url = "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Fabric/capacities/$dedicatedCapacityName/suspend?api-version=$apiVersion"

#Write-Host "Constructed URL: $url"

# Include the Authorization header
$headers = @{
  'Authorization' = "Bearer $($token.access_token)"
}

# Make the request
$response = Invoke-RestMethod -Uri $url -Method Post -Headers $headers

# Output the response
$response

Resume:

$subscriptionId = ""
$resourceGroupName = "SQL-HA-RG-Li"
$dedicatedCapacityName = ""
$apiVersion = "2022-07-01-preview"


# Define your Azure subscription, resource group, and other parameters
$grant_type="client_credentials"
$ClientId = ""
$ClientSecret = ""


# Construct the authentication URL
$authUrl = "https://login.microsoftonline.com/<Replace by TENANT Domain>/oauth2/token"


# Request an access token
$token = Invoke-RestMethod -Uri $authUrl -Method Post -Body @{
  grant_type   = 'client_credentials'
  client_id     = $ClientId
  client_secret = $ClientSecret
  resource     = 'https://management.azure.com/'
}


##Write-Host "Constructed token: $ClientId"


# Construct the API request URL
$url = "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Fabric/capacities/$dedicatedCapacityName/resume?api-version=$apiVersion"

#Write-Host "Constructed URL: $url"

# Include the Authorization header
$headers = @{
  'Authorization' = "Bearer $($token.access_token)"
}

# Make the request
$response = Invoke-RestMethod -Uri $url -Method Post -Headers $headers

# Output the response
$response

Adding Permissions to the Workspace

Once you create the Workspace you can add permissions with this API. For example I am using the SP ID I just created to add as Admin in the Workspace I jsut created. Example is using Python.

principal_id = " Replace by Client ID"###Client ID

workspace_id = "Replace by Workspace ID"

url_patch= f"https://api.fabric.microsoft.com/v1/workspaces/{workspace_id}/roleAssignments/{principal_id}"

jsonPayload = {

"role": "admin"

}

response_status = requests.patch(url_patch,json=jsonPayload, headers=headers)

response_data = response_status.json()

status = response_data.get('status')

print (response_data)

Summary:

In this post, I’ve demonstrated diverse options for utilizing REST APIs in Fabric. These include creating workspaces, adding permission, dropping, creating, and executing data pipelines from that workspace and finally, how to pause/resume Fabric activities using the management API. The pause and resume would be particularly relevant for users who have capacities in Fabric and are no longer under the trial option.

Zugehörige Blogbeiträge

Fabric Change the Game: Revolutionizing Fabric with REST API, Part 1

Oktober 9, 2024 von Misha Desai

At Fabric, we’re passionate about contributing to the open-source community, particularly in areas that advance the usability and scalability of machine learning tools. One of our recent endeavors has been making substantial contributions back to the FLAML (Fast and Lightweight AutoML) project, a robust library designed to automate the tedious and complex process of machine … Continue reading “Enhancing Open Source: Fabric’s Contributions to FLAML for Scalable AutoML”

September 25, 2024 von Santhosh Kumar Ravindran

We’re excited to introduce high concurrency mode for notebooks in pipelines, bringing session sharing to one of the most popular orchestration mechanisms for enterprise data ingestion and transformation. Notebooks will now automatically be packed into an active high concurrency session without compromising performance or security, while paying for a single session. Key Benefits: Why Use … Continue reading “Introducing High Concurrency Mode for Notebooks in Pipelines for Fabric Spark”