The Art of Data warehouse recovery within Microsoft Fabric
In the ever-evolving landscape of data management, data warehouses play a pivotal role in storing and processing vast volumes of data. This enables organizations to derive meaningful insights and make well-informed data driven decisions. The necessity for a quick and reliable data warehouse recovery solution becomes increasingly crucial to not only safeguard against data corruption but also to ensure business continuity. We are thrilled to preview with you the ability to perform the restore-in-place of the data warehouse within Microsoft Fabric.
Restore in-place is an essential part of data warehouse recovery which allows to restore the data warehouse to a prior known reliable state by replacing or over-writing the existing data warehouse from which the restore point was created. Restore points are recovery points that can be leveraged to restore the data warehouse. From the moment the data warehouse is created, system generated restore points are created every 8 hours. There will be a total of 42 system generated restore points at any given point in time. Furthermore, you have the flexibility to create any number of user-defined restore points aligned with your specific business or organizational recovery strategy. Both the system-generated and user-defined restore points come with a retention period of seven calendar days, after which they will expire. Restore in-place can be performed using a REST API call.
The restore in-place demonstrated below utilizes the capabilities of Postman to create user-defined restore points, list, and delete as well as perform restore in-place of the data warehouse within Microsoft Fabric.
Prerequisites:
The following prerequisites are required to perform restore in-place.
- Navigate to the data warehouse within Microsoft Fabric in focus and obtain the workspace and data warehouse GUID.
2. Create an account and a workspace within Postman for calling the REST API. Then, add a ‘New Collection’ and give it the name of your choice.
3. To send requests, a bearer token is required for authentication. Obtain bearer token by clicking try it located above the ‘URI parameters’ in the page. Once try it is clicked, it navigates to ‘REST API Try It’ page. Scroll down to ‘Request Preview’ and copy the bearer token present in ‘Authorization’ line.
Note: Copy the token after the keyword ‘Bearer’.
- Creating user-defined restore point
User-defined restore points are the data warehouse recovery points that is created by the users who have ‘Administrator’ privileges to the Fabric workspace.
- Create a ‘New request’ within the Collection in Postman.
- Submit a ‘POST’ request through Postman using the below URL template by replacing the values as shown below:
https://{baseurl}/v1.0/myorg/groups/{workspaceGUID}/datawarehouses/{datawarehouseGUID}/
- baseurl: api.powerbi.com
- workspaceGUID : The workspace GUID in which the data warehouse resides. Refer to the prerequisites section.
- datawarehouseGUID: The GUID- of the data warehouse for which user-defined restore point is to be created. Refer to the prerequisites section.
3. Paste the bearer token in the ‘Authorization’ tab by selecting the ‘Bearer Token’ as the type of the authorization from the dropdown menu.
4. Select ‘raw’ and ‘JSON’ format in the ‘body’ of the request and insert the following in the body tab.
The Body tab of the request contains the following JSON:
{
"commands":[
{
"$type":"WarehouseCreateRestorePointCommand"
}
]
}
5. The user-defined restore point gets created once the ‘POST’ request is submitted. Click ‘Send’ to submit the request.
2. Listing system-generated and user-defined restore points.
To list all the system-generated and user-defined restore points, repeat the ‘creating user defined restore point’ steps. Replace the ‘body’ of the request with the below.
The Body tab of the request contains the following JSON:
{
"commands":[
{
"$type":"WarehouseListRestorePointsCommand"
}
]
}
All the restore points that are associated with the data warehouse are returned when the ‘POST’ request is submitted. Click ‘Send’ to submit the request.
3. Deleting the user-defined restore point
The user-defined restore points that are created for the data warehouse can be deleted anytime by the workspace administrator. The restore points that are in scope of deletion can be identified by listing the restore points as described in the previous step. The value associated with ‘createTime’ uniquely identifies the restore point. Any number of user-defined restore points can be deleted with a single request. Once the restore point is identified, repeat the ‘create user-defined restore point’ steps and replace the ‘body’ of the request with the below.
The Body tab of the request contains the following JSON:
{
"commands":[
{
"$type":"WarehouseDeleteRestorePointsCommand",
"RestorePointsToDelete": [
"{RestorePoint}"
]
}
]
}
The restore point that is submitted through the POST request gets deleted from the list of data warehouse restore points. Click ‘Send’ to submit the request.
4. Restore the data warehouse using the restore point
The data warehouse can be restored to any previous state by using either the system-created or user-defined restore points. All the ongoing transactions will be terminated upon submission of this request. The restore point that can be leveraged for restore in place can be identified by listing the restore points as described in the previous step. The value associated with ‘createTime’ uniquely identifies the restore point. Once the restore point is identified, repeat the ‘create user-defined restore point’ steps and replace the ‘body’ of the request with the below.
The Body tab of the request contains the following JSON:
{
"commands":[
{
"$type":"WarehouseRestoreInPlaceCommand",
"RestorePoint":"{RestorePoint}"
}
]
}
The data warehouse will be restored as of the point-in-time associated with the restore point that is leveraged to perform restore in place. Click ‘Send’ to submit the request.
To poll for the status of the restore, obtain the ‘batchID’ that gets generated when the POST request is submitted. Append ‘batches/ batchID’ at the end of the request URL as shown below. Empty the body and submit a ‘GET’ request.
Once the ‘GET’ request is submitted, the ‘progressState’ will return the status of the restore. A status value of ‘success’ represents successful completion of the data warehouse restore within Microsoft Fabric.
Conclusion
Data warehouse restores play a pivotal role in effective data management, ensuring organizations can swiftly recover from unforeseen challenges. They embody a proactive investment in organizational resilience. In an era where the resilience of data corresponds with resilience of business, having a well-established restore plan for data warehouse is crucial for organizations. Explore the capability to perform restore-in-place using Rest API.
