Microsoft Fabric Updates Blog

Data Warehouse SKU Guardrails for Burstable Capacity

We are excited to announce that SKU guardrails for burstable capacity is now active for Data Warehouse and SQL Endpoint on Microsoft Fabric.

Burstable Capacity

A Fabric capacity is a distinct pool of resources that’s size (or SKU) determines the amount of computational power available. Warehouse and SQL Endpoint provide burstable capacity that allows workloads to use additional resources to achieve better performance.

For example, a typical customer workload may require more capacity while running nightly loads than at any other time in a 24-hour period. There are also high demand reporting periods that span just a few hours and sparse usage in the late evening. We understand that customer workload patterns are not uniform, so we’ve designed a solution that caters to those variable demands with bursting.

Bursting is automatic and it happens when the workloads being executed demand more resources to run optimally. Instead of running a large nightly load job on 16 capacity units (CU) and completing it in 60 minutes, bursting can run it on 48 CU and complete it in 20 minutes. It is the same amount of work, just completed faster.

Smoothing

Burstable capacity is made possible by allowing the over-usage of a capacity to simply be averaged out over time. In the Fabric capacities blog post earlier this month, it was shown how smoothing helps streamline your capacity management by allowing you to plan for average usage instead of peak.

For SQL Endpoint and Warehouse in Microsoft Fabric, we classify all activity as background so that users get the maximum smoothing period possible which is 24 hours. All of your query activity gets the same 24-hour smoothing period as your ETL jobs. This allows for large ad-hoc queries to “burst” for a short period of time and not be penalized by interactive throttling, therefore providing optimal performance.

SKU Guardrails

With the introduction of throttling earlier this month, burstable capacity should be held within reasonable constraints as related to the baseline capacity units of the SKU size purchased.

Bursting is great, but without guardrails in place, it can lead to overconsumption in a short period of time when your Fabric capacity is undersized for your workload.

For an illustration, let’s review the below scenarios where a workload consumes 88,000 capacity units and is running on a capacity with a SKU of F8.

SKU ScenarioWorkload runtimeWorkload CUCU/sBurstable Scale FactorHours before Overconsumption
F8 (No Bursting)~3 hours88,00080N/A
F8 (Bursting with no Guardrails)5 minutes88,00029436x< 1 hour
F8 (Bursting with Guardrails)30 minutes88,000496x~4 hours

With no bursting, an F8 would take 3 hours to run this workload. It can only consume 8 capacity units (CU) a second as that is the compute capacity of the SKU. However, the capacity never enters a throttling state if continuously running this workload due to overconsumption.

With bursting, but no guardrails, the workload is blazing fast and runs in 5 minutes. However, it greatly overconsumes resources at the rate of 294 CU a second and therefore hits overconsumption in less than an hour if running this workload continuously.

With guardrails in place, the workload still completes in a reasonable amount of time, and overconsumption can easily be smoothed by the natural peaks and valleys of daily usage.

Note that some current SQL Endpoint and Warehouse workloads currently in public preview may experience a performance degradation if the workspace they are running on is assigned to a capacity that is undersized.

For more information on SKU guardrails for burstable capacity, please check out our documentation https://learn.microsoft.com/fabric/data-warehouse/burstable-capacity

To learn more about Fabric Capacity, see the blog Fabric Capacities – Everything you need to know about what’s new and what’s coming | Microsoft Fabric Blog | Microsoft Fabric

Related blog posts

Data Warehouse SKU Guardrails for Burstable Capacity

July 17, 2024 by Ambika Jagadish

In the rapidly evolving world of generative artificial intelligence, historical data plays a significant role in influencing the decision-making process and shaping organizational strategies. Data retention within data warehouse refers to the practice of preserving and managing previous iterations of the data encompassing any inserts, updates or deletes made to the warehouse for a specified … Continue reading “Announcing the General availability of time travel and 30 days of data retention in Fabric Warehouse”

July 17, 2024 by Sowmya Sivaraman

In today’s rapidly evolving data management landscape, maintaining the resilience and continuity of your data infrastructure is essential. Unplanned system failures and scheduled maintenance alike demand the ability to restore data warehouses swiftly and seamlessly. This capability is no longer just a feature – it’s a critical necessity in modern analytics environments. A quick and … Continue reading “Seamless Data Recovery through Warehouse Restoration within Fabric Query Editor”