Microsoft Fabric Updates Blog

Fabric Espresso – Episodes about Performance Optimization & Compute Management in Microsoft Fabric

For the past 1.5 years, the Microsoft Fabric Product Group Product Managers have been publishing a YouTube series featuring deep dives into Microsoft Fabric’s features. These episodes cover both technical functionalities and real-world scenarios, providing insights into the product roadmap and the people driving innovation. With over 80+ episodes, the series serves as a valuable resource for anyone looking to understand and optimize their use of Microsoft Fabric.

All episodes are available at  https://aka.ms/fabric-espresso making it easy to explore the entire catalog. However, to enhance the learning experience, we are launching a short series of blog posts that will categorize the episodes into thematic groups, providing explanations and key takeaways for each.

Fabric Espresso & Performance Optimization & Compute Management

This week, we focus on Performance Optimization & Compute Management in Microsoft Fabric. Below is a curated list of episodes that explore various techniques for optimizing compute resources, tuning queries, and improving efficiency within Microsoft Fabric.

Key Episodes on Performance Optimization & Compute Management:

  1. High Concurrency Mode for Notebooks in Pipelines for Fabric Spark
    • Learn how shared, high-performance sessions cut down job runtimes dramatically.
  2. ML based Autotune for Apache Spark Jobs in MS Fabric performance optimization for recurrent jobs
    • Discover how predictive autotuning refines Spark configurations for optimized performance.
  3. Native execution engine for Apache Spark in Fabric
    • Explore the vectorized execution engine designed to boost query speed and efficiency.
  4. Spark Compute in Fabric Data Engineering and Data Science – Starter Pools vs Custom Pools Unveiled!
    • Compare resource provisioning options and understand the trade-offs between quick-start and tailored compute pools.
  5.  Fabric Apache Spark Autotune and Run Series Job Analysis in Monitoring Hub
    • Gain insights into automated tuning techniques and job performance diagnostics.
  6. Fabric Apache Spark Jobs monitoring capabilities – Resource Usage
    • Understand how detailed monitoring helps identify performance bottlenecks and resource inefficiencies.
  7. Fabric Spark Compute Capabilities – Azure VM’s and their impact on performance
    • See how leveraging Azure VM configurations can drive enhanced Spark performance.
  8. Performance best practices
    • Review essential strategies for optimizing query performance in your workspace.
  9. Microsoft Fabric Capacity Smoothing and Data Warehouse Throttling
    • Learn how capacity smoothing and throttling techniques ensure consistent performance under load.
  10. Caching in data warehousing
    • Discover how in-memory and SSD caching can significantly reduce query latency.
  11. Performance at Scale with Microsoft Fabric: Concurrency!
    • Explore how Fabric handles concurrency to maintain high performance even at scale.
  12. Performance at Scale with Microsoft Fabric: Query Optimizations!
    • Dive into techniques for optimizing query execution to boost efficiency.
  13. Performance at Scale with Microsoft Fabric: Query Processing!
    • Understand the underlying mechanics of query processing and performance tuning.

Related blog posts

Fabric Espresso – Episodes about Performance Optimization & Compute Management in Microsoft Fabric

July 17, 2025 by Sunitha Muthukrishna

Microsoft Fabric has introduced new features for its User Data Functions (UDFs), enhancing Python-based data processing capabilities within the platform. These updates include support for asynchronous functions and the use of pandas DataFrame and Series types for input and output, enabling more efficient handling of large-scale data. • Async function support: Developers can now write async functions in Fabric UDFs to improve responsiveness and efficiency, especially for managing high volumes of I/O-bound operations, such as reading files asynchronously from a Lakehouse. • Pandas DataFrame and Series integration: UDFs can accept and return pandas DataFrames and Series, allowing batch processing of rows with improved speed and performance in data analysis tasks. An example function calculates total revenue by driver using pandas groupby operations. • Usage in notebooks: These functions can be invoked directly from notebooks using pandas objects, facilitating efficient aggregation and analysis of large datasets interactively within Microsoft Fabric. • Getting started and benefits: Users can enable these features by updating the fabric-user-data-functions library to version 1.0.0. The enhancements reduce I/O operations, enable concurrent task handling, and improve performance on datasets with millions of rows.

July 15, 2025 by Brian Kernan

In August 2025, Microsoft Fabric will introduce workspace access limits to improve service quality, reliability, and to encourage workspace access control hygiene. This limit will be permanent once it is rolled out – each Fabric & Power BI workspace will be limited to a maximum of 1,000 users or groups in workspaces roles (Admin, Member, … Continue reading “Introduction of access limits in a Fabric workspace”