Microsoft Fabric Updates Blog

Introducing High Concurrency Mode in Notebooks for Data Engineering and Data Science workloads in Microsoft Fabric

We are excited to announce a new high concurrency mode in Fabric for Data Engineering and Data Science. This allows users to share Spark compute across multiple notebooks within a workspace which means that you can run multiple Spark notebooks simultaneously on the same Spark session without compromising performance or security when paying for a single session. High concurrency mode offers an instant run experience avoiding session start delays and ~30X faster session start experience for the shared notebooks when running on custom pools.

What is High Concurrency Mode?

High concurrency mode allows sharing of Spark compute across multiple notebooks and allows their queries to execute in parallel. In this shared mode, the resources and configurations of each notebook are isolated from each other. As the session sharing is always within a single user boundary, users cannot access or modify the data or variables of another user’s high concurrency session.

High concurrency mode also leverages FAIR allocation to optimize the resource utilization and performance of the notebooks and ensures that each notebook gets a fair share of the executors available for the Spark application.

Why Use High Concurrency Mode?

High concurrency mode offers several benefits for Fabric Spark users, such as:

  • Faster and easier Spark session start: You don’t have to wait for the Spark pool to spin up or configure the node sizes when you start a Spark session. As the session is already warmed up and running, attaching a new notebook to an existing Spark session gives a session start experience within ~5 seconds. You can also use custom Spark pools, which allow you to size the nodes, enable autoscaling, and dynamically allocate executors based on your Spark job requirements and with custom pools you would get a 30X faster session start experience for shared notebooks.
  • Enhanced security and isolation: You can ensure that each user or query has its own isolated Spark session, which prevents data leakage or tampering.
  • Do more by paying less: Achieve better compute cost savings by sharing a single session across multiple notebooks for your Data Engineering or Data Science workloads and only get billed only for the single session.

How to Enable High Concurrency Mode?

To enable high concurrency mode for your Fabric Spark workspace, you need to follow these steps:

  1. Go to the workspace settings in your Fabric workspace.

A screenshot of a computer

Description automatically generated

  1. Navigate to the Data Engineer/Science section.
  2. Select the Spark Compute menu
  3. Enable the High Concurrency Mode Option if its disabled (This option should be enabled by default for all Fabric Workspaces)

4. Save your changes.

Once you enable high concurrency mode, you can run your notebooks in High Concurrency mode from the notebook menu ribbon.

To learn more about using high concurrency in notebooks read: Sharing Spark Compute across Notebooks with High Concurrency Mode in Fabric.

For more information on high concurrency mode, please read Overview of High Concurrency Mode in Microsoft Fabric

Related blog posts

Introducing High Concurrency Mode in Notebooks for Data Engineering and Data Science workloads in Microsoft Fabric

July 1, 2025 by Ye Xu

Copy job has been a go-to tool for simplified data ingestion in Microsoft Fabric, offering a seamless data movement experience from any source to any destination. Whether you need batch or incremental copying, it provides the flexibility to meet diverse data needs while maintaining a simple and intuitive workflow. We continuously refine Copy job based … Continue reading “Simplifying Data Ingestion with Copy job – Incremental Copy GA, Lakehouse Upserts, and New Connectors”

June 25, 2025 by Patrick LeBlanc

Welcome to the June 2025 update. The June 2025 Fabric update introduces several key enhancements across multiple areas. Power BI celebrates its 10th anniversary with a range of community events, contests, expert-led sessions, and special certification exam discounts. In Data Engineering, Fabric Notebooks now support integration with variable libraries in preview, empowering users to manage … Continue reading “Fabric June 2025 Feature Summary”