Fabric February 2025 Feature Summary
Welcome to the Fabric 2025 update!
There are a lot of exciting features for you this month! Here are some highlights: In Power BI, Explore from Copilot visual answers which lets you do easy ad-hoc exploration. In Data Warehouse, Browse files with OPENROWSET (Preview) and Copilot for Data Warehouse Chat (Preview). For Data Science, AI Skill is now conversational.
These are just some of the great features this month, keep reading to learn about all of what’s happened in Fabric this month.
Save your spot at Microsoft Fabric Conference 2025
Workshops are selling out, rooms are selling out, flights are selling out. Don’t wait! Register now for the biggest-ever Fabric learning event – March 31 to April 2, Las Vegas. Need help convincing your boss? Use this letter.
We’re also thrilled to announce exclusive pricing for Fabric users! Use code FABINSIDER to save $400 and reserve your spot. Limited tickets available. Valid for new registrations only. Cannot be combined with any other offers.
Limited-time 50% discount on Exam DP-700
Get ready to fast-track your career in data and AI by earning your Microsoft Certified: Fabric Data Engineer Associate certification (Exam DP-700). There’s no faster way to climb the ladder than with industry-recognized Microsoft certifications. Don’t wait to claim your discount—this offer ends on March 31, 2025.
LIVE NOW! The first-ever Power BI DataViz World Championships!
Data enthusiasts from around the world are battling it out and flexing their Power BI skills like true data wizards! The top four winners will earn their spot in the Grand Finale at the MGM in Las Vegas—LIVE at the Fabric Community Conference!
Think you have what it takes? There are two more chances for you to show off your skills, connect with an amazing community, and win epic prizes! Don’t miss out on this legendary event!
Contents
- Fabric Platform
- Data Engineering
- Managed private endpoint and private link support for Native Execution Engine in Spark
- New Capacity settings for better compute governance on your Data Engineering Workloads
- T-SQL Notebook enhancements
- Full CI/CD support for T-SQL notebook including Git and Deployment pipeline
- New Lakehouse samples
- Monitoring for High Concurrency
- Notebook Snapshots enhancements
- Fabric Spark Resource Analysis (Generally available)
- Real-Time Intelligence
- Configure latency of Eventhouse OneLake availability
- Real-Time Dashboard & Power BI templates for Eventhouse Monitoring
- New Real-Time Dashboard customization features: more control, better usability, and improved performance
- New and improved Data Source Tree in KQL Queryset
- Data exploration is now available for raw KQL table data
Power BI
General
Upgrade Power BI Desktop to 64-bit version
The 32-bit version of Power BI Desktop will no longer be supported after 6/30/2025. Upgrade to the 64-bit version Power BI Desktop to keep receiving updates and support.
Upgrade Power BI Desktop to February 2025 version
To continue using Report view Copilot chat pane, you will need to upgrade to February 2025 version. Report view Copilot chat pane might not work as expected for versions before February 2025 after 4/30/2025.
Copilot and AI
Copilot in the Power BI mobile apps: now available on iPads and Android tablets (Preview)
Copilot support is now available in the Power BI Mobile apps on iPads and Android tablets, extending the functionality of AI to more mobile devices. Introduced a few months ago for phones, Copilot in the Power BI Mobile apps empowered users to quickly analyze data, gain insights, and make informed decisions on the go. Now, the same powerful capabilities have come to iPads and Android tablets, making it easier than ever for you to explore your data anytime, anywhere.
Getting started with Copilot on your mobile app is easy, simply tap the Copilot button located in the report header (for reports that meet Copilot requirements in Power BI). From there, you can choose whether to receive a summary or uncover insights. Copilot will deliver a response based on your request, which you can then copy, share, or continue interacting with. Suggestions at the bottom of the screen help you refine your request or create new ones, making it simple to explore your data further.
Updated section – ‘How Copilot arrived at this’
We’ve heard an overwhelming amount of feedback that users would like more transparency into how Copilot generates visual answers. This month, we’ve revamped our previous ‘show reasoning’ section to a new detailed section we’re calling ‘How Copilot arrived at this’.
When receiving a visual answer from Copilot, expand this new section to dig into what data (fields/measures) and filters were used to generate the answer. You can also click on the field to see more information like the aggregation or home table.
This is especially helpful when there could be multiple fields with the same name, like having multiple Dates in your data, verifying Copilot picked the correct one.
As a reminder, Copilot does not currently incorporate existing filters from the report when generating a visual answer. This list only contains filters that were used at the time of generating the answer.
This is a step towards increasing transparency in how Copilot understands and generates responses. If you have any additional suggestions, please share your thoughts in the comments.
Explore from Copilot visual answers
Sometimes in Copilot, you may want to do a bit more with the visual answer, like drill down, filter it further, maybe even swap it to a different visual type. Until now, the only way to take action on the Copilot generated visual was to add it to the page. This of course was limited to ‘edit mode’ only. But we have great news! This month we’ve replaced the previous ‘expand view’ action with our fully interactive Explore feature. Even better, you can use the feature in both read and edit modes of a report.
With Explore, quick actions are now possible from Copilot. Simply open the Explore experience to filter, sort, swap fields, or change visual types easily for ad-hoc exploration.
You can choose to save the new explore visual to a report or an exploration to come back to.
Note: Saving the new visual to the original report is not yet available.
Explore does have certain limitations, some users may still see the ‘expand view’ button if requirements to use Explore are not met.
Reporting
Conditional formatting support for visual calculations (Preview)
Another month, more updates to visual calculations. This month we are introducing another customer request: support for conditional formatting! You can now use visual calculations to set up conditional formatting rules for columns and measures on your visuals. Also, you can now apply condition formatting rules to visual calculations.
Once you open the conditional formatting dialog you will notice there is a new section that shows the data that’s on your visual. This section includes any column, measure and visual calculations on your visual and will include any hidden fields as well.
For example, here I have a simple visual that shows sales by quarter using a Total Sales measure and the Fiscal Quarter column. I have added a visual calculation that for each quarter calculates the difference with the first quarter:
VSFirst = [Total Sales] - FIRST( [Total Sales] )
I have hidden the VSFirst calculation and am going to use it to set a conditional background color for the fiscal quarter. In the conditional formatting dialog, we see the data in this visual section:
All your model data is still accessible, it is in the ‘All data’ section.
Next, select VSFirst visual calculation and set up the rule as usual:
And the conditional formatting is applied:
As mentioned, you can also format the visual calculations themselves, as done here with the same visual calculation that was made visible to show the result:
Learn more about visual calculations in our documentation and please let us know what you think!
Publish to Web Support for the Azure Maps visual (Coming soon)
We’re excited to announce that Azure Maps support for Publish to Web reports is rolling out. With this update you don’t need to take any extra steps to take advantage of your Azure Map visuals in Publish to Web reports. Just get your embedded code as you normally would using the Publish to Web (public) option under the Embed Report section of the File menu.
From there, you can use the embedded report as you normally would, now with your maps showing.
This update is in the process of rolling out, so you should see the change within the next couple of weeks.
Modeling
Live edit of semantic models in Direct Lake mode with Power BI Desktop – updates (Preview)
Improved modeling performance
We have significantly enhanced the modeling performance for live editing of Direct Lake semantic models using Power BI Desktop. Upgrading to the latest version will result in at least a 50% improvement in each modeling change.
More details on the feature, including requirements, considerations, and limitations can be found in the documentation. We highly value your feedback on this feature and encourage you to share it through our feedback form or the Power BI Community.
Data connectivity
Performance improvement in Snowflake connector (Preview)
We sincerely appreciate your trial and feedback on the new Snowflake connector released as preview in January. This month, we have enhanced this connector with performance improvements and bug fixes to provide a smoother experience. We invite you to upgrade to the latest version to evaluate these enhancements. Learn more about the Snowflake connector from the Snowflake connector documentation.
Use the official ODBC driver to connect to Vertica database (Preview)
This month, we are introducing a new option for getting data from the Vertica database using the official Vertica ODBC driver, currently available in preview. This feature allows you to take advantage of the native client tool provided by the data source. More details about this feature, including the driver installation and configuration instructions, can be found in the documentation.
Visualizations
Drill Down Donut PRO by ZoomCharts
Donut PRO is more than just a beautiful and customizable donut chart visual. It is also a way to instantly make your reports more interactive and user-friendly. Simply click directly on a slice to drill down a level and reveal more details. Donut PRO will also seamlessly cross-filter with other visuals, allowing users to quickly filter data and instantly uncover valuable insights that empower confident business decisions.
Main features:
- Interactivity: Donut PRO provides equally great experience for mouse and touch input – click/tap to drill down, hold to select, triple click/tap to invert selection.
- Animated navigation: Each user interaction has smooth animation feedback that aids with navigation and shows exactly what changed in the chart.
- Multi-level drill down: Create up to nine levels of hierarchy and drill down with just a click.
- Automatic ‘others’ slice: Declutter the chart by grouping smaller slices into one larger slice. Simply click to reveal its contents.
- Cross-chart filtering: Enhance data storytelling by dynamically filtering data across multiple visuals.
- Full customization: Finetune your chart with more than 80 settings, including donut appearance, slice colors, detail labels, legend, and tooltip customization.
- Get on AppSource
Heatmap by Powerviz
The Powerviz Heatmap visualizes data density and value distribution using color gradients, transforming complex datasets into an intuitive matrix format. It is ideal for identifying patterns, correlations, and outliers, aiding data-driven decision-making.
Key features:
- Shapes: Choose from Default, Fixed, or Diverging built-in shapes.
- Data colors: Offers 30+ palettes, color-blind mode, and custom color options.
- Data labels: Select from different styles available or add custom labels with formatting.
- Grid lines: Improve data readability with visual grid lines.
- Reference lines: Highlight key points using X-axis and Y-axis lines.
- Totals: Display row and column totals with advanced bar customization.
- Null values: Customize null values (Blank /NA / Null/ ”0”) and their styling.
- Small multiples: Split visuals into smaller ones using selected fields.
- Conditional formatting: Spot outliers with Value/ Percentage/ Ranking based rules.
- Ranking: Filter Top/Bottom N rows or columns.
Other features included Axis Settings, Grid View, Sorting, Show Condition and more.
Business use cases:
Sales Analysis, Performance Evaluation, Risk Assessment.
- Try Heatmap visual for FREE from App Source
- Check out all features of the visual: Demo
- Step-by-step instructions: Documentation
- YouTube video: Video Link
- Learn more about visuals: https://powerviz.ai/visuals
- Follow Powerviz: https://lnkd.in/gN_9Sa6U
Maximize data efficiency with accoMASTERDATA writenback for Power BI
Unlock the power of seamless masterdata management with accoMASTERDATA for Power BI. Streamline your data processes and enhance your writeback capabilities for smarter business decisions.
Key features:
- Effortless writeback: Easily configure writeback to SQL and FABRIC databases. Track changes in a dedicated log for full transparency.
- Full control: Create, update, and delete rows in Masterdata tables to keep your data dynamic and accurate.
- Custom validation: Use regular expressions for custom data validation and ensure integrity.
- Dropdown list validation: Restrict input to predefined options for consistent data entry.
- Conditional formatting & rules: Ensure quality data with powerful validation and formatting tools.
- Enhanced copy-paste: Copy and paste from Excel directly into Power BI while maintaining validation rules.
- Customizable themes: Create visually stunning reports with flexible theme options.
- Tags & column grouping: Organize data with multi-selected tags and improve navigation with collapsible column sections.
Integrating accoMASTERDATA with Power BI will supercharge your data management, improve reporting, and streamline your operations. Start transforming your data handling today!
Visit the accoTOOL website for more.
Other
The OneLake catalog is now available in Microsoft Teams
The OneLake catalog is now integrated into the Power BI app experience in Microsoft Teams, enabling seamless data discovery and exploration within the Microsoft Office ecosystem. This integration empowers business users and professionals to interact with their data – explore it, take actions, and more – all without leaving Teams.
Monitoring Hub enhancement for Semantic models
In Fabric Monitoring Hub, you can centrally monitor Microsoft Fabric activities. It displays refresh activities for all semantic models, each showing one line with the status of the last refresh.
Previously, clicking on an activity name in the Monitoring Hub directed you to the semantic model detail page. This month, we’re introducing the Semantic model refresh detail page. This new page shows comprehensive details of a selected refresh activity, including capacity, gateway, start and end times, error details, and multiple refresh attempts.
For each refresh attempt, you can view the execution metrics by clicking on the ‘Show’ link in the ‘Execution details’ column. This information can assist with troubleshooting or optimizing the semantic model refresh. Previously, this data was accessible through Log Analytics or Fabric Workspace Monitoring.
It’s also possible to link refresh details from external applications. The semantic model refresh detail page can be accessed from other locations by constructing a URL with the workspace, semantic model, and refresh ID:
https://app.powerbi.com/groups/{workspaceId}/datasets/{semanticModelId}/refreshdetails/{refreshId}
For instance, this Fabric Notebook uses semantic link sempy and Power BI API Get Refresh History to create a refresh detail URL for each run of a semantic model:
import sempy
import sempy.fabric as fabric
import pandas as pd
workspaceId = "[Your Workspace Id]"
semanticModelId = "[Your semantic model Id]"
client = fabric.FabricRestClient()
response = client.get(f"/v1.0/myorg/groups/{workspaceId}/datasets/{semanticModelId}/refreshes")
refreshHistory = pd.json_normalize(response.json()['value'])
refreshHistory["refreshLink"] = refreshHistory.apply(lambda x:f"https://msit.powerbi.com/groups/{workspaceId}/datasets/{semanticModelId}/refreshdetails/{x['requestId']}", axis=1)
displayHTML(refreshHistory[["requestId", "refreshLink"]].to_html(render_links=True, escape=False))
More details on the feature can be found in the data refresh documentation.
Fabric Platform
Fabric API update: control cross-region deployments in Fabric Deployment pipelines
We are excited to announce a significant update to the Fabric Deployment pipelines APIs, which now includes a new feature for managing cross-region deployments. This change is designed to enhance security and compliance while providing developers with greater control over their deployment processes.
What’s new?
The latest update introduces a new flag in the fabric deployment request API that allows developers to specify whether cross-region deployments are permitted. This flag, known as allowCrossRegionDeployment, must be explicitly set by the developer to enable cross-region deployments in fabric. By default, this flag is set to false for security reasons.
Why the change?
Cross-region deployments can pose security and compliance risks, especially in highly regulated environments. By requiring developers to explicitly state whether cross-region deployments are allowed, we aim to mitigate these risks and ensure that deployments adhere to organizational policies and regulatory requirements.
How does it work?
When creating or updating a deployment pipeline, developers can include the allowCrossRegionDeployment flag in their API request. If this flag is set to false (the default value), any attempt to deploy across regions will be blocked. If set to true, the deployment will proceed, provided all other conditions are met.
Here is an example of how to use the new flag in an API request:
{
"options":{
"allowCrossRegionDeployment": true
}
}
What is the impact?
The introduction of the allowCrossRegionDeployment flag in the Fabric Deployment pipelines APIs brings several changes that impact existing deployment pipelines. Here are the key points to consider:
- Default behavior: By default, the allowCrossRegionDeployment flag is set to false for security reasons. This means that unless explicitly set to true, cross-region deployments will be blocked.
- Explicit permission required: Developers must now explicitly set the allowCrossRegionDeployment flag to true in their deployment requests if they wish to enable cross-region deployments. This change ensures that cross-region deployments are intentional and comply with organizational policies.
- Impact on existing pipelines: Existing deployment pipelines that do not include the allowCrossRegionDeployment flag will default to blocking cross-region deployments. Developers will need to update their pipelines to include this flag if cross-region deployments are required.
- Error handling: If a deployment request attempts to perform a cross-region deployment without setting the allowCrossRegionDeployment flag to true, the deployment will fail, and an error message will be returned. This message will inform the user that the deployment failed because cross-region deployment is not allowed.
- Security and compliance: This update enhances security and compliance by ensuring that cross-region deployments are explicitly authorized. It helps organizations adhere to regional data residency requirements and other regulatory guidelines.
Overall, the introduction of the allowCrossRegionDeployment flag provides greater control over cross-region deployments, ensuring that they are secure and compliant with organizational policies. Developers will need to update their existing deployment pipelines to include this flag if cross-region deployments are necessary.
Permission tab in the OneLake catalog
For data owners, security and compliance are top priorities. To support this, we’re expanding the OneLake catalog’s detailed view of all item types to include permission management.
Users with the necessary privileges will be able to view and manage access to items directly within the catalog. This feature will provide a more efficient and streamlined way to manage permissions without leaving the item’s context.
Govern your individual data estate in OneLake catalog (Preview)
We are excited to introduce as preview a centralized data governance experience in the OneLake catalog.
In this tab, data owners can view aggregated insights on the items they created, consider improving their governance by taking recommended actions, and access more information along with all available tools in Fabric. This functionality facilitates data creators in ensuring their items are secure and compliant with organizational policies.
Fabric Capacities Management APIs
In the ever-evolving landscape of cloud computing, the introduction of new application programming interfaces (APIs) can significantly enhance the capabilities and efficiency of developers. We are excited to announce the availability of a new set of ARM APIs designed for comprehensive management of Fabric capacities. These APIs are tailored to streamline operations, enable precise control, and provide unprecedented flexibility in managing Fabric resources.
Overview of the new ARM APIs
The new APIs for Fabric capacities management encompass a wide range of functionalities. Below, we provide an overview of each API and its primary operations:
Check Name Availability: Implements local CheckNameAvailability operations to verify if a particular name is available for use. This API is crucial for ensuring that your chosen names for Fabric capacities are unique and compliant with naming conventions.
Create Or Update: This API allows for the creation or updating of a FabricCapacity. It provides developers with the flexibility to either establish a new Fabric capacity or modify an existing one based on their evolving needs.
Delete: The Delete API facilitates the removal of a FabricCapacity. This operation is essential for maintaining an optimal and efficient resource environment by eliminating obsolete or unnecessary capacities.
Get: This API retrieves details about a specific FabricCapacity. It allows developers to access comprehensive information about their Fabric capacities, ensuring transparency and informed decision-making.
List By Resource Group: Lists all FabricCapacity resources within a specified resource group. This functionality is particularly useful for organizing and managing resources in a structured manner.
List By Subscription: This API lists FabricCapacity resources by subscription ID, offering a bird’s-eye view of all capacities under a particular subscription. It aids in efficient subscription-level resource management.
List Skus: Lists eligible SKUs for the Microsoft Fabric resource provider. This API enables developers to explore and select from various SKUs that best meet their requirements.
List Skus For Capacity: Similar to the previous API, but specifically lists eligible SKUs for a particular Microsoft Fabric resource. This ensures that developers can tailor their capacities with the most appropriate SKUs.
Resume: This API resumes the operation of a specified Fabric capacity instance. It is vital for restoring the functionality of capacities that have been temporarily suspended.
Suspend: Conversely, the Suspend API allows for the temporary suspension of a Fabric capacity instance. This can be useful for maintenance or other operational considerations.
Update: The Update API provides the means to modify an existing FabricCapacity, ensuring that capacities can be adjusted and optimized as needed.
Benefits of the New APIs
The introduction of these new ARM APIs offers several significant benefits to developers and organizations alike:
Enhanced Control and Flexibility
These APIs offer fine-grained control over Fabric capacities, allowing developers to create, update, suspend, and resume capacities as needed. This flexibility ensures that resources can be dynamically managed to meet changing demands and operational requirements.
Improved Resource Management
With APIs such as List By Resource Group and List By Subscription, developers can easily organize and manage their resources. This structured approach to resource management aids in maintaining an efficient and optimized cloud environment.
Streamlined Operations
The ability to quickly check name availability, list eligible SKUs, and retrieve detailed information about Fabric capacities simplifies the operational workflow. Developers can now perform these tasks with ease, reducing the time and effort required for resource management.
Seamless Integration
These APIs are designed to integrate seamlessly with existing ARM tools and processes. This ensures that developers can leverage the new functionalities without the need for significant changes to their existing workflows.
More information can be found in the Fabric Capacities documentation and Fabric REST API documentation
OneLake
Announcing support for storage accounts that are behind a firewall for Mirrored Azure Databricks catalog items (Preview)
A significant number of enterprise customers have their ADLS storage accounts behind a firewall, which houses their Unity Catalog data. Previously, Mirrored Azure Databricks catalog items were unable to access storage accounts behind a firewall, preventing these customers from accessing their catalog data via Fabric.
We are excited to announce the preview of support for storage accounts behind a firewall for Mirrored Azure Databricks catalog items. This new feature enables enterprise customers to securely access their Unity Catalog data, even when it is stored in ADLS accounts protected by a firewall.
- Navigate to the Network Security tab once you have picked a catalog that you want to access from Fabric.
2. Select an existing ADLS connection if you have one configured or create a new one. Authorization types supported are Workspace Identity (recommended), Organizational Account, and Service principal.
3. Provide access rights to the ADLS account based on the authentication type you picked in the previous step.
4. Firewall enables your ADLS storage account using Trusted Workspace Access.
Data Engineering
Managed private endpoint and private link support for Native Execution Engine in Spark
Organizations heavily rely on cloud data platforms for processing data at scale and enterprise operating in healthcare, financial and other domains where data security becomes critical requirement enforcing strict data security standards on all layers. One of the primary approaches has been through blocking public access from untrusted networks using Private Links for inbound access restrictions.
Now when it comes to outbound access enterprise also stores these sensitive data in data sources behind a firewall and would want their analytics systems to connect using a secure channel like managed private endpoints to block public access on these data sources. We are excited to announce the support for Private Links and Managed Private Endpoints for Native Execution Engine which is Fabric for your data engineering workloads.
Native Execution Engine is the vectorized engine optimizes the performance and efficiency of your Spark queries by running them directly on your lakehouse infrastructure. The engine’s seamless integration means it requires no code modifications and avoids vendor lock-in. It supports Apache Spark APIs and is compatible with Runtime 1.3 (Apache Spark 3.5) and works with both Parquet and Delta formats.
Users can enable Native Execution Engine from the Acceleration tab in the Environment item.
With this support users can now run their jobs using Native Execution Engine in their tenants where they have enabled Private Links and run their Spark Jobs using Native Execution Engine when they are connecting to data sources using Managed Private Endpoints.
Learn more about Native Execution Engine from our documentation: Native execution engine for Fabric Spark.
New Capacity settings for better compute governance on your Data Engineering Workloads
We have added new controls as part of the Data Engineering/Science Capacity settings including the option to Disable Starter Pool for workspaces to help capacity admins have better compute restrictions on workspaces attached to their Fabric Capacity.
Use the following steps to manage the Data Engineering/Science settings for Microsoft Fabric capacity:
1. Select the Settings option to open the setting pane for your Fabric account.
2. Select Admin portal under Governance and insights section.
3. Choose the Capacity settings option to expand the menu and select Fabric capacity tab. Here you should see the capacities that you have created in your tenant. Choose the capacity that you want to configure.
4. In the Spark Settings, disable ‘Customized Workspace Pools’. This option enables Capacity administrators to prevent Workspace administrators from creating and managing Spark compute on their Workspaces.
5. Now by selecting the option to Disable Starter Pool for the workspace, capacity admins can also disable Starter Pools for the workspaces, allowing workspace users to utilize Capacity Pools created based on their workload requirements for better controls over resource governance.
T-SQL Notebook enhancements
We’ve made some updates to incremental updates the T-SQL Notebook, including:
- More T-SQL code snippets
- Open the full data set within Excel
- Traceability of Primary warehouse
- CI/CD support
Inside the Notebook Code snippet library, a new TSQL category is added. You can find a list of T-SQL code templates based on different use cases.
Data ingestion: Copy Into, create table as select, Insert into.
Business continuity: Create clone table, create clone table at a point in time.
Design and develop: Create schema, create table, create view, create parameterized stored procedure, create stored procedure with output parameter, create statistics, create inline table-valued function, drop schema, drop table, drop view, drop store procedure, drop function.
Monitoring: Active queries, historical queries, session history, long running queries, most frequent queries.
Open in Excel feature is also available in T-SQL notebook now. To get the full dataset from the submitted query, you can click the ‘Open in Excel’ button in the result view:
Learn more about setting up authentication inside Excel.
In T-SQL notebooks, users can add multiple data warehouse/SQL endpoints and set one as the primary warehouse. The primary warehouse name appears in the code cell status bar after execution, with traceability of its usage enabled, even if the primary warehouse is changed later.
Full CI/CD support for T-SQL notebook including Git and Deployment pipeline
With Git integration, you can commit your T-SQL code and the bindings between Notebook and warehouse into Git repo for history checking. Once you export the repo source into a different workspace, a new T-SQL notebook and warehouse item will be created.
With deployment pipeline integration, you can deploy the T-SQL notebook across different stages/workspaces.
New Lakehouse samples
We’ve listened to your feedback and are excited to release two new Lakehouse samples, designed to support deeper analysis and flexibility for both schema-enabled and non-schema Lakehouses.
New samples
- Retail data model (Wide World Importers): a dataset from a fictional novelty goods distributor, great for retail and supply chain analytics.
- NYC Taxi: taxi trip data, including pick-up/drop-off details, distances, and fares—ideal for pattern and trend exploration.
How to try these, you can get started in two ways:
- From the Workload page: navigate to Data Engineering > Explore a sample, and a schema-enabled lakehouse is automatically created.
2. From the Lakehouse experience: start in an empty lakehouse, and the samples will adapt to your schema settings.
Larger datasets provide a more realistic foundation for analysis, enabling you to test complex queries, simulate real-world scenarios, and uncover richer insights.
Start exploring today and unlock the full potential of your Lakehouse!
Monitoring for High Concurrency
We are excited to introduce several enhancements for monitoring Notebook runs within high concurrency Spark sessions. These improvements allow for better tracking and troubleshooting of Notebook executions, whether they run directly in high concurrency Spark sessions or share the same Spark sessions from a pipeline.
Key enhancements:
- Log segmentation for improved troubleshooting.
- Troubleshooting through logs is a critical task for data engineers. We now support log segmentation for each Notebook running in a high concurrency Spark session.
- This feature allows you to filter driver logs at the Notebook level, making it easier to identify failures or performance issues specific to an individual Notebook run.
2. Auto-mapping Spark Jobs, stages, and tasks to Notebooks.
- Spark jobs, stages, and tasks within a Spark session are now automatically mapped to their respective Notebooks.
- You can easily navigate Spark jobs and stages associated with a Notebook and access the corresponding code snippet when needed.
Notebook Snapshots enhancements
Notebook Snapshots feature has been a widely used feature, enabling users to view the original notebook code at the time of execution, examine outputs based on different parameters, and drill down into each notebook cell to gain deeper insights into execution status and cell-level outputs. To better address customer needs, we have introduced several enhancements to support additional Notebook run scenarios.
Support for Notebook runs triggered by NotebookUtils
- Users can now view snapshots for Notebook runs triggered by NotebookUtils methods, such as notebookutils.notebook.run() or notebookutils.notebook.runMultiple(), once the reference Notebook runs are complete.
- The snapshot tree explorer now provides a hierarchical view, making it easier to understand parent-child relationships and track which parent notebooks have triggered child notebooks via NotebookUtils.
Snapshots for running Notebooks
- Users can now access snapshots of Notebook runs from a Pipeline or direct schedule while they are still in progress.
- This enhancement allows you to monitor execution status in real-time, view progress, and check output or cell-level errors as they occur.
Enhanced visibility into parallel Notebook runs in High-Concurrency Spark Sessions
- An expandable and collapsible tree-structured explorer provides a snapshot of all parallel Notebook runs within a high-concurrency Spark session.
- You can view all Notebooks associated with the same Spark session and click on each Notebook to access its corresponding snapshot, including the original code, input parameters, and cell outputs.
Fabric Spark Resource Analysis (Generally available)
We are thrilled to announce the general availability of the Fabric Spark Resource Utilization Analysis feature, designed to provide in-depth insights into your Apache Spark applications within Microsoft Fabric. This feature helps identify potential bottlenecks by analyzing the distribution of running and idled cores and enables you to better understand executor allocation to optimize resource usage and improve application performance.
Key features:
- Real-time executor cores Monitoring – Access the Resources tab to view a comprehensive graph displaying four distinct metrics:
-
- Running: Actual number of cores utilized by the Spark application for executing jobs and tasks.
- Idled: Cores that are available but currently unused during the application’s runtime.
- Allocated: Total cores allocated for the Spark application’s operation.
- Maximum Instances: The upper limit of cores that can be allocated to the Spark application.
2. Interactive resource utilization graph:
-
- Interactively explore these metrics to gain a clear understanding of resource allocation and utilization.
- Hover over the running executor cores graph to view summaries of cores and corresponding executor information.
- Click on specific points in the graph to access detailed information about executors and jobs at that moment, facilitating precise performance analysis.
This feature is supported in Spark runtime version 3.4 and above. For a comprehensive guide on utilizing this new capability, please refer to our official documentation:
Monitor Apache Spark Applications Resource Utilization
We encourage you to explore the Fabric Spark Resource Utilization Analysis feature to optimize your Spark applications and enhance performance monitoring within Microsoft Fabric.
Data Science
Announcing AI functions for LLM-powered data enrichment and transformation (Preview)
We’re thrilled to announce the preview of AI functions in Fabric will be available by the end of February 2025. AI functions provide a simplified API for common LLM-powered enrichments on text-based data in notebooks—whether you’re looking to translate customer reviews from one language into another or to analyze the sentiment of detailed call logs. Now, in just a single line of code, you can harness the power of Fabric’s built-in LLM endpoint for summarization, translation, classification, grammatical correction, custom text generation, and more.
AI functions are designed for ease, with no need for complex config or GPU infrastructure management. But, if you choose, you also have the option to bring in your own AzureOpenAI LLM for more customized solutions. AI functions are initially available in pandas and Spark but stay tuned for the ability to invoke them with SQL—and much more.
Support for multiple data sources in AI Skill
The AI skill in Fabric now supports multiple data sources, giving users even more flexibility when exploring their data. Users can now add up to five data sources, select the relevant tables, and ask natural language questions—all in one seamless experience. The AI skill intelligently determines the best data source to answer each question, ensuring more accurate and relevant responses.
Additionally, users can provide instructions to fine-tune results and guide the AI skill on which sources to prioritize. This enhancement makes it easier than ever to gain insights from diverse datasets, all within Fabric.
AI Skill is now conversational
The AI skill in Fabric is now fully conversational, making interactions more seamless and intuitive. Your conversations are preserved between sessions, allowing you to pick up right where you left off without losing context. Additionally, the AI skill now remembers your previous questions and answers, making it easier to ask for follow-ups or clarifications without having to re-explain your query.
Whether you’re analyzing data, refining insights, or iterating on queries, this update makes engaging with the AI skill feel more like a natural conversation, streamlining your workflow and enhancing productivity.
Semantic model support with AI Skill
The AI Skill now supports Semantic models, a new data source that enables AI Creators to integrate structured business data for natural language querying. Semantic Models provide a curated data layer that defines business logic, relationships, and aggregations, making it easier for users to analyze and explore data without needing deep technical expertise.
AI Creators can add Semantic models and selectively scope which tables are available for querying. Once configured, both AI Creators and consumers can ask natural language questions, which the AI Skill translates into DAX (Data Analysis Expressions) queries using LLMs. Users can trace the entire query process in the chat canvas, viewing how their input is interpreted, which transformations occur, and the exact DAX queries executed.
KQL databases support in AI Skill
Users can now link their KQL databases as a data source in AI Skill, enhancing capabilities for real-time streaming and data analysis. AI Creators can configure their AI Skill with an Eventhouse Data Source by utilizing AI Instructions to provide the AI Skill agent with relevant context, providing Few Shots sample KQL queries, and selecting relevant tables for the creators use case.
After configuring the AI Skill, consumers can submit natural language questions about the data set and have the AI Skill generate a curated response presenting its interpretation of the prompt and the Kusto query used to generate the response. This powerful combination leverages the real-time streaming and analysis capabilities of Kusto with the customizability of the AI Skill to empower users to gain valuable insights from their data without technical expertise.
Reason over multiple steps with AI Skill
The AI skill in Fabric is now even more intelligent, breaking down complex questions into multiple steps to find the best answer. Instead of just responding to a question in one go, the AI skill can now figure out when it needs to pull information from different sources and take the right steps to get a complete answer.
For example, if your question requires checking both your Lakehouse and a KQL database, the AI skill will automatically query each one. You’ll also be able to see exactly how it got to the final answer—every step, every query, and every result—all clearly displayed in the AI skill chat experience. This update makes the AI skill more transparent, reliable, and smarter in helping you get the answers you need.
Data Warehouse
Collate clause support in Warehouse (Generally available)
All Fabric warehouses by default are configured with case-sensitive (CS) collation Latin1_General_100_BIN2_UTF8. You can also create warehouses with case-insensitive (CI) collation Latin1_General_100_CI_AS_KS_WS_SC_UTF8 via REST API. In a SELECT statement, the COLLATE clause is unrestricted and can be applied with unsupported collations.
You will now be able to use the COLLATE clause in the following T-SQL statements to define the collation for your VARCHAR or CHAR fields, enhancing your database management capabilities.
It’s important to note that only CS collation Latin1_General_100_BIN2_UTF8 and CI collation Latin1_General_100_CI_AS_KS_WS_SC_UTF8 are supported for these statements:
-
- CREATE TABLE
- ALTER TABLE ADD Nullable COLUMN
- CTAS (Create Table as Select)
- SELECT INTO
Sample Syntax:
CREATE TABLE TableName ( ColumnName VARCHAR(length) COLLATE CollationName );
This feature allows you to specify collations on individual columns, offering more granular control than artifact level collation. It ensures precise text data handling by consistently applying collation settings across various operations. It enhances data integrity, minimizes potential collation conflicts, and simplifies the management and querying of text data in mixed-case environments.
This release also includes support for the DATABASE_DEFAULT collation, giving you even more flexibility in managing your data. Upgrade your database management with these new capabilities and experience the difference!
Learn more about Collate Clause in the collate documentation.
Copilot for Data Warehouse Chat (Preview)
We are excited to release the chat experience within Copilot for Data Warehouse!
You will now see a ‘Copilot’ button in the ribbon which will allow you to interact with your own personalized AI assistant for all data warehousing tasks.
In addition to the Copilot code completions and quick actions, chat now enables you to talk to Copilot directly in natural language. You can ask it anything from creating SQL queries from natural language to asking questions about your warehouse, like:
- ‘How can I monitor query performance’?
- ‘How should I secure my tables’?
- ‘Create a relationship between table [x] and table [y]’.
The possibilities are endless! For more information on how to use chat within Copilot for Data Warehouse, learn more: How to: Use the Copilot chat pane for Fabric Data Warehouse
Browse files with OPENROWSET (Preview)
We are enabling you to read the content of parquet and csv files directly from the Fabric DW and SQL endpoints using the OPENROWSET function. The OPENROWSET function enables you to implement new scenarios in Fabric DW:
- Ad-hoc file browsing and exploration: the OPENROWSET function enables you to read the content of external files without ingesting them in DW.
- Improving ingestion (ETL/ELT) scenarios: the OPENROWSET function enables you to explore the file schema before the load and inspect rejected rows files in case of errors.
With the OPENROWSET function, you can easily read the content of a parquet or csv file and return a set of rows representing the content of the file. You just need to provide the URL of a file that you can read:
SELECT TOP 10 *
FROM OPENROWSET( BULK ‘https://pandemicdatalake.blob.core.windows.net/public/curated/covid-19/bing_covid-19_data/latest/bing_covid-19_data.parquet’ ) AS r
The OPENROWSET function enables you to use the ‘schema inference’ to automatically identify the columns and their types, or to explicitly specify the schema with the types that match your source schema. You can use the * and /** wildcards in the path to reference multiple files, access the parts of URL using filepath() and filename() function.
The overview of supported and unsupported functionalities is shown in the table:
| Supported | Not supported | |
| File formats | Parquet, CSV | Delta, CosmosDB |
| Authentication | EntraID passthrough, public storage | SAS/SAK, SPN, Managed Identity |
| Storage | Azure Blob storage, ADLS | OneLake |
The OPENROWSET function is currently in preview, and more enhancements will be added in the future.
Use the BULK INSERT to load your data (Preview)
We are happy to announce that we are starting a preview of BULK INSERT statement in Fabric Data Warehouse. Fabric Data Warehouse enables you to use the well-known Transact-SQL BULK INSERT statement for loading data from external files. With the BULK INSERT statement, you can easily load the CSV files from ADLS and specify the options such as row or field terminators:
BULK INSERT ecdc_cases
FROM ‘https://pandemicdatalake.blob.core.windows.net/public/curated/covid-19/ecdc_cases/latest/ecdc_cases.csv’
WITH (FIRSTROW = 2, FIELDTERMINATOR = ',', ROWTERMINATOR = '\n');
With the BULK INSERT statement, you can use the same code that you are using in the existing SQL Server warehouses for loading data.
Note that Fabric DW provides COPY INTO statement for loading data, and these two statements are almost identical – you can load the same file formats, they have the same performance, etc. In most cases, you will use the COPY INTO statement to load data in the warehouse. But if your warehouse is already in SQL Server and you are planning to migrate it to Fabric DW, there are scenarios where you cannot change the code that uses BULK INSERT and rewrite it to COPY INTO.
If you have a large codebase, or external tools or components that use BULK INSERT and you cannot modify their code. To enable easier migration of your warehouses from SQL Server and Azure SQL to Fabric DW, we enabled the same BULK INSERT statement that you are using in the existing warehouses so you can reuse your code with no or minimal changes.
BULK INSERT statement provides some options that are not available in the COPY INTO statement such support for loading non-Unicode file sources or FORMATFILE for defining the column mapping schema. If you are using these options in your warehouse solutions, you can use the same code and the file without any changes.
Nested common table expression (Generally available)
Common Table Expressions (CTEs) can simplify complex queries by deconstructing ordinarily complex queries into reusable blocks enhancing query’s readability and making troubleshooting easier. With this release, Microsoft Fabric Warehouse now supports three types of CTEs in general availability.
- Standard CTE: This type doesn’t reference or define another CTE in its definition.
- Sequential CTE: This CTE’s definition can reference an existing CTE but can’t define another CTE.
- Nested CTE: This CTE’s definition includes defining another CTE.
Real-Time Intelligence
Configure latency of Eventhouse OneLake availability
Eventhouse offers a robust mechanism that intelligently batches incoming data streams into one or more Parquet files, structured for analysis. Batching data streams is important when dealing with trickling data. Writing many small Parquet files into the lake can be inefficient resulting in higher costs and poor performance.
Eventhouse’s adaptive mechanism can delay writing operations if there isn’t enough data to create optimal Parquet files. This ensures Parquet files are optimal in size and adhere to Delta Lake best practices. The Eventhouse adaptive mechanism ensures that the Parquet files are primed for analysis and balances the need for prompt data availability with cost and performance considerations.
By default, the write operation can take up to 3 hours or until files of sufficient size (typically 200-256 MB) are created. For scenarios where you desire a lower latency, you can now adjust the delay to a value of between 5 minutes and 3 hours.
For example, use the following command to set the delay to 5 minutes:
.alter-merge table <TableName> policy mirroring dataformat=parquet with (IsEnabled=true, TargetLatencyInMinutes=5);
Please consider this change with caution as adjusting the delay to a shorter period may result in a suboptimal delta table with many small files, which can lead to inefficient query performance. The resultant table in OneLake is read-only and cannot be optimized after creation.
To learn more, refer to the Eventhouse OneLake Availability documentation.
Real-Time Dashboard & Power BI templates for Eventhouse Monitoring
Following the release of Fabric Workspace Monitoring we are happy to announce the release of Real-Time Dashboard and Power BI Templates designed to easily manage your Fabric Eventhouses.
The templates were designed to provide an out-of-the-box graphical management experience. With these templates you can easily monitor your query performance and data ingestions.
To leverage these reporting templates, only two steps are required:
- Enable Workspace monitoring – this can be done by turning on a toggle in the Fabric Workspace settings. Once enabled, a Monitoring KQL database that captures performance and ingestion metrics and events is automatically created.
- Download the templates – go to github and download the templates.
Review the Readme file to get a detailed description of how to import the templates into the Fabric experience.
You can see a demo in the fabric-toolbox github repo.
New Real-Time Dashboard customization features: more control, better usability, and improved performance
We’re introducing new customization features that give real-time dashboard authors and viewers more control over visuals, layout, and performance.
From managing data series visibility to improving navigation and map behavior, these updates help create a clearer, more efficient dashboard experience.
- Legend number configuration – Limit the number of data series displayed on load, reducing visual clutter and improving performance. Additional series remain accessible via the legend.
- Adjustable panel width – Resize the pages pane in both edit and view modes for better navigation, especially when working with long page names.
- Map centering configuration – Control whether a map visualization maintains its zoom level on refresh or resets based on new data.
4. Crosshair tooltip number configuration – Set a limit on the number of data points displayed in a line or time chart’s tooltip for improved readability.
New and improved Data Source Tree in KQL Queryset
The enhanced Data Source Tree now shows not only the data source connected to the current query tab but also all other data sources previously connected in your Queryset.
This enhancement makes it easier than ever to switch between sources and work with multiple datasets without needing to open new tabs.
Multiple data source types are supported, including:
- Fabric sources (Eventhouse/KQL database).
- Azure sources (Azure Data Explorer, Application Insights, Log Analytics clusters).
Key features:
- Switch seamlessly: switch between data sources connected to the current Queryset tab without opening a new tab.
- Create cross-cluster queries: browse through schemas from different clusters and double-click on item names for faster editing and query creation.
- Search for Data Sources: At the top of the Data Source Explorer pane, you can now use the search bar to quickly find any data source you need.
- Perform actions on Data Sources items: Contextual action menu is available for data sources items.
Data exploration is now available for raw KQL table data
We’ve made it even easier to explore your data – no code required – with the Explore Data feature, which is now available in additional places across the UI.
What’s new?
- In addition to the entry point in from every Real-Time Dashboard tile, you can now explore raw KQL table data directly.
|
 |
|
 |
|
 |
These new entry points make it easier than ever to dive into your data, whether you’re starting from a dashboard or working directly with raw tables. Try it out today and uncover new insights effortlessly!
Databases
Performance Dashboard in SQL database in Microsoft Fabric
The Performance Dashboard in SQL database in Microsoft Fabric offers comprehensive diagnostic tools to identify and address performance issues, providing detailed metrics to detect bottlenecks and optimize database performance. It includes features like CPU consumption analysis, user connections monitoring, requests per second tracking, and automatic indexing, making it an essential tool for database administrators and developers.
For a detailed guide on troubleshooting and optimizing your SQL database performance, refer to the Speed up your SQL databases with the Performance Dashboard blog.
Data Factory
Mirroring in Fabric now supports workspace monitoring (Preview)
Workspace monitoring is an observability feature in Fabric that enables Fabric developers and admins to access detailed logs and performance metrics for their workspaces.
We are thrilled to announce that we have added support for mirrored database operation logs in workspace monitoring to provide a more comprehensive monitoring experience for mirroring. You can leverage these logs to monitor execution and performance of your mirrored database, including data replication, table changes, mirroring status, failures, and replication latency for mirrored databases and tables.
To get started, enable monitoring in your workspace. The mirrored database execution logs will automatically be ingested into the MirroredDatabaseTableExecution table in the monitoring KQL database. You can then access a monitoring experience that allows you to:
- Derive insights on-demand: Query the granular operation logs directly using KQL in the monitoring database to get instant access to the information you need.
- Build customized monitoring dashboards: Create Real-Time Dashboards or Power BI reports against the data in the monitoring database, tailored to your needs.
- Set up alerting: Set up alerts based on the logs and metrics you’re tracking.
To learn more about this feature, refer to Mirrored database operation logs.
New regions available for Mirroring in Fabric
To meet the increasing customer demand, Mirroring in Fabric has expanded its support to include three additional regions: Central US, Poland Central, and Italy North. For detailed information about the Fabric regions that support mirroring, please refer to the What is Mirroring in Fabric? documentation.
Open mirroring for SAP sources – Simplement and SNP
With Simplement and SNP, two more partners have joined our open mirroring partner ecosystem: Open mirroring (Preview) Partner Ecosystem – Microsoft Fabric | Microsoft Learn.
Simplement Roundhouse is an SAP-certified solution for data extraction from various SAP source systems including SAP ECC, SAP CRM, SAP CFIN, SAP S/4HANA. It supports near real-time data integration into Microsoft Fabric via open mirroring.
For more information, refer to: Simplement integration into open mirroring in Microsoft Fabric.
SNP Glue offers robust integration capabilities with SAP systems, including SAP ECC, SAP S/4HANA, and SAP BW. Based on an SAP-certified ABAP Add-On, it facilitates near real-time data ingestion through open mirroring.
For more information, refer to: SNP Group.
Closing
We hope that you enjoy the update! Be sure to join the conversation in the Fabric Community and as always, keep voting on Fabric Ideas to help us determine what to build next. We are looking forward to hearing from you!
