Please enter your information to subscribe to the Microsoft Fabric Blog.

Thanks for subscribing to the Microsoft Fabric Blog. You’ll receive email notifications when we publish new posts.

Microsoft Fabric Updates Blog

  1. Updates
  2. Connie Xu

Run Spark Job Definitions in Pipelines with Service Principal or Workspace Identity

  • Headshot of article author December 18, 2025 by Connie Xu 53,352 Views

The Spark job definition activity in Microsoft Fabric Data Factory pipelines now supports connection property, unlocking a more secure and production-ready way to run your SJDs. What’s New? With this update, you can configure Notebook activities to run as Service Principal (SPN) or Workspace Identity (WI). These authentication methods are our recommended approach for production … Continue reading “Run Spark Job Definitions in Pipelines with Service Principal or Workspace Identity”

Run Notebooks in Pipelines with Service Principal or Workspace Identity

  • Headshot of article author December 4, 2025 by Connie Xu 24,601 Views

Update: Support for using Workspace Identity (WI) with the Notebook activity in Pipelines is currently rolling out. There was a deployment delay in this feature, but we expect this to be fully available by the end of January. The setup guidance below will apply once support is enabled in your environment. Notebook activity in Microsoft … Continue reading “Run Notebooks in Pipelines with Service Principal or Workspace Identity”

Natural Language to Generate and Explain Pipeline Expressions with Copilot (Preview)

  • Headshot of article author November 21, 2025 by Connie Xu 33,262 Views

Say goodbye to complex syntax—Copilot now makes building pipeline expressions as easy as having a conversation. Introducing a new Copilot capability (Preview) that transforms how you create and understand pipeline expressions in Fabric Data Factory. Why This Matters Pipeline expressions are powerful tools for dynamic data movement and transformation. But writing them often means memorizing … Continue reading “Natural Language to Generate and Explain Pipeline Expressions with Copilot (Preview)”

Apache Airflow Job File Management APIs

  • Headshot of article author November 20, 2025 by Connie Xu 8,821 Views

Introducing Apache Airflow Job File Management APIs—a significant advancement in workflow orchestration, designed to make processes more efficient, secure, and developer-friendly. These APIs are designed to give you full programmatic control over job files in your Apache Airflow environments, enabling seamless automation and integration across your data workflows. The File Management APIs provide endpoints to: … Continue reading “Apache Airflow Job File Management APIs”

Copy job Activity in pipelines (Preview)

We’re thrilled to announce that the Copy job Activity is now in Preview!  This new orchestration activity brings the simplicity of the Copy job item directly into your Microsoft Fabric Data Factory pipelines, enabling you to manage data movement alongside transformations, notifications, and more; all in one place.  The Copy job activity includes a monitoring link that gives you … Continue reading “Copy job Activity in pipelines (Preview)”