Microsoft Fabric Updates Blog

Data Factory Adds CI/CD to Fabric Data Pipelines

When building successful data analytics projects, it is very important to have source control, continuous integration, continuous deployment, and collaborative development environments. Many Fabric engineers with previous Azure Synapse Analytics and Azure Data Factory experience have utilized the Git integration included in those PaaS offerings for those important capabilities. Now, we’re excited to share that we have added Git Integration and integration with built-in Deployment Pipelines to Data Factory data pipelines in Fabric as a public preview!

CI/CD features to utilize your own Git repo in Azure DevOps, or to use the built-in Deployment Pipelines in Fabric, will eventually light-up and become available to all Fabric items. Now that data pipelines can be used with these features, read more about the current preview capabilities and limitations at the online documentation here.

Connect to your Git repo

If you are familiar with ADF’s Git integration set-up and capabilities, then you will find Fabric’s Git integration easy and with a number of nice enhancements! ADF’s approach to CI/CD included the use of ARM templates to simplify the process, but also meant that the entire factory was treated as a single entity. This made cherry-picking elements of a factory very difficult. In Fabric’s approach, each item in our workspace can optionally partake in the check-in process so this is no longer challenging.

In addition to using your own ADO Git repo, you can optionally use the built-in Deployment Pipelines feature that enables code graduation through Dev, Test, and Prod environments through a super-easy and intuitive UI, yet another nice enhancement over ADF and Synapse.

Built-in deployment pipelines

The documentation link found here has more details on current limitations, including deployment rules in deployment pipelines, which we will hook up to data pipelines soon. Other Data Factory and Fabric items like Dataflows and Data Warehouse will also become part of the CI/CD process in Fabric over time.

Have any questions or feedback? Leave a comment below!

Gerelateerde blogberichten

Data Factory Adds CI/CD to Fabric Data Pipelines

oktober 30, 2024 door Patrick LeBlanc

Welcome to the October 2024 Update! Here are a few, select highlights of the many we have for Fabric this month. API for GraphQL support for Service Principal Names (SPNs). Introducing a powerful new feature in Lakehouses: Sorting, Filtering, and Searching capabilities. An addition to KQL Queryset that will revolutionize the way you interact with … Continue reading “Fabric October 2024 Monthly Update”

oktober 29, 2024 door Leo Li

We’re excited to announce several powerful updates to the Virtual Network (VNET) Data Gateway, designed to further enhance performance and improve the overall user experience. These new features allow users to better manage increasing workloads, perform complex data transformations, and simplify log management. Expanded Cluster Size from 5 to 7 One of the key improvements … Continue reading “New Features and Enhancements for Virtual Network Data Gateway”