Dataflow Gen2 CI/CD and GIT source control integration are now in preview!
We are delighted to share that CI/CD and GIT integration support for Dataflow Gen2 is now available in preview!
With this new set of features, you can now seamlessly integrate your Dataflow with your existing CI/CD pipelines and version control of your workspace in Fabric. This integration allows for better collaboration, versioning, and automation of your deployment process across dev, test, and production environments.
Key benefits
- Automated Deployments: Streamline your deployment process by integrating Dataflow with your CI/CD processes in Fabric.
- Version Control: Use GIT to manage and version your Dataflow Gen2, ensuring you have a history of changes and can easily roll back if needed.
- Collaboration: Enhance team collaboration by leveraging GIT’s branching and merging capabilities.
These new features will significantly improve your workflow and productivity when working with Dataflows Gen2 in Fabric. We look forward to hearing your feedback and suggestions as we continue to enhance this feature.
Get started with CI/CD and GIT integration in Dataflow Gen2
To get started, follow the instructions below to set up your Dataflow Gen2 with CI/CD and GIT integration.
- Enable GIT integration in your workspace settings before proceeding.
- When creating a new Dataflow Gen2, there is now an option to enable CI/CD and GIT integration.
- Once enabled, you will be using the new Dataflow Gen2 with CI/CD and GIT integration.
If you have dataflows that you would like to move over to the new Dataflows Gen2 with CI/CD and GIT integration, you can do so by following the steps below.
- Open the Dataflow Gen2 you would like to migrate.
- Copy the queries or export a PQT template of the Dataflow Gen2.
- Create a new Dataflow Gen2 with CI/CD and GIT integration enabled.
- Import the queries or PQT template into the new Dataflow Gen2.
In the near future, we will further streamline this process, soon we will make this even easier by providing a ‘save as’ option to copy an existing Dataflow Gen2 to a new Dataflow Gen2 with CI/CD and GIT integration enabled. In the meantime, you can leverage the Import/Export Power Query templates capability as described in this article: Power Query template (preview) – Power Query | Microsoft Learn
Note: Pricing for Dataflow Gen2 with CI/CD capabilities will be consistent with the already established pricing for Dataflow Gen2. However, Dataflow Gen2 with CI/CD usage will start being charged a few days after the initial availability of the Preview feature. Please refer to this article for more information about Dataflow Gen2 Pricing: Pricing for Dataflow Gen2 – Microsoft Fabric | Microsoft Learn
Also note that newly created items opted into the new capabilities will eventually be consumable through Public APIs, but the APIs will not be available until early 2025. Stay tuned for further updates on this blog.
You can expect these capabilities to be enabled in your Fabric tenant regions in January. We hope you find this new feature beneficial and are eager to hear your feedback.*
Learn more about Dataflow Gen2, CI/CD and Git integration capabilities in Fabric:
- What is Data Factory – Microsoft Fabric | Microsoft Learn
- Differences between Dataflow Gen1 and Dataflow Gen2 – Microsoft Fabric | Microsoft Learn
- Introduction to the CI/CD process as part of the ALM cycle in Microsoft Fabric – Microsoft Fabric | Microsoft Learn
- Overview of Fabric Git integration – Microsoft Fabric | Microsoft Learn
*Previously, we communicated in this blog that CI/CD and Git integration support would be available in a few days. However, we have updated this timeline to January to ensure the quality of the release and ensure that there is no impact to existing operations. This decision was not easy, but it is necessary to ensure the best possible experience for our users.
