Dataflow Gen2 CI/CD, GIT integration and Public APIs (Generally Available)
We’re excited to announce the General Availability of Dataflow Gen2 CI/CD & Git integration support!
With this set of features, you can now seamlessly integrate your Dataflow Gen2 items with your existing CI/CD pipelines and version control of your workspace in Fabric. This integration allows for better collaboration, versioning, and automation of your deployment process across dev, test, and production environments.
You can also easily upgrade your existing Dataflow Gen2 items to become CI/CD enabled by leveraging the Save as Dataflow Gen2 (CI/CD) feature.
Key benefits
- Automated deployments: streamline your deployment process by integrating Dataflow Gen2 items with your CI/CD processes in Fabric.
- Version control: use GIT to manage and version your Dataflow Gen2 items, ensuring you have a history of changes and can easily roll back if needed.
- Collaboration: enhance team collaboration by leveraging GIT’s branching and merging capabilities.
- Multitasking support: you can now also have multiple Dataflow Gen2 items open at the same time as other Fabric items, enabling a more streamlined authoring experience across items.
Additionally, we’re also glad to announce the availability of Dataflow Gen2 (CI/CD) Public APIs, providing a robust set of APIs that enable users to automate and manage their dataflows efficiently. These APIs allow for seamless integration with various data sources and services, enabling users to create, update, and monitor their data workflows programmatically. The APIs support a wide range of operations — including dataflows CRUD (Create, Read, Update, and Delete), scheduling, and monitoring — making it easier for users to manage their data integration processes.
The APIs for dataflows in Fabric Data Factory can be used in various scenarios:
- Automated deployment: Automate the deployment of dataflows across different environments (development, testing, production) using CI/CD practices.
- Monitoring and alerts: Set up automated monitoring and alerting systems to track the status of dataflows and receive notifications in case of failures or performance issues.
- Data integration: Integrate data from multiple sources, such as databases, data lakes, and cloud services, into a unified dataflow for processing and analysis.
- Error handling: Implement custom error handling and retry mechanisms to ensure dataflows run smoothly and recover from failures.
Learn more about these features for Dataflow Gen2:
We hope these new features will significantly improve your workflow and productivity when working with Dataflows Gen2 in Fabric.
Thank You for your feedback, keep it coming!
We wanted to thank you for your support, usage, excitement, and feedback around Data Factory in Fabric. We’re very excited to continue learning from you regarding your Data Integration needs and how Data Factory in Fabric can be enhanced to empower you to achieve more with data.
Please continue to share your feedback and feature ideas with us via our official Community channels, and stay tuned to our public roadmap page for updates on what will come next:
