Data Factory pipeline developer create exciting and interesting data integration and ETL workflows for their data analytics projects. Because Data Factory is a platform service that is shared across ADF, Synapse, and Fabric, we had been limiting the number of activities in a single pipeline to 40 as a way to avoid resource exhaustion. However, … Continue reading “Data Factory Increases Maximum Activities Per Pipeline to 80”
Microsoft releases the public preview of CRUD REST APIs for Fabric Data Factory pipelines.
Fabric Data Factory Git Integration and Deployment Pipelines are now available as public preview.
Microsoft Fabric provides a cloud-scale data warehouse to store your analytical data for dashboards and historical reporting. In Fabric Data Factory, there are a number of ways to query data, retrieve data, and execute commands from your warehouse using pipeline activities that can then be easily automated for unattended background execution. I’ll go left-to-right from … Continue reading “Automate Fabric Data Warehouse Queries and Commands with Data Factory”
Microsoft Fabric Data Factory’s data pipelines enable data engineers to build complex workflows that can orchestrate many different types of data processing, data movement, data transformation, and other activity types. In this post, I want to focus on some good practices when building Fabric Spark Notebook workflows using Data Factory in Fabric with data pipelines. … Continue reading “Use Fabric Data Factory Data Pipelines to Orchestrate Notebook-based Workflows”
