Microsoft Fabric Updates Blog

Updated Invoke Pipeline Activity Coming to Fabric Data Factory

NOTE: We are currently working on a bug fix for the updated invoke pipeline activity and will release this update later in CY24.

The Invoke Pipeline activity in Data Factory data pipelines in Fabric is a very popular activity that is used to build patterns like child/parent, metadata-driven, or to trigger multiple pipelines upon a scheduled invocation based on conditional execution. Unfortunately, in Fabric, it was challenging to find the child execution runs in the Monitoring Hub and you were unable to invoke pipelines across Fabric workspaces.

To make it easier for you to see all of your child pipeline runs, we are releasing an updated preview of the Invoke Pipeline activity that you will see light-up in your Fabric Data Factory over the next 2 weeks. When you have an existing pipeline that uses the previous version of the Invoke Pipeline activity, you will be prompted to upgrade to the new version.

As you can see from the screenshot above, there are now new properties for Workspace and Connection. The Connection can be created once and then re-used. This is needed for Fabric to store your identity in our secure store so that you can invoke pipelines across Fabric. For Workspace, you can keep the default workspace and call pipelines from any workspace that you have access to.

Both of these new and exciting features will start to roll-out later this week so that you can easily view your child pipeline runs in monitoring hub as well as invoke pipelines across workspaces in Fabric!

Have any questions or feedback? Leave a comment below!

Related blog posts

Updated Invoke Pipeline Activity Coming to Fabric Data Factory

June 21, 2024 by Marc Bushong

Developing ETLs/ELTs can be a complex process when you add in business logic, large amounts of data, and the high volume of table data that needs to be moved from source to target. This is especially true in analytical workloads involving relational data when there is a need to either fully reload a table or incrementally update a table. Traditionally this is easily completed in a flavor of SQL (or name your favorite relational database). But a question is, how can we execute a mature, dynamic, and scalable ETL/ELT utilizing T-SQL with Microsoft Fabric? The answer is with Fabric Pipelines and Data Warehouse.

June 18, 2024 by RK Iyer

✎ Co-Author – Abhishek Narain Overview Building an effective Lakehouse starts with establishing a robust ingestion layer. Ingestion refers to the process of collecting, importing, and processing raw data from various sources into the data lake. Data ingestion is fundamental to the success of a data lake as it enables the consolidation, exploration, and processing … Continue reading “Demystifying Data Ingestion in Fabric: Fundamental Components for Ingesting Data into a Fabric Lakehouse using Fabric Data Pipelines”