Please enter your information to subscribe to the Microsoft Fabric Blog.

Thanks for subscribing to the Microsoft Fabric Blog. You’ll receive email notifications when we publish new posts.

Microsoft Fabric Updates Blog

  1. Updates
  2. All

Ingest, transform, and route real-time events with Microsoft Fabric event streams

See Arun Ulagaratchagan’s blog post to read the full Microsoft Fabric preview announcement.
 
You can now ingest, capture, transform and route real-time events to various destinations in Microsoft Fabric with a no-code experience using Microsoft Fabric event streams. It enables customers to ingest real-time event data from external event sources into the data stores in Fabric. The events could be transformed into the native formats required for target destination. For example, Eventstream could transform the events into Delta Lake format for the Lakehouse, into SQL columns based on the table schema, or filter events so that homogenous data can be sent to a KQL table.
 
To create a new instance of Microsoft Fabric event streams, navigate to Real-time Analytics in your Fabric workspace where you will find Eventstream.

After creating a new Eventstream item, you will land into a no code & easy to use experience where you can configure your eventstream from source to destination.
  

What is Fabric event streams

  1. Centralized place for real-time events: It provides the capability to capture, modify, and direct your streaming data in real-time using a fully managed and scalable infrastructure.
  2. Multiple source connectors: It enables you to ingest your real-time streaming data from 3 source types today: Azure Event Hubs, Sample data or Custom application.
  3. Multiple destinations: It enables you to transform/capture/route real-time streaming data to 3 destination types today: KQL database, Lakehouse, or Custom application.
  4. No code experience: It provides an intuitive and easy to use drag & drop experience with end-to-end data visibility and monitoring.


 
Let’s dive into each point stated above in detail:

1. Centralized place for real-time events 

Microsoft Fabric event streams utilize a scalable infrastructure that efficiently manages scaling and resourcing automatically. This enables you to effortlessly capture your real-time events, facilitating streaming ETL (Extract, Transform, Load) operations, as well as transforming and routing the events to your desired destination.
 

2. Multiple source connectors

Microsoft Fabric event streams enable you to ingest your real-time event data from 3 source types today:

    • Azure Event Hubs: You can seamlessly ingest events from an Azure event hub configured using Fabric Data connections.
    • Sample data: You can easily configure your eventstream to continuously ingest sample events from either Yellow Taxi data or Stock Market data.
    • Custom application: With this source, a streaming endpoint is created that allows customers to have their custom streaming application communicate directly with Fabric with either a Kafka client or an AMQP client.

 

3. Multiple destinations

Microsoft Fabric event streams enable you to transform and/or route your real-time event data to 3 destination types today: 

    • Fabric KQL Database: This destination helps with direct ingestion of your real-time events into KQL database. Once the data lands in KQL database, you can then perform additional queries/analysis to gain deeper insights or build Power BI reports.
    • Fabric Lakehouse: With this destination, you can transform (manipulate, filter, aggregate etc.) your real-time events before routing into a Lakehouse table. Real-time events are converted into Delta Lake format and then stored in the designated Lakehouse tables.
    • Custom application: With this destination, you can configure your custom applications to use their Kafka or AMQP client application to pull the events directly from Fabric.

Evenstream sources and destination

 

4. No code experience

There is a super simple & easy to use drag & drop experience with end-to-end data visibility. You can view the shape of your data at every step from source to transformation to destination. You can also monitor each step by looking at meaningful insights.

Drag & drop no-code experience. Transformation logic and data preview.

End-to-end data visibility

 
To see detailed documentation, please see link.  If you have any questions or feedback, please reach out to us at askeventstreams@microsoft.com  
  

Get started with Microsoft Fabric

Microsoft Fabric is currently in preview. Try out everything Fabric has to offer by signing up for the free trial—no credit card information required. Everyone who signs up gets a fixed Fabric trial capacity, which may be used for any feature or capability from integrating data to creating machine learning models. Existing Power BI Premium customers can simply turn on Fabric through the Power BI admin portal. After July 1, 2023, Fabric will be enabled for all Power BI tenants.
 
Sign up for the free trial. For more information read the Fabric trial docs.
  

Other resources

If you want to learn more about Microsoft Fabric, consider:

Related blog posts

Ingest, transform, and route real-time events with Microsoft Fabric event streams

Blog's featured image

Introducing Code-First AutoML and Hyperparameter Tuning: Now in Public Preview for Fabric Data Science
April 23, 2024 by Misha Desai

At the recent Fabric Conference, we announced that both code-first automated machine learning (AutoML) and hyperparameter tuning are now in Public Preview, a key step in making machine learning more complete and widely accessible in the Fabric Data Science. Our system seamlessly integrates the open-source Fast Library for Automated Machine Learning & Tuning (FLAML), offering … Continue reading “Introducing Code-First AutoML and Hyperparameter Tuning: Now in Public Preview for Fabric Data Science”

Read more
Blog's featured image

Introducing Optimistic Job Admission for Fabric Spark
April 18, 2024 by Santhosh Kumar Ravindran

We are excited to announce a new feature which has been a long ask from Synapse Spark customers, Optimistic Job Admission for Spark in Microsoft Fabric.This feature brings in more flexibility to optimize for concurrency usage (in some cases ~12X increase) and prevents job starvation. This job admission approach aims to reduce the frequency of … Continue reading “Introducing Optimistic Job Admission for Fabric Spark”

Read more