Please enter your information to subscribe to the Microsoft Fabric Blog.

Thanks for subscribing to the Microsoft Fabric Blog. You’ll receive email notifications when we publish new posts.

Microsoft Fabric Updates Blog

  1. Updates
  2. Announcements

Ingest, transform, and route real-time events with Microsoft Fabric event streams

See Arun Ulagaratchagan’s blog post to read the full Microsoft Fabric preview announcement.
 
You can now ingest, capture, transform and route real-time events to various destinations in Microsoft Fabric with a no-code experience using Microsoft Fabric event streams. It enables customers to ingest real-time event data from external event sources into the data stores in Fabric. The events could be transformed into the native formats required for target destination. For example, Eventstream could transform the events into Delta Lake format for the Lakehouse, into SQL columns based on the table schema, or filter events so that homogenous data can be sent to a KQL table.
 
To create a new instance of Microsoft Fabric event streams, navigate to Real-time Analytics in your Fabric workspace where you will find Eventstream.

After creating a new Eventstream item, you will land into a no code & easy to use experience where you can configure your eventstream from source to destination.
  

What is Fabric event streams

  1. Centralized place for real-time events: It provides the capability to capture, modify, and direct your streaming data in real-time using a fully managed and scalable infrastructure.
  2. Multiple source connectors: It enables you to ingest your real-time streaming data from 3 source types today: Azure Event Hubs, Sample data or Custom application.
  3. Multiple destinations: It enables you to transform/capture/route real-time streaming data to 3 destination types today: KQL database, Lakehouse, or Custom application.
  4. No code experience: It provides an intuitive and easy to use drag & drop experience with end-to-end data visibility and monitoring.


 
Let’s dive into each point stated above in detail:

1. Centralized place for real-time events 

Microsoft Fabric event streams utilize a scalable infrastructure that efficiently manages scaling and resourcing automatically. This enables you to effortlessly capture your real-time events, facilitating streaming ETL (Extract, Transform, Load) operations, as well as transforming and routing the events to your desired destination.
 

2. Multiple source connectors

Microsoft Fabric event streams enable you to ingest your real-time event data from 3 source types today:

    • Azure Event Hubs: You can seamlessly ingest events from an Azure event hub configured using Fabric Data connections.
    • Sample data: You can easily configure your eventstream to continuously ingest sample events from either Yellow Taxi data or Stock Market data.
    • Custom application: With this source, a streaming endpoint is created that allows customers to have their custom streaming application communicate directly with Fabric with either a Kafka client or an AMQP client.

 

3. Multiple destinations

Microsoft Fabric event streams enable you to transform and/or route your real-time event data to 3 destination types today: 

    • Fabric KQL Database: This destination helps with direct ingestion of your real-time events into KQL database. Once the data lands in KQL database, you can then perform additional queries/analysis to gain deeper insights or build Power BI reports.
    • Fabric Lakehouse: With this destination, you can transform (manipulate, filter, aggregate etc.) your real-time events before routing into a Lakehouse table. Real-time events are converted into Delta Lake format and then stored in the designated Lakehouse tables.
    • Custom application: With this destination, you can configure your custom applications to use their Kafka or AMQP client application to pull the events directly from Fabric.

Evenstream sources and destination

 

4. No code experience

There is a super simple & easy to use drag & drop experience with end-to-end data visibility. You can view the shape of your data at every step from source to transformation to destination. You can also monitor each step by looking at meaningful insights.

Drag & drop no-code experience. Transformation logic and data preview.

End-to-end data visibility

 
To see detailed documentation, please see link.  If you have any questions or feedback, please reach out to us at askeventstreams@microsoft.com  
  

Get started with Microsoft Fabric

Microsoft Fabric is currently in preview. Try out everything Fabric has to offer by signing up for the free trial—no credit card information required. Everyone who signs up gets a fixed Fabric trial capacity, which may be used for any feature or capability from integrating data to creating machine learning models. Existing Power BI Premium customers can simply turn on Fabric through the Power BI admin portal. After July 1, 2023, Fabric will be enabled for all Power BI tenants.
 
Sign up for the free trial. For more information read the Fabric trial docs.
  

Other resources

If you want to learn more about Microsoft Fabric, consider:

Related blog posts

Ingest, transform, and route real-time events with Microsoft Fabric event streams

Blog's featured image

Announcing public preview of VARCHAR(MAX) and VARBINARY(MAX) types in Fabric Data Warehouse
October 31, 2024 by Jovan Popovic

Fabric Data Warehouse is a modern data warehouse optimized for analytical data models, primarily focused on the smaller numeric, datetime, and string types that are suitable for analytics. For the textual data, Fabric DW supports the VARCHAR type that can store up to 8KB of text, which is suitable for most of the textual values … Continue reading “Announcing public preview of VARCHAR(MAX) and VARBINARY(MAX) types in Fabric Data Warehouse”

Read more
Blog's featured image

APIs for Managed Private Endpoint are now available
October 29, 2024 by Dandan Zhang

Managed private endpoints allow Fabric experiences to securely access data sources without exposing them to the public network or requiring complex network configurations. We announced General Availability for Managed Private Endpoint in Fabric in May of this year. Learn more here: Announcing General Availability of Fabric Private Links, Trusted Workspace Access, and Managed Private Endpoints. … Continue reading “APIs for Managed Private Endpoint are now available”

Read more