Microsoft Fabric Updates Blog

From raw data to insights: How to ingest data from Eventstream into a KQL database for powerful analytics

Synapse Real-Time Analytics in Microsoft Fabric can help you capture, transform, route, analyze and visualize huge amounts of real-time data from different sources in a matter of seconds. In this blog post, we’ll learn about the integration between Eventstream and a KQL database, both of which are a part of the Real-Time Analytics experience.
 
The Eventstream feature allows you to capture real-time events from a variety of sources such as Azure Event Hubs, custom applications, and even a sample dataset of taxi rides data and stock rates. It also offers a drag and drop experience that enables you to create event data processing, transforming, and routing logic without requiring any coding experience. The end-to-end data flow diagram in Eventstream provides a comprehensive understanding of the data flow and organization.
 
A KQL database is a fast, powerful, scalable, and highly available tool that is designed to handle large volumes of data. The Kusto Query Language (KQL) provides a simple yet powerful syntax that allows anyone to query, analyze, and visualize data to get insights in real-time.
 
KQL allows data to be queried as it’s ingested, and can quickly analyze structured, semi-structured, and unstructured data, including free text, using built-in operators and functions for trend analysis, pattern recognition, anomaly detection, forecasting, and machine learning capabilities.
 
This allows, for example, businesses to quickly respond to changes in their operations and make informed decisions based on the latest data. KQL Databases can ingest data from a wide range of sources, including Eventstream, making it easy to get data into the service and start analyzing it.
 
The integration between Eventstream and your KQL database is straightforward. You can add a KQL database as a destination to your eventstream and route the event data to your KQL database in real-time, using a friendly wizard. If desired, Eventstream provides the option to transform and filter the data according to your needs.


Screenshot of Eventstream Data Preview showing successful ingestion of Taxi data into a KQL database
 
Once your data is ingested into your KQL database, you can utilize KQL queries to gain valuable insights into your data.

Screenshot of a KQL queryset showing an example query analyzing Yellow Taxi sample data received from Eventstream
 

Example of using KQL queries to analyze sample Yellow Taxi sample data received from Eventstream
 
The video below provides an end-to-end example, which includes several KQL queries for analyzing the sample Yellow Taxi sample data stream.
 

 
 

Get started with Microsoft Fabric

Microsoft Fabric is currently in preview. Try out everything Fabric has to offer by signing up for the free trial—no credit card information required. Everyone who signs up gets a fixed Fabric trial capacity, which may be used for any feature or capability from integrating data to creating machine learning models. Existing Power BI Premium customers can simply turn on Fabric through the Power BI admin portal. After July 1, 2023, Fabric will be enabled for all Power BI tenants.
 
Sign up for the free trial. For more information read the Fabric trial docs.
 
Check out the additional resources:

โพสต์ในบล็อกที่เกี่ยวข้อง

From raw data to insights: How to ingest data from Eventstream into a KQL database for powerful analytics

มิถุนายน 26, 2567 โดย Jason Himmelstein

Welcome to the June 2024 update. Here are a few, select highlights of the many we have for Fabric. Fabric Spark connector for Fabric Data Warehouse in Spark runtime is now available. Query activity, a one-stop view of your running and completed SQL queries for workspace admins is being announced. You can now run queries … Continue reading “Microsoft Fabric June 2024 Update”

มิถุนายน 26, 2567 โดย Amir Jafari

We are excited to share guidance for how you can use Microsoft Fabric to turn your data into knowledge for Generative AI applications. This guide will walk you through implementing a RAG (Retrieval Augmented Generation) system in Microsoft Fabric using Azure OpenAI and Azure AI Search. By the end, you’ll be more familiar with how to … Continue reading “Building Custom AI Applications with Microsoft Fabric: Implementing Retrieval Augmented Generation for Enhanced Language Models”