Microsoft Fabric Updates Blog

Get Data in Real-Time Analytics: A New and Improved Experience

We’re thrilled to unveil a brand new Get Data experience that simplifies the data ingestion process in your KQL database! With simplicity and efficiency in mind, this update streamlines the way you bring data into your KQL database. Whether you’re a seasoned data professional or just starting your data exploration journey, this new user experience is crafted to empower you every step of the way.

Simplify your data flow

We understand the importance of efficiency in data exploration, and that’s why we reimagined the process in order to minimize the number of clicks required. This ensures that you can bring data from all your familiar data sources including local files, Azure storage, Amazon S3, Event Hubs, Eventstream, and OneLake.

Guided Wizard

The Get Data experience is powered by a new Guided Wizard. This step-by-step assistant takes you through the entire data ingestion process, ensuring that you’re never left in the dark. Whether you’re extracting data from various sources, transforming it to suit your needs, or loading it into your KQL database, the Guided Wizard is your knowledgeable co-pilot, offering insights and guidance at every turn.

A screenshot of a computer

Description automatically generated

Available data sources

Currently, Real-Time Analytics supports six different data sources from which you can ingest data into your KQL database.

Eventstream

Eventstream in Microsoft Fabric is a centralized place to capture, transform, and route real-time events to various destinations with a no-code experience. You can either select an existing eventstream from a workspace or create a new one in the Get Data experience and ingest data into your KQL database with just few clicks. Eventstream is the preferred option for streaming data into your KQL database.

A screenshot of a computer

Description automatically generated

OneLake

OneLake is a single, unified, logical data lake for your whole organization. It’s the OneDrive for data. OneLake comes automatically with every Microsoft Fabric tenant and is designed to be the single place for all your analytics data. You can seamlessly ingest data from OneLake into your KQL database with just a few clicks. All you need is your data file path.

A screenshot of a computer

Description automatically generated

Local File

This is the simplest way to ingest data into a KQL Database. You can upload a file from your local machine using the Get Data UX. This is useful when you have a small amount of data that you want to analyze quickly.

A screenshot of a computer

Description automatically generated

Azure storage

If your data is stored in Azure Blob Storage or Azure Data Lake Storage Gen2, you can use the built-in support for Azure storage to ingest your data.

You can get data by pasting either of the following two URI types:

  • Blob container: Ingest up to 5000 blobs from a single container. This is preferred when ingesting a large amount of data or historical backfill.
  • Individual blob: You can add up to 10 individual blobs with a maximum size of 1GB of uncompressed data per blob.

A screenshot of a computer

Description automatically generated

Amazon S3

If your data is stored in Amazon S3, you can use the built-in S3 support to ingest it into your KQL database. This is useful when you have data stored in both Azure and Amazon Web Services, and you want to centralize your data in your KQL database for analysis. All you need is the URI of an S3 bucket or individual objects.
A screenshot of a computer

Description automatically generated

Event Hubs

If you’re using Azure Event Hubs to stream your data, you can seamlessly create a connection to your Azure resource from the wizard. This is useful when you want to analyze data in real-time, such as streaming telemetry data from IoT devices or log data from applications. By default, the system starts ingesting new data, but you can optionally choose to backfill the data by specifying an Event retrieval start date.

We hope you find the new Get Data experience more efficient. We look forward to your feedback and suggestions. To get started, see Get data from Eventstreams.

Related blog posts

Get Data in Real-Time Analytics: A New and Improved Experience

July 14, 2024 by Nimrod Shalit

GitHub and GitHub Enterprise are now available for source control integration. Connect your workspace and you can start syncing your fabric content into your repository.

July 12, 2024 by Ed Lima

During Microsoft Build, in May 2024, we announced the worldwide public preview of API for GraphQL in Microsoft Fabric. With the preview, data engineers and scientists can create a GraphQL data API to connect to different Fabric data sources in seconds, use the APIs in their workflows, or share the API endpoints with app development … Continue reading “Announcing Fabric API for GraphQL pricing”