Microsoft Fabric Updates Blog

Get Data in Real-Time Analytics: A New and Improved Experience

We’re thrilled to unveil a brand new Get Data experience that simplifies the data ingestion process in your KQL database! With simplicity and efficiency in mind, this update streamlines the way you bring data into your KQL database. Whether you’re a seasoned data professional or just starting your data exploration journey, this new user experience is crafted to empower you every step of the way.

Simplify your data flow

We understand the importance of efficiency in data exploration, and that’s why we reimagined the process in order to minimize the number of clicks required. This ensures that you can bring data from all your familiar data sources including local files, Azure storage, Amazon S3, Event Hubs, Eventstream, and OneLake.

Guided Wizard

The Get Data experience is powered by a new Guided Wizard. This step-by-step assistant takes you through the entire data ingestion process, ensuring that you’re never left in the dark. Whether you’re extracting data from various sources, transforming it to suit your needs, or loading it into your KQL database, the Guided Wizard is your knowledgeable co-pilot, offering insights and guidance at every turn.

A screenshot of a computer

Description automatically generated

Available data sources

Currently, Real-Time Analytics supports six different data sources from which you can ingest data into your KQL database.

Eventstream

Eventstream in Microsoft Fabric is a centralized place to capture, transform, and route real-time events to various destinations with a no-code experience. You can either select an existing eventstream from a workspace or create a new one in the Get Data experience and ingest data into your KQL database with just few clicks. Eventstream is the preferred option for streaming data into your KQL database.

A screenshot of a computer

Description automatically generated

OneLake

OneLake is a single, unified, logical data lake for your whole organization. It’s the OneDrive for data. OneLake comes automatically with every Microsoft Fabric tenant and is designed to be the single place for all your analytics data. You can seamlessly ingest data from OneLake into your KQL database with just a few clicks. All you need is your data file path.

A screenshot of a computer

Description automatically generated

Local File

This is the simplest way to ingest data into a KQL Database. You can upload a file from your local machine using the Get Data UX. This is useful when you have a small amount of data that you want to analyze quickly.

A screenshot of a computer

Description automatically generated

Azure storage

If your data is stored in Azure Blob Storage or Azure Data Lake Storage Gen2, you can use the built-in support for Azure storage to ingest your data.

You can get data by pasting either of the following two URI types:

  • Blob container: Ingest up to 5000 blobs from a single container. This is preferred when ingesting a large amount of data or historical backfill.
  • Individual blob: You can add up to 10 individual blobs with a maximum size of 1GB of uncompressed data per blob.

A screenshot of a computer

Description automatically generated

Amazon S3

If your data is stored in Amazon S3, you can use the built-in S3 support to ingest it into your KQL database. This is useful when you have data stored in both Azure and Amazon Web Services, and you want to centralize your data in your KQL database for analysis. All you need is the URI of an S3 bucket or individual objects.
A screenshot of a computer

Description automatically generated

Event Hubs

If you’re using Azure Event Hubs to stream your data, you can seamlessly create a connection to your Azure resource from the wizard. This is useful when you want to analyze data in real-time, such as streaming telemetry data from IoT devices or log data from applications. By default, the system starts ingesting new data, but you can optionally choose to backfill the data by specifying an Event retrieval start date.

We hope you find the new Get Data experience more efficient. We look forward to your feedback and suggestions. To get started, see Get data from Eventstreams.

Gerelateerde blogberichten

Get Data in Real-Time Analytics: A New and Improved Experience

oktober 7, 2024 door Alex Lin

Introducing Managed VNet Support for Fabric Eventstream! By creating a Fabric’s Managed Private Endpoint, you can now securely connect Eventstream to your Azure services, such as Azure Event Hubs or IoT Hub, within a private network or behind a firewall. This integration ensures your data is securely transmitted over a private network, enabling you to … Continue reading “Secure Data Streaming with Managed Private Endpoints in Eventstream (Preview)”

oktober 4, 2024 door Jason Himmelstein

We had an incredible time in our host city of Stockholm for FabCon Europe! 3,300 attendees joined us from our international community, and it was wonderful to meet so many of you in person. Throughout the week of FabCon Europe, our teams published a wealth of valuable content, and we want to ensure you have … Continue reading “Fabric Community Conference Europe Recap”