Expanding Real-Time Intelligence data sources with Cribl source (Preview)
The exchange of real-time data across different data platforms is becoming increasingly popular. The Cribl source (preview) is now available in Real-Time Intelligence, allowing real-time data to flow into Fabric RTI Eventstream through our collaboration with Cribl, enabling you to take full advantage of Fabric Real-Time Intelligence’s robust analytics tools for their real-time needs.
Collaborating to broaden data source coverage
Microsoft Fabric provides a unified platform for data storage, processing, and analytics. Real-Time Intelligence empowers organizations to analyze and visualize streaming data, offering an end-to-end solution for real-time and event-driven scenarios. Within it, Eventstream serves as the central place in Fabric for capturing, transforming, and routing real-time data from diverse sources to multiple destinations.
Cribl helps organizations regain control over their telemetry data flows by providing a unified way to collect data from a wide variety of sources and process it efficiently before delivery to its destinations. It enables enterprises to ingest logs, metrics, and traces originating from cloud services, on-premises infrastructure, and edge environments for IT and security teams. A key strength of Cribl lies in its broad data source coverage. Through its flagship product, Cribl Stream, the platform supports both push and pull ingestion models. It can receive data through sources such as Syslog, Datadog Agent, Splunk, Open Telemetry, etc., or pull data from systems such as Amazon SQS, AWS S3, CrowdStrike FDR, Prometheus Scrape, etc. (sources available in Cribl Stream). Cribl also supports on-premises and edge-based sources—including local log files and system metrics— enabling end-to-end telemetry collection from edge to cloud.
By integrating Cribl as a Fabric RTI source, organizations can efficiently ingest diverse telemetry and log data from multiple environments into Fabric, a unified tool for storing, processing, and analyzing data. This setup boosts data processing flexibility and supports rapid decisions through Fabric’s Real-Time Intelligence capabilities, including large-scale analysis, real-time alerts, data transformation, and event-driven application development.
Better together: Simplifying Real-Time data integration
The collaboration aims to help you transfer real-time data from Cribl to Fabric Real-Time Intelligence, offering improved visibility and increased efficiency by:
- Creating a dedicated source type for Cribl in Fabric Real-Time Hub so that you can easily create the Kafka endpoint in Eventstream to receive the data pushed from Cribl with the properties needed to help you connect Cribl to Eventstream.
- Adding a dedicated destination type for ‘Fabric Real-Time Intelligence’ in Cribl, enabling you to easily discover and add the ‘Fabric Real-Time Intelligence’ destination to push the data to Fabric via the Kafka protocol.
Routing data from Cribl to Fabric
This integration streamlines data routing to Fabric Real-Time Intelligence Eventstream, enabling faster and simpler configuration. Follow these steps to use the feature.
Go to Real-Time hub, select Data source in the left pane, and choose Cribl. Give the source a name to complete the configuration and then the Eventstream Kafka endpoint for receiving data will be created with the detailed information shown in Details.
Log in to the Cribl worker group you want to send from and select Fabric Real-Time Intelligence from the Cribl Data Destination list to add it. You will be prompted to enter the Kafka connection details, which are generated in Eventstream’s Cribl source as shown in Figure 5.
If a Cribl source like Syslog is already added, use QuickConnect to link the source and destination. This allows data to flow from the Cribl source to the Cribl destination you configured, i.e., Fabric Real-Time Intelligence.
As a result, you will see the data available in your Eventstream.
To learn more and get started, refer to the Cribl source documentation.
Analyzing data in Fabric Real-Time Intelligence: insights, notifications, and beyond
When real-time data from Cribl flows into Fabric Eventstream, you can fully leverage the rich suite of real-time analytics tools available in Fabric Real-Time Intelligence (RTI) to power your real-time scenarios. For example,
- Using Eventhouse, you can analyze and explore large volumes of real-time data with Kusto Query Language (KQL) queries. The KQL queries can be exported to a Real-Time Dashboard as visuals to monitor and visualize the data, enhancing data exploration, query performance, and visualization.
- With Activator, you can be alerted across multiple channels such as emails and Teams when specific patterns or conditions are detected in real-time data.
- The Derived Stream feature enables reshaping and transforming real-time data for further analysis or reuse by other teams and applications within the organization.
- Custom endpoint capability allows developers to build event-driven applications that consume and act on events directly from Eventstream.
Try it now and share your feedback
Cribl source is now available in Fabric Real-Time Intelligence (Preview). Try it with your Fabric account. If you don’t have one, sign up for Power BI with a new Microsoft 365 trial and start a free Fabric trial capacity. Learn more about Eventstream. We welcome your feedback through the community forum, idea submission, or via email.
