Eventhouse Monitoring (Preview)
We are excited to announce Eventhouse monitoring in preview!
Fabric Workspace Monitoring is the centralized event logging solution of Fabric. Workspace monitoring is designed to provide a seamless and consistent monitoring experience with end-to-end visibility across all Fabric items.
Workspace Monitoring is based on the Real-time Intelligence Eventhouse KQL database. Once Workspace monitoring is enabled, a KQL database is seamlessly created. Event logs are then automatically routed and stored in the database, requiring no additional configuration or permissions setup. KQL databases are ideal for time series logs and metrics monitoring solutions.
For each one of the supported items, one or more events or metrics tables are created. Eventhouse Monitoring offers five events and metrics tables:
- EventhouseQueryLogs – logs all Eventhouse KQL queries.
- EventhouseCommandLogs- logs all Eventhouse commands.
- EventhouseDataOperations – logs all successful data operations including Batch ingestions, Streaming seal operations (operations that store streaming data to database extents), Materialized views updates, and Update policy table updates.
- EventhouseIngestionResultLogs – logs all successful and failed ingestions, including the file path of the ingested data.
- EventhouseMetrics- set of metrics that provide in depth monitoring of ingestions, materialized views, and continuous exports.
Users can explore and directly query workspace monitoring tables using KQL or SQL. Example queries are available in the documentation. Here is an example of monitoring queries stored in an Eventhouse KQL Queryset.
The workspace monitoring solution centrally monitors all the Power BI semantic models and Eventhouse items created in the workspace. A Power BI report can read data directly from an Eventhouse KQL database. In such cases, users can troubleshoot, and correlate events found in both the Semantic Model and its underling KQL database.
Fabric Real-Time Dashboards can be created on top of the Workspace Monitoring KQL database, providing an easy graphical monitoring user experience. Pre-defined Real-Time dashboard templates can be imported to provide an out-of-the-box Fabric monitoring UX experience.
In this example Real-Time monitoring dashboard, a group of Power BI Semantic Models are accessing data from an underlying KQL database. All of the Semantic Models and Evenhouse items are reporting log events to the Workspace monitoring database.
Here is an example real-time dashboard of the Semantic Model CPU usage reporting. The list of the top Semantic Models consuming a high level of CPU are displayed. When a specific Semantic Model is selected, its CPU consumption over time and its underlying log events are displayed. Users can easily zoom in to the specific time frame of an identified peak.
Users can then review the KQL Query logs at the exact peak time selected by the user and investigate who is consuming these resources. They can then drill down to the specific query log record.
Users can also monitor their data ingestion performance. They can see which databases and tables are being heavily ingested.
They can then deep dive and analyze the specific ingestion patterns of a selected table.
In summary, Workspace Monitoring offers a centralized monitoring solution, allowing users to efficiently monitor and troubleshoot their workspace items. With a click of a button, the KQL monitoring database is created. User can then easily access Eventhouse event logs with KQL queries, a Power BI report or a Real-Time Dashboard.
Eventhouse Monitoring demo
What’s next?
There will be many more exciting developments as we continue to innovate and expand the capabilities of Real-Time Intelligence. Learn more about all the features and follow a step-by-step tutorial. Join the conversation and vote for your favorite features.
Over the next month we’ll be releasing a series of blog posts that dive into all the capabilities further. Stay tuned for more!
