Fabric Real-Time Analytics Integrates with Newly Announced Database Watcher for Azure SQL
Users of Azure SQL can now take advantage of an enhanced monitoring solution for their databases and leverage integration with Microsoft Fabric. With the introduction of the new Database Watcher for Azure SQL (preview), users gain access to advanced monitoring capabilities. Through integration with Microsoft Fabric, they can effortlessly stream, store, and analyze monitoring data in real-time using an Eventhouse KQL database in Fabric Real-Time Analytics. This integration enables users to stay informed about their database performance and make timely, informed decisions.
Database watcher is a new managed monitoring solution for database services in the Azure SQL family. It supports Azure SQL Database and Azure SQL Managed Instance.
Database watcher collects in-depth workload monitoring data to give you a detailed view of database performance, configuration, and health. Monitoring data from the selected databases, elastic pools, and SQL managed instances is collected in near real-time and stored in a central data store. To store and analyze SQL monitoring data, database watcher utilizes an Eventhouse KQL database in Fabric Real-Time Analytics (as a complement option, Azure Data Explorer is also supported). This provides direct access to your monitoring data, allowing you to join it with other data sources and generate insights.
The out-of-the-box dashboards in Azure Portal offer a comprehensive view of your Azure SQL estate, providing both a single-pane-of-glass overview and detailed insights into each database, elastic pool, and SQL managed instance.
The monitoring data is stored in Fabric Real-Time Analytics.
Dashboards in the Azure portal show you a single-pane-of-glass view across your Azure SQL estate, and a detailed view of each database, elastic pool, and managed instance.
Making the Most of Fabric Real-Time Analytics Integration
Once the data is in Eventhouse KQL database in Fabric, you can query it using KQL or T-SQL. Analyze and model your data using Real-time Analytics KQL Querysets to execute queries, share, view, and customize query results on data from a KQL database. Additionally, you can utilize Fabric Notebooks, which serve as a primary tool for developing Apache Spark jobs and machine learning experiments. With a Fabric notebook, you can connect to data in a KQL Database and run queries using native KQL.
Furthermore, you can build custom visualizations using KQL built-in visualizations or Power BI.
Optimal Data Analysis with KQL (Kusto Query Language)
To analyze collected monitoring data, the recommended method is to use KQL. KQL is optimal for querying telemetry, metrics, and logs. It provides extensive support for text search and parsing, time-series operators and functions, analytics and aggregation, and many other language constructs that facilitate data analysis. You can find examples of analyzing database watcher monitoring data with KQL in the documentation.
Evolution of Azure SQL Monitoring
Over the past decade, our customers have emphasized the importance of reliable, low-latency, and comprehensive monitoring for cloud database services. Initially, many Azure SQL users found that Azure Monitor metrics and diagnostic telemetry were sufficient. However, as large and complex mission-critical applications became prevalent, it became evident that more robust monitoring was necessary. While Azure SQL Analytics and SQL Insights were introduced to address this need, feedback revealed gaps in the monitoring data coverage, an excessively high data latency, and high cost at scale. Although efforts were made to enhance SQL Insights, concerns continued to linger, especially regarding the setup and maintenance of IaaS VMs to monitor PaaS database services. Reliability issues further underscored the need for a better solution.
Thus, database watcher was born.
This managed monitoring solution offers extensive data coverage, collecting information from over 70 SQL system views and presenting it directly in the Azure portal. With minimal latency, typically in single-digit seconds, and leveraging Fabric Real-Time Analytics, database watcher empowers users to derive actionable insights from rich, real-time data that database watcher collects.
Summary
The dashboards, complemented by KQL queries, enable you to delve deep into the performance and configuration of your databases. This means you can detect, investigate, and troubleshoot a wide variety of database performance and health issues.
Whether you’re tackling resource bottlenecks or fine-tuning your Azure SQL resources for the best balance of cost and performance, database watcher equips you with the insights needed to make informed decisions. It’s your pathway to optimizing your Azure SQL setup for peak efficiency and cost-effectiveness.
Next steps
One effective approach to grasp the potential and the power of database watcher is to give it a try yourself. Set up your Azure SQL resources with database watcher, explore the dashboards, and start running some queries with KQL.
To read more about database watcher, check out the documentation.
