Understanding Real-Time Intelligence usage reporting and billing
Intro
Microsoft Fabric Real-Time Intelligence (RTI) offers an end-to-end solution for event-driven insight-to-action scenarios, streaming time-based data from various sources using no-code connectors and data in motion visualization.
The current landscape is highly fragmented, requires complex solutions integrating products from multiple vendors, and often requires highly skilled professionals to build and maintain, resulting in a high total solution cost. In comparison, RTI enables a SaaS-based solution for efficient transformation, querying, and storage of large volumes of structured or unstructured data, and immediate visual insights and rule-based actions.
It is useful across all verticals and industries, for example automotive, manufacturing, IoT, fraud detection, business operations management, anomaly detection, service health monitoring and many more.
Billing in Microsoft Fabric Real-Time Intelligence is based on the usage of various resources.
Key points:
- Microsoft Fabric uses Capacity Units per second (CU(s)) – to measure and bill for resource usage.
- Everything you do in Fabric consumes CU(s) and is detailed in the Microsoft Fabric Capacity Metrics app down to the dedicated operations emitted and reported by each Fabric feature.
- These operations are then accumulated and reported into global Real-Time Intelligence capacity consumption meters and billed through your Azure subscription.
Learn about Real-Time Intelligence Eventstream, Eventhouse, storage, Fabric Events and Activator consumption utilization, capacity meters, and costs. To better understand Real-Time Intelligence capabilities and how capacity utilization is reported, let’s consider the following example:
Real-Time Inventory Management for a Retail Store
Imagine a large retail store that sells a wide range of products, both online and in physical locations. The store faces challenges in distributing the sales reports in a timely manner, keeping track of inventory levels, predicting customer demand, and ensuring that popular products are always in stock. To address these challenges, the store implements a Real-Time Intelligence solution to gain real-time insights into their inventory and streamline their operations.
This real-time approach to inventory management helps the retail store maintain optimal stock levels, reduce the risk of being out-of-stock, and improve overall customer satisfaction. Additionally, it enables the store to make data-driven decisions, optimize their operations, and stay ahead of the competition.
In the solution implemented by this store you can see the following components:
1. Eventstream
The retail store uses Eventstream to ingest data from various sources such as warehouse, point-of-sale (POS) systems, online orders, and inventory sensors. These events include sales transactions, stock levels, and customer interactions.
Learn more about Eventstream.
2. Activator
Activator has rules configured to monitor inventory levels on items stored in the warehouse and send an email to the relevant buyer when the inventory drops below the configured threshold.
Learn more about Activator.
3. Eventhouse
The ingested events are stored in Eventhouse, allowing for efficient querying and analysis: for example, understanding which products currently trending or which products are frequently returned.
The data is stored in a structured and unstructured formats, supporting both historical and real-time analysis.
Learn more about Eventhouse.
4. Fabric Events
The sales system writes sales into Azure Blob storage, and to process the data for business analytics, a data pipeline reads the Azure Blob files and transforms and loads it into the Fabric lakehouse. Fabric events are used to automate the process by subscribing to the Azure storage events, and automatically starting the pipeline when files are created.
Learn more about Fabric events.
Real-Time Intelligence also include the Real-Time Hub which is a single place to discover and consume all streaming data from your organization through Fabric. The Real-Time Hub does not consume any Capacity Units.
Resources consumption for the Retail Store example
In our example the Retail Store generates the following resources usage:
- The eventstream ingests data from an Azure Event Hub source that is coming in at 1 MB/minute. The data is transformed and filtered before being sent to a KQL DB (Eventhouse) destination. The data is also routed to an Activator destination.
- In the Activator, this solution uses 3 rules/triggers watching for different conditions.
- The Azure blob storage events generate 10K events every hour, with an event size of 2 KB. Both Eventstream and Activator items subscribe to these events.
- Evenstream and Activator combined capacity usage adds up to 45 CU.
- The Eventstream sends 1MB/minute of data to Eventhouse. In this solution, the eventhouse’s hot cache data is stored for 7 days and cold storage of 30 days, with the database being used for 100% of the time. This would attribute 25 CU consumption to Eventstream.
- Thus, the total capacity consumption of this solution adds up to 4.7 CU.
Assuming that this Real-Time Intelligence solution is the only thing that will be utilizing this customer’s Fabric capacity, and based on the available Fabric Capacity SKUs our Retail Store customer should consider purchasing the F8 SKU, providing 8 Capacity units. This SKU accommodates the Retails Store capacity needs with some room to spare. As the customer grows their operations or adds more components and complexity to their solution, they will to reevaluate their capacity utilization and, if needed, consider upgrading their Fabric Capacity SKU.
Capacity Reporting
This solution will consume compute resources as well as storage. The detailed usage will show up in Microsoft Fabric Capacity Metrics app as operations reported by each solution component. These operations are then reported and billed through meters in the Azure Subscription.
The Azure Portal Cost analysis view details the usage under Microsoft Fabric service and break it down by Fabric Product and specific meters reported by the Fabric components used in the solution.
These are the Fabric operations, and the consumption meters the Retail Store will see in their Azure bill:
Operations reported for Eventstream:
- Eventstream Per Hour: This is a flat base charge, if there is no data coming in or going out for over two hours, this operation will not incur any charges.
- Eventstream Data Traffic per GB: Bills for data ingress and egress in both default and derived streams, calculated hourly per gigabyte.
- Eventstream Processor Per Hour: Charges for operations including 1) retrieving real-time stream data from Azure EH and Azure IoT hub sources, and 2) processing and routing the stream data to derived streams and destinations (excluding the ‘Direct ingestion’ mode in the Eventhouse destination). The CU consumption rate of the operator is correlated to the throughput of event traffic and the complexity of the event processing logic.
- Eventstream Connectors Per vCore Hour: The CU consumption of the operation is used for charging computing resources when retrieving real-time data from connector sources (excluding Azure Event Hubs, Azure IoT Hub, Fabric events, and Custom endpoints). The CU consumption rate is directly related to throughput. As throughput increases, the number of vCore required also increases (autoscale), leading to higher CU consumption. Currently, connector autoscaling is unavailable; therefore, only one vCore is used per connector source.
| Meter Name | Operation name | Operation unit of measure | Fabric consumption rate (CU hours) |
| eventstream Capacity Usage CU | Eventstream Per Hour | Per hour | 0.222 |
| eventstream Data Traffic Capacity Usage CU | Eventstream Data Traffic per GB | Per GB | 0.342 |
eventstreams Processor Capacity Usage CU |
Eventstream Processor Per Hour | Per hour | Starts at 0.778 , and autoscales per throughput |
| eventstreams connectors Capacity Usage CU | Eventstream Connectors Per vCore Hour | Per vCore Per hour | 0.611 |
The retention of events in Fabric eventstreams incurs separate charges from your Fabric or Power BI premium capacity units. It utilizes OneLake Standard Storage that’s used to persist and store all data. When the retention setting is configured for more than one day (24 hours), charges are applied according to OneLake Standard storage rates. For more details of OneLake storage/month price reference the Microsoft Fabric pricing page.
To learn more about the billing meters of Eventstream, please visit: Microsoft Fabric event streams capacity consumption – Microsoft Fabric | Microsoft Learn
Operations reported for Activator
Activator billing is based on the following operations:
- Rule Uptime per Hour: A flat base charge, as long as the rule is active, the account will be charged an hourly uptime cost.
- Event Ingestion: This is accrued when an activator processes incoming real time events data.
- Event Computations: Evaluating an incoming event’s data to see whether the defined condition was met. This cost is calculated based on the compute resources your activator consumes in order to evaluate the rule. If the condition was met, the specified action will be activated.
- Storage: All events stored within Fabric storage incur corresponding Fabric storage costs.
| Meter name | Operation name | Unit of measure | Fabric consumption rate in CU(hr) |
| Real-Time Intelligence – Event Listener & Alert | Rule Uptime per Hour | Per hour | 0.02222 |
| Real-Time Intelligence – Event Operations | Event Ingestion | Per Event | 0.000011111 |
| Data Activator – Event Analytics | Event Computations | Per computation | 0.00000278 |
| n/a | Storage | Per GB per Hour | 0.00177 |
Activators retain all your event data and the information about the resulting actions for 30 days.
There are ways to optimize your Activator capacity usage and reduce your cost. You can learn about possible optimizations in Activator Billing blog. The details about activator pricing and meters reporting are documented in the Understanding Activator Billing and consumption reporting page.
Operations reported for Fabric Events
- Event Listener & Alert meter charges for the duration that the consumer to Fabric events exist, and it is charged to the consumer’s Fabric capacity.
- Event Operations meter charges for publishing, filtering and delivery operations on events. The publishing operations are charged to the publisher’s Fabric capacity, while the filtering and delivery operations are charged to the consumer’s Fabric capacity. For example, when a Fabric Activator rule is created to take an action on a workspace item event, the source workspace’s capacity is charged for the publish operations, and the Activator’s capacity is charged for the filtering and delivery operations.
The charge is on a unit of 64KB. For example, if the size of the event is 100KB, this is counted as 2 event operations. For Fabric events generated by Fabric artifacts (e.g. workspace item events), the publishing operation charge doesn’t kick in until a consumer is established for these events.
Learn more about Fabric events and Fabric Events billing.
| Meter name | Operation name | Operation unit of measure | Fabric consumption rate (CU hours) |
| Real-Time Intelligence – Event Listener & Alert | Event listener | Per hour | 0.0222 |
| Real-Time Intelligence – Event Operations | Event operations | Per event operation | 0.000011111 |
Operations reported for Eventhouse
Eventhouse Capacity
- Eventhouse UpTime is the number of seconds that your eventhouse is active in relation to the number of virtual cores used by your eventhouse. An auto scale mechanism is used to determine the size of your eventhouse. This mechanism ensures cost and performance optimization based on your usage pattern. An eventhouse with multiple KQL databases attached to it only shows Eventhouse UpTime for the eventhouse item, you will not see usage for the KQL database sub-item. Eventhouse automatically scales and adjusts its configuration based on usage and load.
Any query, command, or ingestion is considered activity and will cause your eventhouse to report Eventhouse UpTime.
For example, an eventhouse with 4 KQL databases using 4 virtual cores that are active for 30 minutes in an hour will require 2 Capacity Units to operate.
| Meter name | Operation name | Operation unit of measure | Fabric consumption rate (CU hours) |
| Eventhouse Capacity Usage CU | Eventhouse UpTime | Per core per hour | 1 |
Eventhouse Storage
Eventhouse Storage is billed separately from your Fabric or Power BI Premium Capacity units. Data ingested into a KQL database is stored in two tiers of storage:
- OneLake Cache Storage is premium storage that is utilized to provide the fastest query response times. The amount of cache storage is determined by how much compressed data you ingest along with the cache policy, which defines how many days of cache data you retain. For instance, if you typically query back seven days then you can set the cache retention to seven days for best performance. This storage tier is comparable to the Azure ADLS (Azure Data Lake Storage) premium tier.
Note : Enabling minimum consumption means that you aren’t charged for OneLake Cache Storage. When minimum capacity is set, the eventhouse is always active resulting in 100% Eventhouse UpTime.
- OneLake Standard Storage is used to persist and store all queryable data. The amount of standard storage is determined by how much compressed data you ingest along with the retention policy, which defines how many days you retain your data. For instance, if you need to maintain 365 days of queryable data you can set the retention to 365 days. This storage tier is comparable to the Azure ADLS (Azure Data Lake Storage) hot tier.
What’s next?
There will be many more exciting developments as we continue to innovate and expand the capabilities of Real-Time Intelligence. Learn more about all the features and follow a step-by-step tutorial. Join the conversation and vote for your favorite features.
Over the next month we’ll be releasing a series of blog posts that dive into all the capabilities further. Stay tuned for more!
