Building an analytical web application with Microsoft Fabric
Imagine a retail company that wants to gain insights into customer sentiment for each of its products. They also want to find their top-selling and least-selling products. Using Microsoft Fabric, they can build a powerful analytical application to transform their raw data into actionable insights. The process starts with ingesting raw data, such as customer reviews and sales figures, and ends with providing refined data through an API for internal use. This helps the company efficiently process customer feedback and make informed decisions to improve their products.
In this blog post, we’ll dive into the architecture of an analytical application powered by Microsoft Fabric, as shown in the image, and provide a step-by-step guide on how to build it.
Figure: Analytical application using Microsoft Fabric
The architecture is divided into two parts:
- Data transformation using Medallion architecture: Bronze, Silver and Gold tiers stand for various stages of data processing and transformation.
- Application layer: Once the data is in a clean state in the gold tier warehouse it can be consumed by a front-end application, in a language of your choice and use GraphQL to fetch data.
Sign in to your Power BI account and sign up for the free Microsoft Fabric trial. If you do not have a Power BI license, sign up for a Power BI free license and then you can start the Fabric trial.
Architecture overview
- Bronze tier: this is Retail Sales Data Lakehouse, where raw retail sales data is stored. Raw product review CSV files are copied into the system. Learn how to use medallion architecture with Lakehouse.
- Silver tier: sales Warehouse that stores the data in a Warehouse table called Product Reviews after first processing. Learn more about Warehouse in Fabric.
- Gold tier: the refined sales data is stored here after added processing in a warehouse in Fabric.
- Data pipelines: to orchestrate the transformation we use Data pipelines and use notebooks, user data functions and copy activity to enable this workflow.
- User data functions: there are two user data functions used here.
- Remove missing values: function to remove missing values in CSV file and ingest in silver tier. This function is the invoked from within a Notebook in Fabric.
- Sentiment score calculation: function to add Sentiment Scores to a table in the gold tier.
- API layer: the refined data is exposed through a Fabric API for GraphQL endpoint through GraphQL queries and update data using mutations.
- Web application: this is an internal portal that can give insights from the refined data in gold tier. This application can run on premises or Azure web app.
Analytical application architecture using Microsoft Fabric
The steps below describe how each part in these solutions helps you transform data from bronze tier to gold tier ready to be consumed by the front-end applications.
- Data pipelines: create a workflow using data pipelines and create the activity for each as mentioned below:
- Get the CSV file into the lakehouse for product reviews.
- Add a notebook activity that invokes the function to clean the data and ingest it into the Notebook.
- Add a functions activity that uses a user data function to compute the sentiment of each comment for product reviews table.
Configure the pipeline to automate this process. Let’s break down each part in the pipelines below.
- Data storage: store raw retail sales data in Lakehouse in Fabric. In this solution we will store all the CSV files with data related to all the comments received from customers about each product. This CSV file can be added to the lakehouse from an external system or from within Fabric.
- Initial processing: create a Silver tier Warehouse in Fabric to store all the clean data that was transformed from the lakehouse. To do this first processing, we will use Data pipelines in Fabric to piece together all the activities needed to process and transform the data.
- Data cleaning and ingestion: create a notebook in Microsoft Fabric to clean and ingest the data into the silver tier Sales Warehouse. In this solution you can use user data functions to clean the CSV files and remove any records where there are no comments for a product and then ingest this data into silver tier to ensure data quality and consistency. User data function securely connects to the warehouse and lakehouse in Fabric without having to deal with storing connection strings for these data sources. This makes it easier to write functions with security in mind.
- Advanced processing: create multiple functions in a user data function. Create a new function that takes the data in the Product Reviews table in silver tier warehouse and calculates the sentiment of each product review comment. This sentiment can then be saved in Gold Tier Sales warehouse to be consumed by the front-end application.
- API development: create a Fabric API for GraphQL endpoint to expose this refined data in the gold tier. The Fabric API offers a seamless way to integrate data into your applications through queries as well as use mutations to change this data. It ensures that your data is always up-to-date and accessible, helping you make informed decisions quickly and efficiently. Use Fabric API for GraphQL endpoint to join data from multiple warehouses or query data using a view or stored procedure. This gives you flexibility to query what data you need for your application.
- Web application: create an internal web application that can provide the stakeholders in your business with clear insights into the customers’ sentiment about their products. You can further use Power BI or a similar tool to visualize the data and generate insights.
Conclusion
To wrap things up, building an end-to-end solution in Microsoft Fabric can help you set up data applications. If you no longer need it, you clean up resources by removing the workspace and other items within in.
You can build a powerful analytical application using Microsoft Fabric, enabling your business to make data-driven decisions with ease. Start building your analytical application with Microsoft Fabric today and get a start with Free trial. Keep exploring, keep innovating, and happy coding!
Submit your feedback on Fabric Ideas and join the conversation on the Fabric Community.
