Streamline Data Engineering & Data Science with Copilot in Fabric
In today’s data-driven world, it’s important to be able to read & understand your data to enable you to gain insights on which patterns/trends you want to monitor. After a data driven decision is taken, you can understand which visualizations you want to build and which machine learning models you want to train to give further insight into your data.
Contoso Retailers, a fictitious company, wants to learn how they can unlock the full potential of their data science and data engineering efforts with Copilot for Data Science and Data Engineering using Fabric.
Using Copilot for Data Science & Data Engineering
Contoso Retailers aims to apply data science and data engineering techniques to track sales patterns and gain valuable insights. By analyzing these trends, they can implement new strategies to enhance their operations and stay ahead in the competitive market. Leveraging Copilot for Data Science & Data Engineering in Microsoft Fabric, they plan to generate precise visualizations and sophisticated machine learning models to make predictions on customer behavior so they can build a targeted marketing campaign.
With Copilot for Data Science & Data Engineering you can explore, apply predictive analytics and use it as a learning tool for your dataset.
Enhancing Data Engineering for predictive analysis with Copilot
Before you can prepare your data for predictive analysis, you need to create a Notebook that is connected/using a Lakehouse where your data sits. You can learn more about Notebooks in Microsoft Fabric by referring to – Explore the data in your lakehouse with a notebook documentation.
- Inside your Notebook, you need to make sure that Copilot is enabled to use it on the chat panel and/or as cell magics on the notebook and that your data is loaded into a dataframe.
- For this requirement you can import a notebook from the Copilot in Fabric tutorial files resource. As an initial step you can use Copilot to assist your team to better understand the code in the imported notebook or a shared notebook through chat magics in the notebook cell.
Example chat magics and prompt:
%%chat
Explain this block of code step by step
- After running the code cell, Copilot will provide an explanation of what the code does as an output of the code cell, and it may include concepts that might be unfamiliar to a beginner in the team. You can use the Copilot Chat panel or chat magics to define what those concepts are to make sure that team understands the implemented code and concepts.
- You can also get an overview of the data loaded into the dataframe using the chat magics to describe it. This allows you to better understand how the data is structured and may get further insights on which columns have empty values/need further transformations.
Example chat magics and prompt:
%describe
df_cust_details
- Within this scenario, you can add other columns like ‘age’ to enable Contoso Retailers to gain better insights to monitor the purchasing behavior of their customers based on their age groups and which products are more popular. The added benefit of using the Copilot chat panel is that you get a breakdown of what the code does line by line.
Example prompt: Add a column ‘age’ to df_cust_details using 2012 as the current year
Build visualizations and a predictive analysis model with Copilot
Using the same notebook you can use Copilot for Data Science to build visualizations and train machine learning models based on your data. This enables you to continue from your data engineering work and transition into data science without reworking your dataset.
- For this requirement, you can use Copilot chat panel to enable the team to gain better insights of the distribution of the age within their customer base so they can tailor marketing efforts accordingly.
Example prompt: Visualize the distribution of customer ages using a histogram to understand age demographics
- Within this scenario, Contoso Retailers wants to build an analytical model to predict the likelihood of a customer to purchase a bike, you need to transform the data to add a new column that indicates if a customer has purchased a bike or not.
Example prompt: Add a new column ‘IsBikeBuyer’ with a value of 1 for rows where ‘ProductCategory’ is ‘Bikes’, and 0 otherwise.
- Lastly, Copilot can enable the team to build an analytical model for this scenario by suggesting a predictive machine learning model using the data loaded in a dataframe.
Example prompt: Suggest how we can build a predictive machine learning model using df_cust_details to predict if a customer is likely to buy a bike or not to help Adventure Works, the bike shop, build a targeted marketing campaign, the ‘IsBikeBuyer’ column is the target column.
Tip: As an added benefit, Copilot will explain exactly how the code works and why it suggested the predictive model so you can see if it fits your scenario.
Next Steps
There’s more you can do for a retail scenario with Copilot for Data Science & Data Engineering, this is a baseline of the thought process of understanding your dataframes, code and training predictive machine learning models. This can be applied to other scenarios; you have to mindful of what is needed based on your requirements.
You can find a deep dive of using Copilot for Data Science & Data Engineering in our Copilot Learning Hub for Data professional’s tutorial.
Resources
Start your Copilot learning journey
Copilot for Data Science & Data Engineering overview
