Fabric Espresso – Episodes about Data Integration and Data Engineering in Microsoft Fabric
For the past 1.5 years, the Microsoft Fabric Product Group Product Managers have been publishing a YouTube series featuring deep dives into Microsoft Fabric’s features. These episodes cover both technical functionalities and real-world scenarios, providing insights into the product roadmap and the people driving innovation. With over 80+ episodes, the series serves as a valuable resource for anyone looking to understand and optimize their use of Microsoft Fabric.
All episodes are available at https://aka.ms/fabric-espresso making it easy to explore the entire catalog. However, to enhance the learning experience, we are launching a short series of blog posts that will categorize the episodes into thematic groups, providing explanations and key takeaways for each.
Fabric Espresso & Data Engineering
This week, we focus on Data Integration and Engineering in Microsoft Fabric. We are providing a curated list of episodes that delve into various aspects of data engineering, including Spark optimizations, data pipelines, storage solutions, and performance tuning.
Click the link to view each episode:
1. Security in Fabric Data Engineering
2. Microsoft Fabric Spark Diagnostic Emitter for Logs and Metrics
3. High Concurrency Mode for Notebooks in Pipelines for Fabric Spark
4. Microsoft Fabric environment – A consolidated item for all your hardware and software settings
5. Job Queueing for Notebook in Microsoft Fabric | Spark Compute
6. Apache Airflow Job in Microsoft Fabric
7. Lifecycle of Apache Spark Runtimes in MS Fabric Unpacking Experimental Preview, GA & End of Support
8. Spark in Microsoft Fabric and Spark SQL for DW Users
9. Interoperability of Delta Lake table format in Fabric
10. Apache Spark Runtimes in Microsoft Fabric – Runtime 1.3 based on Apache Spark 3.5
11. ML based Autotune for Apache Spark Jobs in MS Fabric performance optimization for recurrent jobs
12. Microsoft Fabric Optimistic Job Admission for Apache Spark
13. Native execution engine for Apache Spark in Fabric
14. Fabric Apache Spark Autotune and Run Series Job Analysis in Monitoring Hub
15. Fabric Apache Spark Jobs monitoring capabilities – Resource Usage Analysis – watch here
16. Microsoft Fabric Data Engineering – Notebooks Monitoring and Apache Spark Jobs Advisor
17. Microsoft Fabric Data Engineering – Apache Spark Jobs Monitoring
18. Getting the most out of the Staging Mechanisms in Dataflows Gen2 Fabric Data Factory
19. Dataflow Gen2 data destinations in Fabric Data Factory
20. What are Dataflows Gen2 in Fabric Data Factory?
21. Medallion Architecture Data Design and Lakehouse Patterns | Microsoft Fabric Data Factory
22. Microsoft Fabric Data Engineering – How to make the reference file work in Spark Job Definitions?
23. Fabric Spark Compute Capabilities – Azure VM’s and their impact on performance
24. Data Engineering Starter Kit – Quick Start with Fabric Product Group
25. Microsoft Fabric Spark Utilities – mssparkutils
26. Microsoft Fabric Spark integration and VS Code
27. Microsoft Fabric Notebooks – Showcase with advanced features
28. Spark Data Engineering Patterns Optimizing Delta Tables for Power BI in Microsoft Fabric
29. Spark Data Engineering Patterns – Shortcuts and external tables
30. High Concurrency Mode in Microsoft Fabric
31. Spark Compute in Fabric Data Engineering and Data Science – Starter Pools vs Custom Pools Unveiled!
32. Data Engineer perspective on loading data for Fabric: Uploads, Data Flows, Copy Tool or Shortcuts?
33. Top Microsoft Fabric features that every Data Engineer should know
34. Fabric Espresso: Loading existing Delta Tables to Silver!
35. Fabric Espresso: Loading Data from Landing to Bronze!
36. Fabric Espresso: Building a Landing zone in Microsoft Fabric
37. Fabric Espresso: Tap into existing data with Shortcuts
38. Fabric Espresso: OneLake as the glue In Microsoft Fabric
39. Enabling Data Mesh with OneLake on Microsoft Fabric
40. Data Engineer’s perspective on storing data in Fabric: Lakehouse or Warehouse?
Stay tuned!
We will be sharing a weekly post for the next few weeks sharing more of our Espresso video series.
