Microsoft Fabric Updates Blog

Running Apache Airflow jobs seamlessly in Microsoft Fabric

Apache Airflow is a powerful platform to programmatically author, schedule, and monitor workflows. It is widely adopted for its flexibility, scalability, and ability to handle complex workflows with ease. With Apache Airflow, you can orchestrate your data pipelines, ensuring they run smoothly and efficiently. In May 2024, we launched the preview of Apache Airflow job … Continue reading “Running Apache Airflow jobs seamlessly in Microsoft Fabric”

High Concurrency mode for notebooks in pipelines (Generally Available)

High Concurrency mode for notebooks in pipelines is now generally available (GA)! This powerful feature enhances enterprise data ingestion and transformation by optimizing session sharing within one of the most widely used orchestration mechanisms. With this release, we’re also introducing Comprehensive Monitoring for High-Concurrency Spark Applications, bringing deeper visibility and control to your workloads. Key … Continue reading “High Concurrency mode for notebooks in pipelines (Generally Available)”

Supercharge your workloads: write-optimized default Spark configurations in Microsoft Fabric

Introducing predefined Spark resource profiles in Microsoft Fabric—making it easier than ever for data engineers to optimize their compute configurations based on workload needs. Whether you’re handling read-heavy, write-heavy, or mixed workloads, Fabric now provides a property bag-based approach that streamlines Spark tuning with just a simple setting. With these new configurations, users can effortlessly … Continue reading “Supercharge your workloads: write-optimized default Spark configurations in Microsoft Fabric”