We have made Azure Synapse Runtime for Apache Spark 3.5 for our Azure Synapse Spark customers generally available for customers to start using it for their production workloads, while they get ready and prepare for migrating to Microsoft Fabric Spark. What This Mean for You You can now create Azure Synapse Runtime for Apache Spark … Continue reading “Azure Synapse Runtime for Apache Spark 3.5 (Generally Available)”
Set It and Forget It Target File Size Optimization What if you could enable a single setting and never worry about file size tuning again? Or if your tables automatically adjusted their optimal file sizes as they grew from megabytes to terabytes, without any manual intervention? Today’s data teams face a familiar challenge. Too small, … Continue reading “Adaptive Target File Size Management in Fabric Spark”
We’re introducing a new feature that gives you more granular control over your Spark compute resources in Microsoft Fabric: The Job-Level Bursting Switch. This highly anticipated addition empowers capacity administrators to fine-tune how Spark jobs utilize burst capacity, optimizing for either peak performance or higher concurrency based on your specific workload needs. Microsoft Fabric’s Compute … Continue reading “Introducing the Job-Level Bursting Switch in Microsoft Fabric”
Microsoft OneLake is the unified data lake for your entire organization, built into Microsoft Fabric. It provides a single, open, and secure foundation for all your analytics workloads – eliminating data silos and simplifying data management across domains. The preview of Microsoft OneLake Table APIs, a new way to programmatically manage and interact with your … Continue reading “OneLake Table APIs (Preview)”
End Write Amplification and Automate Your Table Maintenance Compaction is one the most necessary but also challenging aspects of managing a Lakehouse architecture. Similar to file systems and even relational databases, unless closely managed, data will get fragmented over time, and can lead to excessive compute costs. The OPTIMIZE command exists to solve for this … Continue reading “Introducing Optimized Compaction in Fabric Spark”
