Azure Synapse Runtime for Apache Spark 3.5 (Generally Available)
We have made Azure Synapse Runtime for Apache Spark 3.5 for our Azure Synapse Spark customers generally available for customers to start using it for their production workloads, while they get ready and prepare for migrating to Microsoft Fabric Spark.
What This Mean for You
You can now create Azure Synapse Runtime for Apache Spark 3.5. The essential changes include features which come from upgrading Apache Spark to version 3.5 and Delta Lake 3.2. Please review the official release notes for Apache Spark 3.5 to check the complete list of fixes and features. In addition, review the migration guidelines between Spark 3.4 and 3.5 to assess potential changes to your applications, jobs and notebooks.
For additional details check Azure Synapse Runtime for Apache Spark 3.5 documentation.
What’s next?
We offer Azure Synapse Runtime for Apache Spark 3.5 to our Azure Synapse Spark customers. However, we strongly recommend that customers plan to migrate to Microsoft Fabric Spark to benefit from the latest innovations and optimizations exclusively available to Microsoft Fabric Spark. For example, the Native Execution Engine (NEE) significantly enhances query performance at no additional cost. Starter pools allow the creation of a Spark session within seconds, unified security in the lakehouse enables the definition of RLS (Row-Level Security) and CLS (Column-Level Security) for objects in the lakehouse. Additionally, newly announced Materialized Views and many other features are available.
