Azure Synapse Runtime for Apache Spark 3.5 (Preview)
We’re thrilled to announce that we have made Azure Synapse Runtime for Apache Spark 3.5 for our Azure Synapse Spark customers in preview, while they get ready and prepare for migrating to Microsoft Fabric Spark.
Apache Spark 3.5
You can now create Azure Synapse Runtime for Apache Spark 3.5. The essential changes include features which come from upgrading Apache Spark to version 3.5 and Delta Lake 3.2. Please review the official release notes for Apache Spark 3.5 to check the complete list of fixes and features. In addition, review the migration guidelines between Spark 3.4 and 3.5 to assess potential changes to your applications, jobs and notebooks.
For additional details check Azure Synapse Runtime for Apache Spark 3.5 documentation.
Azure Synapse Users
We offer Azure Synapse Runtime for Apache Spark 3.5 to our Azure Synapse Spark customers. However, we strongly recommend that customers plan to migrate to Microsoft Fabric Spark to benefit from the latest innovations and optimizations exclusive to Microsoft Fabric Spark. For example, the Native Execution Engine (NEE) significantly enhances query performance at no additional cost. Starter pools allow the creation of a Spark session within seconds, unified security in the lakehouse enables the definition of RLS (Row-Level Security) and CLS (Column-Level Security) for objects in the lakehouse. Additionally, newly announced Materialized Views and many other features are available.
