1.7.0 Release notes
Spark 4.0 and Scala 2.13 Support
GeoAnalytics Engine 1.7.0 includes added support for Apache Spark 4.0. This is a major release of Spark with significant new features.
Beginning with GeoAnalytics Engine 1.7.0, install files are provided for both Scala 2.12 and Scala 2.13. You can choose which install files to use depending on which version of Scala your Spark runtime was built for. For example, Spark 4.0.0 is built only for Spark 2.13, so you would use the following GeoAnalytics Engine 1.7.0 jars with that runtime:
geoanalytics_2.13-1.7.0.jar geoanalytics-natives_2.13-1.7.0.jar
With Spark 3.2.x through Spark 3.5.x, Spark can be built for either Scala 2.12 or Scala 2.13. For example, say you wanted to use Spark 3.5.x built for Scala 2.12 - to use GeoAnalytics Engine 1.7.0 with that runtime, you would use the following jars:
geoanalytics_2.12-1.7.0.jar geoanalytics-natives_2.12-1.7.0.jar
New Spark and cloud runtime support
In 1.7.0, GeoAnalytics Engine added support for Spark 4.0.0, 3.5.6, and the latest of the following cloud runtimes:
- Databricks (16.4 LTS Scala 2.13, 17.0, 17.1, and 17.2 (Beta))
- AWS EMR (7.10.0)
- Google Dataproc (2.3-debian12, 2.3-ubuntu22, 2.3-rocky9)
- Azure Synapse (3.5)
Performance enhancements
GeoAnalytics Engine 1.7.0 enhanced the geocoding process to better handle failures, ensuring geocoding tasks run more reliably in cloud environments.
Left-outer join in the Nearest Neighbors tool
GeoAnalytics Engine 1.7.0 added support for left-outer join in the Nearest Neighbors tool. You can specifies whether all query records (left-outer join) or only those that have the nearest neighbors (inner join) to be returned in the result DataFrame.