1.2.1 Patch Release notes
The 1.2.1 patch release of ArcGIS GeoAnalytics Engine fixes issues related to spatial left joins, H3 binning in the Aggregate Points tool, and correctness of the Group By Proximity tool. The patch also includes improvements to feature service read and write, such as improved error reporting and improved resiliency when writing large datasets.
1.2.0 Release notes
SQL functions
Version 1.2.0 adds 8 new SQL functions to GeoAnalytics Engine:
-
ST_Centerline—Creates linestrings at the approximate center of long, thin polygons. This is especially useful for polygons that represent hydrographic features such as streams and rivers.
-
ST_Flip, ST_Rotate, ST_Scale, ST_Shear, and ST_Translate—Perform affine transformations on point, line, polygon, and multipoint geometries. Affine transformations are geographic transformations that preserve parallelism but may not preserve distances or angles.
-
ST_Aggr_Linestring—Creates a linestring from the points in each group of a grouped DataFrame. A numeric field is used to order the points.
-
ST_BboxIntersects—Checks if a geometry intersects an axis-aligned rectangle (also known as an envelope or bounding box) defined using minimum and maximum x,y coordinates. This was formerly the behavior of ST_EnvIntersects, which has been changed to check whether the envelopes of two geometries intersect.
Tools
GeoAnalytics Engine 1.2.0 includes added support for performing geodesic distance calculations in the Nearest Neighbors tool. Geodesic calculations are recommended for accurate analysis of larger, global areas and when your data is not in a local projection. For more information see Planar vs. geodesic distance calculations.
Also new is the option to return the near distance and duration from one-to-many near joins in the Spatiotemporal Join tool. This can be useful if the distance between your joined geometries is important, and you don't want to calculate it in a separate step.
In version 1.2.0, H3 has been added to the Aggregate Points tool as a bin type option. You can now use Aggregate Points as an alternative to H3 binning functions for generating H3 bins and summarizing the points located within each bin. You can also aggregate into time-stepped H3 bins to see both spatial and temporal patterns using H3 indices.
Data sources
GeoAnalytics Engine 1.2.0 adds the ability to read from file geodatabases so that you can easily load feature classes and tables into PySpark DataFrames. Like other data sources included with GeoAnalytics Engine, geometry columns are created automatically when reading from a file geodatabase feature class so that you can connect to your spatial data with just one line of code.
This release also includes support for more authentication schemes when connecting to ArcGIS Online or ArcGIS Enterprise. You can now register a GIS protected with SAML, PKI, or OAuth 2.0 when reading or writing feature services.
The feature service data source was also improved to include support for reading 64-bit signed integers
(esri in ArcGIS) and several bug fixes.
Additional documentation
The new Data Storage section in the GeoAnalytics Engine guide provides guidance for bringing your big data into Spark from common storage locations like HDFS, Amazon S3, Azure Blob Storage, Azure Data Lake Storage Gen2, and Google Cloud Storage.
Documentation and tutorials on connecting to databases with Java Database Connectivity (JDBC) demonstrates how to load your spatial data into Spark DataFrames from databases in Microsoft SQL Server, Oracle, and PostgreSQL. For step-by-step instructions see Read and write to databases.
Spark 3.4 and new cloud runtime support
With every release, support for the latest Spark versions and cloud runtimes are added if possible. In 1.2.0, GeoAnalytics Engine adds support for Spark 3.4.1 and the latest of the following cloud runtimes:
- Databricks (12.2 LTS, 13.0, 13.1)
- AWS EMR (6.10.0, 6.11.0)
- Azure Synapse (3.3)
Reorganized Projection Engine data jars
At 1.2.0, the supplemental Projection Engine data jars that come with GeoAnalytics Engine have been reorganized so that each jar contains the transformation data for a specific region. This provides more flexibility to only install the data that you need. For more information see the README included with the Projection Engine data jars, as well as Geographic transformations.