Summarize data using binning functions
Spatial binning functions (spatial aggregation) give you a way to summarize spatial data for visualization and analysis. Summarizing spatial data is useful for both visualization of large datasets, and analysis. Many GeoAnalytics Engine tools use binning functionality as a core component of analysis, such as Summarize Within and Aggregate Points. In this tutorial you will learn how to use spatial binning functions such as ST_SquareBin, ST_SquareBins, ST_HexBin, ST_HexBins, ST_H3Bin, ST_H3Bins, ST_BinCenter and ST_BinGeometry, and Spark SQL aggregate functions to recreate and customize the workflow of many GeoAnalytics Engine aggregation tools.
Steps
Setup modules and data
Import
geoanalytics
and PySpark SQL functions and read in the LA Ozone dataset.PythonUse dark colors for code blocks Copy PythonUse dark colors for code blocks Copy ResultUse dark colors for code blocks Copy PythonUse dark colors for code blocks Copy ResultUse dark colors for code blocks Copy Read in the US census data and create a DataFrame that is filtered to only show LA county.
PythonUse dark colors for code blocks Copy ResultUse dark colors for code blocks Copy PythonUse dark colors for code blocks Copy ResultUse dark colors for code blocks Copy
Count the points within bins
It's common to use binning functions with Spark groupBy expressions to statistics such as the count of points within each bin. The example below shows
how to count points within bins using the square_
function.
Bin the point geometry column of Ozone data with the specified bin size and aggregate the count of points by bin ID.
PythonUse dark colors for code blocks Copy ResultUse dark colors for code blocks Copy
Calculate summary statistics for bins
Binning functions also can leverage the aggregate functions in PySpark, and perform additional statistics and summarize by bin area. The example below shows how to calculate basic statistics on the attribute columns for each bin area.
Specify the
bin_
based on your data and use asize group
expression to add statistics in the resulting DataFrame.B y PythonUse dark colors for code blocks Copy ResultUse dark colors for code blocks Copy
Summarize nearby records using bin centers
In addition to generating summary statistics within bins, you can calculate statistics on neighboring bins, or bins withing a specified distance of another bin's centroid. Below is an example of calculating statistics on nearby bins. The example below calculates summary bins within a distance of 35.5 degrees from the bin centroid.
Calculate the bin centroids of the bins that cover California.
PythonUse dark colors for code blocks Copy ResultUse dark colors for code blocks Copy Use a specified distance and bin center to filter the bin geometries that are within the distance to the centroid of the California bins. Then use PySpark functions such
as.describe()
to generate basic summary statistics on the attribute columns of the bins nearby.PythonUse dark colors for code blocks Copy ResultUse dark colors for code blocks Copy
Visualize bins results
Visualizing bins results provides a quick representation of how your data is distributed spatially. You can visualize bins overlaid with geometry records to identify spatial patterns or attribute characteristics. You can also visualize with basemaps to put the bin results in geographical context.
Call
st.plot
on the DataFrame containing the bins column.PythonUse dark colors for code blocks Copy Overlay the point geometry with the bins to better visualize the spatial relationships.
PythonUse dark colors for code blocks Copy To learn more about visualization, see the tutorial on visualizing results using st.plot().
Add a color ramp to the plotted bins to represent the count values.
PythonUse dark colors for code blocks Copy Close out the temp directories created at the beginning of the tutorial.
PythonUse dark colors for code blocks Copy