Summarize Within calculates statistics in areas where geometries from the input DataFrame are within or overlap specified boundaries. The boundaries can be a polygon geometry column or you can summarize within hexagonal or square bins.
Summarize Within takes a DataFrame and a boundary, and stacks them on top of each other. After stacking, you can look down through the stack and count the number of summarized input records that fall within the input boundaries. You can also calculate simple statistics about the attributes of the input summary records, such as sum, mean, minimum, maximum, and so on.
There are two ways to specify the boundaries:
- Use a polygon DataFrame by specifying
- Use a square or hexagonal bin of a specified size that is generated
when the analysis is run by specifying
- Use a polygon DataFrame by specifying
The bin size specifies how large the bins are. If you are summarizing into hexagons, the size is the height of each hexagon, and the width of the resulting hexagon will be two times the height divided by the square root of three. If you are aggregating into squares, the bin size is the height of the square, which is equal to the width.
Use Summarize Within to calculate standard statistics. Standard statistics summarize the statistical values without weighting, and use the value of the records that intersect an area of interest.
You can calculate the lengths and areas of the summarized geometries within each polygon using the options in the table below. Options are based on the geometry of the summarized DataFrame.
Input geometry Description Option Points The count of summarized points within each boundary. None Linestrings The length of summarized linestrings within or intersecting each boundary.
Polygons The area of summarized polygons within or intersecting each boundary.
- Square Miles
- Square Yards
- Square Feet
- Square Kilometers
- Square Meters
For standard statistics, there are eight options: count, sum, mean, minimum, maximum, range, standard deviation, and variance. There are two options for string statistics: count and any.
Analysis with binning requires that your input DataFrame's geometry has a projected coordinate system. If your data is not in a projected coordinate system, the tool will transform your input to a World Cylindrical Equal Area (SRID: 54034) projection. You can transform your data to a projected coordinate system by using ST_Transform.
Optionally, specify a field name using
setso statistics are calculated separately for each unique field value. When a group by field value is specified, a summary table listing each record and statistic is also created.
minor_ major_ fields
include_are part of the group by option (
set). The minority and majority will be the least and most dominant value from the group field, respectively, where dominance is determined using the count of points, total length, or total area of each value.
When the specified value for
include_is True, two fields will be added to the result DataFrame. The fields will list the values from the group field that are the minority and majority for each result.
minor_ major_ fields
include_option can only be used when you specify a value of True for
include_. When the value specified for
minor_ major_ fields
True, two fields will be added to the result DataFrame listing the percentage of the count of points, total length, or total area that belong to the minority and majority values for each input record. A percentage field will also be added to the result table listing the percentage of the count of points, total length, or total area that belong to all values from the group by field for each input record.
The output DataFrame always contains polygons. Only polygons that intersect the summarized geometries will be returned. Other polygons will be completely removed from the result.
- You can only calculate statistics on the records that intersect your boundary. No proportional weighting is applied.
The following fields are included in the output polygon DataFrame:
|The result bin geometries.|
|The count of summarized records that intersect each boundary.|
|The total length of linestrings within or intersecting the boundary or total area of summarized polygons within or intersecting a polygon. These values are returned when you specify a value of True for |
|Specified statistics will each create a field named in the following format: |
|This value is returned when you create a group-by table and specify minority and majority calculations. This represents the values for the specified field that is the minority in each polygon. For example, there are five points within a polygon with a field called color and values of red, blue, blue, green, green. If you create a group by the color field, the value for the minority_color field is red.|
|This value is returned when you create a group-by table and specify minority and majority calculations. This represents the values for the specified field that is the majority in each polygon. For example, there are five points within a polygon with a field called color and values of red, blue, blue, green, green. If you create a group by the color field, the value for the majority_color field is blue;green.|
|This value is returned when you create a group-by table and specify percent shapes. This represents the percentages of the count for the specified field that is the minority in each polygon. For example, there are five points within a polygon with a field called color and values of red, blue, blue, green, green. If you create a group by the color field, the value for the minority_color_percent field is 20 (calculated as 1/5).|
|This value is returned when you create a group-by table and specify percent shapes. This represents the percentages of the count for the specified field that is the majority in each polygon. For example, there are five points within a polygon with a field called color and values of red, blue, blue, green, green. If you create a group by the color field, the value for the majority_color_percent field is 40 (calculated as 2/5).|
|This value is returned when you create a group-by table. This is an ID to link records to the group-by table. Every join_id field corresponds to one or more records in the group-by table.|
The following fields are included in the output group-by DataFrame:
|This is an ID to link records to the polygon DataFrame. Each polygon will have one or more records with the same ID that represent all of the group-by values. For example, there are five points within a polygon with a field called color and values of red, blue, blue, green, green. The group-by table will have three records representing that polygon (same join ID), one for each of the colors red, blue, and green.|
|The count of the specified group within the joined polygon. For example, red is 1 for the selected polygon.|
|Any specified statistic calculated for each group.|
|The percentage each group contributes to the total count in the polygon. Using the above example, red contributes 1/5 = 20, blue contributes 2/5 = 40, and green contributes 2/5 = 20.|
Improve the performance of Summarize Within by doing one or more of the following:
Only analyze the records in your area of interest. You can pick the records of interest by using one of the following SQL functions:
- If you are using bins, larger bins will perform better than smaller bins. If you are unsure which size to use, start with a larger bin to prototype.
For more details, go to the GeoAnalytics Engine API reference for summarize within.
|Adds a summary statistic of a field in the input DataFrame to the result DataFrame.||No|
|Sets to the tool to calculate statistics based on the geometry type of the primary geometry column in the input DataFrame, such as the length of lines or areas of polygons within each summary polygon.||No|
|Runs the Summarize Within tool using the provided DataFrame.||Yes|
|Sets a field from the input DataFrame that will be used to calculate statistics for each unique value. When ||No|
|Sets the size and shape of bins that the input DataFrame will be summarized into.||One of |
|Sets the DataFrame containing a column of polygons that the input DataFrame will be summarized into.||One of |
# Log in import geoanalytics geoanalytics.auth(username="myusername", password="mypassword") # Imports from geoanalytics.tools import SummarizeWithin from geoanalytics.tools import ReconstructTracks from geoanalytics.sql import functions as ST # Path to the hurricane tracks dataset hurricanes_data_path = r"https://services2.arcgis.com/FiaPA4ga0iQKduv3/arcgis/rest/" \ "services/IBTrACS_ALL_list_v04r00_points_1/FeatureServer/0" # Create a hurricanes tracks DataFrame and filter to a smaller extennt of area hurricanes_df = spark.read.format("feature-service").load(hurricanes_data_path) \ .withColumn("env_intersects", ST.env_intersects("shape",-10512137.72, -9527997.38,3278846.39,4303954.46)) \ .where("env_intersects == 'true'") \ .where("BASIN == 'NA'") # Use Reconstruct Tracks to create hurricane paths rt_result = ReconstructTracks() \ .setTrackFields("NAME") \ .setDistanceMethod(distance_method="Planar") \ .run(dataframe=hurricanes_df) # Use Summarize Within to summarize hurricane tracks into bins to # visualize a track heat map result = SummarizeWithin() \ .setSummaryBins(bin_size=200, bin_size_unit="Kilometers", bin_type='hexagon') \ .includeShapeSummary(include=True, units="Kilometers") \ .run(dataframe=rt_result)
# Plot the summarized result with shorelines (continent outlines) near Florida continents_path = "https://services.arcgis.com/P3ePLMYs2RVChkJx/ArcGIS/rest/" \ "services/World_Continents/FeatureServer/0" shoreline_df = spark.read.format("feature-service").load(continents_path) result_plot = result.output.st.plot(cmap_values="COUNT1", cmap="coolwarm", figsize=(16,10), legend=True) shoreline_plot = shoreline_df.st.plot(edgecolors="black", alpha=0.3, figsize=(16,10),ax=result_plot) result_plot.set_title("Hurricane track heat map near Florida and the Gulf of Mexico") result_plot.set_xlabel("X (Meters)") result_plot.set_ylabel("Y (Meters)"); result_plot.set_xlim(left=-10512137, right=-8027997) result_plot.set_ylim(bottom=2688846, top=4303954)