Performing Data Enrichment

The arcgis.geoanalytics.data_enrichment submodule contains functions that are used for data enrichment using geoanalytics

Table of contents

This toolset uses distributed processing to complete analytics on your GeoAnalytics Server.

Note: The purpose of the notebook is to show examples of the different tools that can be run on an example dataset.

In [2]:
# connect to Enterprise GIS
from arcgis.gis import GIS
import arcgis.geoanalytics

portal_gis = GIS("your_enterprise_profile")
In [3]:
search_result1 = portal_gis.content.search("bigDataFileShares_hurricanes", item_type = "big data file share")[0]
search_result1
Out[3]:
bigDataFileShares_all_hurricanes
Big Data File Share by admin
Last Modified: October 28, 2021
0 comments, 0 views
In [5]:
search_result2 = portal_gis.content.search("bigDataFileShares_ServiceCallsOrleans", item_type = "big data file share")[0]
search_result2
Out[5]:
bigDataFileShares_ServiceCallsOrleans
Big Data File Share by portaladmin
Last Modified: October 05, 2019
0 comments, 0 views
In [6]:
calls = search_result1.layers[0]
In [4]:
grid_item = portal_gis.content.get('f61cf232eecb49f89a089aedbcc55c51')
grid_item
Out[4]:
build_multivariable_grid
build_multivariable_gridFeature Layer Collection by arcgis_python
Last Modified: June 24, 2021
0 comments, 2 views
In [7]:
grid_lyr = grid_item.layers[0]

Enrich From Grid

The enrich_from_grid tool joins attributes from a multivariable grid to a point layer. The multivariable grid must be created using the build_multivariable_grid task. Metadata from the multivariable grid is used to efficiently enrich the input point features, making it faster than the Join Features task. Attributes in the multivariable grid are joined to the input point features when the features intersect the grid.

The attributes in the multivariable grid can be used as explanatory variables when modeling spatial relationships with your input point features, and this task allows you to join those attributes to the point features quickly.

In [2]:
from arcgis.geoanalytics.data_enrichment import enrich_from_grid
In [8]:
enrich_from_grid(calls, grid_lyr, output_name='enrich from grid test') 
Attaching log redirect
Log level set to DEBUG
Detaching log redirect
Out[8]:
enrich_from_grid_test
enrich_from_grid_testFeature Layer Collection by arcgis_python
Last Modified: June 25, 2021
0 comments, 0 views

Calculate Motion Statistics

The Calculate Motion Statistics tool calculates motion statistics and descriptors for time-enabled points that represent one or more moving entities. The time-enabled point data must include features that represent an instant in time.

Points are grouped together into tracks representing each entity using a unique identifier. Motion statistics are calculated at each point using one or more points in the track history. Calculations include summaries of distance traveled, duration, elevation, speed, acceleration, bearing, and idle status. The result is a new point layer enriched with the requested statistics.

For example, a city could be monitoring snowplow operations and want to better understand vehicle movement. The Calculate Motion Statistics tool could be used to determine idle locations, time spent idling, average and maximum speeds over time, total distance covered, and other statistics.

In [8]:
from arcgis.geoanalytics.data_enrichment import calculate_motion_statistics
In [4]:
hurricanes = search_result2.layers[0]
In [7]:
hurricanes.properties
Out[7]:
{
  "dataStoreID": "2d305bf2-3bd4-4e33-ac95-7bb5d987b494",
  "fields": [
    {
      "name": "serial_num",
      "type": "esriFieldTypeString"
    },
    {
      "name": "season",
      "type": "esriFieldTypeInteger"
    },
    {
      "name": "num",
      "type": "esriFieldTypeInteger"
    },
    {
      "name": "basin",
      "type": "esriFieldTypeString"
    },
    {
      "name": "sub_basin",
      "type": "esriFieldTypeString"
    },
    {
      "name": "name",
      "type": "esriFieldTypeString"
    },
    {
      "name": "iso_time",
      "type": "esriFieldTypeString"
    },
    {
      "name": "nature",
      "type": "esriFieldTypeString"
    },
    {
      "name": "latitude",
      "type": "esriFieldTypeDouble"
    },
    {
      "name": "longitude",
      "type": "esriFieldTypeDouble"
    },
    {
      "name": "wind_wmo_",
      "type": "esriFieldTypeDouble"
    },
    {
      "name": "pres_wmo_",
      "type": "esriFieldTypeInteger"
    },
    {
      "name": "center",
      "type": "esriFieldTypeString"
    },
    {
      "name": "wind_wmo1",
      "type": "esriFieldTypeDouble"
    },
    {
      "name": "pres_wmo1",
      "type": "esriFieldTypeDouble"
    },
    {
      "name": "track_type",
      "type": "esriFieldTypeString"
    },
    {
      "name": "size",
      "type": "esriFieldTypeString"
    },
    {
      "name": "Wind",
      "type": "esriFieldTypeInteger"
    }
  ],
  "name": "hurricanes",
  "geometryType": "esriGeometryPoint",
  "type": "featureClass",
  "spatialReference": {
    "wkid": 4326
  },
  "geometry": {},
  "time": {
    "timeType": "instant",
    "timeReference": {
      "timeZone": "UTC"
    },
    "fields": [
      {
        "name": "iso_time",
        "formats": [
          "yyyy-MM-dd HH:mm:ss"
        ]
      }
    ]
  },
  "currentVersion": 10.91,
  "children": []
}
In [9]:
hurricanes_motion_stats = calculate_motion_statistics(hurricanes, 
                                                      motion_statistics=["speed", 
                                                                         "acceleration", 
                                                                         "bearing"],
                                                      track_fields='track_type',
                                                      track_history_window=5,
                                                      dist_method='Geodesic',
                                                      output_name="Hurricanes_MotionStats")
hurricanes_motion_stats
Out[9]:
Hurricanes_MotionStats
Hurricanes_MotionStatsFeature Layer Collection by admin
Last Modified: January 09, 2022
0 comments, 0 views

image.png

In this guide, we learned about data enrichment tools. In the next guide, we will learn how to manage big data using arcgis.geoanalytics.manage_data.

Tool
Description

enrich_from_grid

The Enrich From Multi-Variable Grid tool joins attributes from a multivariable grid to a point layer.

calculate_motion_statistics

The Calculate Motion Statistics tool calculates motion statistics and descriptors for time-enabled points that represent one or more moving entities. The time-enabled point data must include features that represent an instant in time.

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.