Executing big data tools

The GeoAnalytics tools are presented through set of sub modules within the geoanalytics module. To view the list of tools available, refer to the page titled Working with big data. In this page, we will learn how to execute big data tools.

Ensuring your GIS supports GeoAnalytics

Before executing a tool, we need to ensure the ArcGIS Enterprise GIS is set up with a licensed GeoAnalytics server. To do so, call the is_supported() method.

In [ ]:
# connect to Enterprise GIS
from arcgis.gis import GIS
import arcgis.geoanalytics

gis = GIS("http://dev003246.esri.com/portal", "arcgis_python_api", "sharing.1")
In [ ]:
# check if GeoAnalytics is supported
arcgis.geoanalytics.is_supported()
Out[ ]:
True

When no parameters are specified is_supported() method like the rest of the methods in the geoanalytics module, uses the active GIS connection from the arcgis.env module. However if you are working with more than one GIS objects, you can specify the desired GIS object to the gis parameter of this method. For instance, let us create an anonymous connection to ArcGIS Online and check if GeoAnalytics is supported.

In [ ]:
ago_gis = GIS()
arcgis.geoanalytics.is_supported(ago_gis)
Out[ ]:
False

Executing a GeoAnalytics tool

Looking for big data file share items

When you add a big data file share datastore, a corresponding item gets created on your portal. You can search for it like a regular item and query its layers.

In [ ]:
search_result = gis.content.search("", item_type = "big data file share")
search_result
Out[ ]:
[<Item title:"bigDataFileShares_hurricanes_1m_168yrs" type:Big Data File Share owner:admin>,
 <Item title:"bigDataFileShares_NYC_taxi" type:Big Data File Share owner:admin>,
 <Item title:"bigDataFileShares_hurricanes" type:Big Data File Share owner:admin>,
 <Item title:"bigDataFileShares_Chicago_accidents" type:Big Data File Share owner:admin>,
 <Item title:"bigDataFileShares_hurricanes_all" type:Big Data File Share owner:admin>,
 <Item title:"bigDataFileShares_NYCdata" type:Big Data File Share owner:admin>]
In [ ]:
data_item = search_result[5]
data_item
Out[ ]:
bigDataFileShares_NYCdata
Big Data File Share by admin
Last Modified: November 26, 2016
0 comments, 0 views
In [ ]:
data_item.layers
Out[ ]:
[<Layer url:"https://dev003247.esri.com/gax/rest/services/DataStoreCatalogs/bigDataFileShares_NYCdata/BigDataCatalogServer/sampled">]
In [ ]:
year_2015 = data_item.layers[0]
year_2015
Out[ ]:
<Layer url:"https://dev003247.esri.com/gax/rest/services/DataStoreCatalogs/bigDataFileShares_NYCdata/BigDataCatalogServer/sampled">

Executing the Aggregate Points tool

The aggregate_points() tool can be accessed through the summarize_data module. In this example, we are using this tool to aggregate the numerous points into 1 Kilometer square blocks. The tool creates a feature layer as an output which can be accessed once the processing is complete.

In [ ]:
from arcgis.geoanalytics.summarize_data import aggregate_points
In [ ]:
arcgis.env.process_spatial_reference=3857
In [ ]:
agg_result = aggregate_points(year_2015, bin_size=1, bin_size_unit='Kilometers')
Submitted.
Executing...
Executing (AggregatePoints): AggregatePoints "Feature Set" # 1 Kilometers # # # # # # # {"serviceProperties":{"name":"Aggregate_Points_Analysis_DJY1VK","serviceUrl":"http://Dev003246.esri.com/server/rest/services/Hosted/Aggregate_Points_Analysis_DJY1VK/FeatureServer"},"itemProperties":{"itemId":"1066499553a64bf683e2883f9953bcd6"}} {"processSR":{"wkid":3857}}
Start Time: Sat Nov 26 22:27:38 2016
Using URL based GPRecordSet param: https://dev003247.esri.com/gax/rest/services/DataStoreCatalogs/bigDataFileShares_NYCdata/BigDataCatalogServer/sampled?token=lGt4v6vo8qoRMfmh9c45OBMiZs8BzPDOY7WpcNEydTM4bi-JWJisZUm_dlLQiGTtEo-9rfTEqzFo3PCtEyaGbXBmtM2U5vzqb7FOtD96kG3JBeo0vV4YIhrQquYWUpPZ-LRCGRJm1BRnISzQdqZmiaWs-Rh1Updy1kCo-tuphic.
{"messageCode":"BD_101033","message":"'pointLayer' will be projected into the processing spatial reference.","params":{"paramName":"pointLayer"}}
{"messageCode":"BD_101028","message":"Starting new distributed job with 6 tasks.","params":{"totalTasks":"6"}}
{"messageCode":"BD_101029","message":"1/6 distributed tasks completed.","params":{"completedTasks":"1","totalTasks":"6"}}
  extent = Some(Envelope: [-74.27270769100203, 0.0, 0.008983152841195214, 40.8687394594905])
  interval = None
  count = 152
{"messageCode":"BD_0","message":"Feature service layer created: http://Dev003246.esri.com/server/rest/services/Hosted/Aggregate_Points_Analysis_DJY1VK/FeatureServer/0","params":{"serviceUrl":"http://Dev003246.esri.com/server/rest/services/Hosted/Aggregate_Points_Analysis_DJY1VK/FeatureServer/0"}}
Succeeded at Sat Nov 26 22:27:46 2016 (Elapsed Time: 7.97 seconds)

The aggregate points tool, just like may other GeoAnalytics tools returns a feature layer item which contains the processed results.


Feedback on this topic?