ArcGIS Developers
Dashboard

ArcGIS API for Python

Download the samples Try it live

Exploring OpenStreetMap using Pandas and the Python API

This notebook is based around a simple tool named OSM Runner that queries the OpenStreetMap (OSM) Overpass API and returns a Spatial Data Frame. Using the Python API inside of a Jupyter Notebook, we can develop map-driven tools to explore OSM with the full capabilities of the ArcGIS platform at our disposal. Be sure to update the GIS connection information in the cell below before proceeding.

This Notebook was written for an environment that does not have access to arcpy.

In [1]:
!pip install osm-runner
Collecting osm-runner
  Downloading osm_runner-0.0.3-py3-none-any.whl (3.8 kB)
Requirement already satisfied: arcgis in /opt/conda/lib/python3.6/site-packages (from osm-runner) (1.8.1)
Requirement already satisfied: pandas in /opt/conda/lib/python3.6/site-packages (from osm-runner) (1.0.3)
Requirement already satisfied: requests in /opt/conda/lib/python3.6/site-packages (from osm-runner) (2.23.0)
Requirement already satisfied: python-dateutil>=2.6.1 in /opt/conda/lib/python3.6/site-packages (from pandas->osm-runner) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.6/site-packages (from pandas->osm-runner) (2020.1)
Requirement already satisfied: numpy>=1.13.3 in /opt/conda/lib/python3.6/site-packages (from pandas->osm-runner) (1.18.1)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->osm-runner) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->osm-runner) (1.25.8)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.6/site-packages (from requests->osm-runner) (2020.4.5.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->osm-runner) (3.0.4)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.6/site-packages (from python-dateutil>=2.6.1->pandas->osm-runner) (1.15.0)
Installing collected packages: osm-runner
Successfully installed osm-runner-0.0.3
In [2]:
import time

from osm_runner import Runner  # pip install osm-runner
import pandas as pd

from arcgis.features import FeatureLayer, GeoAccessor, GeoSeriesAccessor
from arcgis.geoenrichment import enrich
from arcgis import dissolve_boundaries
from arcgis.geometry import project
from arcgis.gis import GIS

# Organization Login
gis = GIS('home')

Build Data Frames from Feature Layers & Extract Bounding Box

Let's assume we want to compare recycling amenities in OSM across 2 major cities. The first step will be to turn the boundaries for each city into a Data Frame via the GeoAccessor method from_layer(). Once we have a Data Frame for each city, we will use the Project operation of the Geometry Service in our GIS to get the envelope required to fetch data from Open Street Map.

In [2]:
dc_fl = FeatureLayer('https://maps2.dcgis.dc.gov/dcgis/rest/services/DCGIS_DATA/Administrative_Other_Boundaries_WebMercator/MapServer/10')
dc_df = GeoAccessor.from_layer(dc_fl)
display(dc_df.head())

dc_extent = dc_df.spatial.full_extent
dc_coords = project([[dc_extent[0], dc_extent[1]], [dc_extent[2], dc_extent[3]]], in_sr=3857, out_sr=4326)
dc_bounds = f"({dc_coords[0]['y']},{dc_coords[0]['x']},{dc_coords[1]['y']},{dc_coords[1]['x']})"

pr_fl = FeatureLayer('https://carto2.apur.org/apur/rest/services/OPENDATA/QUARTIER/MapServer/0')
pr_df = GeoAccessor.from_layer(pr_fl)
display(pr_df.head())

pr_extent = pr_df.spatial.full_extent
pr_coords = project([[pr_extent[0], pr_extent[1]], [pr_extent[2], pr_extent[3]]], in_sr=2154, out_sr=4326)
pr_bounds = f"({pr_coords[0]['y']},{pr_coords[0]['x']},{pr_coords[1]['y']},{pr_coords[1]['x']})"
OBJECTID CITY_NAME STATE_CITY CAPITAL WEB_URL AREAKM AREAMILES Shape_Length Shape_Area SHAPE
0 1 Washington 1150000 Y http://www.dc.gov 177.47 68.52 67608.276922 1.774562e+08 {'rings': [[[-8584936.334474642, 4712272.26069...
OBJECTID N_SQ_QU C_QU C_QUINSEE L_QU C_AR N_SQ_AR SHAPE_Length SHAPE_Area SHAPE
0 1 750000046 46 7511202 Picpus 12 750000012 18260.602587 7.205014e+06 {'rings': [[[656779.2524999976, 6859005.5504],...
1 2 750000047 47 7511203 Bercy 12 750000012 6154.591415 1.902932e+06 {'rings': [[[655300.0428000018, 6858622.629000...
2 3 750000048 48 7511204 Quinze-Vingts 12 750000012 4509.226476 1.235916e+06 {'rings': [[[653996.0221000016, 6860240.466299...
3 4 750000049 49 7511301 Salpêtrière 13 750000013 4758.777675 1.181560e+06 {'rings': [[[652751.3332000002, 6859190.603100...
4 5 750000050 50 7511302 Gare 13 750000013 7069.818989 3.044177e+06 {'rings': [[[653571.8592000008, 6857669.765999...

Overview of the area in Washington DC to be Collected

In [3]:
dc_map = gis.map('Washington DC')

dc_map.draw(dc_df.iloc[0].SHAPE)
dc_map.draw(dc_df.spatial.bbox)

display(dc_map)

print(f'Searching Area: {round(dc_df.spatial.bbox.area / 1000000)} Square Kilometers')
Out[3]: