ArcGIS Developers
Dashboard

ArcGIS API for Python

Automatic road extraction using deep learning

  • 🔬 Data Science
  • 🥠 Deep Learning and pixel-based classification

Introduction

Road network is a required layer in a lot of mapping exercises, for example in Basemap preparation (critical for navigation), humanitarian aid, disaster management, transportation, and for a lot of other applications it is a critical component.

This sample shows how ArcGIS API for Python can be used to train a deep learning model (Multi-Task Road Extractor model) to extract the road network from satellite imagery. The models trained can be used with ArcGIS Pro or ArcGIS Enterprise and even support distributed processing for quick results.

Further details on the Multi-Task Road Extractor implementation in the API (working principle, architecture, best practices, etc.), can be found in the Guide, along with instructions on how to set up the Python environment.

Before proceeding through this notebook, it is advised to go through the API Reference for Multi-Task Road Extractor (prepare_data(), MultiTaskRoadExtractor()). It will help in understanding the Multi-Task Road Extractor's workflow in detail.

Objectives:

  1. Classify roads, utilizing API's Multi-Task Road Extractor model.

Area of Interest and data pre-processing

For this sample, we will be using a subset of the publically available SpaceNet dataset. Vector labels as 'road centerlines' are available for download along with imagery, hosted on AWS S3 [1].

The area of interest is Paris, with 425 km of 'road centerline' length (As shown in Figure. 1). Both of these inputs, Imagery, and vector layer (for creating image chips and labels as 'classified tiles') are used to create data that is needed for model training.

Figure 1: SpaceNet dataset - AOI 3 - Paris

Downloaded data has 4 types of imagery: Multispectral, Pan, Pan-sharpened Multispectral, Pan-sharpened RGB. 8-bit RGB imagery support and 16-bit RGB imagery experimental support is available with Multi-Task Road Extractor Model (Multispectral imagery will be supported in the subsequent release). In this sample, Pan-sharpened RGB is used, after converting it to 8-bit imagery.

Pre-processing steps:

  • Downloaded vector labels, in .geojson format, are converted to feature class/shapefile. (Refer to ArcGIS Pro's JSON To Features GP tool.)
  • The converted vector data is checked and repaired if any invalid geometry is found. (Refer to ArcGIS Pro's Repair Geometry GP tool.)
  • 'Stretch function' is used to convert 16-bit imagery to 8-bit imagery. (Refer to ArcGIS Pro's Stretch raster function.)
  • 'Projected coordinate system' is applied to imagery and road vector data, for ease in the interpretation of results and setting the values of tool parameters.

Now, the data is ready for Export Training Data For Deep Learning GP tool (As shown in Figure. 2). It is used to export data that will be needed for model training. This tool is available in ArcGIS Pro as well as ArcGIS Enterprise.

Here, we exported the data in 'Classified Tiles' format using a Cell Size of '30 cm'. Tile Size X and Tile Size Y are set to '512', while Stride X and Stride Y are set to '128'. If Road centerlines are directly used as an input, then based on the area of interest and types of roads in that region, the appropriate buffer size can be set. Alternatively, ArcGIS Pro's Create Buffers GP tool can be used to convert road centerlines to road polygons and buffer value can be decided iteratively by checking the results of the Create Buffers GP tool.

Figure 2: Export Training Data For Deep Learning GP tool

This tool will create all the necessary files needed in the next step, at the Output Folder's directory.

Data preparation

Imports:

In [ ]:
from arcgis.learn import prepare_data, MultiTaskRoadExtractor

Preparing the exported data:

_Some of the frequently used parameters that can be passed in prepare_data() are described below:_

path: the path of the folder containing training data. (Output generated by the "Export Training data for deep learning GP tool")

chip_size: Images are cropped to the specified chip_size.

batch_size: No. of images your model will train on each step inside an epoch, it directly depends on the memory of your graphic card.

val_split_pct: Percentage of training data to keep as validation.

resize_to: Resize the cropped image to the mentioned size.

In [ ]:
output_path = r'path/to/exported/data/from/Pro-GP-tool'
In [ ]:
data = prepare_data(output_path, chip_size=512, batch_size=10)
In [ ]:
data.classes

Try it live:

Alternatively, this notebook can be 'tried' live. The following steps will download a sample 'exported data' and pass it to prepare_data():

Note: Data meant for 'Try it Live' is a very small subset of the actual data that was used for this sample notebook, so the training time, accuracy, visualization, etc. will change, from what is depicted below.

In [ ]:
import os, zipfile
from pathlib import Path
from arcgis.gis import GIS
gis = GIS('home')
In [ ]:
training_data = gis.content.get('b7bbf2f5f4184960890afeabbdb51a32')
training_data
Out[ ]:
automatic_road_extraction_using_deep_learning
Image Collection by api_data_owner
Last Modified: December 04, 2020
0 comments, 0 views
In [ ]:
filepath = training_data.download(file_name=training_data.name)
In [ ]:
with zipfile.ZipFile(filepath, 'r') as zip_ref:
    zip_ref.extractall(Path(filepath).parent)
In [ ]:
output_path = Path(os.path.join(filepath.split('.')[0]))
In [ ]:
data = prepare_data(output_path, chip_size=512, batch_size=10)
In [ ]:
data.classes

Visualization of prepared data

show_batch() can be used to show the prepared data. Where input imagery is shown with labels overlayed on them.

alpha is used to control the transparency of labels.

In [ ]:
data.show_batch(alpha=1)