ArcGIS Developers
Dashboard

ArcGIS API for Python

Download the samples Try it live

Streams extraction using Multi-Task Road Extractor

  • 🔬 Data Science
  • 🥠 Deep Learning and Pixel Classification

Introduction

In this notebook, we will use 3 different geo-morphological characteristics derived from a 5 m resolution DEM in one of the watersheds of Alaska to extract streams. These 3 characteristics are Topographic position index derived from 3 cells, Geomorphon landform, and topographic wetness index.

We created a composite out of these 3 characteristic rasters and used Export raster tool to convert scale the pixels values to 8 bit unsigned. Subsequently, the images are exported as "Classified Tiles" to train a Multi-Task Road Extractor model provided by ArcGIS API for Python for extracting the streams.

Before proceeding through this notebook, it is advised to go through the API Reference for Multi-Task Road Extractor. It will help in understanding the Multi-Task Road Extractor's workflow in detail.

Necessary imports

In [1]:
import os
import zipfile
from pathlib import Path

from arcgis.gis import GIS
from arcgis.learn import prepare_data, MultiTaskRoadExtractor

Connect to your GIS

In [2]:
ent_gis = GIS('https://pythonapi.playground.esri.com/portal')
In [3]:
gis = GIS("home")

Get the data for analysis

Here is the composite with 3 bands representing the 3 geo-morphological characteristics namely Topographic position index, Geomorphon landform, and Topographic wetness index.

In [4]:
composite_raster = ent_gis.content.get('43c4824dd4bf41ee886be5042262f192')
composite_raster
Out[4]:
composite_3bands_6BC_8BC_8bitunsigned
Image by api_data_owner
Last Modified: March 04, 2021
0 comments, 3 views
In [5]:
BeaverCreek_Flowlines = ent_gis.content.get('2b198075e53748b4ab84197bc6ddd478')
BeaverCreek_Flowlines
Out[5]:
BeaverCreek_Flowlines
Feature Layer Collection by api_data_owner
Last Modified: March 04, 2021
0 comments, 7 views

Export training data

Export training data using 'Export Training data for deep learning' tool, click here for detailed documentation:

  • Set 'composite_3bands_6BC_8BC_8bitunsigned' as Input Raster.
  • Set a location where you want to export the training data in Output Folder parameter, it can be an existing folder or the tool will create that for you.
  • Set the 'BeaverCreek_Flowlines' as input to the Input Feature Class Or Classified Raster parameter.
  • Set Class Field Value as 'FClass'.
  • Set Image Format as 'TIFF format'
  • Tile Size X & Tile Size Y can be set to 256.
  • Stride X & Stride Y can be set to 128.
  • Select 'Classified Tiles' as the Meta Data Format.
  • In 'Environments' tab set an optimum Cell Size. For this example, as we have performing the analysis on the geo-morphological characteristics with 5 m resolution, so, we used '5' as the cell size.

arcpy.ia.ExportTrainingDataForDeepLearning("composite_3bands_6BC_8BC_8bitunsigned.tif", r"D:\Stream Extraction\Exported_3bands_composite_8bit_unsigned", "BeaverCreek_Flowlines", "TIFF", 256, 256, 128, 128, "ONLY_TILES_WITH_FEATURES", "Classified_Tiles", 0, "FClass", 5, None, 0, "MAP_SPACE", "PROCESS_AS_MOSAICKED_IMAGE", "NO_BLACKEN", "FIXED_SIZE")

Alternatively, we have provided a subset of training data containing a few samples. You can use the data directly to run the experiments.

In [6]:
training_data = gis.content.get('3a95fd7a25d54898bddabf1989eea87d')
training_data
Out[6]:
streams_extraction_using_connectnet
Image Collection by api_data_owner
Last Modified: March 08, 2021
0 comments, 1 views
In [7]:
filepath = training_data.download(file_name=training_data.name)
In [8]:
with zipfile.ZipFile(filepath, 'r') as zip_ref:
    zip_ref.extractall(Path(filepath).parent)
In [9]:
output_path = Path(os.path.join(os.path.splitext(filepath)[0]))

Prepare data

We will specify the path to our training data and a few hyperparameters.

  • path: path of the folder containing training data.
  • batch_size: Number of images your model will train on each step inside an epoch, it directly depends on the memory of your graphic card. 8 worked for us on a 11GB GPU.
In [5]:
data = prepare_data(output_path, chip_size=512, batch_size=4)

Visualize a few samples from your training data

To get a sense of what the training data looks like, arcgis.learn.show_batch() method randomly picks a few training chips and visualizes them.

  • rows: number of rows we want to see the results for.
In [6]:
data.show_batch()