Streams extraction using Multi-Task Road Extractor¶
- 🔬 Data Science
- 🥠 Deep Learning and Pixel Classification
Table of Contents¶
- Necessary imports
- Connect to your GIS
- Get the data for analysis
- Export training data
- Prepare data
- Visualize a few samples from your training data
- Train the model
- Model inference
In this notebook, we will use 3 different geo-morphological characteristics derived from a 5 m resolution DEM in one of the watersheds of Alaska to extract streams. These 3 characteristics are Topographic position index derived from 3 cells, Geomorphon landform, and topographic wetness index.
We created a composite out of these 3 characteristic rasters and used Export raster tool to convert scale the pixels values to 8 bit unsigned. Subsequently, the images are exported as "Classified Tiles" to train a Multi-Task Road Extractor model provided by ArcGIS API for Python for extracting the streams.
Before proceeding through this notebook, it is advised to go through the API Reference for Multi-Task Road Extractor. It will help in understanding the Multi-Task Road Extractor's workflow in detail.
import os import zipfile from pathlib import Path from arcgis.gis import GIS from arcgis.learn import prepare_data, MultiTaskRoadExtractor
ent_gis = GIS(url='https://pythonapi.playground.esri.com/portal', username='arcgis_python', password='amazing_arcgis_123')
gis = GIS("home")
Here is the composite with 3 bands representing the 3 geo-morphological characteristics namely Topographic position index, Geomorphon landform, and Topographic wetness index.
composite_raster = ent_gis.content.search("title: composite_3bands_6BC_8BC_8bitunsigned owner:api_data_owner", "Image") composite_raster
BeaverCreek_Flowlines = ent_gis.content.search('title:BeaverCreek_Flowlines owner:api_data_owner', "Feature Layer") BeaverCreek_Flowlines
Export training data using 'Export Training data for deep learning' tool, click here for detailed documentation:
- Set 'composite_3bands_6BC_8BC_8bitunsigned' as
- Set a location where you want to export the training data in
Output Folderparameter, it can be an existing folder or the tool will create that for you.
- Set the 'BeaverCreek_Flowlines' as input to the
Input Feature Class Or Classified Rasterparameter.
Class Field Valueas 'FClass'.
Image Formatas 'TIFF format'
Tile Size X&
Tile Size Ycan be set to 256.
Stride Ycan be set to 128.
- Select 'Classified Tiles' as the
Meta Data Format.
- In 'Environments' tab set an optimum
Cell Size. For this example, as we have performing the analysis on the geo-morphological characteristics with 5 m resolution, so, we used '5' as the cell size.
arcpy.ia.ExportTrainingDataForDeepLearning("composite_3bands_6BC_8BC_8bitunsigned.tif", r"D:\Stream Extraction\Exported_3bands_composite_8bit_unsigned", "BeaverCreek_Flowlines", "TIFF", 256, 256, 128, 128, "ONLY_TILES_WITH_FEATURES", "Classified_Tiles", 0, "FClass", 5, None, 0, "MAP_SPACE", "PROCESS_AS_MOSAICKED_IMAGE", "NO_BLACKEN", "FIXED_SIZE")
Alternatively, we have provided a subset of training data containing a few samples. You can use the data directly to run the experiments.
training_data = gis.content.get('3a95fd7a25d54898bddabf1989eea87d') training_data
filepath = training_data.download(file_name=training_data.name)
with zipfile.ZipFile(filepath, 'r') as zip_ref: zip_ref.extractall(Path(filepath).parent)
output_path = Path(os.path.join(os.path.splitext(filepath)))
We will specify the path to our training data and a few hyperparameters.
path: path of the folder containing training data.
batch_size: Number of images your model will train on each step inside an epoch, it directly depends on the memory of your graphic card. 8 worked for us on a 11GB GPU.
data = prepare_data(output_path, chip_size=512, batch_size=4)
To get a sense of what the training data looks like,
arcgis.learn.show_batch() method randomly picks a few training chips and visualizes them.
rows: number of rows we want to see the results for.