Finetuning Pre-trained Building Footprint Model

  • 🔬 Data Science
  • 🥠 Deep Learning and Instance Segmentation

Introduction

ArcGIS Living Atlas hosts a variety of pre-trained models. While these models work well on geography that the model's training data was exported from, they may not perform well on other geographies.

However, we can improve the performance of these models on different geographies by finetuning the model on our own training data. When compared to training a similar model from scratch, this process will save time, is computationally less intensive, and will provide more accurate results.

In this workflow, we will perform three broad steps.

  • Load the training data
  • Finetune a pre-trained model
  • Deploy the model and extract footprints

This workflow requires deep learning dependencies to be installed. Documentation is available here that outlines how to install and setup an appropriate environment.

Load training data

Input
from arcgis.gis import GIS
gis = GIS('home')
portal = GIS('https://pythonapi.playground.esri.com/portal')
Input
training_data = gis.content.get('5351aca735604197ac8d8ede45f6cc4b')
training_data
Output
building_footprints_kuwait_osm_sample
building_footprints_kuwait_osm_sampleImage Collection by api_data_owner
Last Modified: August 10, 2021
0 comments, 0 views
Input
filepath = training_data.download(file_name=training_data.name)
Input
import zipfile
from pathlib import Path
with zipfile.ZipFile(filepath, 'r') as zip_ref:
    zip_ref.extractall(Path(filepath).parent)
Input
data_path = Path(filepath).parent / 'building_footprints'
Input
from arcgis.learn import prepare_data
data = prepare_data(data_path, 
                    batch_size=16, 
                    chip_size=400)
Please check your dataset. 3 images dont have the corresponding label files.

Visualize training data

To get a sense of what the training data looks like, use the show_batch() method to randomly pick a few training chips and visualize them. The chips are overlaid with masks representing the building footprints in each image chip.

Input
data.show_batch(rows=4)

Model finetuning

Load a pre-trained building footprint model

We can search ArcGIS Living Atlas for Pre-trained models.

From a model's page on living atlas, we can either directly download the model from the page or find the itemid in the URL to download it using the ArcGIS Python API as follows.

Input
model_item = gis.content.get('a6857359a1cd44839781a4f113cd5934')
model_item
Output
Building Footprint Extraction - USA
Deep learning model to extract building footprints from high-resolution aerial and satellite imagery.Deep Learning Package by esri_analytics
Last Modified: July 12, 2021
19 comments, 8,772 views

Next, we download the model.

Input
model_path = model_item.download(file_name=model_item.name)

Once the model is downloaded, we can then load the model.

Input
from arcgis.learn import MaskRCNN
model = MaskRCNN.from_model(model_path, data)

Test the model on our dataset

In this sample, our dataset has been curated from Kuwait, which has a very different geography when compared to the data from the United States that was used to train the 'Building Footprint Extraction - USA' model.

We will run the {model}.show_results() method to check the performance of the model on our dataset, and as the model has not yet been trained on the Kuwaiti data, it is expected that the model will not perform well.

Input
model.show_results()

The learning rate is one of the most important hyperparameters in training a model. We will use the lr_find() method to find an optimal learning rate that will allow us to fine tune the model.

Input
lr = model.lr_find()
lr
Output
5.248074602497728e-05

Train the model

Next, we will use the learning rate suggested above to train our model for 10 epochs.

Input
model.fit(10,lr=lr)
epoch train_loss valid_loss time
0 1.443777 1.424972 05:56
1 1.259852 1.282082 05:56
2 1.207860 1.285075 05:56
3 1.148594 1.203295 05:57
4 1.126693 1.172327 05:58
5 1.094930 1.153458 05:56
6 1.071475 1.145979 05:58
7 1.044148 1.137563 05:58
8 1.062895 1.128028 05:57
9 1.058062 1.126796 05:57
Input
 

Visualize detected building footprints

The model.show_results() method can be used to display the detected building footprints. Each detection is visualized as a mask by default.

Input
model.show_results()

We can set the mode parameter to bbox_mask to visualize both masks and bounding boxes.

Input
model.show_results(mode='bbox_mask')

Save the model

As we can see, with 10 epochs, we are already seeing reasonable results. More improvements can be achieved by training the model further or by adding more training data. Let's save the model, so that it can be used for inference or further training. By default, it will be saved into the path that you specified in the very beginning of this notebook, in the prepare_data call.

Input
model.save('Building_footprint_10epochs')

Model inference

The saved model can now be used to extract building footprint masks using the 'Detect Objects Using Deep Learning' tool available in ArcGIS Pro or ArcGIS Enterprise. For this sample, we will use high satellite imagery to detect footprints.

image.png

You can also achieve this using arcpy.

with arcpy.EnvManager(
    extent="799956.322438235 3234305.33935078 801796.850070329 3235284.04198516", 
    cellSize=0.3, 
    processorType="GPU"
):
    arcpy.ia.DetectObjectsUsingDeepLearning(
        "im_2019_12_05", 
        r"C:\building_footprints\building_footprints.gdb\building_footprints", 
        r"C:\building_footprints\Building_footprint_10epochs\Building_footprint_10epochs.emd", 
        "padding 100;batch_size 4;threshold 0.9;return_bboxes False;tile_size 400", 
        "NMS", 
        "Confidence", 
        "Class", 
        0, 
        "PROCESS_AS_MOSAICKED_IMAGE"
    )

The output of the model is a layer of detected building footprints that need to be post-processed using the Regularize Building Footprints tool. This tool normalizes the footprint of building polygons by eliminating undesirable artifacts in their geometry. The post-processed building footprints are shown below:

Capture.JPG

A subset of detected building footprints

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.