Change Detection of Buildings from Satellite Imagery

Introduction

The World is changing every day and monitoring that change on ground can be a tedious and labor intensive task. so, is there a way to automate it?

This notebook will walk you through how deep learning can be used to perform change detection using satellite images.

One of the popular models available in the arcgis.learn module of ArcGIS API for Python, ChangeDetector is used to identify areas of persistent change between two different time periods using remotely sensed images. It can help you identify where new buildings have come up for instance. This model is based upon the latest research in deep learning and works well with objects of various sizes. The ChangeDetector model workflow consists of three parts:

  • Preparing the training data,
  • training a model
  • using the trained model for inferencing.

Let’s first prepare the training data.

Export training data for deep learning

In the cells below, we have provided the input rasters and input mask polygons needed to export training data.

from arcgis.gis import GIS
gis = GIS('home')
input_data = gis.content.get('3ebf8ca5f6c245d69e2e0f4358986ed3')
input_data
change_detection_input_rasters
Image Collection by api_data_owner
Last Modified: December 10, 2020
0 comments, 85 views
mask_polygons = gis.content.get('6ee0b48611c44b31b499f6cbe202686f')
mask_polygons
cd_input_mask_polygons
Feature Layer Collection by api_data_owner
Last Modified: December 10, 2020
0 comments, 153 views

ChangeDetector model requires data in this folder format : images_after, images_before and labels folder. The label indicates where there are changes in the before and after images. These images are too large to process in the GPU memory for training the model, so we need to create small image chips or tiles. Training data can be exported by using the Export Training Data For Deep Learning tool available in ArcGIS Pro as well as ArcGIS Image Server.

  • Input Raster: 2014rasters
  • Tile Size X & Tile Size Y: 256
  • Stride X & Stride Y: 64
  • Meta Data Format: 'Export Tiles'
  • Environments: 0.3

As shown below, the above tool needs to be run thrice each for image before, image after and the change labels in order to create training data.

Model training

This step would be done using jupyter notebook and documentation is available here to install and setup environment.

Necessary imports

import os
from pathlib import Path
from arcgis.learn import prepare_data, ChangeDetector

Get training data

We have already exported the data which can be directly downloaded using the steps below:

training_data = gis.content.get('d284e2083b254f6b8508f9cf41f53713')
training_data
change_detection_training_data
Image Collection by api_data_owner
Last Modified: December 10, 2020
0 comments, 4 views
filepath = training_data.download(file_name=training_data.name)
import zipfile
with zipfile.ZipFile(filepath, 'r') as zip_ref:
    zip_ref.extractall(Path(filepath).parent)
data_path = Path(os.path.join(os.path.splitext(filepath)[0]))

prepare_data function takes path to training data and creates a fast.ai databunch with specified transformation, batch size, split percentage, etc.

data = prepare_data(data_path,
                    chip_size=256,
                    dataset_type='ChangeDetection', 
                    batch_size=4
                   )

Visualize training data

To get a sense of what the training data looks like, use the show_batch() method to randomly pick a few training chips and visualize them. The chips are overlaid with masks representing the building footprints in each image chip.

data.show_batch()
<Figure size 1440x1440 with 12 Axes>

Load model architecture

arcgis.learn provides the ChangeDetector model for identifying areas of persistent change tasks, which is based on a pretrained convnet, like ResNet that acts as the 'backbone'. More details about ChangeDetector can be found here.

cd = ChangeDetector(data, backbone='resnet50')

Train the model

Learning rate is one of the most important hyperparameters in model training. We will use the lr_find() method to find an optimum learning rate at which we can train a robust model.

lr = cd.lr_find()
<Figure size 432x288 with 1 Axes>

We are using the suggested learning rate above to train the model for 100 epochs.

cd.fit(epochs=100, lr=lr)
epochtrain_lossvalid_lossprecisionrecallf1time
03.6385553.9913940.0000000.0000000.00000000:20
13.5821453.9036150.0100810.0019510.00326900:20
23.4666303.7886590.0307160.0061060.00762200:20
33.3588373.6397810.1783600.0238840.02830000:20
43.2229723.5825980.0986450.0228280.02719100:20
53.0946263.5140140.0592110.0125160.01521000:20
62.9528463.3279570.1211380.0172890.02627900:20
72.7816973.1879270.2110840.0113770.02009900:20
82.6277992.9624560.4625560.0235960.04236100:20
92.4744192.9663770.3741130.0093250.01757600:20
102.3261282.8378910.5848110.0130850.02458900:20
112.2016962.6785290.8579920.0449940.07916400:20
122.0890432.3628210.8325360.1459340.22725700:20
131.9699672.2833650.9452650.1897070.29550500:20
141.8443721.9520050.9143960.3207960.45812200:20
151.7224061.8218770.9424920.3461170.49206900:20
161.6199691.7159590.9162300.3920460.53167600:20
171.5280371.7130270.9268090.3838660.53253000:20
181.4488761.7049920.9235340.3941540.54459000:20
191.3765491.6394030.9107120.4265270.57442300:20
201.3079741.6505380.9572500.4131840.57054100:20
211.2399001.5578450.9387000.4509390.60374900:20
221.1892041.5041760.9192000.4708040.61688900:20
231.1338611.3357760.8854090.5463200.67073400:20
241.0811921.1956590.9267010.5924520.72096800:20
251.0262581.1171000.9276870.6224000.74356400:20
260.9678151.1112880.9295990.6241230.74408900:20
270.9133461.2365370.9322940.5788530.71176200:20
280.8620911.1232690.9427850.6222890.74770500:20
290.8134870.9786350.9190880.6835850.78177400:20
300.7674920.9927730.9458530.6721120.78413400:20
310.7305050.9181340.9405090.7041690.80402300:20
320.6909500.9052170.9268570.7198950.80860000:20
330.6570390.7949990.9255160.7605220.83384200:20
340.6300160.8670780.9477970.7316030.82443200:20
350.6011950.6999930.9201420.7951240.85204900:20
360.5765200.6445060.9157470.8196730.86406400:20
370.5550320.7331630.9442610.7800360.85338200:20
380.5388260.6595370.9193980.8214230.86699900:21
390.5204520.7607950.9449950.7800050.85358100:20
400.5120400.6045870.8920460.8375530.86292600:20
410.5042150.7718750.9413430.7683840.84456000:20
420.4919120.7943730.9497770.7664830.84716700:21
430.4786840.7063380.9429260.8007820.86540800:20
440.4677120.5699060.9267190.8463040.88411900:20
450.4569670.8736390.9634690.7401130.83574600:20
460.4478680.8433720.9367050.7498070.83098200:20
470.4407950.7206380.9434600.7963710.86275100:20
480.4379120.6898290.9390670.7986970.86170900:20
490.4330790.7930260.9440430.7612860.84192100:20
500.4296850.7530050.9513770.7761670.85392200:20
510.4223481.0803900.9777770.6417480.77162300:20
520.4181870.8695610.9663350.7300300.83074300:20
530.4129410.9656460.9707140.7005010.81241600:20
540.4068950.7957810.9700920.7666020.85586000:20
550.4057980.8330270.9585390.7399120.83341700:20
560.4021461.0527490.9799800.6541340.78185800:20
570.3976040.8948210.9648450.7297750.82940800:20
580.3953700.9598870.9597180.7057230.81177200:20
590.3908401.0089450.9781210.6757900.79739000:20
600.3866280.9028250.9577370.7287810.82643000:20
610.3855150.8341280.9646580.7557740.84676700:20
620.3828370.9996410.9791290.6819380.80217800:20
630.3792090.9306040.9682490.7070380.81569800:20
640.3779851.1493860.9809990.6144130.75306100:20
650.3742651.1413540.9772680.6220480.75712500:20
660.3730870.9975410.9750620.6838490.80191000:20
670.3699950.9651050.9773230.6941940.81014100:20
680.3686430.9656320.9758830.6956900.81060400:20
690.3675861.0291960.9836880.6679320.79380400:20
700.3633481.1034620.9812770.6347390.76853600:20
710.3629181.1367310.9821680.6197850.75763100:20
720.3607721.2506510.9821840.5663970.71490300:20
730.3611541.0951960.9826240.6345770.76815600:20
740.3574801.1692300.9823010.6019070.74270700:20
750.3549701.2432480.9861440.5653080.71409900:20
760.3536631.2580470.9863080.5571710.70752500:20
770.3526311.1692110.9815290.6045920.74427400:20
780.3523121.1388290.9850360.6148920.75357400:20
790.3497141.1542100.9850810.6076030.74802700:20
800.3486251.1686010.9848470.5987270.74092200:20
810.3488151.2007200.9833230.5875940.73142800:20
820.3475881.1205810.9830280.6229890.75964000:20
830.3456401.1988890.9864340.5833400.72911500:20
840.3445551.2465380.9866280.5641640.71371200:20
850.3463261.1767820.9856890.5950280.73845900:20
860.3451761.2177010.9859820.5770310.72417200:20
870.3429481.2591890.9862950.5583960.70842300:20
880.3423291.2679150.9880480.5527180.70444700:20
890.3428351.2971850.9855510.5412810.69401600:20
900.3432461.2352770.9872630.5679470.71702500:20
910.3445471.2199060.9848940.5764650.72335800:20
920.3433861.1827620.9847200.5940580.73760600:20
930.3431031.2266890.9852290.5726330.72028900:20
940.3435831.1696140.9855580.6010310.74354800:20
950.3425151.2134480.9847350.5809920.72708700:20
960.3427561.1566600.9832090.6083910.74860200:20
970.3434461.1924470.9849510.5885870.73325900:20
980.3434121.2146100.9829240.5795340.72537300:20
990.3423111.1635220.9838790.6044650.74575200:20

We have further trained the model for 100 more epochs to improve model performance. For the sake of time, the cell below is commented out.

# cd.fit(100)

Visualize detected changes

It's a good practice to see results of the model viz-a-viz ground truth. The code below picks random samples and shows us ground truth and model predictions, side by side. This enables us to preview the results of the model within the notebook.

cd.show_results(rows=8) 
<Figure size 1440x2880 with 32 Axes>

Evaluate model performance

As we have 2 classes for this change detection task, we need to do accuracy assessment for each of those. For that ArcGIS API for Python provides precision_recall_score function that will calculate precision and recall for each class.

cd.precision_recall_score()
NoChangeChange
precision0.9369540.926067
recall0.9701000.848920
f10.9531470.885244

Save model

We will save the model which we trained as a 'Deep Learning Package' ('.dlpk' format). Deep Learning package is the standard format used to deploy deep learning models on the ArcGIS platform.

We will use the save() method to save the trained model. By default, it will be saved to the 'models' sub-folder within our training data folder.

cd.save('change_detection_model_e200')

Model inference

Using predict function, we can apply the trained model on a test image/area to detect changes that occurred in the satellite images during two different time periods.

inference_data = gis.content.get('6b32a534228b44b284c14e75b3d3f5f5')
inference_data
cd_inference_data
Image Collection by api_data_owner
Last Modified: December 10, 2020
0 comments, 3 views
test_path = inference_data.download(file_name=inference_data.name)
import zipfile
with zipfile.ZipFile(test_path, 'r') as zip_ref:
    zip_ref.extractall(Path(test_path).parent)
test_images = Path(os.path.join(os.path.splitext(test_path)[0]))
before_img = os.path.join(test_images, 'before.tif')
after_img = os.path.join(test_images, 'after.tif')

The predict function takes in before and after image path as required variables. You can optionally pass visualize=True if you want to see the results in the notebook. Additionally, you can pass save=True function in order to save the image to the local disk.

cd.predict(before_image=before_img,
           after_image=after_img,
           visualize=True)
tensor([[[0, 0, 0,  ..., 0, 0, 0],
         [0, 0, 0,  ..., 0, 0, 0],
         [0, 0, 0,  ..., 0, 0, 0],
         ...,
         [0, 0, 0,  ..., 0, 0, 0],
         [0, 0, 0,  ..., 0, 0, 0],
         [0, 0, 0,  ..., 0, 0, 0]]])
<Figure size 1440x360 with 3 Axes>

Conclusion

In this notebook, we learned how to solve various problems like identifying new construction. The same workflow can also be used to find out which new roads have come up in the past five years for instance. With just some labeled data and with little to no human involvement by using deep learning, we can now perform change detection using satellite images.

References

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.