Detecting Swimming Pools using Satellite Imagery and Deep Learning¶

🔬 Data Science

🥠 Deep Learning and Object Detection

Deep Learning has achieved great success with state of the art results, but taking it to the field and solving real-world problems is still a challenge. Integration of the latest research in AI with ArcGIS opens up a world of opportunities. This notebook demonstrates an end-to-end deep learning workflow in using ArcGIS API for Python. The workflow consists of three major steps: (1) extracting training data, (2) train a deep learning object detection model, (3) deploy the model for inference and create maps. To better illustrate this process, we choose detecting swmming pools in Redlands, CA using remote sensing imagery.

Part 1 - export training data for deep learning¶

Import ArcGIS API for Python and get connected to your GIS¶

In [1]:

from arcgis import GIS

In [2]:

gis = GIS("your_portal", "username", "password")

Prepare data that will be used for training data export¶

To export training data, we need a labeled feature class that contains the bounding box for each object, and a raster layer that contains all the pixels and band information. In this swimming pool detection case, we have created feature class by hand labelling the bounding box of each swimming pool in Redlands using ArcGIS Pro and USA NAIP Imagery: Color Infrared as raster data.

In [3]:

pool_bb = gis.content.search("SwimmingPoolLabels", item_type='Feature Layer Collection')[0]
pool_bb

Out[3]:

SwimmingPoolLabels

Feature Layer Collection by portaladmin
Last Modified: January 29, 2019
0 comments, 8 views

In [4]:

pool_bb_layer = pool_bb.layers[0]
pool_bb_layer.url

Out[4]:

'https://datascienceadv.esri.com/server/rest/services/Hosted/SwimmingPoolLabels/FeatureServer/0'

In [5]:

m = gis.map("Prospect Park, Redlands, CA")
m

In [6]:

m.basemap = 'gray'

Now let's retrieve the NAIP image layer.

In [7]:

naip_item = gis.content.search("naip_ml", item_type="Imagery Layer")[0]
naip_item

Out[7]:

naip_ml
Naip data or swimming pool detection

Imagery Layer by portaladmin
Last Modified: February 08, 2019
0 comments, 1 views

In [8]:

from arcgis.raster.functions import extract_band

naiplayer = naip_item.layers[0]
naiplayer

Out[8]:

In [10]:

m.add_layer(naiplayer)

Specify a folder name in raster store that will be used to store our training data¶

Make sure a raster store is ready on your raster analytics image server. This is where where the output subimages, also called chips, labels and metadata files are going to be stored.

In [15]:

from arcgis.raster import analytics

In [ ]:

ds = analytics.get_datastores(gis=gis)
ds

In [17]:

ds.search()

Out[17]:

[<Datastore title:"/nosqlDatabases/AGSDataStore_bigdata_bds_4c9tuc3o" type:"nosql">,
 <Datastore title:"/nosqlDatabases/AGSDataStore_nosqldb_tcs_l6mh5mhm" type:"nosql">,
 <Datastore title:"/enterpriseDatabases/AGSDataStore_ds_b6108wk9" type:"egdb">,
 <Datastore title:"/rasterStores/LocalRasterStore" type:"rasterStore">]

In [18]:

rasterstore = ds.get("/rasterStores/LocalRasterStore")
rasterstore

Out[18]:

<Datastore title:"/rasterStores/LocalRasterStore" type:"rasterStore">

In [ ]:

samplefolder = "pool_chips"
samplefolder

Export training data using `arcgis.learn`¶

With the feature class and raster layer, we are now ready to export training data using the export_training_data() method in arcgis.learn module. In addtion to feature class, raster layer, and output folder, we also need to speficy a few other parameters such as tile_size (size of the image chips), strid_size (distance to move in the X when creating the next image chip), chip_format (TIFF, PNG, or JPEG), metadata format (how we are going to store those bounding boxes). More detail can be found here.

Depending on the size of your data, tile and stride size, and computing resources, this opertation can take 15mins~2hrs in our experiment. Also, do not re-run it if you already run it once unless you would like to update the setting.

In [11]:

pool_bb

Out[11]:

SwimmingPoolLabels

Feature Layer Collection by portaladmin
Last Modified: January 29, 2019
0 comments, 8 views

In [12]:

pool_bb_layer

Out[12]:

<FeatureLayer url:"https://datascienceadv.esri.com/server/rest/services/Hosted/SwimmingPoolLabels/FeatureServer/0">

In [13]:

import arcgis
from arcgis import learn
arcgis.env.verbose = True

In [ ]:

export = learn.export_training_data(input_raster=naiplayer,
                                           output_location=samplefolder,
                                           input_class_data=pool_bb_layer, 
                                           chip_format="PNG", 
                                           tile_size={"x":448,"y":448}, 
                                           stride_size={"x":224,"y":224}, 
                                           metadata_format="PASCAL_VOC_rectangles",                                        
                                           classvalue_field = "Id",
                                           buffer_radius = 6,
                                           context={"startIndex": 0, "exportAllTiles": False},
                                           gis = gis)

export

Now let's get into the raster store and look at what has been generated and exported.

In [20]:

from arcgis.raster.analytics import list_datastore_content

samples = list_datastore_content(rasterstore.datapath + '/' + samplefolder + "/images", filter = "*png")
# print out the first five chips/subimages
samples[0:5]

Submitted.
Executing...
Start Time: Thursday, February 21, 2019 9:44:27 PM
Running script ListDatastoreContent...

Out[20]:

['/rasterStores/LocalRasterStore/pool_chips_yongyao/images/000000000.png',
 '/rasterStores/LocalRasterStore/pool_chips_yongyao/images/000000001.png',
 '/rasterStores/LocalRasterStore/pool_chips_yongyao/images/000000002.png',
 '/rasterStores/LocalRasterStore/pool_chips_yongyao/images/000000003.png',
 '/rasterStores/LocalRasterStore/pool_chips_yongyao/images/000000004.png']

In [21]:

labels = list_datastore_content(rasterstore.datapath + '/' + samplefolder + "/labels", filter = "*xml")
# print out the labels/bounding boxes for the first five chips
labels[0:5]

Start Time: Thursday, February 21, 2019 9:44:29 PM
Running script ListDatastoreContent...
Completed script ListDatastoreContent...
Succeeded at Thursday, February 21, 2019 9:44:29 PM (Elapsed Time: 0.05 seconds)

Out[21]:

['/rasterStores/LocalRasterStore/pool_chips_yongyao/labels/000000000.xml',
 '/rasterStores/LocalRasterStore/pool_chips_yongyao/labels/000000001.xml',
 '/rasterStores/LocalRasterStore/pool_chips_yongyao/labels/000000002.xml',
 '/rasterStores/LocalRasterStore/pool_chips_yongyao/labels/000000003.xml',
 '/rasterStores/LocalRasterStore/pool_chips_yongyao/labels/000000004.xml']

We can also create a image layer using one of this images and look at what it looks like. Note that a chip may or may not have a bounding box in it and one chip might have multiple boxes as well.

Part 2 - model training¶

If you've already done part 1, you should already have both the training chips and swimming pool labels. Please change the path to your own export training data folder that contains "images" and "labels" folder.

In [22]:

from arcgis.learn import SingleShotDetector, prepare_data

In [23]:

data_path = r'to_your_data_folder'

In [24]:

data = prepare_data(data_path, {0:'Pool'}, batch_size=32)
data.classes

Out[24]:

['background', 'Pool']

Visualize training data¶

To get a sense of what the training data looks like, arcgis.learn.show_batch() method randomly picks a few training chips and visualize them.

In [28]:

%%time
data.show_batch()

CPU times: user 3.43 s, sys: 1.14 s, total: 4.57 s
Wall time: 26.5 s

Detecting Swimming Pools using Satellite Imagery and Deep Learning¶

Table of Contents

Introduction and objective¶

Part 1 - export training data for deep learning¶

Import ArcGIS API for Python and get connected to your GIS¶

Prepare data that will be used for training data export¶

Specify a folder name in raster store that will be used to store our training data¶

Export training data using `arcgis.learn`¶

Part 2 - model training¶

Visualize training data¶

ArcGIS API for Python Sample Notebooks

Detecting Swimming Pools using Satellite Imagery and Deep Learning¶

Table of Contents

Introduction and objective¶

Part 1 - export training data for deep learning¶

Import ArcGIS API for Python and get connected to your GIS¶

Prepare data that will be used for training data export¶

Specify a folder name in raster store that will be used to store our training data¶

Export training data using arcgis.learn¶

Part 2 - model training¶

Visualize training data¶

Export training data using `arcgis.learn`¶