Detecting Airplanes in Satellite Imagery Using Deep Learning

• 🔬 Data Science
• 🥠 Deep Learning and Object Detection

• Introduction
• Necessary Imports
• Prepare the data
• Visualize training data
• Train the model
• Detect and visualize airplanes in validation set
• Inference
• Conclusion
• References

Introduction

With the rapid increase in high-resolution satellite imagery, detecting and tracking objects such as airplanes has become a vital task in applications like

• ✈️ Airport management
• 🛰️ Air traffic monitoring
• 🛡️ Security and surveillance
• 🚨 Disaster response and logistics

Airplane detection is a valuable process across multiple industries, as it allows for the identification and monitoring of airports at a global scale. Manual identification of airplanes in satellite images is time-consuming, error-prone, and not scalable. To overcome these challenges, we use deep learning-based object detection, specifically the Faster R-CNN model provided in the ArcGIS Learn API.

In this notebook, we will demonstrate how to train a deep learning model to detect airplanes in satellite imagery using the arcgis.learn module—a powerful deep learning API built into the ArcGIS platform. You can read more about the learn module here.

🎯 Objective

To train and deploy a high-accuracy airplane detection model using Faster R-CNN—a two-stage object detection network known for its balance between precision and speed. We use the arcgis.learn module from the ArcGIS API for Python for this task.

🧪 What You’ll Learn

By the end of this notebook, you will learn how to:

• Understand how object detection works in remote sensing using deep learning.
• Preprocess and prepare annotated satellite imagery for training.
• Use ArcGIS's arcgis.learn module to work with PASCAL VOC formatted data.
• Visualize and verify training samples using show_batch().
• Train a Faster R-CNN model for detecting airplanes in imagery.
• Evaluate the model’s predictions on unseen data.
• Save and reuse a trained model for further training or inferencing.
• Understand the available inferencing methods.

🧰 Tools & Technologies

• ArcGIS API for Python (arcgis.learn)
• Faster R-CNN: A deep learning model for object detection
• PASCAL VOC format: For training data annotation
• Satellite imagery: As the input data source

📂 Dataset Description

The dataset used in this notebook contains satellite images with labeled airplanes, annotated in the PASCAL VOC format. Each airplane is assigned a class label (1 through 7), which we will merge into a single category "airplane" to simplify the detection task. The dataset we used in this notebook was downloaded from RarePlanes. RarePlanes is an open-source machine learning dataset from CosmiQ Works and AI.Reverie that incorporates both real and synthetically generated satellite imagery. This data is licensed under the CC-BY-SA 4.0 license. We are going to use real training data that contains 5,815 training images. The images have dimensions of 512x512 and a resolution of 30cm. Tiled images have also been put into an images folder, and the annotations have been converted to the PASCAL_VOC_rectangles format and have been put into a labels folder. The dataset can be downloaded in the necessary format here.

🗺️ Workflow Overview

The notebook follows a step-by-step pipeline:

1. Data Preparation

   • Load annotated satellite images in PASCAL VOC format.
   • Remap multiple class labels to a single unified class: "airplane".

2. Data Inspection

   • Visually verify image-label alignment and class balance using show_batch().

3. Model Creation

   • Initialize a Faster R-CNN model with the prepared data.

4. Training

   • Train the model for 10 epochs using an auto-selected learning rate.

5. Model Evaluation

   • Display predicted bounding boxes on validation data to assess model performance.

6. Model Deployment

   • Save the trained model for future inference or deployment in ArcGIS Pro.

🚀 Let’s get started by importing required modules and preparing our data!

Necessary Imports

# 📁 Standard Python Libraries
# -----------------------------

import os                                                              # Interact with the operating system (e.g., directory operations, environment variables).
import glob                                                            # Search for files using wildcard patterns (e.g., "*.tif", "*.zip").
import zipfile                                                         # Work with .zip archives: extracting and creating compressed files.
import tempfile                                                        # Create secure temporary files and directories for intermediate data handling.
from pathlib import Path                                               # Modern and safer way to handle filesystem paths using object-oriented syntax.
from datetime import datetime                                          # Handle dates and times — useful for logging, naming, or tracking processes.

# 🌍 ArcGIS Deep Learning Libraries
# -----------------------------

import arcpy                                                           # ArcPy: Provides access to ArcGIS geoprocessing tools and functions within Python.
from arcgis.gis import GIS                                             # Connect to ArcGIS Online or Enterprise portal to manage and interact with GIS content.
from arcgis.learn import prepare_data, FasterRCNN, detect_objects      # prepare_data  : Prepares training data (e.g., from feature layers or labeled imagery) for deep learning workflows.
                                                                       # FasterRCNN    : Implements the Faster R-CNN deep learning model for object detection tasks.
                                                                       # detect_objects: Performs inference using trained models to detect objects in raster imagery.

Connect to your GIS

GIS("home") is typically used when you're running the script inside ArcGIS Pro or ArcGIS Enterprise Notebooks, where you're already signed in.

# 🔐 Connect to Your GIS
# -----------------------------

# This connects to your active ArcGIS session (e.g., within ArcGIS Pro or an Enterprise Notebook). 
# No need to enter a username or password — it uses your current portal authentication.
gis = GIS("home")

# ✅ Verify the connection by printing the signed-in username
print(f"Connected to ArcGIS as: {gis.users.me.username}")

Connected to ArcGIS as: gis_python

Alternative: Manual login (e.g., for use outside ArcGIS Pro or in standalone scripts)

To connect using ArcGIS Online or Enterprise credentials manually, use the following:

• Replace 'url' with your portal URL — e.g., 'https://www.arcgis.com' for ArcGIS Online.
• Replace 'your_username' and 'your_password' with valid credentials.

gis = GIS("https://www.arcgis.com", "your_username", "your_password")

📚 More Info: Learn more about the GIS class here: https://developers.arcgis.com/python/api-reference/arcgis.gis.toc.html#gis

# 📦 Fetch the Hosted Training Data
# -----------------------------

# Retrieve the training data item from ArcGIS Online or Enterprise using its unique item ID
# This dataset contains labeled airplane imagery in Pascal VOC format
training_data = gis.content.get("9982c9a749f94f7797e82358a90527c1")

# ✅ Optional: Display the item (works best in notebooks) to confirm it's the correct dataset
training_data

Airplanes_detection_training_data
Rareplanes dataset from https://www.cosmiqworks.org/rareplanes/ in Pascal VOC rectangles format.
Image Collection by api_data_owner
Last Modified: August 26, 2025
0 comments, 0 views

# 💾 Download the Training Dataset
# -----------------------------

# Downloads the item and saves it using its original name (e.g., "Airplanes_detection_training_data.zip")
filepath = training_data.download(file_name=training_data.name)

# 📂 Unzip the Dataset to Local Directory
# -----------------------------

# Extract the contents of the ZIP file into the same directory where it was saved
with zipfile.ZipFile(filepath, 'r') as zip_ref:
    zip_ref.extractall(Path(filepath).parent)

# Define the extracted data path by removing the ".zip" extension
data_path = Path(os.path.splitext(filepath)[0])
data_path

WindowsPath('C:/Users/Neh13173/AppData/Local/Temp/airplanes_30cm_512px_pascal_voc')

Prepare the data

Prepare the data using prepare_data() function

Here, we will specify the path to our training data and a few hyperparameters.

• path: Directory containing the exported labeled tiles.
• dataset_type: The annotation format used in the dataset.
• class_mapping: Dictionary that maps original class IDs to the desired class labels. This dataset contains multiple classes of planes; however, we are only interested in detecting planes and not classifying their categories. So, we will map the planes to one class using class_mapping.

This function returns a data object that can be fed into a model for training. Once the data is prepared using prepare_data(), you can proceed to initialize and train the FasterRCNNmodel.

# 🧪 Prepare the Training Data
# -----------------------------

# Define a mapping to convert multiple plane-related class IDs into a single class label: "airplane"
# This simplifies the classification task to "detect all airplanes", ignoring subclass types
class_mapping = {
    **dict.fromkeys(['1', '2', '3', '4', '5', '6', '7'], 'airplane')
}

# Use ArcGIS Learn's `prepare_data()` to load and preprocess the training data
data = prepare_data(
    path=data_path,                         # Path to the extracted training dataset
    dataset_type="PASCAL_VOC_rectangles",   # Annotation format: Pascal VOC (bounding boxes as rectangles)
    class_mapping=class_mapping             # Custom class mapping: merge all aircraft types into "airplane"
)

✅ Data is now prepared and can be used to initialize and train the FasterRCNN model.

Visualize training data

🔍 To inspect the quality and variety of your training data, you can use the show_batch() method provided by arcgis.learn. This method:

• Randomly selects a batch of labeled image chips (or features).
• Displays them in a grid along with their associated class labels.
• Helps you visually verify that:
  • Images are properly aligned with their labels.
  • There's enough class variation. Check for class balance across categories.
  • Data quality is suitable for training.
  • Verify that the tiles are correctly labeled.
• Especially useful during early experimentation to catch issues like:
  • Mislabeling
  • Missing classes
  • Poor image quality

✅ What to look for:

• Are airplanes correctly labeled?
• Are bounding boxes placed accurately?
• Is there variety in object size, orientation, and position?
• Is class distribution balanced (if using multiple classes)?

# Display a random batch of image tiles with their labels
# Useful for verifying class labels, bounding boxes, and image quality

data.show_batch()

Train the model

arcgis.learn provides many deep learning algorithms that we can use to detect airplanes. You can read more about those models here.

Here, we are going to use FasterRCNN. It is a powerful two-stage object detection model, ideal for precise object localization and classification.

# Create a Faster R-CNN model object with the prepared data. This will automatically download the pretrained COCO weights
model = FasterRCNN(data)

Downloading: "https://download.pytorch.org/models/fasterrcnn_resnet50_fpn_coco-258fb6c6.pth" to C:\Users\Neh13173/.cache\torch\hub\checkpoints\fasterrcnn_resnet50_fpn_coco-258fb6c6.pth
100%|███████████████████████████████████████████████████████████████████████████████| 160M/160M [00:06<00:00, 26.1MB/s]

Next, we can simply call the model.fit() method and pass the number of epochs to it. The model will then begin to train after finding the optimal learning rate itself.

# 🚀 Train the FasterRCNN Model
# -----------------------------

# Train the model for 10 epochs.
# The model automatically finds the optimal learning rate and tracks training progress.
# Early stopping is enabled to halt training if no improvement is observed.

model.fit(10, early_stopping=True)

📝 Training output explained:

• train_loss : Loss calculated on training data (lower is better)
• valid_loss : Loss on validation data (helps avoid overfitting)
• average_precision: How well the model detects airplanes (higher is better)
• time : Gives you an idea of computational cost per epoch.

📊 Interpretation of Metrics:

• train_loss: Decreases as the model learns to fit the training data.
• valid_loss: Measures generalization; it should decrease or stabilize.
• average_precision (AP): Indicates detection accuracy — closer to 1 means better detection.
• Training time: Duration per training epoch (typically ~6.5 minutes here)

# 📉 Visualize Training Metrics
# -----------------------------

# Plot the training and validation losses over epochs
# This helps you identify overfitting, underfitting, or plateaus in learning

model.plot_losses()

Detect and visualize airplanes on the validation set

Now that we have the trained model, we will examine how the model performs on data it has not yet seen.

# 👁️ Detect & Visualize Airplanes on Validation Set
# -----------------------------

# Display model predictions alongside ground truth annotations
# This helps visually assess the performance — how well the model is detecting airplanes
model.show_results()

📌 Note:

Even with just 10 epochs, we're seeing promising results. You can improve performance further with:

• More training epochs
• Data augmentation
• Hyperparameter tuning

We will now save the model for further training or later inferences. By default, the model should save into a models folder in your data folder.

Save the model

# 💾 Save the Trained Model
# -----------------------------

# Save the model for later use (e.g., resuming training or running inference on new data)
# This saves the model in a subfolder named 'models' within your training data directory

model.save("airplane_10e")

# 🗂 Example save path:
# WindowsPath('C:/Users/YourUser/AppData/Local/Temp/airplanes_30cm_512px_pascal_voc/models/airplane_10e')

Computing model metrics...

WindowsPath('C:/Users/Neh13173/AppData/Local/Temp/airplanes_30cm_512px_pascal_voc/models/airplane_10e')

Inference

🔎 Overview of Inference Options

Once your deep learning model is trained, there are three primary ways to perform object detection inferencing using ArcGIS:

1. ArcGIS API for Python

• Environment: Jupyter Notebook, or Python scripts in ArcGIS Online or Enterprise
• Function: detect_objects()
• Use Case: Automated workflows, cloud notebooks

2. ArcGIS Pro Geoprocessing Tool

• Environment: ArcGIS Pro
• Tool: Detect Objects Using Deep Learning
• Use Case: Visual workflows for users who prefer no coding

3. ArcPy (Python + ArcGIS Pro)

• Environment: ArcGIS Pro Python window or script
• Function: arcpy.ia.DetectObjectsUsingDeepLearning()
• Use Case: Automating inference inside ArcGIS Pro or from standalone scripts

Each method offers flexibility depending on your workflow — from notebooks to production-scale GIS environments.

Get test imagery and published model for inferencing.

# 📥 Load Imagery for Inference
# -----------------------------

# Fetch input raster (imagery) from ArcGIS Online using its item ID
inf_raster = gis.content.get("0a6ac9b3937c4893a2318d1efe1eae81")

inf_raster

airplanes_inferencing_imagery
Input imagery for referencing
Tiled Imagery Layer by api_data_owner
Last Modified: August 30, 2025
0 comments, 26 views

# Select the image layer from the imagery item (usually the first layer) to supply to inferencing methods
image_layer = inf_raster.layers[0]
image_layer

We have trained the model for more epochs and provided it as an item in ArcGIS Online. We can fetch the model below and try inferencing on unseen imagery.

# 🔍 Get model from ArcGIS Online 
fetched_model = gis.content.get('f28b2715ca714df5880e04c7d847edb5') 

# ✅ Display the retrieved model item to confirm it loaded correctly
fetched_model

airplane_47e
Trained deep learning model
Deep Learning Package by api_data_owner
Last Modified: August 30, 2025
0 comments, 47 views

# 📥 Download the published model to your system
model_path = fetched_model.download(file_name=fetched_model.name)
model_path

'C:\\Users\\Neh13173\\AppData\\Local\\Temp\\airplane_47e.dlpk'

1. Inferencing using ArcGIS API for Python

📌 The model is scale-sensitive, so using the correct cell size is crucial for accurate detection.

Use the detect_objects() function from learn module to run object detection in ArcGIS Online or Enterprise.

# 📦 Import necessary libraries
from arcgis.learn import detect_objects                                         # Function to run inference using a deep learning model in ArcGIS Online or Enterprise
import datetime                                                                 # Used to generate unique output names using timestamps

# 🧠 Run airplane detection using the pretrained Faster R-CNN model
detected_planes = detect_objects(
    input_raster=image_layer,                                                   # 🌍 Input raster layer (satellite imagery) to perform object detection on
    model=fetched_model,                                                        # 🧠 Fetched trained deep learning model (.dlpk) that will detect airplanes in the image
    output_name="detected_planes_" + str(datetime.datetime.now().microsecond),  # 🏷️ Unique name for the output layer to avoid overwriting results

    # 🔧 Model arguments – tune detection behavior and performance
    model_arguments={
        "padding": "56",                                                        # 🧱 Adds 56-pixel context padding to each tile
        "batch_size": "64",                                                     # 📦 Number of tiles to process in one forward pass (depends on GPU memory)
        "threshold": "0.5",                                                     # 🎯 Minimum confidence score (50%) to accept a detection
        "return_bboxes": "False",                                               # 🔁 If True, returns bounding boxes; if False, returns full polygon geometry
        "tile_size": "224",                                                     # 📏 Size of image chips extracted for inference (in pixels)
        "nms_overlap": "0.3",                                                   # 🤝 Overlap threshold for non-maximum suppression (to eliminate duplicate detections) and to define how much overlap between boxes is allowed
        "exclude_pad_detections": "True",                                       # ❌ Exclude detections that fall within padding areas
        "test_time_augmentation": "True",                                       # 🔄 Enables TTA: runs inference at multiple augmentations for robustness
        "tta_scales": "1,1.5,0.5"                                               # 🔍 Scales used for TTA (original, 150%, and 50% sizes)
    },

    # ⚙️ Execution context settings
    context={
        "processorType": "GPU" ,                                                # 🚀 Use GPU acceleration if available for faster inference
        "cellSize": "0.5"                                                       # 📏 Pixel resolution (in map units); adjust to match model training scale
    }
)

✅ Output:

The function returns a hosted feature layer (FeatureLayer object) containing:

• Polygon geometries representing detected airplanes
• Attributes: Class = "airplane", Confidence = model score (0 to 100)

This output can be visualized in ArcGIS Pro, ArcGIS Online, or used in further analysis.

2. Inferencing in ArcGIS Pro

You can also use the Detect Objects Using Deep Learning geoprocessing tool in ArcGIS Pro or ArcGIS Image Server to apply the model.

🗺️ Deployment Setup

We will apply the model to a validation region, using an input raster image. The model has been trained on high-resolution imagery and is optimized to detect airplanes at a specific scale. You can use the downloaded model and perform inferencing in ArcGIS Pro.

Parameters Used:

• Input Raster: High-resolution satellite image
• Model Definition: Trained model dlpk file
• Output Detected Objects: Feature class (e.g., Detected_airplanes)
• Extent: Use imagery bounds or custom AOI
• Cell Size: Between 0.5 and 1.0 (in map units)

3. Inferencing using Arcpy

You can use the ArcGIS Image Analyst extension to perform object detection directly on raster imagery using the DetectObjectsUsingDeepLearning() function.

import arcpy

# 🔍 Run inference on new satellite imagery using the trained Faster R-CNN model
arcpy.ia.DetectObjectsUsingDeepLearning(
    in_raster = inf_raster.url,                                 # Input imagery (URL or local path)
    out_detected_objects = r"<out_path>\DetectPlanes_222.shp",  # Output shapefile or feature class
    in_model_definition = model_path,                           # Path to .dlpk model file
    arguments = (
        "padding 56;"
        "threshold 0.5;"
        "nms_overlap 0.1;"
        "batch_size 64;"
        "exclude_pad_detections True;"
        "test_time_augmentation False;"
        "tta_scales 1,1.5,0.5"
    ),
    run_nms = "NO_NMS",                                         # Let ArcGIS skip internal NMS if external one is used
    confidence_score_field = "Confidence",                      # Output field for detection confidence
    class_value_field = "Class",                                # Output field for object type
    max_overlap_ratio = 0.3,                                    # Max allowed overlap between detected objects
    processing_mode = "PROCESS_AS_MOSAICKED_IMAGE",             # Ensures tiles are processed as a whole
    use_pixelspace = "NO_PIXELSPACE",                           # Keeps spatial reference (not pixel coordinates)
    in_objects_of_interest = ""                                 # Optional: specify AOI (Area of Interest)
)

🖼️ Visual Inspection in ArcGIS Pro

Once the model runs successfully, the output will be a vector feature class containing bounding boxes for each detected airplane in the scene.

You can add this layer (Detected_airplanes) to your ArcGIS Pro map to visually inspect detection performance:

Each polygon represents a detected airplane. Attribute fields include:

• Class: Type of detected object (e.g., "airplane")
• Confidence: Detection confidence score

Symbolize by confidence score or class to assess accuracy.

This image shows the results of airplane detection inference using the trained deep learning model (Faster R-CNN here) in ArcGIS Pro. The aerial/satellite test imagery shows a runway/apron area at an airport. Each bounding box corresponds to a detection with an associated confidence score. The highlighted cyan box indicates the selected feature in the attribute table.

This attribute table lists details about each detected airplane (total of 76 features detected).

The detections are spatially accurate—bounding boxes tightly fit around the airplanes.

Confidence scores vary (some as low as ~66%, others near 98%), which can be used to:

• Filter out false positives
• Prioritize high-certainty detections

This layer can now be:

• Used for further GIS analysis like airport infrastructure monitoring or asset tracking.
• Symbolized based on confidence or shape area.

Conclusion

In this notebook, we demonstrated a complete deep learning workflow using ArcGIS API for Python and ArcGIS Pro to:

• Prepare and preprocess remote sensing data using prepare_data()
• Visualize and verify data with show_batch()
• Train a Faster R-CNN object detection model using model.fit()
• Evaluate model performance
• Run inferencing using three ways
• Visualize results in ArcGIS Pro

This workflow shows the power of combining geospatial analysis with deep learning, especially for large-scale object detection tasks such as airplane detection in satellite imagery.

References

[1] Dataset source https://www.cosmiqworks.org/rareplanes/