Use MMDetection with arcgis.learn

Introduction

MMdetection is an open-source library containing many popular and state-of-the-art object detection models. Through arcgis.learn we have provided a bridge to use the growing list of models provided by the MMDetection library.

In 2018, the MMdet team won the COCO object detection challenge. Their codebase, which is built with Pytorch, has gradually evolved to include various models and methods. Although the detection task can be complex, the creators of MMDetection decomposed the models into different general components - Backbone, Neck, DenseHead, ROIExtractor, and ROIHead. With this abstraction, the library is able to provide a multitude of single-stage (e.g. GHM, FCOS), two-stage (e.g. Double-Head R-CNN), multi-stage (e.g. Cascade R-CNN), and other detection models. This abstraction-based approach also allows the development of models either by changing the different components or even by adding new ones. Read the MMDetection paper or checkout its github repository, for more details.

Setting up the environment

Follow the steps here to install deep learning dependencies in ArcGIS Pro or the Anaconda environment respectively.

ArcGIS Pro 2.8 users additionally need to install the CUDA toolkit (version 11), mmcv-full, and mmdet libraries. Follow these steps to do so:

• Download and install the latest CUDA toolkit version from here.
• Add the installed CUDA toolkit's bin folder path (typically, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin) to the (user or system) Path Environment Variables.
• Run the following command in a cloned environment:
  conda install -c esri mmcv-full mmdet

Implementation in arcgis.learn

Using MMDetection with arcgis.learn is as simple as using any other object detection model in the library. The only additional step is providing the name of the model to be used when initializing the MMDetection model object.

model = MMDetection(data, model='dcn')

The parameters required are:

• data - the data object prepared using prepare_data
• model - the name of the models from the list of supported models.

The following MMDetection models are supported through arcgis.learn:

MMDetection.supported_models

['atss',
 'carafe',
 'cascade_rcnn',
 'cascade_rpn',
 'dcn',
 'detectors',
 'double_heads',
 'dynamic_rcnn',
 'empirical_attention',
 'fcos',
 'foveabox',
 'fsaf',
 'ghm',
 'hrnet',
 'libra_rcnn',
 'nas_fcos',
 'pafpn',
 'pisa',
 'regnet',
 'reppoints',
 'res2net',
 'sabl',
 'vfnet']

The model argument can also accept the path to a config file for a variation of one of the supported models. These files can be found here. The configs directory in the mmdetection repository needs to be downloaded for other model variants to work.

Additionally, model_weight can be provided for these config files. The link to the corresponding weights can be found in the README file for each model.

cfg = r'C:\mmdetection\configs\atss\atss_r101_fpn_1x_coco.py'
wts = r'C:\mmdetection\atss\atss_r101_fpn_1x_coco\atss_r101_fpn_1x_20200825-dfcadd6f.pth'
model = MMDetection(data, model=cfg, model_weight=wts)

Training and inference

The MMdetection models can be trained using the fit method.

model.fit()

Alternatively, Train Deep Learning Model tool in ArcGIS Pro can be used to train the models. Models trained through the API or ArcGIS Pro can be used for inferencing using the Detect Objects Using Deep Learning tool in ArcGIS Pro.

For more information about the API, visit the API reference for MMDetection. For a detailed object detection workflow, refer to a sample notebook.

References

[1] OpenMmlab, “openmmlab/mmdetection: OpenMMLab Detection Toolbox and Benchmark.,” GitHub. [Online]. Available: https://github.com/open-mmlab/mmdetection/ [Accessed: 20-Jul-2021].