Skip To Content ArcGIS for Developers Sign In Dashboard

ArcGIS API for Python

Point Cloud Segmentation using PointCNN

Introduction

The arcgis.learn module includes PointCNN [1], to efficiently classify and segment points from a point cloud dataset. Point cloud datasets are typically collected using Lidar sensors (light detection and ranging) – an optical remote-sensing technique that uses laser light to densely sample the surface of the earth, producing highly accurate x, y, and z measurements. These Lidar sensor produced points, once post-processed and spatially organized are referred to as a 'Point cloud' and are typically collected using terrestrial (both mobile or static) and airborne Lidar.

Point clouds are large collections of 3D elevation points, which include x, y, and z, along with additional attributes such as, but not limited to: 'GPS timestamps', 'intensity' - the return strength of the laser pulse that generated the Lidar point, and 'number of returns' - the total number of returns for a given pulse. Additionally, with further post-processing, the Lidar data can be further attributed with RGB (red, green, and blue) bands - often derived from imagery collected at the same time as the lidar survey, as well as point classification – where each point gets assigned a classification (class code) that defines the type of object that has reflected the laser pulse. Lidar points can be classified into several categories including elevations for the ground, buildings, forest canopy, highway overpasses, water and anything else that the laser beam encounters during the survey. The different classes are defined using numeric integer codes. Point cloud data is typically stored in the industry standard file format - LAS.

It is in this aspect of point classification where deep learning and neural networks provide an efficient and scalable architecture for object classification, segmentation and classification of the input point cloud data, and with huge potential to render manual or semi assisted segmentation of point clouds obsolete. With this background let’s look at how the PointCNN model in arcgis.learn can be used for segmentation of point cloud data.

Though the PointCNN network emulates traditional convolution neural networks and is a generalization of CNNs such as those that operate to extract features from imagery, it also introduces a novel approach to feature learning from point clouds – by accounting for the irregular and unordered nature of point clouds, which is not typically encountered when processing data represented in dense grids – for example in the case of images. This key ability of PointCNN, where it simultaneously weights and permutes the input features before applying a typical convolution on the transformed features, is known as X-Conv. Another feature of PointCNN that makes it appealing is its capability to consider the point shapes while being invariant to ordering – making it ideal for classification of point clouds, and in some cases even better suited than neural networks designed specifically for point clouds (such as PointNet++).

In the remainder of this guide, we will discuss the architecture of PointCNN network and how to use it in arcgis.learn.


Figure 1. Visualization of point cloud dataset with RGB values [3]. The features apart from x, y, and z values, such as intensity and number of returns are quite valuable for the task of classification, but at the same, time they are sensor dependent and could become the main reasons for loss of generalization.

Point Cloud Segmentation

Point cloud segmentation is a task where each point in the point cloud is assigned a label, representing a real-world entity as described above. It is different from point cloud categorization where the complete point cloud dataset is given one label.