Lidar (light detection and ranging) is a remote sensing technology. Lidar sensors emit laser pulses that reflect off objects, allowing them to perceive the structure of their surroundings. The sensors record the reflected light energy to determine the distances to objects to create a 2D or 3D representations of the surroundings.

With lidar technology a point cloud is created, that is a collection of data points plotted in 3-D space, where each point represents the X-, Y-, and Z-coordinates of a location on a real-world object’s surface, and the points collectively map the entire surface. Lidar is used in many industries including forestry, infrastructure analysis, and mining.

Deep learning adds precision and speed to the processing of point cloud data. Watch the following five videos that walk you through the steps of a lidar processing workflow with deep learning. You can also find the videos here.

For more examples, see lidar with deep learning examples. To learn more about how MATLAB users can use deep learning, computer vision, and image processing for addressing lidar challenges, check out this story: Spacesium Creates Deep Learning System to Segment Large Lidar Point Clouds with MATLAB.

 

Load Point Cloud Data

Load point cloud data as a data store using the pcread function and load the bounding box labels using the boxLabelDatastore function.

 

Preprocess Data

Split the data into training and testing sets. Synthetically increase the size of the training data set by randomly adding a fixed number of car and truck class objects to every point cloud and by using augmentation techniques such as flipping, scaling, rotation, and translation. To learn more about typical data augmentation techniques for 3-D object detection workflows with lidar data, see Data Augmentations for Lidar Object Detection Using Deep Learning.

 

Define Deep Neural Network

Define network parameters, such as anchor boxes and pillars, for the PointPillars network. Then, define the PointPillars detection network using the pointPillarObjectDetector function.

 

Train Network

To train a network, you must specify training options. Because training a network can be time consuming, a pretrained PointPillars model is used for this workflow.

 

Object Detection

Test the PointPillars network on a test dataset and display the detected output point cloud with bounding boxes.