Connell D’Souza is back guest-blogging and tells us about object detection in MATLAB.

—

A few weeks ago, I visited Florida Atlantic University’s Team Owltonomous, who compete in RoboNation student competitions like RoboBoat, RobotX and from 2019 onwards RoboSub as well! Our discussions spanned a range of topics including designing object detection algorithms in MATLAB. Object detectors are critical to allow an autonomous system to identify what is in its surroundings. The team thought the workflow would help reduce the time needed to develop object detectors given their 1-year development cycle. So, I thought I would share some of our discussions in this post! The code discussed in this example can be found in this File Exchange entry.

What is an Object Detector?

An object detector is a computer program that employs computer vision, image processing and/or artificial intelligence algorithms to detect features of interest in images or a video stream. As Sebastian argues in this post about sensors for autonomous systems, a camera is a cheap and important perception sensor employed by autonomous systems. You can use an object detection algorithm to make sense of what your camera “sees”. I like to classify object detectors into 3 broad categories based on the technology used:

1. Classical Computer Vision: Employs classical computer vision techniques such as image segmentation and feature detection and matching to identify objects of interest. Features could include colors, shapes, edges, etc. Check out our online tutorial series on Computer Vision in MATLAB to learn more.
   e.g. Color Thresholding, Blob Analysis, Histogram of Gradients, Speeded-Up Robust Features
2. Machine Learning: Machine learning is an effective way to classify data. These detectors use classical computer vision algorithms to extract features or data points from the image and then employ machine learning techniques like support vector machines to classify the features.
   e.g. Cascade Object Detector (Viola-Jones Algorithm), Aggregate Channel Features (ACF)
3. Deep Learning: Deep Learning detectors use data in the form of labelled images to teach a convolutional neural network (CNN) features of interest. You can train a network from scratch or perform transfer learning on pre-trained networks. Check out the Deep Learning Onramp to learn how you can get started!
   e.g. YOLO v2, R-CNN, Fast R-CNN and Faster R-CNNs

We will discuss designing an ACF Object Detector which is a machine learning detector. However, as you will see, by replacing a few functions and with the right compute power you can follow the same process for deep learning-based detectors as well.

Design that Detector

The workflow for using ground truth for object detection is shown in the graphic below. I will use the next few sections to explain them briefly.

Generating Ground Truth

Ground Truth refers to information provided by empirical evidence or observation. In our case this is a set of labeled images. Labeled images contain, images, object class descriptors like bigRedBuoy, smallGreenBuoy, as shown above and locations of regions of interest (ROIs) in those images. The designer needs to supply this dataset to train the detector. There are many publicly available, open-source labeled data sets, but there could be a chance where no dataset is available for your specific application. This will warrant you to create your own ground truth.

MATLAB provides you with a tool – Ground Truth Labeler to automate this process. This app gives you an easy way to label rectangular ROI’s, polyline ROIs, pixels, and scenes. You can also automate this process using built-in automation algorithms or providing your own algorithm. Once you have finished labeling the images or video you can export the ground truth as a ground truth data object. Watch the video below to learn how you can automate ground truth labelling!

[VIDEO] MATLAB and Simulink Robotics Arena: Using Ground Truth for Object Detection, Part 1

Training Object Detectors

Now that you have a labeled dataset or ground truth, Computer Vision Toolbox provides built-in training functions that can be used to train machine learning or deep learning detectors. The graphic below shows the functions and the workflow that can help you train these detectors.

The trainACFObjectDetector function is used to train an ACF object detector which as we discussed earlier is a machine learning detector. This function call can be replaced by other similar functions like

• trainRCNNObjectDetector – R-CNN deep learning object detector
• trainFastRCNNObjectDetector – Fast R-CNN deep learning object detector
• trainFasterRCNNObjectDetector – Faster R-CNN deep learning object detector

One caveat is you will need to provide some training options that are specific to these deep learning detectors which you can read about it in the documentation links above.

Now that you have a trained detector you can use the detect method of the object detector object to identify the object of interest an image!

Evaluating Object Detectors

Once you have a trained detector and have visually confirmed that it is detecting what it is intended to, you may want to evaluate these detectors with some numerical metrics. This could be in the form of a confusion matrix or other common metrics like Miss Rate and Precision. MATLAB offers built-in functions to carry out these evaluations.

When it comes to the miss rate and precision, an important parameter used is threshold. The threshold parameter determines the extent of overlap of the bounding box around an object of interest given by the detector over the bounding box of the same object in the ground truth. It is calculated as the Intersection over Union (IoU) or Jaccard index. As shown in the plots below for the same detection and ground truth data, changing the value of the threshold parameter drastically changes the value of the evaluation metric. Pick an overlap threshold value that best suits your application and keep in mind a higher threshold means you are expecting your detection results to overlap a larger area of the ground truth.

When selecting a dataset to test your detector make sure you use a data set that is independent of the one used to train the detector. This will help ensure you are not overfitting your detector to a particular dataset. Watch this video below to see how this code works!

[VIDEO] MATLAB and Simulink Robotics Arena: Using Ground Truth for Object Detection, Part 2

Generate C/C++ Code

To use this detector on your robot’s/vehicle’s computer, you will need to convert the MATLAB code to a low-level language like C/C++, that can be executed on an embedded system. In R2019a, we added code generation support for some object detectors discussed above, including the ACF object detector that is used in this example. To generate C/C++ code, MATLAB code must be packaged in a function. The ACFObjectDetector object, cannot be passed through the function interface as an argument in the generated code as it is a MATLAB object, you will have to construct the object inside the function by calling the constructor method of the acfObjectDetector class with the Classifier and TrainingOptions properties as arguments. This can be done by converting the object into a structure with the properties as fields and save it as a MAT-file as shown below.

s = toStruct(detector);
save('detectorCodegen.mat','-struct','s','Classifier','ModelName',...
'NumWeakLearners','ObjectTrainingSize','TrainingOptions') 

Next load the MAT-file inside the function using the coder.load function as shown below and call the constructor. You will want to declare it as persistent, so it is stored in memory and does not need to be constructed at every call to the function. Once you have modified your code you can follow the MATLAB Coder app workflow to obtain C/C++ code. Not familiar with the MATLAB Coder app?  Check out this tutorial series on code generation.

function [boxes,scores ] = ACFDetector(img) 
%#codegen 

persistent s detector 
if isempty(detector)
  	s = coder.load('detectorCodegen.mat');
	detector = acfObjectDetector(s.Classifier,s.TrainingOptions);
end
[boxes, scores] = detect(detector,img);) 

To conclude, I would encourage you to download the code and try it out. See how a few lines of MATLAB code can help you develop robust object detectors as well as convert it into C/C++ .