Brett's Pick this week is MTCNN Face Detection, by Justin Pinkney, a MathWorks colleague from our Consulting Department.

We have provided the functionality to detect faces using MathWorks' tools since 2012. Our face detectors, which live in the Computer Vision Toolbox, rely on the Viola-Jones algorithm, and are very easy to implement:

img = imread('visionteam.jpg');
imshow(img)
faceDetector = vision.CascadeObjectDetector();
bboxes = step(faceDetector, img);
figure
imshow(img)
title('Face Detection using the Cascade Object Detector')
for ii = 1:size(bboxes, 1)
    drawrectangle('Position', bboxes(ii, :), 'facealpha', 0);
end

The default model for the (tunable) cascade object detector uses classification and regression tree analysis (CART) of "Haar-like" features. As you can see above, the models often work quite well. Additionally, you can engage a 'FrontalFaceLBP' model to detect faces using "Local Binary Patterns." Both models work well for the image above, to detect faces that are upright and forward facing.

There are other detection models you can invoke with the CascadeObjectDetector, including 'UpperBody', 'EyePair' (big and small), 'LeftEye' and 'RightEye' (two options), 'ProfileFace', 'Mouth', and 'Nose'. (In addition, you can readily train your own!)

Which brings me to today's Pick.

Justin's submission implements "Multi-task Cascaded Convolutional Networks" (MTCCN) to solve the face detection problem. Having options is good; the built-in models don't always work well:

Note the large number of false positives, in both instances.

In contrast, the MTCCN model works remarkably well on this image:

[bboxes, scores, landmarks] = mtcnn.detectFaces(img);
figure
imshow(img)
title('MTCNN')
for ii = 1:size(bboxes, 1)
    drawrectangle('Position', bboxes(ii, :), 'facealpha', 0);
end

Another thing I appreciate is that facial landmarks are provided automatically as outputs of the MTCNN Model:

hold on
xs = landmarks(:, :, 1);
ys = landmarks(:, :, 2);
for ii = 1:size(xs, 1)
    for jj = 1:size(xs, 2)
        plot(xs(ii, jj), ys(ii, jj), 'r.')
    end
end

The MTCNN model also appears to be fairly robust; it even works on the 'cameraman' image, despite the fact that the face in the image is clearly not forward facing. (Both vision.CascadeObjectDetector face detectors fail on the image, at least with default parameterizations.)

I should also note that whereas the built-in Toolbox functions can adapt to RGB or grayscale images, the MTCNN detector operates on only on RGB images. But that's an easy problem to address:

if size(img, 3) == 1
    img = repmat(img, [1 1 3]);
end

Thanks, Justin! This is a nice addition to our detection tools!

As always, I welcome your thoughts and comments.

Get the MATLAB code

Published with MATLAB® R2021a