File Exchange Pick of the Week

Our best user submissions

Detect and Track Multiple Faces

Avi's pick of the week is Detect and Track Multiple Faces by Dima Lisin. One of the projects I was working on recently required the detection and tracking of multiple objects (pedestrians in my case) over time in a video stream. While I know how to implement this myself I was hoping someone had some code I could modify for my problem, as I often do I called my friend Dima who is a developer on the Computer Vision System Toolbox team. Dima pointed me to his excellent File Exchange submission that I was able to modify to solve my problem in under a minute. There are three basic steps required to solve the object detection and tracking problem:
  1. Detect object of interest in current frame
  2. Match detections to current list of objects the system is tracking
  3. Update the status of each track, create new tracks, and delete lost tracks
First, let's look at how this is done in Dima's code. We start by connecting to a webcam using the USB Webcam support package.
vidObj = webcam;
The next step is to create an object that is used to detect faces in input frames, in this case I use vision.CascadeOjectDetector that detects faces by default.
faceDetector = vision.CascadeObjectDetector(); % Finds faces by default
The last object I create is used to "track" all the detected faces.
tracker = MultiObjectTrackerKLT;

% We then acquire new frames till we succesully detect the first
% object/face.
videoPlayer  = vision.DeployableVideoPlayer;

% Iterate until we have successfully detected a face
bboxes = [];
while isempty(bboxes)
    framergb = snapshot(vidObj);
    frame = rgb2gray(framergb);
    bboxes = faceDetector.step(frame);
Now lets look at the first detected object or face.
framergb = insertObjectAnnotation(framergb,'Rectangle',bboxes,'Face');
imshow(framergb);title('First Detected Face');
The next thing Dima's code does it adds a detection to the tracker object. This object does most of the heavy lifting and decided when to create a new track or if a detection is associated with an existing track. The tracker object in this File Exchange submission manages the new detections and the state of objects being tracked.
tracker.addDetections(frame, bboxes);
After this the detection and tracking code is run in a loop till the video window is closed. Note that the detector is run only once every 10 frames to improve execution performance.
%And loop until the player is closed
frameNumber = 0;
keepRunning = true;
while keepRunning

    framergb = snapshot(vidObj);
    frame = rgb2gray(framergb);

    if mod(frameNumber, 10) == 0
        %(Re)detect faces.
        % NOTE: face detection is more expensive than imresize; we can
        % speed up the implementation by reacquiring faces using a
        % downsampled frame:
        % bboxes = faceDetector.step(frame);
        bboxes = 2 * faceDetector.step(imresize(frame, 0.5));
        if ~isempty(bboxes)
            tracker.addDetections(frame, bboxes);
        %Track faces

    %Display bounding boxes and tracked points.
    displayFrame = insertObjectAnnotation(framergb, 'rectangle',...
        tracker.Bboxes, tracker.BoxIds);
    displayFrame = insertMarker(displayFrame, tracker.Points);

    frameNumber = frameNumber + 1;
    keepRunning = isOpen(videoPlayer);

%Clean up
This is what it looks like when I run the loop above.

Finally, I modified Dima's code to solve by problem simply by replacing the object detector used with a model used to detect pedestrians by changing this code. The detector models to locate faces and pedestrians are both part of the Computer Vision System Toolbox.
faceDetector = vision.CascadeObjectDetector(); % Finds faces by default

% to this
pedestrianDetector = vision.PeopleDetector;
If you are working on a project that involves detecting and tracking objects in images and video, I'd highly recommend trying out this pick.
  • print


To leave a comment, please click here to sign in to your MathWorks Account or create a new one.