Avi's pick of the week is Detect and Track Multiple Faces by Dima Lisin. One of the projects I was working on recently required the detection and tracking of multiple objects (pedestrians in my case) over time in a video stream. While I know how to implement this myself I was hoping someone had some code I could modify for my problem, as I often do I called my friend Dima who is a developer on the Computer Vision System Toolbox team. Dima pointed me to his excellent File Exchange submission that I was able to modify to solve my problem in under a minute. There are three basic steps required to solve the object detection and tracking problem: First, let's look at how this is done in Dima's code. We start by connecting to a webcam using the USB Webcam support package.

vidObj = webcam;

The next step is to create an object that is used to detect faces in input frames, in this case I use vision.CascadeOjectDetector that detects faces by default.

faceDetector = vision.CascadeObjectDetector(); % Finds faces by default

The last object I create is used to "track" all the detected faces.

tracker = MultiObjectTrackerKLT;

% We then acquire new frames till we succesully detect the first
% object/face.
videoPlayer  = vision.DeployableVideoPlayer;

% Iterate until we have successfully detected a face
bboxes = [];
while isempty(bboxes)
    framergb = snapshot(vidObj);
    frame = rgb2gray(framergb);
    bboxes = faceDetector.step(frame);
end

Now lets look at the first detected object or face.

figure;
framergb = insertObjectAnnotation(framergb,'Rectangle',bboxes,'Face');
imshow(framergb);title('First Detected Face');

 
The next thing Dima's code does it adds a detection to the tracker object. This object does most of the heavy lifting and decided when to create a new track or if a detection is associated with an existing track. The tracker object in this File Exchange submission manages the new detections and the state of objects being tracked.

tracker.addDetections(frame, bboxes);

After this the detection and tracking code is run in a loop till the video window is closed. Note that the detector is run only once every 10 frames to improve execution performance.

%And loop until the player is closed
frameNumber = 0;
keepRunning = true;
while keepRunning

    framergb = snapshot(vidObj);
    frame = rgb2gray(framergb);

    if mod(frameNumber, 10) == 0
        %(Re)detect faces.
        %
        % NOTE: face detection is more expensive than imresize; we can
        % speed up the implementation by reacquiring faces using a
        % downsampled frame:
        % bboxes = faceDetector.step(frame);
        bboxes = 2 * faceDetector.step(imresize(frame, 0.5));
        if ~isempty(bboxes)
            tracker.addDetections(frame, bboxes);
        end
    else
        %Track faces
        tracker.track(frame);
    end

    %Display bounding boxes and tracked points.
    displayFrame = insertObjectAnnotation(framergb, 'rectangle',...
        tracker.Bboxes, tracker.BoxIds);
    displayFrame = insertMarker(displayFrame, tracker.Points);
    videoPlayer.step(displayFrame);

    frameNumber = frameNumber + 1;
    keepRunning = isOpen(videoPlayer);
end

%Clean up
release(videoPlayer);

This is what it looks like when I run the loop above.

Finally, I modified Dima's code to solve by problem simply by replacing the object detector used with a model used to detect pedestrians by changing this code. The detector models to locate faces and pedestrians are both part of the Computer Vision System Toolbox.

faceDetector = vision.CascadeObjectDetector(); % Finds faces by default

% to this
pedestrianDetector = vision.PeopleDetector;

If you are working on a project that involves detecting and tracking objects in images and video, I'd highly recommend trying out this pick.

Published with MATLAB® R2015a