Contents

Avi's pick of the week is a hardware support package Image Acquisition Toolbox Support Package for Kinect For Windows Runtime, by The Image Acquisition Toolbox Team. The hardware support package enables you to acquire RGB, depth images, and 3-D point clouds from a Kinect v2 sensor. If you have a Kinect for Windows v2 you can start using it in MATLAB by downloading the hardware support package from File Exchange.

Using the Kinect for Windows v2 in MATLAB

Once you have sucessfuly installed the hardware support package you can connect to the device by defining a imaq.VideoDevice system object. Remember the Kinect v2 returns both RGB and depth images so we create a one system object for each

colorDevice = imaq.VideoDevice('kinect',1)
depthDevice = imaq.VideoDevice('kinect',2)

colorDevice = 

  imaq.VideoDevice with properties:

                Device: 'Kinect V2 Color Sensor (kinect-1)'
           VideoFormat: 'BGR_1920x1080'
                   ROI: [1 1 1920 1080]
    ReturnedColorSpace: 'rgb'
      ReturnedDataType: 'uint8'
      DeviceProperties: [1x1 imaq.internal.DeviceProperties]


depthDevice = 

  imaq.VideoDevice with properties:

                Device: 'Kinect V2 Depth Sensor (kinect-2)'
           VideoFormat: 'Depth_512x424'
                   ROI: [1 1 512 424]
    ReturnedColorSpace: 'grayscale'
      ReturnedDataType: 'uint16'
      DeviceProperties: [1x1 imaq.internal.DeviceProperties]

Initializing the device

Now lets initialize the device and grab an input RGB image and a depth image.

step(colorDevice); % Initialize color/RGB sensor
step(depthDevice); % Initialize depth sensor

colorImage = step(colorDevice); % Load one color/RGB frame
depthImage = step(depthDevice); % Load one depth frame

Get a 3-D point cloud from the device

Since the sensor has both color and depth information you can combine information from both sensor to create a 3-D point cloud.

ptCloud = pcfromkinect(depthDevice,depthImage,colorImage);

Viewing the point cloud stream from the Kinect v2

Lets initialize the pcplayer that will let us visualize a live stream of 3-D point clouds. We first initialize the player.

player = pcplayer(ptCloud.XLimits,ptCloud.YLimits,ptCloud.ZLimits,...
    'VerticalAxis','y','VerticalAxisDir','down');

xlabel(player.Axes,'X (m)');
ylabel(player.Axes,'Y (m)');
zlabel(player.Axes,'Z (m)');

% Then stream a live 3-D point cloud from the Kinect v2.
for i = 1:500
    colorImage = step(colorDevice);
    depthImage = step(depthDevice);

    ptCloud = pcfromkinect(depthDevice,depthImage,colorImage);

    view(player,ptCloud);
end

Detect planes in the 3-D point cloud

Once we have aquired a 3-D point cloud from the Kinect v2 we can process it using functionality in the Computer Vision System Toolbox. Here I simply extract and display planes in the 3-D point cloud.

colorImage = step(colorDevice);
depthImage = step(depthDevice);
ptCloud = pcfromkinect(depthDevice,depthImage,colorImage); % Grab a new point cloud
maxDistance = 0.02;
referenceVector = [0,0,1];
maxAngularDistance = 5;
[model,inlierIndices,outlierIndices] = pcfitplane(ptCloud,maxDistance,referenceVector,maxAngularDistance);
figure;
pcshow(ptCloud);
hold on;
plot(model)

Release the device

When you are all doen don't forget to release the device.

release(colorDevice);
release(depthDevice);

Published with MATLAB® R2016a