{"id":7702,"date":"2016-07-22T09:00:32","date_gmt":"2016-07-22T13:00:32","guid":{"rendered":"https:\/\/blogs.mathworks.com\/pick\/?p=7702"},"modified":"2017-11-17T12:52:36","modified_gmt":"2017-11-17T17:52:36","slug":"hardware-support-for-image-aquisition-from-kinect-v2","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/pick\/2016\/07\/22\/hardware-support-for-image-aquisition-from-kinect-v2\/","title":{"rendered":"Hardware Support for Image Aquisition from Kinect v2"},"content":{"rendered":"<h2>Contents<\/h2>\r\n<div>\r\n<ul>\r\n \t<li><a href=\"#2\">Using the Kinect for Windows v2 in MATLAB<\/a><\/li>\r\n \t<li><a href=\"#3\">Initializing the device<\/a><\/li>\r\n \t<li><a href=\"#4\">Get a 3-D point cloud from the device<\/a><\/li>\r\n \t<li><a href=\"#5\">Viewing the point cloud stream from the Kinect v2<\/a><\/li>\r\n \t<li><a href=\"#6\">Detect planes in the 3-D point cloud<\/a><\/li>\r\n \t<li><a href=\"#7\">Release the device<\/a><\/li>\r\n<\/ul>\r\n<\/div>\r\nAvi's pick of the week is a hardware support package <a href=\"https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/40445-image-acquisition-toolbox-support-package-for-kinect-for-windows-sensor\">Image Acquisition Toolbox Support Package for Kinect For Windows Runtime<\/a>, by <a href=\"https:\/\/www.mathworks.com\/matlabcentral\/profile\/authors\/3215751-mathworks-image-acquisition-toolbox-team\">The Image Acquisition Toolbox Team<\/a>. The hardware support package enables you to acquire RGB, depth images, and 3-D point clouds from a Kinect v2 sensor.\r\n\r\nIf you have a Kinect for Windows v2 you can start using it in MATLAB by downloading the hardware support package from File Exchange.\r\n<h2><\/h2>\r\n<h2><strong>Using the Kinect for Windows v2 in MATLAB<\/strong><a name=\"2\"><\/a><\/h2>\r\nOnce you have sucessfuly installed the hardware support package you can connect to the device by defining a imaq.VideoDevice system object. Remember the Kinect v2 returns both RGB and depth images so we create a one system object for each\r\n<pre class=\"codeinput\">colorDevice = imaq.VideoDevice(<span class=\"string\">'kinect'<\/span>,1)\r\ndepthDevice = imaq.VideoDevice(<span class=\"string\">'kinect'<\/span>,2)\r\n<\/pre>\r\n<pre class=\"codeoutput\">colorDevice = \r\n\r\n  imaq.VideoDevice with properties:\r\n\r\n                Device: 'Kinect V2 Color Sensor (kinect-1)'\r\n           VideoFormat: 'BGR_1920x1080'\r\n                   ROI: [1 1 1920 1080]\r\n    ReturnedColorSpace: 'rgb'\r\n      ReturnedDataType: 'uint8'\r\n      DeviceProperties: [1x1 imaq.internal.DeviceProperties]\r\n\r\n\r\ndepthDevice = \r\n\r\n  imaq.VideoDevice with properties:\r\n\r\n                Device: 'Kinect V2 Depth Sensor (kinect-2)'\r\n           VideoFormat: 'Depth_512x424'\r\n                   ROI: [1 1 512 424]\r\n    ReturnedColorSpace: 'grayscale'\r\n      ReturnedDataType: 'uint16'\r\n      DeviceProperties: [1x1 imaq.internal.DeviceProperties]\r\n\r\n<\/pre>\r\n<h2><strong>Initializing the device<\/strong><a name=\"3\"><\/a><\/h2>\r\nNow lets initialize the device and grab an input RGB image and a depth image.\r\n<pre class=\"codeinput\">step(colorDevice); <span class=\"comment\">% Initialize color\/RGB sensor<\/span>\r\nstep(depthDevice); <span class=\"comment\">% Initialize depth sensor<\/span>\r\n\r\ncolorImage = step(colorDevice); <span class=\"comment\">% Load one color\/RGB frame<\/span>\r\ndepthImage = step(depthDevice); <span class=\"comment\">% Load one depth frame<\/span>\r\n<\/pre>\r\n<h2><strong>Get a 3-D point cloud from the device<\/strong><a name=\"4\"><\/a><\/h2>\r\nSince the sensor has both color and depth information you can combine information from both sensor to create a 3-D point cloud.\r\n<pre class=\"codeinput\">ptCloud = pcfromkinect(depthDevice,depthImage,colorImage);\r\n<\/pre>\r\n<h2><strong>Viewing the point cloud stream from the Kinect v2<\/strong><a name=\"5\"><\/a><\/h2>\r\nLets initialize the pcplayer that will let us visualize a live stream of 3-D point clouds. We first initialize the player.\r\n<pre class=\"codeinput\">player = pcplayer(ptCloud.XLimits,ptCloud.YLimits,ptCloud.ZLimits,<span class=\"keyword\">...<\/span>\r\n    <span class=\"string\">'VerticalAxis'<\/span>,<span class=\"string\">'y'<\/span>,<span class=\"string\">'VerticalAxisDir'<\/span>,<span class=\"string\">'down'<\/span>);\r\n\r\nxlabel(player.Axes,<span class=\"string\">'X (m)'<\/span>);\r\nylabel(player.Axes,<span class=\"string\">'Y (m)'<\/span>);\r\nzlabel(player.Axes,<span class=\"string\">'Z (m)'<\/span>);\r\n\r\n<span class=\"comment\">% Then stream a live 3-D point cloud from the Kinect v2.<\/span>\r\n<span class=\"keyword\">for<\/span> i = 1:500\r\n    colorImage = step(colorDevice);\r\n    depthImage = step(depthDevice);\r\n\r\n    ptCloud = pcfromkinect(depthDevice,depthImage,colorImage);\r\n\r\n    view(player,ptCloud);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n\r\n<a href=\"https:\/\/blogs.mathworks.com\/pick\/files\/Kinect_POTW_Post_01.png\"><img decoding=\"async\" loading=\"lazy\" width=\"1368\" height=\"795\" class=\"aligncenter size-full wp-image-7704\" src=\"https:\/\/blogs.mathworks.com\/pick\/files\/Kinect_POTW_Post_01.png\" alt=\"Kinect_POTW_Post_01\" \/><\/a><\/pre>\r\n<h2>Detect planes in the 3-D point cloud<a name=\"6\"><\/a><\/h2>\r\nOnce we have aquired a 3-D point cloud from the Kinect v2 we can process it using functionality in the Computer Vision System Toolbox. Here I simply extract and display planes in the 3-D point cloud.\r\n<pre class=\"codeinput\">colorImage = step(colorDevice);\r\ndepthImage = step(depthDevice);\r\nptCloud = pcfromkinect(depthDevice,depthImage,colorImage); <span class=\"comment\">% Grab a new point cloud<\/span>\r\nmaxDistance = 0.02;\r\nreferenceVector = [0,0,1];\r\nmaxAngularDistance = 5;\r\n[model,inlierIndices,outlierIndices] = pcfitplane(ptCloud,maxDistance,referenceVector,maxAngularDistance);\r\nfigure;\r\npcshow(ptCloud);\r\nhold <span class=\"string\">on<\/span>;\r\nplot(model)\r\n\r\n<a href=\"https:\/\/blogs.mathworks.com\/pick\/files\/Kinect_POTW_Post_02.png\"><img decoding=\"async\" loading=\"lazy\" width=\"1368\" height=\"795\" class=\"aligncenter size-full wp-image-7705\" src=\"https:\/\/blogs.mathworks.com\/pick\/files\/Kinect_POTW_Post_02.png\" alt=\"Kinect_POTW_Post_02\" \/><\/a><\/pre>\r\n<h2><strong>Release the device<\/strong><a name=\"7\"><\/a><\/h2>\r\nWhen you are all doen don't forget to release the device.\r\n<pre class=\"codeinput\">release(colorDevice);\r\nrelease(depthDevice);\r\n<\/pre>\r\n<p class=\"footer\">\r\n<a href=\"https:\/\/www.mathworks.com\/products\/matlab\/\">Published with MATLAB\u00ae R2016a<\/a><\/p>","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/pick\/files\/Kinect_POTW_Post_01.png\" onError=\"this.style.display ='none';\" \/><\/div><p>Contents\r\n\r\n\r\n \tUsing the Kinect for Windows v2 in MATLAB\r\n \tInitializing the device\r\n \tGet a 3-D point cloud from the device\r\n \tViewing the point cloud stream from the Kinect v2\r\n \tDetect planes in... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/pick\/2016\/07\/22\/hardware-support-for-image-aquisition-from-kinect-v2\/\">read more >><\/a><\/p>","protected":false},"author":132,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[16],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/7702"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/users\/132"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/comments?post=7702"}],"version-history":[{"count":9,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/7702\/revisions"}],"predecessor-version":[{"id":8984,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/7702\/revisions\/8984"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/media?parent=7702"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/categories?post=7702"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/tags?post=7702"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}