{"id":49,"date":"2017-11-17T08:00:23","date_gmt":"2017-11-17T08:00:23","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=49"},"modified":"2021-04-06T15:52:40","modified_gmt":"2021-04-06T19:52:40","slug":"deep-learning-for-automated-driving-part-2-lane-detection","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2017\/11\/17\/deep-learning-for-automated-driving-part-2-lane-detection\/","title":{"rendered":"Deep Learning for Automated Driving (Part 2) – Lane Detection"},"content":{"rendered":"This is the second post in the series on using deep learning for automated driving. In the first post<\/a> I covered object detection (specifically vehicle detection). In this post I will go over how deep learning is used to find lane boundaries.\r\n

Lane Detection<\/h2>\r\nLane detection is the identification of the location and curvature of lane boundaries of visible lanes on a roadway. This is useful to help a vehicle center it's driving path and safely navigate lane changes. Unlike the previous post where the algorithm had to predict the class of the vehicle (classification) as well as its location (bounding box), in this case I need the algorithm to output a set of numbers that represent the coefficients of parabolas that represent the right and left lane boundaries. To solve this, I will construct a CNN that performs regression to output the coefficients.\r\n\r\nSimilar to my previous post, the first step in the process is to label a set of training data with the ground truth representing the right and left lane boundaries. As in the previous section,\u00a0 I recommend using the Ground Truth Labeler app in\u00a0MATLAB Automated Driving System Toolbox<\/a>. Notice how I\u2019ve labeled the lane boundaries using poly-lines in the figure below in addition to other objects that are labeled using rectangular bounding boxes.\r\n\r\n\"\"<\/a>\r\n

Labeled objects and lane boundaries.<\/i><\/p>\r\nTo get a little insight on the ground truth for lane boundaries,\u00a0the table shows the table used to store the coefficients. Notice each column represents one of the coefficients of the parabola.\r\n\r\n\"\"<\/a>\r\n

Coefficients of parabolas representing lane boundaries.<\/i><\/p>\r\nYou\u2019ll notice that\u00a0there are just 1225 training samples for this task, which is usually not sufficient to train a deep network. The reason this actually works for us is I used\u00a0transfer learning<\/a>, by starting with a pre-existing network that was trained on a massive set of images and adapting it for our specific task of finding lane boundaries. The network I\u2019ll use as a starting point is AlexNet, trained to recognize 1000 different categories of images.\r\n\r\nYou can load a pre-trained AlexNet model into MATLAB with a single line of code. Note that MATLAB allows you to load other models like GoogLeNet, VGG-16 and VGG-19, or import models from the Caffe ModelZoo.\r\n

originalConvNet = alexnet<\/pre>\r\nOnce I have the network loaded into MATLAB I need to modify its structure slightly to change it from a classification network into a regression network. Notice in the code below that, I have 6 outputs corresponding to the three coefficients for the parabola representing each lane boundary(left and right).\r\n
% Extract layers from the original network\r\nlayers = originalConvNet.Layers\r\n\r\n% Net surgery\r\n% Replace the last few fully connected layers with suitable size layers\r\nlayers(20:25) = [];\r\noutputLayers = [ ...\r\nfullyConnectedLayer(16, 'Name', 'fcLane1');\r\nreluLayer('Name','fcLane1Relu');\r\nfullyConnectedLayer(6, 'Name', 'fcLane2');\r\nregressionLayer('Name','output')];\r\nlayers = [layers; outputLayers]<\/pre>\r\nI used an NVIDIA Titan X (Pascal) GPU to train this network. \u00a0As you can see in the figure below it took 245 seconds to train the network. This time is lower than I expected mostly due to the fact that only a limited number of weights from the new layers are being learned, and also because MATLAB automatically uses CUDA and cuDNN to accelerate the training process when a GPU is available.\r\n\r\n\"\"<\/a>\r\n
Training progress to train lane boundary detection regression network on an NVIDIA Titan X GPU.<\/i><\/p>\r\nDespite the limited number of training samples, the network performs extremely well and is able to accurately detect lane boundaries, as\u00a0the figure below\u00a0 shows.\r\n\r\n\"\"<\/a>\r\n
Output of lane boundary detection network.<\/i><\/p>\r\n\r\n
Conclusion<\/h2>\r\nIn this series of posts I\u00a0covered how to solve some of the common perception tasks for automated driving using deep learning and MATLAB. I hope it has helped you appreciate how ground truth labeling impacts the time required to solve some of these problems, as well as the ease and performance of defining and training neural networks in MATLAB with GPU acceleration.\r\n\r\nTo solve the problems described in this post I used MATLAB R2017b along with\u00a0Neural Network Toolbox<\/a>,\u00a0Parallel Computing Toolbox<\/a>,\u00a0Computer Vision System Toolbox<\/a>, and\u00a0Automated Driving System Toolbox<\/a>.\r\n\r\nVisit the\u00a0MATLAB deep learning page<\/a>\u00a0to learn about all the deep learning capabilities in MATLAB.","protected":false},"excerpt":{"rendered":"
<\/div>
This is the second post in the series on using deep learning for automated driving. In the first post I covered object detection (specifically vehicle detection). In this post I will go over how deep... read more >><\/a><\/p>","protected":false},"author":132,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/49"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/132"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=49"}],"version-history":[{"count":2,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/49\/revisions"}],"predecessor-version":[{"id":54,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/49\/revisions\/54"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=49"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=49"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=49"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}