{"id":400,"date":"2018-06-08T17:46:23","date_gmt":"2018-06-08T17:46:23","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=400"},"modified":"2021-04-06T15:51:57","modified_gmt":"2021-04-06T19:51:57","slug":"semantic-segmentation-using-deep-learning","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2018\/06\/08\/semantic-segmentation-using-deep-learning\/","title":{"rendered":"Semantic Segmentation Using Deep Learning"},"content":{"rendered":"

Today I want to show you a documentation example<\/a> that shows how to train a semantic segmentation network using deep learning and the Computer Vision System Toolbox. <\/p>\r\n

A semantic segmentation network classifies every pixel in an image, resulting in an image that is segmented by class. Applications for semantic segmentation include road segmentation for autonomous driving and cancer cell segmentation for medical diagnosis. To learn more, see Semantic Segmentation Basics<\/a>. <\/p>\r\n

To illustrate the training procedure, this example trains SegNet, one type of convolutional neural network (CNN) designed for semantic image segmentation. Other types networks for semantic segmentation include fully convolutional networks (FCN) and U-Net. The training procedure shown here can be applied to those networks too. <\/p>\r\n

This example uses the CamVid dataset<\/a> from the University of Cambridge for training. This dataset is a collection of images containing street-level views obtained while driving. The dataset provides pixel-level labels for 32 semantic classes including car, pedestrian, and road. <\/p>\r\n

Setup<\/h3>\r\n

This example creates the SegNet network with weights initialized from the VGG-16 network. To get VGG-16, install Neural Network Toolbox\u2122 Model for VGG-16 Network<\/a>. After installation is complete, run the following code to verify that the installation is correct. <\/p>

vgg16();\r\n<\/pre>
In addition, download a pretrained version of SegNet. The pretrained model allows you to run the entire example without having to wait for training to complete. <\/p>
pretrainedURL = 'https:\/\/www.mathworks.com\/supportfiles\/vision\/data\/segnetVGG16CamVid.mat'<\/span>;\r\npretrainedFolder = fullfile(tempdir,'pretrainedSegNet'<\/span>);\r\npretrainedSegNet = fullfile(pretrainedFolder,'segnetVGG16CamVid.mat'<\/span>); \r\nif<\/span> ~exist(pretrainedFolder,'dir'<\/span>)\r\n    mkdir(pretrainedFolder);\r\n    disp('Downloading pretrained SegNet (107 MB)...'<\/span>);\r\n    websave(pretrainedSegNet,pretrainedURL);\r\nend<\/span>\r\n<\/pre>
Downloading pretrained SegNet (107 MB)...\r\n<\/pre>
A CUDA-capable NVIDIA\u2122 GPU with compute capability 3.0 or higher is highly recommended for running this example. Use of a GPU requires Parallel Computing Toolbox\u2122. <\/p>\r\n      
Download CamVid Dataset<\/h3>\r\n      
Download the CamVid dataset from the following URLs.<\/p>
imageURL = 'http:\/\/web4.cs.ucl.ac.uk\/staff\/g.brostow\/MotionSegRecData\/files\/701_StillsRaw_full.zip'<\/span>;\r\nlabelURL = 'http:\/\/web4.cs.ucl.ac.uk\/staff\/g.brostow\/MotionSegRecData\/data\/LabeledApproved_full.zip'<\/span>;\r\noutputFolder = fullfile(tempdir,'CamVid'<\/span>);\r\nif<\/span> ~exist(outputFolder, 'dir'<\/span>)\r\n   \r\n    mkdir(outputFolder)\r\n    labelsZip = fullfile(outputFolder,'labels.zip'<\/span>);\r\n    imagesZip = fullfile(outputFolder,'images.zip'<\/span>);   \r\n    \r\n    disp('Downloading 16 MB CamVid dataset labels...'<\/span>); \r\n    websave(labelsZip, labelURL);\r\n    unzip(labelsZip, fullfile(outputFolder,'labels'<\/span>));\r\n    \r\n    disp('Downloading 557 MB CamVid dataset images...'<\/span>);  \r\n    websave(imagesZip, imageURL);       \r\n    unzip(imagesZip, fullfile(outputFolder,'images'<\/span>));    \r\nend<\/span>\r\n<\/pre>
Note: Download time of the data depends on your Internet connection. The commands used above block MATLAB until the download is complete. Alternatively, you can use your web browser to first download the dataset to your local disk. To use the file you downloaded from the web, change the outputFolder<\/inline> variable above to the location of the downloaded file. <\/p>\r\n      
Load CamVid Images<\/h3>\r\n      
Use imageDatastore<\/a> to load CamVid images. The imageDatastore<\/inline> enables you to efficiently load a large collection of images on disk. <\/p>
imgDir = fullfile(outputFolder,'images'<\/span>,'701_StillsRaw_full'<\/span>);\r\nimds = imageDatastore(imgDir);\r\n<\/pre>
Display one of the images.<\/p>
I = readimage(imds,1);\r\nI = histeq(I);\r\nimshow(I)\r\n<\/pre>
Load CamVid Pixel-Labeled Images<\/h3>\r\n      
Use imageDatastore<\/tt><\/a> to load CamVid pixel label image data. A pixelLabelDatastore<\/inline> encapsulates the pixel label data and the label ID to a class name mapping. <\/p>\r\n      
Following the procedure used in original SegNet paper (Badrinarayanan, Vijay, Alex Kendall, and Roberto Cipolla. \"SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation.\" arXiv preprint arXiv:1511.00561, 201), group the 32 original classes in CamVid to 11 classes. Specify these classes. <\/p>
classes = [\r\n    \"Sky\"\r\n    \"Building\"\r\n    \"Pole\"\r\n    \"Road\"\r\n    \"Pavement\"\r\n    \"Tree\"\r\n    \"SignSymbol\"\r\n    \"Fence\"\r\n    \"Car\"\r\n    \"Pedestrian\"\r\n    \"Bicyclist\"\r\n    ];\r\n<\/pre>
To reduce 32 classes into 11, multiple classes from the original dataset are grouped together. For example, \"Car\" is a combination of \"Car\", \"SUVPickupTruck\", \"Truck_Bus\", \"Train\", and \"OtherMoving\". Return the grouped label IDs by using the supporting function camvidPixelLabelIDs<\/inline>, which is listed at the end of this example. <\/p>
labelIDs = camvidPixelLabelIDs();\r\n<\/pre>
Use the classes and label IDs to create the pixelLabelDatastore.<\/inline> <\/p>
labelDir = fullfile(outputFolder,'labels'<\/span>);\r\npxds = pixelLabelDatastore(labelDir,classes,labelIDs);\r\n<\/pre>
Read and display one of the pixel-labeled images by overlaying it on top of an image.<\/p>
C = readimage(pxds,1);\r\ncmap = camvidColorMap;\r\nB = labeloverlay(I,C,'ColorMap'<\/span>,cmap);\r\nimshow(B)\r\npixelLabelColorbar(cmap,classes);\r\n<\/pre>
Areas with no color overlay do not have pixel labels and are not used during training.<\/p>\r\n      
Analyze Dataset Statistics<\/h3>\r\n      
To see the distribution of class labels in the CamVid dataset, use countEachLabel<\/tt><\/a>. This function counts the number of pixels by class label. <\/p>
tbl = countEachLabel(pxds)\r\n<\/pre>
tbl=11\u00d73 table<\/em>\r\n        Name        PixelCount    ImagePixelCount\r\n    ____________    __________    _______________\r\n    'Sky'            76801167        483148800   \r\n    'Building'      117373718        483148800   \r\n    'Pole'            4798742        483148800   \r\n    'Road'          140535728        484531200   \r\n    'Pavement'       33614414        472089600   \r\n    'Tree'           54258673        447897600   \r\n    'SignSymbol'      5224247        468633600   \r\n    'Fence'           6921061        251596800   \r\n    'Car'            24436957        483148800   \r\n    'Pedestrian'      3402909        444441600   \r\n    'Bicyclist'       2591222        261964800   \r\n<\/pre>
Visualize the pixel counts by class.<\/p>
frequency = tbl.PixelCount\/sum(tbl.PixelCount);\r\nbar(1:numel(classes),frequency)\r\nxticks(1:numel(classes)) \r\nxticklabels(tbl.Name)\r\nxtickangle(45)\r\nylabel('Frequency'<\/span>)\r\n<\/pre>
Ideally, all classes would have an equal number of observations. However, the classes in CamVid are imbalanced, which is a common issue in automotive datasets of street scenes. Such scenes have more sky, building, and road pixels than pedestrian and bicyclist pixels because sky, buildings and roads cover more area in the image. If not handled correctly, this imbalance can be detrimental to the learning process because the learning is biased in favor of the dominant classes. Later on in this example, you will use class weighting to handle this issue. <\/p>\r\n      
Resize CamVid Data<\/h3>\r\n      
The images in the CamVid data set are 720 by 960. To reduce training time and memory usage, resize the images and pixel label images to 360 by 480. resizeCamVidImages<\/inline> and resizeCamVidPixelLabels<\/inline> are supporting functions listed at the end of this example. <\/p>
imageFolder = fullfile(outputFolder,'imagesResized'<\/span>,filesep);\r\nimds = resizeCamVidImages(imds,imageFolder);\r\nlabelFolder = fullfile(outputFolder,'labelsResized'<\/span>,filesep);\r\npxds = resizeCamVidPixelLabels(pxds,labelFolder);\r\n<\/pre>
Prepare Training and Test Sets<\/h3>\r\n      
SegNet is trained using 60% of the images from the dataset. The rest of the images are used for testing. The following code randomly splits the image and pixel label data into a training and test set. <\/p>
[imdsTrain,imdsTest,pxdsTrain,pxdsTest] = partitionCamVidData(imds,pxds);\r\n<\/pre>
The 60\/40 split results in the following number of training and test images:<\/p>
numTrainingImages = numel(imdsTrain.Files)\r\n<\/pre>
numTrainingImages = \r\n   421\r\n<\/pre>
numTestingImages = numel(imdsTest.Files)\r\n<\/pre>
numTestingImages = \r\n   280\r\n<\/pre>
Create the Network<\/h3>\r\n      
Use segnetLayers<\/tt><\/a> to create a SegNet network initialized using VGG-16 weights. segnetLayers<\/inline> automatically performs the network surgery needed to transfer the weights from VGG-16 and adds the additional layers required for semantic segmentation. <\/p>
imageSize = [360 480 3];\r\nnumClasses = numel(classes);\r\nlgraph = segnetLayers(imageSize,numClasses,'vgg16'<\/span>);\r\n<\/pre>
The image size is selected based on the size of the images in the dataset. The number of classes is selected based on the classes in CamVid. <\/p>\r\n      
Balance Classes Using Class Weighting<\/h3>\r\n      
As shown earlier, the classes in CamVid are not balanced. To improve training, you can use class weighting to balance the classes. Use the pixel label counts computed earlier with countEachLayer<\/tt><\/a> and calculate the median frequency class weights. <\/p>
imageFreq = tbl.PixelCount .\/ tbl.ImagePixelCount;\r\nclassWeights = median(imageFreq) .\/ imageFreq\r\n<\/pre>
classWeights = 11\u00d71<\/em>\r\n\r\n   0.318184709354742\r\n   0.208197860785155\r\n   5.092367332938507\r\n   0.174381825257403\r\n   0.710338097812948\r\n   0.417518560687874\r\n   4.537074815482926\r\n   1.838648261914560\r\n   1.000000000000000\r\n   6.605878573155874\r\n      \u22ee\r\n\r\n<\/pre>
Specify the class weights using a pixelClassificationLayer<\/tt><\/a>. <\/p>
pxLayer = pixelClassificationLayer('Name'<\/span>,'labels'<\/span>,'ClassNames'<\/span>,tbl.Name,'ClassWeights'<\/span>,classWeights)\r\n<\/pre>
pxLayer = \r\n  PixelClassificationLayer with properties:\r\n\r\n            Name: 'labels'\r\n      ClassNames: {11\u00d71 cell}\r\n    ClassWeights: [11\u00d71 double]\r\n      OutputSize: 'auto'\r\n   Hyperparameters\r\n    LossFunction: 'crossentropyex'\r\n<\/pre>
Update the SegNet network with the new pixelClassificationLayer<\/inline> by removing the current pixelClassificationLayer<\/inline> and adding the new layer. The current pixelClassificationLayer<\/inline> is named 'pixelLabels'. Remove it using removeLayers<\/tt><\/a>, add the new one using addLayers<\/a>, and connect the new layer to the rest of the network using connectLayers<\/a>. <\/p>
lgraph = removeLayers(lgraph,'pixelLabels'<\/span>);\r\nlgraph = addLayers(lgraph, pxLayer);\r\nlgraph = connectLayers(lgraph,'softmax'<\/span>,'labels'<\/span>);\r\n<\/pre>
Select Training Options<\/h3>\r\n      
The optimization algorithm used for training is stochastic gradient descent with momentum (SGDM). Use trainingOptions<\/tt><\/a> to specify the hyperparameters used for SGDM. <\/p>
options = trainingOptions('sgdm'<\/span>, ...<\/span>\r\n    'Momentum'<\/span>,0.9, ...<\/span>\r\n    'InitialLearnRate'<\/span>,1e-3, ...<\/span>\r\n    'L2Regularization'<\/span>,0.0005, ...<\/span>\r\n    'MaxEpochs'<\/span>,100, ...<\/span>  <\/span>\r\n    'MiniBatchSize'<\/span>,4, ...<\/span>\r\n    'Shuffle'<\/span>,'every-epoch'<\/span>, ...<\/span>\r\n    'VerboseFrequency'<\/span>,2);\r\n<\/pre>
A minibatch size of 4 is used to reduce memory usage while training. You can increase or decrease this value based on the amount of GPU memory you have on your system. <\/p>\r\n      
Data Augmentation<\/h3>\r\n      
Data augmentation is used during training to provide more examples to the network because it helps improve the accuracy of the network. Here, random left\/right reflection and random X\/Y translation of +\/- 10 pixels is used for data augmentation. <\/p>
augmenter = imageDataAugmenter('RandXReflection'<\/span>,true,...<\/span>\r\n    'RandXTranslation'<\/span>,[-10 10],'RandYTranslation'<\/span>,[-10 10]);\r\n<\/pre>
 imageDataAugmenter<\/inline> supports several other types of data augmentation. Choosing among them requires empirical analysis and is another level of hyperparameter tuning. <\/p>\r\n      
Start Training<\/h3>\r\n      
Combine the training data and data augmentation selections using pixelLabelImageDatastore<\/tt><\/a>. The pixelLabelImageDatastore<\/inline> reads batches of training data, applies data augmentation, and sends the augmented data to the training algorithm. <\/p>
pximds = pixelLabelImageDatastore(imdsTrain,pxdsTrain,...<\/span>\r\n    'DataAugmentation'<\/span>,augmenter);\r\n<\/pre>
Start training if the doTraining<\/inline> flag is true. Otherwise, load a pretrained network. Note: Training takes about 5 hours on an NVIDIA\u2122 Titan X and can take even longer depending on your GPU hardware. <\/p>
doTraining = false;\r\nif<\/span> doTraining    \r\n    [net, info] = trainNetwork(pximds,lgraph,options);\r\nelse<\/span>\r\n    data = load(pretrainedSegNet);\r\n    net = data.net;\r\nend<\/span>\r\n<\/pre>
Test Network on One Image<\/h3>\r\n      
As a quick sanity check, run the trained network on one test image. <\/p>
I = read(imdsTest);\r\nC = semanticseg(I, net);\r\n<\/pre>
Display the results.<\/p>
B = labeloverlay(I,C,'Colormap'<\/span>,cmap,'Transparency'<\/span>,0.4);\r\nimshow(B)\r\npixelLabelColorbar(cmap, classes);\r\n<\/pre>
Compare the results in C<\/inline> with the expected ground truth stored in pxdsTest<\/inline>. The green and magenta regions highlight areas where the segmentation results differ from the expected ground truth. <\/p>
expectedResult = read(pxdsTest);\r\nactual = uint8(C);\r\nexpected = uint8(expectedResult);\r\nimshowpair(actual, expected)\r\n<\/pre>
Visually, the semantic segmentation results overlap well for classes such as road, sky, and building. However, smaller objects like pedestrians and cars are not as accurate. The amount of overlap per class can be measured using the intersection-over-union (IoU) metric, also known as the Jaccard index. Use the jaccard<\/tt><\/a> function to measure IoU. <\/p>
iou = jaccard(C, expectedResult);\r\ntable(classes,iou)\r\n<\/pre>
ans=11\u00d72 table<\/em>\r\n      classes              iou        \r\n    ____________    __________________\r\n    \"Sky\"            0.926585343977038\r\n    \"Building\"       0.798698991022729\r\n    \"Pole\"           0.169776501947919\r\n    \"Road\"           0.951766120547122\r\n    \"Pavement\"       0.418766821629557\r\n    \"Tree\"           0.434014251781473\r\n    \"SignSymbol\"     0.325092056812204\r\n    \"Fence\"           0.49200469780468\r\n    \"Car\"           0.0687557042896258\r\n    \"Pedestrian\"                     0\r\n    \"Bicyclist\"                      0\r\n<\/pre>
The IoU metric confirms the visual results. Road, sky, and building classes have high IoU scores, while classes such as pedestrian and car have low scores. Other common segmentation metrics include the Dice index<\/a> and the Boundary-F1<\/a> contour matching score. <\/p>\r\n      
Evaluate Trained Network<\/h3>\r\n      
To measure accuracy for multiple test images, run semanticseg<\/tt><\/a> on the entire test set. <\/p>
pxdsResults = semanticseg(imdsTest,net,'MiniBatchSize'<\/span>,4,'WriteLocation'<\/span>,tempdir,'Verbose'<\/span>,false);\r\n<\/pre>
metrics = evaluateSemanticSegmentation(pxdsResults,pxdsTest,'Verbose'<\/span>,false);\r\n<\/pre>
 evaluateSemanticSegmentation<\/inline> returns various metrics for the entire dataset, for individual classes, and for each test image. To see the dataset level metrics, inspect metrics.DataSetMetrics<\/inline> . <\/p>
metrics.DataSetMetrics\r\n<\/pre>
ans=1\u00d75 table<\/em>\r\n     GlobalAccuracy        MeanAccuracy            MeanIoU            WeightedIoU         MeanBFScore   \r\n    _________________    _________________    _________________    _________________    ________________\r\n    0.882035049405331    0.850970241394654    0.608927281006314    0.797947090677593    0.60980715338674\r\n<\/pre>
The dataset metrics provide a high-level overview of the network performance. To see the impact each class has on the overall performance, inspect the per-class metrics using metrics.ClassMetrics<\/inline>. <\/p>
metrics.ClassMetrics\r\n<\/pre>
ans=11\u00d73 table<\/em>\r\n                      Accuracy                IoU              MeanBFScore   \r\n                  _________________    _________________    _________________\r\n    Sky           0.934932109589398    0.892435212043741    0.881521241030993\r\n    Building      0.797763575866624    0.752633046400693    0.597070806633627\r\n    Pole          0.726347220018996    0.186622256135469    0.522519568793497\r\n    Road          0.936763259117679    0.906720411900943    0.710433513101952\r\n    Pavement      0.906740772559168    0.728650096831083    0.703619961786386\r\n    Tree          0.866574402823008    0.737468334515386    0.664211092196979\r\n    SignSymbol    0.755895966085333    0.345193190798607    0.434011059025598\r\n    Fence         0.828068989656379    0.505920925889568     0.50829520978596\r\n    Car           0.911873566421394    0.750012303035288    0.643524410331899\r\n    Pedestrian     0.84866313766479    0.350461157529184     0.45550879471499\r\n    Bicyclist     0.847049655538425    0.542083155989493    0.468181589716695\r\n<\/pre>
Although the overall dataset performance is quite high, the class metrics show that underrepresented classes such as Pedestrian<\/inline>, Bicyclist<\/inline>, and Car<\/inline> are not segmented as well as classes such as Road<\/inline>, Sky<\/inline>, and Building<\/inline>. Additional data that includes more samples of the underrepresented classes might help improve the results. <\/p>\r\n      
Supporting Functions<\/h3>
function<\/span> labelIDs = camvidPixelLabelIDs()\r\n% Return the label IDs corresponding to each class.<\/span>\r\n%<\/span>\r\n% The CamVid dataset has 32 classes. Group them into 11 classes following<\/span>\r\n% the original SegNet training methodology [1].<\/span>\r\n%<\/span>\r\n% The 11 classes are:<\/span>\r\n%   \"Sky\" \"Building\", \"Pole\", \"Road\", \"Pavement\", \"Tree\", \"SignSymbol\",<\/span>\r\n%   \"Fence\", \"Car\", \"Pedestrian\",  and \"Bicyclist\".<\/span>\r\n%<\/span>\r\n% CamVid pixel label IDs are provided as RGB color values. Group them into<\/span>\r\n% 11 classes and return them as a cell array of M-by-3 matrices. The<\/span>\r\n% original CamVid class names are listed alongside each RGB value. Note<\/span>\r\n% that the Other\/Void class are excluded below.<\/span>\r\nlabelIDs = { ...<\/span>\r\n    \r\n    % \"Sky\"<\/span>\r\n    [\r\n    128 128 128; ...<\/span> % \"Sky\"<\/span>\r\n    ]\r\n    \r\n    % \"Building\" <\/span>\r\n    [\r\n    000 128 064; ...<\/span> % \"Bridge\"<\/span>\r\n    128 000 000; ...<\/span> % \"Building\"<\/span>\r\n    064 192 000; ...<\/span> % \"Wall\"<\/span>\r\n    064 000 064; ...<\/span> % \"Tunnel\"<\/span>\r\n    192 000 128; ...<\/span> % \"Archway\"<\/span>\r\n    ]\r\n    \r\n    % \"Pole\"<\/span>\r\n    [\r\n    192 192 128; ...<\/span> % \"Column_Pole\"<\/span>\r\n    000 000 064; ...<\/span> % \"TrafficCone\"<\/span>\r\n    ]\r\n    \r\n    % Road<\/span>\r\n    [\r\n    128 064 128; ...<\/span> % \"Road\"<\/span>\r\n    128 000 192; ...<\/span> % \"LaneMkgsDriv\"<\/span>\r\n    192 000 064; ...<\/span> % \"LaneMkgsNonDriv\"<\/span>\r\n    ]\r\n    \r\n    % \"Pavement\"<\/span>\r\n    [\r\n    000 000 192; ...<\/span> % \"Sidewalk\" <\/span>\r\n    064 192 128; ...<\/span> % \"ParkingBlock\"<\/span>\r\n    128 128 192; ...<\/span> % \"RoadShoulder\"<\/span>\r\n    ]\r\n        \r\n    % \"Tree\"<\/span>\r\n    [\r\n    128 128 000; ...<\/span> % \"Tree\"<\/span>\r\n    192 192 000; ...<\/span> % \"VegetationMisc\"<\/span>\r\n    ]\r\n    \r\n    % \"SignSymbol\"<\/span>\r\n    [\r\n    192 128 128; ...<\/span> % \"SignSymbol\"<\/span>\r\n    128 128 064; ...<\/span> % \"Misc_Text\"<\/span>\r\n    000 064 064; ...<\/span> % \"TrafficLight\"<\/span>\r\n    ]\r\n    \r\n    % \"Fence\"<\/span>\r\n    [\r\n    064 064 128; ...<\/span> % \"Fence\"<\/span>\r\n    ]\r\n    \r\n    % \"Car\"<\/span>\r\n    [\r\n    064 000 128; ...<\/span> % \"Car\"<\/span>\r\n    064 128 192; ...<\/span> % \"SUVPickupTruck\"<\/span>\r\n    192 128 192; ...<\/span> % \"Truck_Bus\"<\/span>\r\n    192 064 128; ...<\/span> % \"Train\"<\/span>\r\n    128 064 064; ...<\/span> % \"OtherMoving\"<\/span>\r\n    ]\r\n    \r\n    % \"Pedestrian\"<\/span>\r\n    [\r\n    064 064 000; ...<\/span> % \"Pedestrian\"<\/span>\r\n    192 128 064; ...<\/span> % \"Child\"<\/span>\r\n    064 000 192; ...<\/span> % \"CartLuggagePram\"<\/span>\r\n    064 128 064; ...<\/span> % \"Animal\"<\/span>\r\n    ]\r\n    \r\n    % \"Bicyclist\"<\/span>\r\n    [\r\n    000 128 192; ...<\/span> % \"Bicyclist\"<\/span>\r\n    192 000 192; ...<\/span> % \"MotorcycleScooter\"<\/span>\r\n    ]\r\n    \r\n    };\r\nend<\/span>\r\nfunction<\/span> pixelLabelColorbar(cmap, classNames)\r\n% Add a colorbar to the current axis. The colorbar is formatted<\/span>\r\n% to display the class names with the color.<\/span>\r\ncolormap(gca,cmap)\r\n% Add colorbar to current figure.<\/span>\r\nc = colorbar('peer'<\/span>, gca);\r\n% Use class names for tick marks.<\/span>\r\nc.TickLabels = classNames;\r\nnumClasses = size(cmap,1);\r\n% Center tick labels.<\/span>\r\nc.Ticks = 1\/(numClasses*2):1\/numClasses:1;\r\n% Remove tick mark.<\/span>\r\nc.TickLength = 0;\r\nend<\/span>\r\nfunction<\/span> cmap = camvidColorMap()\r\n% Define the colormap used by CamVid dataset.<\/span>\r\ncmap = [\r\n    128 128 128   % Sky<\/span>\r\n    128 0 0       % Building<\/span>\r\n    192 192 192   % Pole<\/span>\r\n    128 64 128    % Road<\/span>\r\n    60 40 222     % Pavement<\/span>\r\n    128 128 0     % Tree<\/span>\r\n    192 128 128   % SignSymbol<\/span>\r\n    64 64 128     % Fence<\/span>\r\n    64 0 128      % Car<\/span>\r\n    64 64 0       % Pedestrian<\/span>\r\n    0 128 192     % Bicyclist<\/span>\r\n    ];\r\n% Normalize between [0 1].<\/span>\r\ncmap = cmap .\/ 255;\r\nend<\/span>\r\nfunction<\/span> imds = resizeCamVidImages(imds, imageFolder)\r\n% Resize images to [360 480].<\/span>\r\nif<\/span> ~exist(imageFolder,'dir'<\/span>) \r\n    mkdir(imageFolder)\r\nelse<\/span>\r\n    imds = imageDatastore(imageFolder);\r\n    return<\/span>; % Skip if images already resized<\/span>\r\nend<\/span>\r\nreset(imds)\r\nwhile<\/span> hasdata(imds)\r\n    % Read an image.<\/span>\r\n    [I,info] = read(imds);     \r\n    \r\n    % Resize image.<\/span>\r\n    I = imresize(I,[360 480]);    \r\n    \r\n    % Write to disk.<\/span>\r\n    [~, filename, ext] = fileparts(info.Filename);\r\n    imwrite(I,[imageFolder filename ext])\r\nend<\/span>\r\nimds = imageDatastore(imageFolder);\r\nend<\/span>\r\nfunction<\/span> pxds = resizeCamVidPixelLabels(pxds, labelFolder)\r\n% Resize pixel label data to [360 480].<\/span>\r\nclasses = pxds.ClassNames;\r\nlabelIDs = 1:numel(classes);\r\nif<\/span> ~exist(labelFolder,'dir'<\/span>)\r\n    mkdir(labelFolder)\r\nelse<\/span>\r\n    pxds = pixelLabelDatastore(labelFolder,classes,labelIDs);\r\n    return<\/span>; % Skip if images already resized<\/span>\r\nend<\/span>\r\nreset(pxds)\r\nwhile<\/span> hasdata(pxds)\r\n    % Read the pixel data.<\/span>\r\n    [C,info] = read(pxds);\r\n    \r\n    % Convert from categorical to uint8.<\/span>\r\n    L = uint8(C);\r\n    \r\n    % Resize the data. Use 'nearest' interpolation to<\/span>\r\n    % preserve label IDs.<\/span>\r\n    L = imresize(L,[360 480],'nearest'<\/span>);\r\n    \r\n    % Write the data to disk.<\/span>\r\n    [~, filename, ext] = fileparts(info.Filename);\r\n    imwrite(L,[labelFolder filename ext])\r\nend<\/span>\r\nlabelIDs = 1:numel(classes);\r\npxds = pixelLabelDatastore(labelFolder,classes,labelIDs);\r\nend<\/span>\r\nfunction<\/span> [imdsTrain, imdsTest, pxdsTrain, pxdsTest] = partitionCamVidData(imds,pxds)\r\n% Partition CamVid data by randomly selecting 60% of the data for training. The<\/span>\r\n% rest is used for testing.<\/span>\r\n    \r\n% Set initial random state for example reproducibility.<\/span>\r\nrng(0); \r\nnumFiles = numel(imds.Files);\r\nshuffledIndices = randperm(numFiles);\r\n% Use 60% of the images for training.<\/span>\r\nN = round(0.60 * numFiles);\r\ntrainingIdx = shuffledIndices(1:N);\r\n% Use the rest for testing.<\/span>\r\ntestIdx = shuffledIndices(N+1:end);\r\n% Create image datastores for training and test.<\/span>\r\ntrainingImages = imds.Files(trainingIdx);\r\ntestImages = imds.Files(testIdx);\r\nimdsTrain = imageDatastore(trainingImages);\r\nimdsTest = imageDatastore(testImages);\r\n% Extract class and label IDs info.<\/span>\r\nclasses = pxds.ClassNames;\r\nlabelIDs = 1:numel(pxds.ClassNames);\r\n% Create pixel label datastores for training and test.<\/span>\r\ntrainingLabels = pxds.Files(trainingIdx);\r\ntestLabels = pxds.Files(testIdx);\r\npxdsTrain = pixelLabelDatastore(trainingLabels, classes, labelIDs);\r\npxdsTest = pixelLabelDatastore(testLabels, classes, labelIDs);\r\nend<\/span>\r\n<\/pre>","protected":false},"excerpt":{"rendered":"
<\/div>
Today I want to show you a documentation example that shows how to train a semantic segmentation network using deep learning and the Computer Vision System Toolbox. \r\n      A semantic segmentation... read more >><\/a><\/p>","protected":false},"author":42,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/400"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/42"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=400"}],"version-history":[{"count":12,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/400\/revisions"}],"predecessor-version":[{"id":692,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/400\/revisions\/692"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=400"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=400"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=400"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
 semanticseg<\/inline> returns the results for the test set as a pixelLabelDatastore<\/inline> object. The actual pixel label data for each test image in imdsTest<\/inline> is written to disk in the location specified by the 'WriteLocation'<\/inline> parameter. Use evaluateSemanticSegmentation<\/tt><\/a> to measure semantic segmentation metrics on the test set results. <\/p>