{"id":18554,"date":"2025-10-08T11:14:42","date_gmt":"2025-10-08T15:14:42","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=18554"},"modified":"2025-10-08T11:15:39","modified_gmt":"2025-10-08T15:15:39","slug":"tennis-analysis-with-ai-object-detection-for-ball-tracking","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2025\/10\/08\/tennis-analysis-with-ai-object-detection-for-ball-tracking\/","title":{"rendered":"Tennis Analysis with AI: Object Detection for Ball Tracking"},"content":{"rendered":"
<\/h6>\r\nThis blog post is from <\/em>Cory Hoi<\/em><\/a>, Engineer at MathWorks Engineering Development Group.<\/em>\r\n
<\/h6>\r\n \r\n
<\/h6>\r\nIn our previous blog post<\/a>, we used the Video Labeler app to segment both the tennis ball and the court, creating ground truth data for training a neural network. This ground truth forms the foundation for the next step: building and evaluating object detectors.\r\n
<\/h6>\r\nPretrained networks offer faster development, require less labeled data, and are well-suited for common tasks. However, they might not generalize well to highly specific applications. Custom networks, on the other hand, offer greater flexibility and control but typically require more effort to design, train, and validate. In this post, we\u2019ll explore both approaches\u2014building a detector from scratch and using a pretrained model\u2014while making use of the labeled ground truth data to improve performance through retraining.\r\n
<\/h6>\r\n \r\n
<\/h6>\r\n

Data Preprocessing<\/strong><\/p>\r\nThe first step is to organize the labeled data into a format suitable for training. Since the annotations were created using the Video Labeler app, MATLAB makes this process straightforward. We begin by loading the ground truth data and storing the image files using an imageDataStore<\/a> object. This object efficiently manages large collections of images, supports batch processing, and integrates well with deep learning workflows such as training.\r\n

load(\"trainingData\\tennisData\\gTruth.mat\")\r\n\r\ninputSize = [540 960 3];\r\n\r\n[imds, pxds] = pixelLabelTrainingData(gTruth,\"WriteLocation\",\"folder\\to\\write\\images\");\r\n\r\npxds = imageDatastore(pxds.Files);\r\nmasks = imageDatastore(pxds.Files);\r\n<\/pre>\r\n
<\/h6>\r\nWe split the dataset into training and validation sets using the partitionData function to help prevent overfitting<\/a>. During training, the network updates its weights to minimize the loss on the training data. If stopping criteria are based on the training set, the model may overfit, that is, it will perform well on seen data but poorly on unseen data. By using the validation set to define stopping criteria, we improve generalization and overall model performance.\r\n
<\/h6>\r\nThe transform<\/a> function is used to preprocess the input images. After the data is processed we combine the training and validation datastore objects by using the combine<\/a> function.\r\n
[trainingImages, trainingMasks, validationImages, validationMasks] = partitionData(imds, masks);\r\n\r\nresizedTrainingImages = transform(trainingImages, @(x) preProcessImages(x, inputSize));\r\nresizedTrainingMasks = transform(masks, @(x) preProcessImages(x, inputSize));\r\n\r\nresizedValidationImages = transform(validationImages, @(x) preProcessImages(x, inputSize));\r\nresizedValidationMasks = transform(validationMasks, @(x) preProcessImages(x, inputSize));\r\n\r\ndsTraining = combine(resizedTrainingImages, resizedTrainingMasks);\r\ndsValidation = combine(resizedValidationImages, resizedValidationMasks);\r\n<\/pre>\r\n
<\/h6>\r\n \r\n
<\/h6>\r\n
Designing Object Detector<\/strong><\/p>\r\nTo design a simple object detector from scratch, we are using five main building blocks: input layer, downsampling layers, bottleneck layers, upsampling layers, and output layer.\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\nNeural network layout<\/em>\r\n
<\/h6>\r\n \r\n
<\/h6>\r\nThe downsampling block reduces the spatial dimensions of the input features and only captures larger, higher-level information. This reduces the computational cost, because smaller feature maps mean fewer operations in subsequent layers. The downsampling block in our network consists of a convolutional layer<\/a>, a batch normalization layer<\/a>, a ReLU<\/a> activation function, and a 2-D max pooling layer<\/a>.\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\nConvolutional neural network structure<\/em>\r\n
<\/h6>\r\n \r\n
<\/h6>\r\nThe convolutional layer applies learnable kernels to the input by sliding them over local regions of the image, performing element-wise multiplication and summation to generate feature maps that capture spatial patterns. The ReLU activation introduces non-linearity by zeroing out negative values, allowing the network to learn complex features. The max pooling layer reduces spatial dimensions by sliding a 2\u00d72 window over the feature map and keeping only the maximum value from each region, which decreases computation and adds robustness to small spatial shifts. Stride and padding settings in both convolution and pooling control how the filters move and how edges are handled.\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\nExample of the max pooling layer<\/em>\r\n
<\/h6>\r\n \r\n
<\/h6>\r\nIn the middle of the neural network are the bottleneck layers, which capture and compress the most relevant features from the downsampling block. This encoded information is then passed to the upsampling block through a sequence of transposed convolutional layers, which reconstruct the spatial dimensions. In this architecture, the number of transposed convolutional layers mirrors the number of convolutional layers in the downsampling block, ensuring that the output image matches the input size.\r\n
<\/h6>\r\nTo create the architecture for the described object detector, use the following code.\r\n
layers = [\r\n    imageInputLayer(inputSize)\r\n\r\n    % Downsampling\r\n    convolution2dLayer(3, 16, 'Padding', 'same', 'Name', 'conv1')\r\n    batchNormalizationLayer\r\n    reluLayer('Name', 'relu1')\r\n    maxPooling2dLayer(2, 'Stride', 2, 'Name', 'maxpool1')\r\n\r\n    convolution2dLayer(3, 32, 'Padding', 'same', 'Name', 'conv2')\r\n    batchNormalizationLayer\r\n    reluLayer('Name', 'relu2')\r\n    maxPooling2dLayer(2, 'Stride', 2, 'Name', 'maxpool2')\r\n\r\n    % Bottleneck\r\n    convolution2dLayer(3, 64, 'Padding', 'same', 'Name', 'conv2')\r\n    batchNormalizationLayer\r\n    reluLayer('Name', 'relu2')\r\n\r\n    % Upsample\r\n    transposedConv2dLayer(3, 32, 'Stride', 2, 'Cropping', 'same', 'Name', 'upsample1')\r\n    reluLayer('Name', 'relu3')\r\n\r\n    transposedConv2dLayer(3, 16, 'Stride', 2, 'Cropping', 'same', 'Name', 'upsample2')\r\n    reluLayer('Name', 'relu4')\r\n\r\n    % Mask output layer\r\n    convolution2dLayer(1, 1, 'Padding', 'same', 'Name', 'conv3')\r\n    sigmoidLayer('Name', 'softmax')\r\n    ];\r\n<\/pre>\r\n
<\/h6>\r\nConstructing a neural network from scratch involves careful architecture design, layer configuration, and parameter initialization. Before proceeding with training, it\u2019s useful to compare this custom approach with leveraging a pretrained network, which can significantly reduce development time and improve performance on certain tasks.\r\n
<\/h6>\r\n \r\n
<\/h6>\r\n
Pretrained Object Detector<\/strong><\/p>\r\nInstead of creating a network from scratch, you can use a pretrained object detector. In MATLAB, there are many available options for pretrained models<\/a> depending on your task. For this task, we will use the You Only Look Once X (YOLOX) object detector from the Automated Visual Inspection Library for Computer Vision Toolbox<\/a> support package.\r\n
<\/h6>\r\nThe YOLOX detector is a pretrained neural network that consists of three parts: the backbone, the neck, and the head.\r\n
<\/h6>\r\n