{"id":18425,"date":"2025-10-01T10:44:50","date_gmt":"2025-10-01T14:44:50","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=18425"},"modified":"2025-10-01T10:45:55","modified_gmt":"2025-10-01T14:45:55","slug":"tennis-analysis-with-ai-interactive-ground-truth-labeling","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2025\/10\/01\/tennis-analysis-with-ai-interactive-ground-truth-labeling\/","title":{"rendered":"Tennis Analysis with AI: Interactive Ground Truth Labeling"},"content":{"rendered":"
<\/h6>\r\nThis blog post is from <\/em>Cory Hoi<\/em><\/a>, Engineer at MathWorks Engineering Development Group.<\/em>\r\n
<\/h6>\r\n \r\n
<\/h6>\r\nWith the rapid advancement of artificial intelligence (AI)<\/a>, harnessing its power is now more accessible than ever. I imagine that the arrival of the personal computer was equally transformative. We are now seeing \u00a0AI advancements in areas like computer vision<\/a> and natural language processing (NLP)<\/a> being applied in chatbots, healthcare research, transportation, education, and sports. It is everywhere we look \u2013 so integrated into our daily lives that many of us hardly notice its presence. For example, just turn on your TV to your favorite sports broadcast.\r\n
<\/h6>\r\nFor me, it\u2019s tennis. At this past Wimbledon, line calls were fully automated, eliminating the need to argue with judges over close calls. Ball tracking and real-time data analysis have also become integral parts of the game, representing a leap forward from sports analytics just five years ago. Players now use AI to study playing patterns and refine their tactics. Yet, as an avid fan, I can\u2019t help but wonder: how does all of this actually work?\r\n
<\/h6>\r\nIn this two-part blog post series, I will show you how to build and leverage deep neural networks in MATLAB for object detection. This first blog post focuses on the initial steps of labeling the data. It will go over the many tools available in MATLAB to ease the typical pains of data labeling.\r\n
<\/h6>\r\n \r\n
<\/h6>\r\n

Object Detection in Sports<\/strong><\/p>\r\nAs you may have guessed, my sport of choice is tennis. However, the object detection<\/a> methods described here have many applications, in other sports like basketball and football, and beyond sports like in autonomous vehicles. Object detection in tennis has greatly improved my viewing experience in recent years. This is evident when looking at instant replays of tennis points with the ball clearly marked and tracked.\r\n

<\/h6>\r\nData preparation is an arguably crucial but often overlooked step in any AI task. It involves labelling the dataset to create ground truth data for training, and also preparing the dataset into the correct form.\r\n
<\/h6>\r\n \r\n
<\/h6>\r\n

Interactive Video Labeling<\/strong><\/p>\r\nIn MATLAB, video labeling is made easy with the Video Labeler<\/a> app. The app allows you to interactively label shapes or regions of interest (ROI) with rectangles, polylines, pixels, and polygon ROI labels. In this post, we are using Video Labeler to label the tennis ball, and also the tennis court in our dataset.\r\n

<\/h6>\r\nOpen the Video Labeler app from the Apps<\/strong> tab, under Image Processing and Computer Vision<\/strong>. Create a new project and import the video from the trainingData folder.\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\n \r\n
<\/h6>\r\nAfter loading the video, the app will display the first frame in the middle of the screen. In the panel just below, you can navigate between frames with the left and right arrows.\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\nVideo Labeler<\/em>\r\n
<\/h6>\r\n \r\n
<\/h6>\r\n

Manual vs Automatic Labeling<\/strong><\/p>\r\nIn MATLAB, there are numerous ways to label a dataset. For example, you can define bounding boxes around the people playing tennis and give them the label name \u201cperson\u201d. You can also define lines to label the lines on the court as linear objects. However, for this project, let\u2019s label pixels by identifying the tennis ball, give it the label name \u201cball\u201d, and assign the Color to green.\r\n

<\/h6>\r\n\"\"\r\n
<\/h6>\r\nAfter adding the label, it will appear in the ROI Label Definitions pane to the left. This allows you to easily switch between object labels if there are multiple objects in a single image. There are multiple manual \u00a0and automatic algorithms available.\r\n
<\/h6>\r\nManual algorithms such as Polygon and Brush, allow you to exactly define the area to label. While automation algorithms leverage a range of techniques to speed up and ease typical pains of the labeling process. Some additional \u00a0automation algorithms include Superpixel, Segment Anything, and Assisted Freehand.\r\n
<\/h6>\r\nFor example, in the following video, the ball and the person are manually labeled with the Brush and the Polygon. The ball is easy to label since it only takes a single click with the brush tool. However, labeling the person is more challenging. The polygon tool uses straight lines that don't automatically snap to the person's edges. This label could be improved by using much shorter line segments, though this would require more time.\r\n
<\/h6>\r\n