Ping Pong Shot Recognition – A STEM Project for Lower Secondary School
Today’s guest blogger is Francesco Bellucci, a PhD student in Mechanical Engineering at Department of Industrial Engineering of the University of Bologna. Francesco has a passion for helping young minds and has been actively involved in mentoring a team of young students, aged 13-14, from a secondary school. He assisted them in developing the “Ping-Pong Shot Recognition” project using Arduino and Deep Learning. Over to you, Francesco
High school students participating in the Problem Solving Olympiad had the opportunity to work on a unique and innovative project: building a system to recognize ping pong shots using technology. This initiative was part of the Makers section of the Olympiad, where students were challenged to apply their creativity and technical skills to solve real-world problems. The outcome was a project that combined engineering, coding, and artificial intelligence – all using MATLAB and Arduino.
Every year the “L. Voluseno” comprehensive school participates in the problem solving Olympics. It’s a small mountain school where the Montessori method is applied. The class that participated in the Olympics is the eigth grade, made up of six students. For the years 2023-2024 the participation in the makers competition required the creation of a project based on Arduino or compatible board concerning one or both of the following themes: IOT with Arduino, AI with Arduino. Thanks to the originality of the idea and the impementation of the solution, the team qualified for the national finals which were held in Cesena.
STEM (Science, Technology, Engineering, and Mathematics) education plays a crucial role in preparing students for the technological demands of the future. This blog showcases a STEM project where high school students used engineering and programming skills to develop a smart ping pong paddle. The project aimed to recognize technical gestures during a game using sensors and artificial intelligence, fostering the spirit of innovation and problem-solving among young minds.
The challenge was to design a system that could classify different shots in ping pong, such as forehand, backhand, topspin, and chop. Traditional methods of tracking these movements often rely on complex video analysis and manual tagging. This project sought to simplify the process using an Arduino platform and an accelerometer.
The students began by designing and 3D printing a custom ping pong paddle that could house an accelerometer. After assembling and wiring the components, they programmed the Arduino to collect data on the paddle’s motion. The collected data was then processed and analyzed in MATLAB, leveraging its libraries and tools to classify different types of shots based on acceleration data.
1. Design and 3D Printing
The students designed a paddle with a custom handle that could hold the accelerometer securely. Using the school’s 3D printer, they fabricated the paddle using biodegradable yellow PLA filament, emphasizing both functionality and sustainability.
We started by drawing the profile of the paddle, which was then extruded to make the model three-dimensional.
Subsequently, a hole was made in the handle for the cables to pass through and a pocket to house the accelerometer. To avoid leaving the accelerometer exposed, a cover was designed, which will also serve as a grip for the paddle.
The components were manufactured using the school’s 3D printer, utilizing a yellow PLA filament, a polymer derived from corn and biodegradable.
2. Data Acquisition and Processing
The paddle’s accelerometer measured accelerations along three axes (x, y, z) as the player moved the paddle.
By connecting the Arduino to MATLAB, the students could read acceleration values in real-time and visualize the data using simple commands such as:
a = arduino(‘/dev/ttyACM0’, ‘Mega2560’, ‘Libraries’, ‘I2C’);
imu = mpu6050(a);
accelData = readAcceleration(imu);
In these 3 images, we can both see the accelerometer signal trend for the 3 Cartesian axes (x, y, z) and next to it the corresponding transformation performed by the continous wavelet transform to create the scalogram.
For the part of Data Preprocessing and the Scalogram the code was taken from this blog: Building a High Five Counter Using Deep Learning
3. Training the Neural Network
The data collected from different shots was used to train a neural network model in MATLAB. A pre-trained neural network, GoogLeNet, was adapted to classify the frequency graphs generated for each ping pong shot. The model was fine-tuned to differentiate between service, forehand, backhand, topspin, and chop strokes.
Starting from the pre-trained network, we replaced the third-to-last layer of the 144-layer network, requesting an output size of 4 to classify 4 categories of images. We also increased the learning rate for both weights and biases so that they could update more significantly a teach cycle, given that this layer needs to be trained from scratch.
4. Performance Evaluation
Using metrics such as accuracy, precision, and recall, the students evaluated the model’s ability to classify shots. Although the initial results were promising, the team realized that more data was needed for better generalization.
Since we have 400 observations available, we set the mini-batch size to 25 and the validation frequency to 16, ensuring even data distribution in training so that the entire dataset is used each epoch. Specifically, repeating the process 16 times with batches of 25 data points each covers all 400 samples.
It would also be possible to use the validation data, as mentioned earlier, to automatically stop training when the loss stops decreasing on that dataset. However, it was decided to let the algorithm run for all 20 epochs, setting ValidationPatience to Inf. The neural network provided as output, to prevent overfitting, is not necessarily from the final iteration but rather the one with the lowest loss on the validation set, setting the OutputNetwork parameter as shown below.
The graph below shows the trends of accuracy and loss during the training process of the neural network.
The project provided students with hands-on experience in multiple disciplines, from mechanical design to software programming and machine learning. One challenge they faced was overfitting, where the model performed well on training data but struggled with unseen test data. To overcome this, they experimented with reducing the number of categories and retrained the model to improve accuracy.
High school students demonstrated enthusiasm and skill in tackling a complex problem, such as pattern recognition, using advanced tools (Pre-Processing + GoogleNet). This was made possible by MATLAB’s intuitive interface, which simplified the approach to sophisticated technologies. My sincere thanks go to the Headmaster, teachers and students for their commitment and cooperation.
The project remains a work-in-progress. The students plan to collect more data to improve the model’s robustness. They are also considering integrating other sensors, such as gyroscopes, to further enhance shot recognition. The ultimate goal is to create a comprehensive system that can be used in real-time during games to provide insights on a player’s performance.
When bright minds collaborate on bold ideas with the best tools, they don’t just learn—they innovate, solve, and inspire others to follow in their footsteps.
The Ping Pong Shot Recognition project is a testament to the potential of combining STEM education with practical problem-solving activities. By leveraging MATLAB and Arduino, high school students were able to explore the fascinating world of artificial intelligence and machine learning. Projects like these not only enhance technical skills but also inspire students to pursue careers in STEM fields.
This project enabled students to approach computational thinking using MATLAB and Simulink, tools that foster the development of critical skills to analyse complex problems, design effective solutions and apply advanced artificial intelligence and machine learning techniques in real-world contexts. Therefore I recommend as an additional resource Learn to Code with MATLAB.
Finally, I would like to remind you that since the beginning of the project I have been supporting students as MATLAB Student Ambassadors. I sincerely hope that our experience can be a model for future projects and an example for other schools.
Therefore, the project remains open to gather more examples to further train the neural network and for subsequent fine-tuning of various parameters.
The link to the video illustrating the various phases described in this paper: Table Tennis Shot Recognition – 3B IC L. Voluseno
- Category:
- deep learning,
- MATLAB
Comments
To leave a comment, please click here to sign in to your MathWorks Account or create a new one.