Our post today is from Jeff Homer of our UK office in Cambridge, England. Jeff is an Application Support Engineer with a keen interest in controls, robotics and physical modelling. He was part of a team of people that built an action-packed crowd-pleasing MATLAB application: pitting tiny rolling robots against each other in a merciless race against time. They took their race track to a museum for some after hours mischief (it’s okay, they were invited). What happened next is what you’re about to read…
Sphero Racing at the Science Museum
by Jeff Homer
Recently an intrepid team of MathWorkers brought Sphero racing to the Science Museum in London during one of their Lates events. The goal was to demonstrate how MATLAB can be used for robotics, real-time control, image processing, and app development, all at the same time.
Throughout the evening, over 300 people raced Spheros controlled by MATLAB! Science Museum Lates are regular evening events for adults only with a great variety of activities, talks, music and bars among the regular exhibits. Team Sphero Racing was the most ambitious MathWorks demo yet, to celebrate our sponsorship of their new maths gallery (Mathematics: The Winton Gallery).
A Sphero is a small, spherical robot that moves around by rolling (sometimes wildly!) from one point to the next. It can be controlled via Bluetooth from a smartphone, tablet, or even from MATLAB as it was in the Lates event.
In each round, six players raced their Sphero robot through an obstacle course with the aim of reaching the finish in the shortest possible time, while simultaneously trying to pass through marked gates to avoid time penalties (think slalom skiing). Players directed their robot by drawing a desired route on a paper map of the course, and then choosing a trade off between accuracy and speed.
Players then placed their paper map under a webcam, and MATLAB detected the route and mapped it onto the actual arena through a geometric transform that was defined by matching the four corners on the paper with the four corners on the arena.
This is an example of the transformation performed on the above image and overlaid on the course:
MATLAB detected and tracked the Spheros using another webcam pointing at the racing arena. Each Sphero was a different colour, and the image recognition algorithm coded in MATLAB located a particular Sphero predominantly by its colour, but also by its size, brightness and previous location.
When all the players were ready, the MATLAB referee app counted down and players started their Spheros at the same time by pressing Enter. MATLAB communicated with each Sphero by Bluetooth to follow the trajectory drawn on the player’s map.
Meanwhile, the referee app tracked all the Spheros and mapped their progress, recorded their finish times, and applied penalty times for missing gates or starting early. Cheats who went straight to the finish without going through any gates got a penalty of 30 seconds so they could not win!
A team of talented MathWorkers wrote the MATLAB code to create the player apps. They coded algorithms to:
• detect and track Spheros
• tell the Sphero where to move next
• tell the Sphero to return to the start
• calibrate the speed and direction of the Sphero after each race
• detect route lines on the paper map and the speed/accuracy level
• draw trajectories live during races
• generate unique player names
• referee the races by timing races and applying penaltes
• record rankings within each race and the overall leaderboard
The team used a Git repository to collaborate and quickly develop a large amount of MATLAB code to create these algorithms and apps for the players to use. To communicate with the Spheros, they used code from the MATLAB Central File Exchange. The code was all prototyped in core MATLAB. The final apps also use several functions from the Image Processing Toolbox and Robotics System Toolbox. To use the webcams, the team used the MATLAB support package for USB Webcams.
The hardware and code was all tested in a completely different environment to the Science Museum gallery with very different lighting conditions. This meant that the image recognition algorithms had to be robust in order to perform on the night. Also, set up time was under an hour for all of the following:
• Construct course (frame, ground, borders)
• Set up laptops and laptop tables
• Mount and position course webcams on microphone stand
• Mount reading lights
• Calibrate all webcams
• Connect and calibrate Spheros
• Manage power for laptops, referee monitor and Sphero charging stations
Happily, getting all the MATLABs, and robots, and Bluetooth, and webcams to communicate went very smoothly indeed. Science Museum Lates always have thousands of visitors and hundreds of people raced Spheros with us, with an average race start every 2 and half minutes!
Sphero racing in actions: “And they’re off!”
But Wait… There’s More!
A number of MATLAB users and Sphero enthusiasts have asked if they can try out the connection between MATLAB and Sphero for themselves. Of course you can! You can download the Sphero Connectivity Package from the MATLAB Central File Exchange to communicate with a Sphero from within MATLAB (and Simulink). You can even run virtual simulations of a Sphero by downloading the Sphero Simulation with Simscape Multibody submission from the File Exchange. With this submission, you can drive a virtual Sphero with user inputs from Simulink dashboard blocks or a joystick, or add your own controller and see how fast your Sphero can navigate its environment.
Now you can build your own MATLAB Sphero Formula Racing league. Better get busy.