Applying Deep Learning for European Rover Challenge purposes

作者 Tanya Kuruvilla, November 8, 2021

3 次查看（过去 30 天） | 0 个赞 | 0 个评论

Today’s guest blogger is Michał Hałoń who has been taking part in the European Rover Challenge for several years as a team member of the SKA Robotics project from the Students’ Space Association at Warsaw University of Technology in Poland. He is going to share with us the Deep Learning techniques that he used in the challenge.

European Rover Challenge

European Rover Challenge (ERC) is an annual international student Mars rover competition attended by teams from around the world. ERC competition has been organized since 2014 in Poland. Like the University Rover Challenge (taking place in the USA), European Rover Challenge is a part of the Rover Challenge Series – the most prestigious robotics competition in the world. During the competition, the teams must complete tasks that include taking and analyzing soil samples, performing manual operations on a control panel (such as setting switches to the right position, making electric measurements), collecting samples from different locations, or navigating autonomously on the Mars Yard. For many years The MathWorks, Inc. is an ERC partner and sponsor.

Despite many unknowns and limitations, European Rover Challenge 2020 turned out to be the only competition of this type during the global COVID-19 restrictions that physically took place. The sixth, unique edition was carried out in a hybrid formula using technologies that enabled teams from around the world to remotely participate in this competition controlling Leo Rover mobile robot.

26 teams from 14 countries participated in the ERC 2020 final. Oprogramowanie Naukowo-Techniczne sp. z o.o. sp. k. (ONT Poland, MathWorks distributor) funded the Special Award for the best use of the MathWorks software to perform advanced scientific calculations and computer simulations. Michał together with SKA Robotics, won MathWorks Special Award receiving Arduino Engineering Kit for using Deep Learning techniques to detect and recognize markers based on which the rover navigates autonomously during the competition. The motivation and methodology of working with Deep Learning algorithms are described below.

Facing new challenge

During preparations for the competition, I was responsible for developing an AR Tag markers detection and recognition algorithm based on which the rover determines its position while driving in an autonomous mode. AR Tag markers are somewhat like QR codes, however, are used for different purposes. Attempts to use the existing and ready-to-use classic solutions based on traditional image processing methods have shown that they tend to falsely detect markers among objects around the competition, such as windows or pictures on the t-shirts of observers. This defect turned out to be difficult to remove by classical methods.

Applying Deep Learning

Therefore, I decided to use artificial intelligence algorithms to solve the above problem. The first task was to increase the training set of marker photos, which was intended to be used to develop deep neural network-based detectors and classifiers in the proposed structure.

Using MATLAB, I developed a conditional GAN (Generative Adversarial Network) network for generating thousands (and even more) of realistic-looking photos of markers, which I called ARTaGAN (Augmented Reality Tag GAN). The animation below presents the learning process of the presented GAN network for an exemplary marker. ARTaGAN network, starting from generating random noise, slowly learns to generate realistic-looking markers.

Results

The figure below presents further sample markers generated by the ARTaGAN network compared with the actual marker’s photos. The data obtained in the presented process was then used to train detector and classifier neural networks, which allowed to distinguish photos with and without markers and to read their code correctly. Thanks to the use of the described networks, it was possible to solve the problem of incorrect marker recognition among the elements of the competition place and its surroundings.