This is the second post of our two–part series on how MathWorks platforms support AV developers who use NVIDIA DRIVE Sim and deploy algorithms to NVIDIA hardware (See Part 1 here).  In this post we’ll cover how to deploy algorithms created in MATLAB and Simulink to NVIDIA DRIVE AGX.  

Simulink provides an environment to integrate and run simulations of the control logic with the vehicle dynamics and environment models. This enables the entire system to be tested early in the design process. GPU Coder and Embedded Coder can then be used to deploy to modern NVIDIA GPUs, including the NVIDIA DRIVE platform. 

To illustrate this workflow, consider a highway lane following system that steers a vehicle to travel within a marked lane. The system typically uses vision processing algorithms to detect lanes and vehicles from a camera. The controller uses the lane detections, vehicle detections, and set speed to control steering and acceleration.

The system level simulation can be run to see that it correctly identifies the lane markers and vehicles on the road. 

Looking inside the vision detector subsystem, the input video is fed to two deep-learning networks running in parallel to detect the left and right lane markers and oncoming vehicles. Pre- and post-processing subsystems prepare the input video data for the two deep learning networks, annotate the lane markers, and draw bounding boxes around detected vehicles prior to displaying the output video. 

Using video captured from a test vehicle, the simulation of the vision detector subsystem can be run on the host machine CPU to ensure it correctly identifies the lane markers and incoming vehicles. 

In this example, running the simulation on the CPU, the framerate appears choppy. Developers can switch to a desktop NVIDIA GPU to speed up the simulation. The output results remain the same and the framerate improves significantly. 

When satisfied with the simulation results, developers can generate code to target the NVIDIA DRIVE from the same Simulink model. Embedded Coder generates optimized C/C++ code running on the ARM processors while GPU Coder generates CUDA kernels for the CUDA cores. GPU Coder takes care of allocating memory on the GPU (using cudaMalloc calls), moving the data between CPU and GPU memory (using cudaMemcpyToSymbol calls), and calling the CUDA kernels, all at the appropriate points in the code. 

GPU Coder then calls the NVIDIA toolchain to compile and download the complete application on the NVIDIA DRIVE. Using Simulink to start the application on the board, the processed video is sent back from the NVIDIA DRIVE on the SDL video display window. The framerate is not quite as snappy as running the simulation on the desktop GPU, but it is to be expected given the more resource-constrained embedded GPU. 

With this workflow set up, the Simulink model can continue to be tweaked and the changes can be seen running on the NVIDIA GPUs within a few minutes. Simulation enables the ability to find and fix bugs earlier in the process, and GPU Coder and Embedded Coder provide an automated workflow to running the entire application on NVIDIA DRIVE. 

To learn more about how to use the different MathWorks platforms for automated driving development, and how other AV developers use MathWorks platforms in their development, register for MATLAB Expo on 4th and 5th May 2021. The events include talks from other MathWorks users in automated driving including: 

• Bosch 
• Cummins 
• General Motors 
• Flux Auto 
• Porsche Engineering 
• Renault Nissan 
• TuSimple 
• ZF 

Technical presentations from MathWorks engineers on automated driving include: 

• What’s New in MATLAB, Simulink and RoadRunner for Automated Driving  
• Designing and Evaluating Sensor Fusion Algorithms for Automated Driving  
• Developing Virtual Scenes for Automated Driving Testing  
• Developing a Motion Planner for Highway Lane Change Maneuvers