{"id":19069,"date":"2026-01-13T10:00:49","date_gmt":"2026-01-13T15:00:49","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=19069"},"modified":"2026-01-07T14:42:58","modified_gmt":"2026-01-07T19:42:58","slug":"reinforcement-learning-on-hardware-explained-by-brian-douglas","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2026\/01\/13\/reinforcement-learning-on-hardware-explained-by-brian-douglas\/","title":{"rendered":"Reinforcement Learning on Hardware, explained by Brian Douglas"},"content":{"rendered":"
\r\n
\r\n
<\/h6>\r\n\r\n\r\n\r\n
\"\"<\/td>\r\nCo-author: Brian Douglas<\/a><\/strong>\r\n\r\nBrian is a Technical Content Creator at MathWorks and control engineer with over 20 years of experience in the field, and a passion for sharing his knowledge with others. He creates and posts engaging videos, drawings, and short writings on various engineering topics. You can find his Tech Talks series on mathworks.com and YouTube.<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<\/div>\r\n<\/div>\r\n
<\/h6>\r\n
\r\n
You've probably seen impressive demos of reinforcement learning agents doing amazing things\u2014balancing poles, playing games, controlling robots. But here's the thing: getting from \"cool simulation\" to \"actually running on hardware\" is where it gets tricky.<\/div>\r\n
My colleague Brian Douglas just released a Tech Talk that tackles exactly this question. Not the specifics of RL algorithms (we have a whole series for that<\/a>), but something that doesn't get nearly enough attention: which approach should you take<\/span> to get a good policy running on your hardware?<\/div>\r\n
I know that doesn't sound terribly exciting at first. But trust me\u2014it matters a lot for hardware safety and how long you'll spend waiting for your policy to converge.<\/div>\r\n
<\/div>\r\n

The setup<\/h3>\r\n
Brian demonstrates this with a Quanser\u00ae Qube-Servo 2<\/a> rotary pendulum, controlled by a policy running on a Raspberry Pi\u00ae, which is connected to a PC running MATLAB\u00ae and Simulink\u00ae. Three pieces of hardware working together\u2014and how you use them depends entirely on your approach to training.<\/div>\r\n
<\/h6>\r\n
If you want to follow along with your own hardware, check out the\u00a0GitHub example<\/a>\u00a0that covers the complete workflow from plant modeling to deployment.<\/div>\r\n<\/div>\r\n
<\/h6>\r\n \r\n
<\/h6>\r\n
\r\n
\"\"<\/div>\r\n

The decision flowchart<\/h3>\r\n

<\/h6>\r\n
Here is the complete decision tree that Brian covers in his video. Let's break it down, step by step:<\/div>\r\n
<\/h6>\r\n
\"\"<\/div>\r\n
<\/div>\r\n
Let me walk you through the key decision points:<\/div>\r\n
<\/div>\r\n
1. Do you have existing data?<\/span><\/div>\r\n
<\/div>\r\n
You can learn a policy offline from data collected by humans or another controller. This is great for bootstrapping\u2014getting a decent starting point before you do any online training.<\/div>\r\n
<\/div>\r\n
2. Where do you get additional data?<\/span><\/div>\r\n
<\/div>\r\n
This is the big fork in the road: train directly on hardware (real environment) or train in simulation (modeled environment). Both have trade-offs. Brian covers the environment setup<\/a> in detail in Part 2 of the RL series.<\/div>\r\n
<\/div>\r\n

Training directly on hardware<\/h3>\r\n<\/div>\r\n
Where does inference happen vs. where does learning happen?<\/span><\/div>\r\n
<\/div>\r\n
This one's subtle but important. You might have a low-power embedded processor (like a Raspberry Pi) and a more powerful PC or cloud server. You can mix and match:<\/div>\r\n\r\n\r\n\r\n\r\n\r\n\r\n
\r\n
Approach<\/div><\/td>\r\n
\r\n
Pros<\/div><\/td>\r\n
\r\n
Cons<\/div><\/td>\r\n<\/tr>\r\n
\r\n
Train & run on embedded<\/div><\/td>\r\n
\r\n
Self-contained, minimal latency<\/div><\/td>\r\n
\r\n
Embedded processor may be too weak for training<\/div><\/td>\r\n<\/tr>\r\n
\r\n
Train & run on remote PC<\/div><\/td>\r\n
\r\n
Faster learning<\/div><\/td>\r\n
\r\n
Susceptible to latency and connection issues<\/div><\/td>\r\n<\/tr>\r\n
\r\n
Train on PC, run on embedded<\/div><\/td>\r\n
\r\n
Best of both worlds<\/div><\/td>\r\n
\r\n
More complex setup<\/div><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
<\/div>\r\n
Brian goes with that third option\u2014and honestly, it's often the sweet spot for real hardware applications.<\/div>\r\n
\r\n
<\/div>\r\n
The results? Well...<\/div>\r\n
<\/div>\r\n
The first episodes are rough. Random commands, motor jittering, the pendulum occasionally banging into the power cord (which let's admit, isn't great for the hardware). After several hours and almost 1,500 episodes, the policy could balance the pendulum\u2014but with noticeable wobble and plenty of room for improvement.<\/div>\r\n
<\/div>\r\n
The takeaway: training on hardware is straightforward but time-consuming, potentially dangerous, and hard to cover all operating states your system might encounter in the real world.<\/div>\r\n

Training in simulation<\/h3>\r\n
Here's where things get more practical. With a model of your environment, you can:\r\n- Train faster than real-time\r\n- Forget about hardware safety during exploration\r\n- Easily test different initial conditions and scenarios<\/div>\r\n
<\/div>\r\n
Brian trained against a simulated Qube model for about 1,200 episodes. The result? A policy that controls the pendulum rock-steady in simulation\u2014reward over 800, nice and settled.<\/div>\r\n
<\/div>\r\n
But the real question: does it work on actual hardware?<\/div>\r\n
For a detailed answer with code, see this example from the documentation: Train Reinforcement Learning Agents to Control Quanser QUBE Pendulum<\/a><\/div>\r\n

The sim2real gap<\/h3>\r\n
When Brian deployed the simulation-trained policy to the real Qube, it worked pretty well! No terrible jittering, no banging into cables. But if you look closely, there's a small amount of wobble that wasn't there in simulation.<\/div>\r\n
<\/div>\r\n
That's the sim2real gap\u2014the difference between what your model predicts and what happens in reality. Now, this isn't necessarily a problem. Models don't have to be perfect; they just have to be useful. If the behavior meets your requirements, you're done.<\/div>\r\n
<\/div>\r\n
But if it's not good enough, you have options:<\/div>\r\n
\u00a01. Improve the model<\/strong>\u00a0\u2014 Add more dynamics, use domain randomization to make your policy more robust<\/div>\r\n
\u00a02. Fine-tune on hardware<\/strong> \u2014 Start from your simulation-trained policy and continue learning on the real system<\/div>\r\n
<\/div>\r\n
<\/div>\r\n
That second approach is particularly nice because you're not starting from scratch. Your policy already knows roughly what to do, so training is faster and puts less stress on the hardware.<\/div>\r\n
<\/div>\r\n
<\/div>\r\n

Why this matters<\/h3>\r\n
There's no single best way to learn a policy for hardware. It depends on how well you can model your environment, how hard it is to reset between episodes, and how much you care about hardware safety (spoiler: probably a lot).<\/div>\r\n
<\/div>\r\n
But this workflow\u2014train in simulation, fine-tune on hardware, run policy on embedded processor, learn on a remote computer\u2014is actually ideal for many situations. It's also the foundation for continuous learning, where your policy keeps adapting as components wear out or conditions change, without requiring a full retraining cycle.<\/div>\r\n
<\/div>\r\n
<\/div>\r\n
Brian goes into much more detail in the video, including live demos of all three approaches and the actual Simulink models involved. Highly recommend watching the full thing:<\/div>\r\n
<\/div>\r\n
\ud83d\udcfd\ufe0f Reinforcement Learning on Hardware<\/a><\/div>\r\n