{"id":17682,"date":"2025-07-07T12:40:13","date_gmt":"2025-07-07T16:40:13","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=17682"},"modified":"2025-07-07T12:40:13","modified_gmt":"2025-07-07T16:40:13","slug":"game-on-building-on-device-ai-with-model-based-design","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2025\/07\/07\/game-on-building-on-device-ai-with-model-based-design\/","title":{"rendered":"Game On: Building On-Device AI with Model-Based Design"},"content":{"rendered":"<h6><\/h6>\r\n<em>This blog post is from <\/em><a href=\"https:\/\/www.linkedin.com\/in\/brenda-zhuang\/\">Brenda Zhuang<\/a><em>, Software Development Manager, and <\/em><a href=\"https:\/\/www.linkedin.com\/in\/haeng-jae-shin-66b780145\/?locale=en_US\"><em>Jayden Shin<\/em><\/a><em>, Senior Application Engineer at MathWorks.<\/em>\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 20px; color: #c04c0b;\"><strong>Ready to pick up a childhood game and play it with a robot\u2014on stage at a tech conference?<\/strong><\/p>\r\nAt this year\u2019s <a href=\"https:\/\/www.matlabexpo.com\/kr\/2025.html\">Korea MATLAB Expo<\/a>, while most demos showcased industrial automation, deep learning pipelines, or high-performance simulations, Jayden brought something a little different: a small blue robot trained to play Cheonggi-Baekgi\u2014a classic Korean game of strategy, speed, and team spirit.\r\n<h6><\/h6>\r\n<div class=\"row\"><div class=\"col-xs-12 containing-block\"><div class=\"bc-outer-container add_margin_20\"><videoplayer><div class=\"video-js-container\"><video data-video-id=\"6375056683112\" data-video-category=\"blog\" data-autostart=\"false\" data-account=\"62009828001\" data-omniture-account=\"mathwgbl\" data-player=\"rJ9XCz2Sx\" data-embed=\"default\" id=\"mathworks-brightcove-player\" class=\"video-js\" controls><\/video><script src=\"\/\/players.brightcove.net\/62009828001\/rJ9XCz2Sx_default\/index.min.js\"><\/script><script>if (typeof(playerLoaded) === 'undefined') {var playerLoaded = false;}(function isVideojsDefined() {if (typeof(videojs) !== 'undefined') {videojs(\"mathworks-brightcove-player\").on('loadedmetadata', function() {playerLoaded = true;});} else {setTimeout(isVideojsDefined, 10);}})();<\/script><\/div><\/videoplayer><\/div><\/div><\/div>\r\n<strong>Video:<\/strong> Robot playing Cheonggi-Baekgi with Jayden\r\n<h6><\/h6>\r\nThe project was more than just a playful throwback. It demonstrated how accessible and expressive embedded AI has become. With tools like MATLAB, Simulink, NVIDIA\u00ae Jetson\u2122, and Arduino\u00ae, Jayden turned a nostalgic idea into a live, interactive demo of real-time AI in action.\r\n<h6><\/h6>\r\nAnd let\u2019s be honest\u2014there\u2019s something undeniably delightful about watching a robot try to master a game that once ruled the schoolyard.\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 20px; color: #c04c0b;\"><strong>What\u2019s the Game?<\/strong><\/p>\r\nCheonggi-Baekgi is a children\u2019s game that is similar to \u201cSimon Says\u201d but with flags, popular in Korea and Japan. Players hold a blue flag and a white flag in each hand. \u201cSimon\u201d calls out commands like \u201cWhite flag up!\u201d or \u201cBlue flag down!\u201d \u00a0The players follow the instructions by raising or lowering their blue and white flags accordingly. If they make a mistake or respond too slowly, they are out of the game. The game continues with increasingly rapid and tricky commands. It is exciting and fun!\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 20px; color: #c04c0b;\"><strong>Behind the Scenes: How the Robot Learns to Play<\/strong><\/p>\r\nAt its core, this demo is an example of an on-device AI agent\u2014a compact system that can understand speech, make decisions, and control the motors in a robot to respond to the voice commands. Like a good companion, the agent is conversational, responsive, and in control of physical systems.\r\n<h6><\/h6>\r\nHere\u2019s how the system works:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li>Automatic Speech Recognition (ASR), <a href=\"https:\/\/www.mathworks.com\/discovery\/natural-language-processing.html\">natural language processing<\/a> (NLP), and language model integration are handled using open-source frameworks.<\/li>\r\n \t<li>The decision-making logic and robot control are implemented using MATLAB and Simulink tools.<\/li>\r\n \t<li>By separating the decision model from the language model, the system reduces training complexity and avoids the unpredictability that can come from relying solely on large language models (LLMs) for control tasks.<\/li>\r\n<\/ul>\r\nThis hybrid approach makes the system more stable, efficient, and easier to adapt for other applications\u2014whether you're building a game-playing robot or a voice-controlled assistant.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-17685 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2025\/06\/FrameWorkScreenshot-scaled.jpg\" alt=\"AI-driven system of game-playing robot\" width=\"2560\" height=\"1353\" \/>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> System of open-source models, AI tools, and Model-Based Design for edge GPU and Arduino\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 20px; color: #c04c0b;\"><strong>Tools at work: Challenges and Iterations<\/strong><\/p>\r\nThe development of the control logic was quickly accomplished with Model-Based Design. Using <a href=\"https:\/\/www.mathworks.com\/products\/gpu-coder.html\">GPU Coder<\/a> and <a href=\"https:\/\/www.mathworks.com\/help\/coder\/nvidia.html\">Jetson Support Package<\/a>, the entire system was easily tested and deployed to the Nvidia Jetson platform. Using Simulink, the decision-making model and the control logic were simulated on desktop and with the <a href=\"https:\/\/www.mathworks.com\/discovery\/hardware-in-the-loop-hil.html\">Hardware-in-the-loop (HIL)<\/a>.\r\n<h6><\/h6>\r\nFor ASR, we used OpenAI\u2122 Whisper. While Whisper provides high recognition accuracy for relatively long sentences, it had difficulty accurately recognizing short and fast voice commands, such as those used in the game. In some cases, short utterances were misrecognized as similar but incorrect words. Since the model could not be fine-tuned, we implemented the exception handling logic to manage these misrecognitions.\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 20px; color: #c04c0b;\"><strong>Beyond the Game<\/strong><\/p>\r\nWhat makes this demo more than just a fun experiment is its potential as a template for <a href=\"https:\/\/www.mathworks.com\/solutions\/deep-learning\/embedded-ai.html\">embedded intelligence<\/a>, that is a modular, scalable architecture that can help you build complex AI-driven systems for robotics, autonomous vehicles, and innovative embedded applications with multimodal interface. The main building blocks in this template are:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li>Perception (e.g., speech recognition, gestures, touch, and <a href=\"https:\/\/www.mathworks.com\/discovery\/computer-vision.html\">computer vision<\/a>)<\/li>\r\n \t<li>Understanding (e.g., NLP and intent recognition)<\/li>\r\n \t<li>Decision-making (e.g., rule-based logic and neural networks)<\/li>\r\n \t<li>Control (e.g., motor commands and system actuation)<\/li>\r\n<\/ul>\r\nThis layered approach mirrors how many intelligent systems are designed in industry, and it\u2019s fully supported by the MathWorks ecosystem.\r\n<h6><\/h6>\r\n<ul>\r\n \t<li><strong>MATLAB &amp; Simulink<\/strong>: Ideal for modeling, simulation, and rapid prototyping of control systems and AI workflows with open-source integration.<\/li>\r\n \t<li><strong>Deep Learning Toolbox<\/strong>: Enables training and deployment of neural networks for tasks like classification, regression, and decision-making.<\/li>\r\n \t<li><strong>Embedded Coder &amp; GPU Coder<\/strong>: Allow you to generate optimized code for deployment on embedded CPUs and GPUs, including platforms like NVIDIA Jetson.<\/li>\r\n \t<li><strong>Hardware Support Packages<\/strong>: Enable direct testing on hardware targets such as Raspberry Pi, Arduino, and Jetson by providing dedicated interfaces.<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<p style=\"font-size: 20px; color: #c04c0b;\"><strong>Try It Yourself<\/strong><\/p>\r\nCurious to build your own AI-powered robot? You can get started by downloading the <a href=\"https:\/\/content.mathworks.com\/viewer\/680af19d4e18ca8f66afdc83\">example files<\/a> and watching the demo introduction video. Once you\u2019ve unzipped the files, open the\u00a0ReadMe.mlx\u00a0file in MATLAB to walk through the setup.\r\n<h6><\/h6>\r\nHere\u2019s what you\u2019ll need:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li>A Jetson board (e.g., Orin)<\/li>\r\n \t<li>MATLAB with Deep Learning Toolbox<\/li>\r\n \t<li>A basic robot platform (or even a simulated one)<\/li>\r\n \t<li>A bit of curiosity and a willingness to experiment<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<p style=\"font-size: 20px; color: #c04c0b;\"><strong>What will you build?<\/strong><\/p>\r\nIn a world where AI is often framed in terms of productivity and performance, it\u2019s easy to forget that it can also be personal. It can reflect who we are, where we come from, and what we care about. Whether it\u2019s a childhood game, a local tradition, or a personal challenge, embedded intelligence gives us the tools to bring those stories to life in new and unexpected ways.\r\n<h6><\/h6>\r\nAnd the best part? With platforms like MATLAB, Jetson, and open-source models and MBD tools, the barrier to entry has never been lower.\r\n<h6><\/h6>\r\nWhat would your robot learn to do? Whatever it is, the tools are here\u2014and the possibilities are wide open.\r\n<h6><\/h6>\r\nLeave your ideas in the comments section!\r\n<h6><\/h6>","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2025\/06\/videoIndexScreenshot.jpg\" class=\"img-responsive attachment-post-thumbnail size-post-thumbnail wp-post-image\" alt=\"\" decoding=\"async\" loading=\"lazy\" \/><\/div><p>\r\nThis blog post is from Brenda Zhuang, Software Development Manager, and Jayden Shin, Senior Application Engineer at MathWorks.\r\n\r\n&nbsp;\r\n\r\nReady to pick up a childhood game and play it with a... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2025\/07\/07\/game-on-building-on-device-ai-with-model-based-design\/\">read more >><\/a><\/p>","protected":false},"author":194,"featured_media":17727,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[36,48,9,68,33,39],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/17682"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/194"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=17682"}],"version-history":[{"count":24,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/17682\/revisions"}],"predecessor-version":[{"id":17993,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/17682\/revisions\/17993"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media\/17727"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=17682"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=17682"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=17682"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}