{"id":11138,"date":"2022-12-05T08:59:44","date_gmt":"2022-12-05T13:59:44","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=11138"},"modified":"2024-08-21T08:28:40","modified_gmt":"2024-08-21T12:28:40","slug":"using-ai-for-reduced-order-modeling","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2022\/12\/05\/using-ai-for-reduced-order-modeling\/","title":{"rendered":"Using AI for Reduced-Order Modeling"},"content":{"rendered":"<em>The following post is from <\/em><a href=\"https:\/\/www.linkedin.com\/in\/lucas-garcia-phd\/\"><em>Lucas Garcia<\/em><\/a><em>, Deep Learning Product Manager at MathWorks.<\/em>\r\n<h6><\/h6>\r\nThis blog discusses the MathWorks\u2019 presence at <a href=\"https:\/\/neurips.cc\/\">NeurIPS 2022<\/a> and my talk on \u2018Using AI for Reduced-Order Modeling\u2019 at the conference.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-11141 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/NeurIPS_MathWorks_team.png\" alt=\"MathWorks team at NeurIPS 2022\" width=\"624\" height=\"351\" \/>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> The MathWorks team at our booth at NeurIPS 2022\r\n<h6><\/h6>\r\nFounded in 1987, the Conference on Neural Information Processing Systems (abbreviated as NeurIPS) is one of the most prestigious and competitive international conferences in machine learning. Last week, the MathWorks team was at NeurIPS 2022 in New Orleans for the in-person portion of the conference.\r\n<h6><\/h6>\r\nDuring the Expo Day at NeurIPS, I presented a talk about \u2018Using AI for Reduced Order Modeling\u2019. This blog post provides an overview of this presentation. If you are interested in learning more, check out the <a href=\"https:\/\/content.mathworks.com\/viewer\/63814a32c8ce21bfab25787d\">Slides: Using AI for Reduced-Order Modeling<\/a>.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-11144 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/NeurIPS_ROM_presentation.png\" alt=\"Presentation on Using AI for Reduced-Order Modeling\" width=\"511\" height=\"329\" \/>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> Presentation on Using AI for Reduced-Order Modeling at NeurIPS 2022\r\n<h6><\/h6>\r\nAdditionally, we had many interesting interactions at the booth on deep learning, reinforcement learning, and interoperability between MATLAB and Python\u00ae. My colleagues <a href=\"https:\/\/www.linkedin.com\/in\/drew-davis-662894113\/\">Drew<\/a> and <a href=\"https:\/\/www.linkedin.com\/in\/naren-srivaths-raman\/\">Naren<\/a> developed an excellent reinforcement learning demo that showcases learning directly from hardware by using a <a href=\"https:\/\/www.quanser.com\/products\/qube-servo-2\/\">Quanser QUBE\u2122-Servo 2<\/a> and <a href=\"https:\/\/www.mathworks.com\/products\/reinforcement-learning.html\">Reinforcement Learning Toolbox<\/a>.\r\n\r\n<div style=\"width: 304px;\" class=\"wp-video\"><!--[if lt IE 9]><script>document.createElement('video');<\/script><![endif]-->\n<video class=\"wp-video-shortcode\" id=\"video-11138-1\" width=\"304\" height=\"540\" preload=\"metadata\" controls=\"controls\"><source type=\"video\/mp4\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/RL_CartPole_Quanser_noaudio.mp4?_=1\" \/><a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/RL_CartPole_Quanser_noaudio.mp4\">https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/RL_CartPole_Quanser_noaudio.mp4<\/a><\/video><\/div>\r\n<h6><\/h6>\r\n<strong>Video:<\/strong> Reinforcement learning demo showcasing learning directly from hardware\r\n<h6><\/h6>\r\n&nbsp;\r\n<p style=\"font-size: 18px;\"><strong>What is ROM and why use it<\/strong><\/p>\r\nIf you are an engineer or have ever worked in solving an engineering problem, you have probably tried to explain the behavior of a system using first principles. In such situations, you must understand the system\u2019s physics to derive a mathematical representation. The real value of a first-principles model is that results typically have a clear, explainable physical meaning. In addition, behaviors can often be parameterized.\r\n<h6><\/h6>\r\nHowever, high-fidelity non-linear models can take hours or even days to simulate. In fact, system analysis and design might require thousands or hundreds of thousands of model simulations to obtain meaningful results. This causes a significant computational challenge for many engineering teams. Moreover, linearizing complex models can result in high-fidelity models that do not contribute to the dynamics of interest in your application. In these situations, AI-based reduced-order models can significantly speed up simulations and analysis of higher-order large-scale systems.\r\n<h6><\/h6>\r\n<a href=\"https:\/\/www.mathworks.com\/discovery\/reduced-order-modeling.html\">Reduced Order Modeling<\/a> (ROM) is a technique for reducing the computational complexity or storage requirement of a computer model, while preserving the expected fidelity within a controlled error.\r\n<h6><\/h6>\r\nEngineers and scientists use ROM techniques to:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li>Speed up system-level desktop simulation<\/li>\r\n \t<li>Perform hardware-in-the-loop testing<\/li>\r\n \t<li>Enable system-level simulation<\/li>\r\n \t<li>Develop virtual sensors and digital twins<\/li>\r\n \t<li>Perform control design<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-13178 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/ROM_for_simulation.png\" alt=\"From high-fidelity model to reduced-order model\" width=\"628\" height=\"618\" \/>\r\n<h6><\/h6>\r\n<strong>Figure: <\/strong>The simulation of a reduced-order model is significantly faster than the simulation of a high-fidelity model.\r\n<h6><\/h6>\r\n&nbsp;\r\n<p style=\"font-size: 18px;\"><strong>AI-based reduced-order modeling<\/strong><\/p>\r\nAI enables to create a model from measured data of your component, so you get the right answer in an accurate, dynamic, and low-cost way. We can\u2019t always make good analytical models of the things in the world. Sometimes the theory and\/or technology isn\u2019t there. For example, estimating a motor\u2019s internal temperature is challenging because no cost-efficient sensors can do this, and methods like FEA and lumped thermal models are either very slow or require domain expertise to set up.\r\n<h6><\/h6>\r\nEven if you already have a high-fidelity first-principles model, you can use data-driven models to create a surrogate model that is potentially simpler and simulates faster. A faster but equally accurate model can help you progress as you design, test, and deploy your system. For this talk, I focused on replacing an existing high-fidelity first-principles model with an AI-based reduced-order model. To create such a reduced-order model, you can follow the steps in an AI-driven system design workflow: data preparation, AI modeling, system simulation, and deployment.\r\n<h6><\/h6>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-11204 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/AI_ROM_system.png\" alt=\"AI system design\" width=\"1572\" height=\"179\" \/><\/h6>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> AI-driven design involves data preparation, AI modeling, system design, and deployment.\r\n<h6><\/h6>\r\nLet\u2019s take a closer look at how you can create a reduced-order model of a vehicle engine for closed-loop control of vehicle speed. In the image below, you can see a system-level model of a vehicle in Simulink. The goal is to control the vehicle\u2019s velocity based on the driver\u2019s input. This virtual model includes components representing a simulated driver, road conditions, the controllers, and vehicle dynamics. The passenger car subsystem models the entire vehicle dynamics: wheels, differential, vehicle body, and the vehicle engine. These have been modeled using first principles, but given the complex nature of high-fidelity models, simulation can notably be slowed down.\r\n<h6><\/h6>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-11210 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/AI_ROM_example_overview.png\" alt=\"Replacing a high-fidelity engine model with an AI-based model\" width=\"1834\" height=\"894\" \/><\/h6>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> Replace a high-fidelity engine model with an AI-based model to reduce complexity and speed up the system simulation.\r\n<h6><\/h6>\r\nYou can replace a high-fidelity model with an AI-based reduced-order model. For example, the AI model may consist of a sequence model, a Neural ODE, a Non-Linear ARX model, etc. The model should input the engine speed, the ignition timing, the throttle position, and the wastegate valve value; and output the engine torque.\r\n<h6><\/h6>\r\nThe first step in the AI-driven system design workflow involves data preparation. Here, you can design a set of experiments for you to generate the required synthetic data. This can be achieved by determining which parameters to vary, running the respective simulations, and log the data you need for training.\r\n<h6><\/h6>\r\nMoving to the AI modeling phase, you can create an <a href=\"https:\/\/www.mathworks.com\/discovery\/lstm.html\">LSTM<\/a> network programmatically with a few lines of code (for an example, see <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/sequence-to-one-regression-using-deep-learning.html\">Sequence-to-One Regression Using Deep Learning<\/a>) or interactively with the <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ref\/deepnetworkdesigner-app.html\">Deep Network Designer<\/a> app. Another exciting approach involves using Neural ODEs. Neural ODEs are especially interesting when you need to model the dynamic behavior of a system, but it might not be clear how to derive the ODE from first principles. Using <a href=\"https:\/\/www.mathworks.com\/help\/ident\/neural-state-space-models.html\">Neural State-Space Models<\/a>, introduced in MATLAB R2022b, you can create a deep learning-based non-linear state-space model using feedforward neural networks. This allows you to train Neural ODE models without being a deep learning expert. Additionally, you can import a pretrained network from TensorFlow\u2122, PyTorch\u00ae, or ONNX\u2122. For more information, see <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/interoperability-between-deep-learning-toolbox-tensorflow-pytorch-and-onnx.html\">Interoperability Between MATLAB, TensorFlow, PyTorch, and ONNX<\/a>.\r\n<h6><\/h6>\r\nOnce the model has been trained, you can simulate and test your AI model with the rest of your components in Simulink. Integrating the AI model in Simulink, couldn\u2019t be easier. Simply drag and drop the corresponding block to your Simulink model and configure it accordingly (by specifying the location of the model or name in the workspace). By using a reduced-order model of the vehicle engine, you speed up the overall simulation time of the system:\r\n<h6><\/h6>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-11225 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/AI_ROM_simulation.gif\" alt=\"System-level simulation of AI-based reduced-order model\" width=\"1840\" height=\"1072\" \/><\/h6>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> System-level simulation of AI-based reduced-order model for vehicle speed control\r\n<h6><\/h6>\r\nFinally, you can deploy the entire plant model (Passenger Car subsystem) by generating C\/C++ code and perform system-level integration and test. <a href=\"https:\/\/www.mathworks.com\/discovery\/hardware-in-the-loop-hil.html\">Hardware-in-the-Loop<\/a> (HIL) serves as a final functional test of the algorithm under design before we move into final system integration. In HIL, you can generate code both for the component or algorithm you are designing (e.g., the controllers) as well as for the plant model. The plant code, which here includes the trained neural network, runs on a real-time computer; mimicking the behavior of our vehicle. The component code or the algorithm (in this case, the controller) runs on the target platform. Simulink can be used to monitor signals and adjust parameters on the deployed components.\r\n<h6><\/h6>\r\nThe same techniques can be applied to a variety of systems, for example, temperature models, turbulence and combustion simulation, and estimating NOx emissions. Check out the following user stories to learn more about these applications:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/videos\/virtual-xcu-calibration-with-neural-networks-1559311099816.html\">Virtual XCU Calibration with Neural Networks<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/company\/mathworks-stories\/speeding-up-engine-development-with-deep-learning.html\">Building Better Engines with AI<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/company\/newsletters\/articles\/using-deep-learning-networks-to-estimate-nox-emissions.html\">Using Deep Learning Networks to Estimate NOX Emissions<\/a><\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n&nbsp;\r\n<p style=\"font-size: 18px;\"><strong>Conclusion<\/strong><\/p>\r\nIt was truly exciting to attend NeurIPS 2022 and have the opportunity to share with the Deep Learning community how AI can be used for reduced-order modeling. Before wrapping up the session, I went back to the AI-based reduced order models I had trained and analyzed different attributes. This allows you to evaluate and make design tradeoffs based on specific requirements you might have. As the following graph describes, the LSTM model provides slightly better accuracy, but the Neural State Space outperforms the LSTM in every other attribute.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-11228\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/AI_ROM_radar_plot.png\" alt=\"Differences between the trained LSTM and Neural State Space models\" width=\"628\" height=\"417\" \/>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> <a href=\"https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/59561-spider_plot\">Radar plot<\/a> highlighting different attributes of the trained LSTM and Neural State Space models. Note that the results shown in this plot are specific to this vehicle engine example.\r\n<h6><\/h6>\r\nIn summary, an AI model may be used to create an AI-based reduced-order model that replaces part of the complex dynamics of a vehicle engine. Using data synthetically generated from the original first-principles model, you can train AI models using various techniques (LSTMs, Neural ODEs, NLARX models, etc.) to mimic the behavior of the vehicle engine. You can then integrate such an AI model into Simulink for system-level simulation (together with the rest of the first-principles components), generate C\/C++ code, and perform HIL testing.\r\n<h6><\/h6>\r\nLeave a comment with anything you\u2019d like to chat about related to AI for Reduced Order Modeling. And don\u2019t forget to look at the <a href=\"https:\/\/content.mathworks.com\/viewer\/63814a32c8ce21bfab25787d\">Slides: Using AI for Reduced-Order Modeling<\/a>, which provide details on how to use AI for ROM at every stage of a complete system!\r\n<h6><\/h6>","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2022\/12\/AI_ROM_thumbnail.png\" class=\"img-responsive attachment-post-thumbnail size-post-thumbnail wp-post-image\" alt=\"\" decoding=\"async\" loading=\"lazy\" \/><\/div><p>The following post is from Lucas Garcia, Deep Learning Product Manager at MathWorks.\r\n\r\nThis blog discusses the MathWorks\u2019 presence at NeurIPS 2022 and my talk on \u2018Using AI for Reduced-Order... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2022\/12\/05\/using-ai-for-reduced-order-modeling\/\">read more >><\/a><\/p>","protected":false},"author":194,"featured_media":11243,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[36,48,9,33],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/11138"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/194"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=11138"}],"version-history":[{"count":43,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/11138\/revisions"}],"predecessor-version":[{"id":15922,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/11138\/revisions\/15922"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media\/11243"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=11138"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=11138"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=11138"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}