{"id":13022,"date":"2023-08-11T12:43:59","date_gmt":"2023-08-11T16:43:59","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=13022"},"modified":"2025-01-06T17:59:22","modified_gmt":"2025-01-06T22:59:22","slug":"virtual-sensors-with-ai-and-model-based-design","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2023\/08\/11\/virtual-sensors-with-ai-and-model-based-design\/","title":{"rendered":"Virtual Sensors with AI and Model-Based Design"},"content":{"rendered":"<h6><\/h6>\r\nVirtual sensors can be used in any system where real-time monitoring and control are required, and where the use of physical sensors might be impractical or costly. Leveraging AI techniques can improve the accuracy and reliability of virtual sensors. This blog post shows how you can use MATLAB and Simulink to design AI virtual sensors.\r\n<h6><\/h6>\r\nVirtual sensors are not new. They are often referred to as \u201cobservers\u201d and engineers have been successfully designing them using <a href=\"https:\/\/www.mathworks.com\/videos\/series\/understanding-kalman-filters.html\">Kalman filtering<\/a> and other techniques (for example, <a href=\"https:\/\/www.mathworks.com\/help\/mcb\/ref\/slidingmodeobserver.html\">sliding mode observers<\/a> for estimating position and speed of brushless motor shaft). AI is a newer option that, in some cases, can improve the design of virtual sensors as an addition or alternative to other methods. With MATLAB and Simulink, you can easily compare methods and combine methods to find the best solution for your workflow.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-13025 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/08\/virtual_sensor_ai_workflow.png\" alt=\"The workflow to design virtual sensors using AI includes data preparation, AI modeling, system simulation, and deployment\" width=\"725\" height=\"185\" \/>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> Workflow for designing virtual sensors using AI\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><strong>Why Virtual Sensors?<\/strong><\/p>\r\nVirtual sensors are software components that mimic the behavior of a physical sensor by using other measurements or data available to estimate the quantity of interest. They are commonly used in many industries and applications to monitor and optimize processes, or estimate environmental conditions.\r\n<h6><\/h6>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-13031 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/08\/virtual_sensors_applications.png\" alt=\"Visual sensors can be used in many industries and applications\" width=\"1042\" height=\"607\" \/><\/h6>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> Industries and applications virtual sensors can be used for\r\n<h6><\/h6>\r\nThe main benefits of virtual sensors, compared to physical sensors, are improved performance and reduced costs. Consider using virtual sensors, when:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li>It is difficult or impractical to use physical sensors.<\/li>\r\n \t<li>Additional information is needed beyond what can be directly measured.<\/li>\r\n \t<li>The physical sensors are unreliable, difficult, or costly to maintain.<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<strong>Real-World Applications of Virtual Sensors<\/strong>\r\n<h6><\/h6>\r\nMATLAB and Simulink users develop virtual sensors by leveraging AI, traditional methods (e.g., Kalman filtering), and hybrid approaches (AI combined with other methods). Check out some of their stories:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/company\/user_stories\/coca-cola-develops-virtual-pressure-sensor-with-machine-learning-to-improve-beverage-dispenser-diagnostics.html\">Coca-Cola Develops Virtual Pressure Sensor with Machine Learning to Improve Beverage Dispenser Diagnostics<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/company\/user_stories\/poclain-hydraulics-develops-soft-sensors-to-measure-motor-remperature-in-real-time-using-deep-learning-and-kalman-filters.html\">Poclain Hydraulics Develops Soft Sensors to Measure Motor Temperature in Real Time Using Deep Learning and Kalman Filters<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/videos\/developing-onboard-soh-estimation-using-dva-and-ica-for-lfp-batteries-1654073089339.html\">Gotion Develops Onboard SOH Estimation Using DVA and ICA for LFP Batteries<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/company\/user_stories\/mercedes-benz-simulates-hardware-sensors-with-deep-neural-networks.html?s_tid=srchtitle_virtual%20sensor_1\">Mercedes-Benz Simulates Hardware Sensors with Deep Neural Networks<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/content\/dam\/mathworks\/mathworks-dot-com\/company\/events\/conferences\/automotive-conference-michigan\/2022\/battery-soh-and-soc-estimation-using-a-hybrid-machine-learning-approach.pdf\">Battery SOC and SOH Estimation using a Hybrid Machine Learning Approach<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/content\/dam\/mathworks\/mathworks-dot-com\/company\/events\/conferences\/automotive-conference-michigan\/2022\/onboard-battery-pack-state-of-charge-estimation-using-a-trained-neural-network.pdf\">Onboard Battery Pack State of Charge Estimation Using a Neural Network<\/a><\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><strong>Virtual Sensors with AI<\/strong><\/p>\r\nThis section talks about using AI to design virtual sensors and shows how:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li>You can quickly train and compare different AI models in MATLAB to implement a virtual sensor or import AI models from other deep learning frameworks.<\/li>\r\n \t<li>Safely, systematically, and thoroughly test your virtual sensor design by simulating different test scenarios before deploying to production.<\/li>\r\n<\/ul>\r\nI will use an example of an AI virtual sensor for battery State-Of-Charge (SOC) estimation to illustrate the design steps and highlight the benefits of using AI. You can see the full example <a href=\"https:\/\/www.mathworks.com\/campaigns\/offers\/next\/integrating-ai-into-system-level-design\/using-ai-for-embedded-algorithms.html\">here<\/a>. An SOC virtual sensor is a key component of a battery management system (BMS) that ensures the safe and efficient operation of a battery. Lithium-ion batteries are everywhere today, from wearable electronics, mobile phones, and laptops to electric vehicles and smart grids.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-13040 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/08\/SoC_video.gif\" alt=\"Measurements of voltage, current, and temperature are used to train AI model that estimates State of Charge\" width=\"1414\" height=\"370\" \/>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> Virtual sensor for estimating battery State-of-Charge (SOC)\r\n<h6><\/h6>\r\nEven though this blog post walks you through the SOC virtual sensor example, it also talks about tools and options that might not be relevant to the SOC example. You might find these tools and options helpful in your workflow.\r\n<h6><\/h6>\r\n<strong>Data Preparation<\/strong>\r\n<h6><\/h6>\r\nThe design process can be limited or constrained by the quality or availability of data. MATLAB can help you improve the data quality by (1) using <a href=\"https:\/\/www.mathworks.com\/solutions\/artificial-intelligence\/ai-signal-processing.html\">signal processing<\/a> techniques to remove noise and (2) extracting meaningful standard and domain-specific <a href=\"https:\/\/www.mathworks.com\/discovery\/feature-extraction.html\">features<\/a> from the data. You can also use low-code apps (e.g., <a href=\"https:\/\/www.mathworks.com\/help\/signal\/ref\/signalanalyzer-app.html\">Signal Analyzer<\/a>) to explore your data.\r\n<h6><\/h6>\r\nDeep learning models in particular require large data sets for training. To overcome the issue of data availability, you can complement real-world data with synthetic data. You can generate synthetic data sets by using generative adversarial networks (<a href=\"https:\/\/www.mathworks.com\/discovery\/generative-adversarial-networks.html\">GANs<\/a>), <a href=\"https:\/\/www.mathworks.com\/content\/dam\/mathworks\/white-paper\/gated\/deep-learning-for-comms-and-radar-white-paper.pdf\">combining waveform types (for communications and radar applications)<\/a>, or creating and simulating a model of the physical system in Simulink.\r\n<h6><\/h6>\r\n<strong>AI Modeling<\/strong>\r\n<h6><\/h6>\r\nTo develop a virtual sensor, you can train an AI model by using a <a href=\"https:\/\/www.mathworks.com\/discovery\/machine-learning-models.html\">machine learning algorithm<\/a> (e.g., SVM, linear regression, and neural network) with a few lines of MATLAB code or by <a href=\"https:\/\/www.mathworks.com\/solutions\/deep-learning\/models.html\">importing a model<\/a> trained in TensorFlow\u2122 or PyTorch\u00ae.\r\n<h6><\/h6>\r\nLet\u2019s take a look at how you can create some useful models for virtual sensors.\r\n<h6><\/h6>\r\n1. Create an LSTM (<a href=\"https:\/\/www.mathworks.com\/discovery\/lstm.html\">long short-term memory<\/a>) model for sequence-to-one regression. LSTMs are particularly effective in working with sequential data that varies in length. For an example on how to predict the remaining useful life (RUL) of engines, see <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/sequence-to-sequence-regression-using-deep-learning.html\">Sequence-to-Sequence Regression Using Deep Learning<\/a>.\r\n<h6><\/h6>\r\n<pre>layers = [sequenceInputLayer(numFeatures)\r\nlstmLayer(numHiddenUnits,OutputMode=\"last\")\r\nfullyConnectedLayer(numResponses)\r\nregressionLayer];\r\n<\/pre>\r\n<h6><\/h6>\r\nThen, you can train the LSMT model by using the <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ref\/trainnetwork.html\">trainNetwork<\/a> function.\r\n<h6><\/h6>\r\n2. Train a feedforward, fully connected neural network for regression.\r\n<h6><\/h6>\r\n<pre>Mdl = fitrnet(X,Y)\r\n<\/pre>\r\n<h6><\/h6>\r\n3. Fit a decision tree for regression.\r\n<h6><\/h6>\r\n<pre>tree = fitrtree(X,Y);\r\n<\/pre>\r\nIf you prefer a more interactive workflow, you can use low-code apps (<a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ref\/deepnetworkdesigner-app.html\">Deep Network Designer<\/a>, <a href=\"https:\/\/www.mathworks.com\/help\/stats\/classificationlearner-app.html\">Classification Learner<\/a>,\u00a0and\u00a0<a href=\"https:\/\/www.mathworks.com\/help\/stats\/regressionlearner-app.html\">Regression Learner<\/a>) for designing, tuning, assessing, and optimizing AI models.\r\n<h6><\/h6>\r\nMATLAB also offers <a href=\"https:\/\/www.mathworks.com\/help\/ident\/nonlinear-model-identification.html\">nonlinear models for capturing system dynamics<\/a>. You can estimate nonlinear system dynamics using Hammerstein-Wiener and Nonlinear ARX models with machine learning techniques such as Gaussian Processes (GP), Support Vector Machines (SVM), and other representations. Alternatively, you can create neural ordinary differential equation (ODE) models using deep learning to capture nonlinear system dynamics.\u00a0For an example, see <a href=\"https:\/\/www.mathworks.com\/help\/ident\/ug\/nonlinear-state-space-model-of-si-engine-torque-dynamics.html\">Neural State-Space Model of SI Engine Torque Dynamics<\/a>.\r\n<h6><\/h6>\r\nYou can use your AI models in combination with other methods, such as Kalman filters. For instance, a neural state space model can be used as the model inside of a nonlinear Kalman filter (example: <a href=\"https:\/\/www.mathworks.com\/help\/ident\/ug\/online-state-estimation-using-identified-models-nonlinear-models.html\">Online State Estimation Using Identified Models - Nonlinear Models<\/a>).\r\n<h6><\/h6>\r\n<strong>System Simulation<\/strong>\r\n<h6><\/h6>\r\nOnce you have created or loaded your AI models, you can integrate them into Simulink and test them in simulations. Using Simulink, you can evaluate tradeoffs, such as accuracy, model size, and inference speed, in the context of the larger system that the virtual sensor is part of (BMS in our example). In the SOC estimation example, four models are compared in the Simulink simulation: an extended Kalman filter (EKF), a regression tree, a feed-forward network (DL-FFN), and an LSTM model.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-13046 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/08\/system_simulation.jpg\" alt=\"Simulink simulation of an extended Kalman filter (EKF), a regression tree, a feed-forward network (DL-FFN), and an LSTM model. \" width=\"1200\" height=\"890\" \/>\r\n<h6><\/h6>\r\n<strong>Figure:<\/strong> Simulating and comparing four virtual sensors (Kalman filter model and 3 AI models) in Simulink\r\n<h6><\/h6>\r\nOne of the main benefits of using virtual sensors based on AI models is that they could be faster than virtual sensors based on physical models (especially, complicated physical models). On the other hand, AI models are less <a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2022\/12\/30\/what-is-explainable-ai\/\">explainable<\/a> than Kalman filter models and additional <a href=\"https:\/\/www.mathworks.com\/discovery\/interpretability.html\">techniques<\/a> need to be applied to understand the inner workings of an AI model. The lack of explainability is not necessarily an issue (according to <a href=\"https:\/\/www.mathworks.com\/company\/user_stories\/poclain-hydraulics-develops-soft-sensors-to-measure-motor-remperature-in-real-time-using-deep-learning-and-kalman-filters.html\">engineers at Poclain Hydraulics<\/a>).\r\n<h6><\/h6>\r\nBy simulating and testing the complete system with the virtual sensors, you can evaluate model performance, such as speed and accuracy, and use results from simulations to inform model selection.\r\n<h6><\/h6>\r\n<strong>Deployment<\/strong>\r\n<h6><\/h6>\r\nUsing MATLAB and\/or Simulink you can generate library-free C\/C++ code for deploying AI virtual sensors to resource-constrained <a href=\"https:\/\/www.mathworks.com\/solutions\/deep-learning\/edge-ai.html\">edge systems<\/a>. You can also deploy virtual sensors to FPGA devices, enterprise systems, or the <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/deep-learning-in-the-cloud.html\">cloud<\/a>.\r\n<h6><\/h6>\r\nAt this stage, you can perform processor-in the-loop (<a href=\"https:\/\/www.mathworks.com\/help\/ecoder\/ug\/about-sil-and-pil-simulations.html\">PIL<\/a>) or hardware-in-the-loop (<a href=\"https:\/\/www.mathworks.com\/discovery\/hardware-in-the-loop-hil.html\">HIL<\/a>) testing. You can deploy the entire BMS software (which includes the virtual sensor to estimate the battery SOC) onto the target hardware and run a real-time closed-loop simulation with the plant model, which in this case is the model of battery dynamics. You can test key design parameters of the virtual sensor, such as accuracy, memory usage, and inference speed. Once testing is finalized, the model is ready to be deployed to the production hardware.\r\n<h6><\/h6>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-13049 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/08\/SOC_deployment.jpg\" alt=\"Deployment of SoC virtual sensor to production hardware\" width=\"1080\" height=\"195\" \/><\/h6>\r\n<strong>Figure: <\/strong>Generate C\/C++ code for your AI model and deploy to hardware, used in a vehicle.\r\n<h6><\/h6>\r\nMATLAB and Simulink also automate <a href=\"https:\/\/www.mathworks.com\/discovery\/mlops.html\">MLOps<\/a> processes for regulating the full lifecycle of AI models, including maintaining and monitoring the operation of AI virtual sensors after deployment to production.\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><strong>Conclusion<\/strong><\/p>\r\nMATLAB and Simulink provide engineers with tools to improve data quality, access to popular machine learning and deep learning algorithms, integration with a system-wide simulation environment, and deployment to the real-world system. So, you can choose the best AI technique (or combination of AI technique with other method) not only for designing an accurate virtual sensor, but for designing and deploying a virtual sensor that will run efficiently on the intended system.","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/08\/virtual_sensor_ai_workflow.png\" onError=\"this.style.display ='none';\" \/><\/div><p>\r\nVirtual sensors can be used in any system where real-time monitoring and control are required, and where the use of physical sensors might be impractical or costly. Leveraging AI techniques can... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2023\/08\/11\/virtual-sensors-with-ai-and-model-based-design\/\">read more >><\/a><\/p>","protected":false},"author":194,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[36,9,12],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/13022"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/194"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=13022"}],"version-history":[{"count":44,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/13022\/revisions"}],"predecessor-version":[{"id":16766,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/13022\/revisions\/16766"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=13022"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=13022"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=13022"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}