Virtual sensors can be used in any system where real-time monitoring and control are required, and where the use of physical sensors might be impractical or costly. Leveraging AI techniques can improve the accuracy and reliability of virtual sensors. This blog post shows how you can use MATLAB and Simulink to design AI virtual sensors.

Virtual sensors are not new. They are often referred to as “observers” and engineers have been successfully designing them using Kalman filtering and other techniques (for example, sliding mode observers for estimating position and speed of brushless motor shaft). AI is a newer option that, in some cases, can improve the design of virtual sensors as an addition or alternative to other methods. With MATLAB and Simulink, you can easily compare methods and combine methods to find the best solution for your workflow.

Figure: Workflow for designing virtual sensors using AI

 

Why Virtual Sensors?

Virtual sensors are software components that mimic the behavior of a physical sensor by using other measurements or data available to estimate the quantity of interest. They are commonly used in many industries and applications to monitor and optimize processes, or estimate environmental conditions.

Figure: Industries and applications virtual sensors can be used for

The main benefits of virtual sensors, compared to physical sensors, are improved performance and reduced costs. Consider using virtual sensors, when:

• It is difficult or impractical to use physical sensors.
• Additional information is needed beyond what can be directly measured.
• The physical sensors are unreliable, difficult, or costly to maintain.

Real-World Applications of Virtual Sensors

MATLAB and Simulink users develop virtual sensors by leveraging AI, traditional methods (e.g., Kalman filtering), and hybrid approaches (AI combined with other methods). Check out some of their stories:

• Coca-Cola Develops Virtual Pressure Sensor with Machine Learning to Improve Beverage Dispenser Diagnostics
• Poclain Hydraulics Develops Soft Sensors to Measure Motor Temperature in Real Time Using Deep Learning and Kalman Filters
• Gotion Develops Onboard SOH Estimation Using DVA and ICA for LFP Batteries
• Mercedes-Benz Simulates Hardware Sensors with Deep Neural Networks
• Battery SOC and SOH Estimation using a Hybrid Machine Learning Approach
• Onboard Battery Pack State of Charge Estimation Using a Neural Network

 

Virtual Sensors with AI

This section talks about using AI to design virtual sensors and shows how:

• You can quickly train and compare different AI models in MATLAB to implement a virtual sensor or import AI models from other deep learning frameworks.
• Safely, systematically, and thoroughly test your virtual sensor design by simulating different test scenarios before deploying to production.

I will use an example of an AI virtual sensor for battery State-Of-Charge (SOC) estimation to illustrate the design steps and highlight the benefits of using AI. You can see the full example here. An SOC virtual sensor is a key component of a battery management system (BMS) that ensures the safe and efficient operation of a battery. Lithium-ion batteries are everywhere today, from wearable electronics, mobile phones, and laptops to electric vehicles and smart grids.

Figure: Virtual sensor for estimating battery State-of-Charge (SOC)

Even though this blog post walks you through the SOC virtual sensor example, it also talks about tools and options that might not be relevant to the SOC example. You might find these tools and options helpful in your workflow.

Data Preparation

The design process can be limited or constrained by the quality or availability of data. MATLAB can help you improve the data quality by (1) using signal processing techniques to remove noise and (2) extracting meaningful standard and domain-specific features from the data. You can also use low-code apps (e.g., Signal Analyzer) to explore your data.

Deep learning models in particular require large data sets for training. To overcome the issue of data availability, you can complement real-world data with synthetic data. You can generate synthetic data sets by using generative adversarial networks (GANs), combining waveform types (for communications and radar applications), or creating and simulating a model of the physical system in Simulink.

AI Modeling

To develop a virtual sensor, you can train an AI model by using a machine learning algorithm (e.g., SVM, linear regression, and neural network) with a few lines of MATLAB code or by importing a model trained in TensorFlow™ or PyTorch®.

Let’s take a look at how you can create some useful models for virtual sensors.

1. Create an LSTM (long short-term memory) model for sequence-to-one regression. LSTMs are particularly effective in working with sequential data that varies in length. For an example on how to predict the remaining useful life (RUL) of engines, see Sequence-to-Sequence Regression Using Deep Learning.

layers = [sequenceInputLayer(numFeatures)
lstmLayer(numHiddenUnits,OutputMode="last")
fullyConnectedLayer(numResponses)
regressionLayer];

Then, you can train the LSMT model by using the trainNetwork function.

2. Train a feedforward, fully connected neural network for regression.

Mdl = fitrnet(X,Y)

3. Fit a decision tree for regression.

tree = fitrtree(X,Y);

If you prefer a more interactive workflow, you can use low-code apps (Deep Network Designer, Classification Learner, and Regression Learner) for designing, tuning, assessing, and optimizing AI models.

MATLAB also offers nonlinear models for capturing system dynamics. You can estimate nonlinear system dynamics using Hammerstein-Wiener and Nonlinear ARX models with machine learning techniques such as Gaussian Processes (GP), Support Vector Machines (SVM), and other representations. Alternatively, you can create neural ordinary differential equation (ODE) models using deep learning to capture nonlinear system dynamics. For an example, see Neural State-Space Model of SI Engine Torque Dynamics.

You can use your AI models in combination with other methods, such as Kalman filters. For instance, a neural state space model can be used as the model inside of a nonlinear Kalman filter (example: Online State Estimation Using Identified Models - Nonlinear Models).

System Simulation

Once you have created or loaded your AI models, you can integrate them into Simulink and test them in simulations. Using Simulink, you can evaluate tradeoffs, such as accuracy, model size, and inference speed, in the context of the larger system that the virtual sensor is part of (BMS in our example). In the SOC estimation example, four models are compared in the Simulink simulation: an extended Kalman filter (EKF), a regression tree, a feed-forward network (DL-FFN), and an LSTM model.

Figure: Simulating and comparing four virtual sensors (Kalman filter model and 3 AI models) in Simulink

One of the main benefits of using virtual sensors based on AI models is that they could be faster than virtual sensors based on physical models (especially, complicated physical models). On the other hand, AI models are less explainable than Kalman filter models and additional techniques need to be applied to understand the inner workings of an AI model. The lack of explainability is not necessarily an issue (according to engineers at Poclain Hydraulics).

By simulating and testing the complete system with the virtual sensors, you can evaluate model performance, such as speed and accuracy, and use results from simulations to inform model selection.

Deployment

Using MATLAB and/or Simulink you can generate library-free C/C++ code for deploying AI virtual sensors to resource-constrained edge systems. You can also deploy virtual sensors to FPGA devices, enterprise systems, or the cloud.

At this stage, you can perform processor-in the-loop (PIL) or hardware-in-the-loop (HIL) testing. You can deploy the entire BMS software (which includes the virtual sensor to estimate the battery SOC) onto the target hardware and run a real-time closed-loop simulation with the plant model, which in this case is the model of battery dynamics. You can test key design parameters of the virtual sensor, such as accuracy, memory usage, and inference speed. Once testing is finalized, the model is ready to be deployed to the production hardware.

Figure: Generate C/C++ code for your AI model and deploy to hardware, used in a vehicle.

MATLAB and Simulink also automate MLOps processes for regulating the full lifecycle of AI models, including maintaining and monitoring the operation of AI virtual sensors after deployment to production.

 

Conclusion

MATLAB and Simulink provide engineers with tools to improve data quality, access to popular machine learning and deep learning algorithms, integration with a system-wide simulation environment, and deployment to the real-world system. So, you can choose the best AI technique (or combination of AI technique with other method) not only for designing an accurate virtual sensor, but for designing and deploying a virtual sensor that will run efficiently on the intended system.