Guy and Seth on Simulink

February 12th, 2011

An important benefit of Model-Based Design is doing early verification of your model and using what you learn to verify your final production software. As your design moves to production a critical question is how do you verify the generated code? In this post I want to provide a high level answer to this question. This question is answered in detail with a clearly defined set of processes in the DO Qualification Kit (for DO-178) and the IEC Qualification Kit (for IEC 61508 and ISO 26262).

Start with a model that meets requirements

The entire code verification and validation process depends on your model meeting your requirements and exactly representing your design.

Verify that your design meets requirements during model development by running simulations. Functionality in the model should be traceable back to model requirements. You can use reviews, analysis of the model, and requirements based tests to prove that your original requirements are met by your design and the design does not contain unintended functionality. The test vectors from these simulations will be used for later verification of the generated code.

Test your code in an environment as close to the real thing as possible

Verification of the generated code can be done on the host computer through running the stand-alone application, using software in the loop (SIL) testing or processor in the loop (PIL) testing. Each of these modes of testing provides additional verification using an environment that is increasingly more similar to the final target environment.

Verify the generated code is equivalent to the model

Test vectors used to verify that the model meets requirements provide a baseline for the behavior of the generated code. Once you exercise the generated code with the test vectors compare the generated code results to the baseline result. If you find any differences in testing results they should be investigated and understood. Some differences may be attributed to implementation changes (converting from an interpreted model to an executable implementation), or could be an indication of bugs. All differences should be understood in order to confirm the behavior of the system is bug free.

Verify there is no unintended functionality

After testing for equivalence, you might think you are done, however, if there are code paths you have not exercised, there may be bugs hidden in that portion of the code. To test for unintended functionality you need to perform a traceability review and/or measure code coverage.

Traceability Review

Measuring the traceability of the generated code requires that you confirm that all of the code generated for your final application traces back to a part of the original model that represents your design. Any non-traceable elements in the code need to be reviewed.

Code Coverage

Measuring code coverage can give you an idea of how comprehensive your test cases are. Of course it would be great to have 100% code coverage just from running the baseline equivalence tests, but when you don’t have full coverage you need to add additional test vectors to increase coverage. Review any portions of the code that are not fully tested.

Tools to help with verification and validation

• Simulink Verification and Validation – enables requirements tracing and custom modeling standards checking through model advisor.
• Simulink Design Verifier – automatically generate test vectors to achieve model coverage, implement tests to prove that the model meets requirements.
• Polyspace – use formal methods to prove your code is bug free.
• PIL (processor in the loop) and SIL (software in the loop) mode testing with Real-Time Workshop Embedded Coder
• Use the Code Generation Verification API in Real-Time Workshop Embedded Coder to verify numerical equivalence.

Now it’s your turn

What do you do? Leave a comment here and tell us how you verify your generated code.

Note: The process described here does not ensure the safety of the software or the system under consideration.

By Seth Popinchalk

20:48 UTC | Posted in Code Generation, Model-Based Design, Real-Time Workshop, Verification & Validation | Permalink | 4 Comments »

You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

4 Responses to “How to Verify Code Generated with Model-Based Design”

1. John Nebel replied on March 8th, 2011 at 22:20 UTC :
   

   I have a more basic questions. Do your customers consider Simulink a programming language? When developing models, do they follow a software development process (unit test, functional test, and integration test)?

2. Seth Popinchalk replied on March 9th, 2011 at 15:05 UTC :
   

   @John Nebel – Absolutely! While some think of modeling in Simulink as “coding,” I don’t think everyone puts it in the same class of programming languages as C or FORTRAN. Anyone developing software should follow some kind of process, appropriate to the kind of application they are developing. The most basic of process could be following a Simulink modeling style guide.

3. John Nebel replied on March 9th, 2011 at 16:04 UTC :
   

   I would classify Simulink as a 4th generation programming language which is the next level above C/C++. I am trying to get my company to create a model development standard. We already have software development standards for C/C++ and Ada, but not for Simulink models. We have always used Simulink and NI LabVIEW for generating testing models, but not for production code. Groups here are starting to push for using Simulink models in released/production code, and those groups want to avoid using our CMMI level 5 Software Development process for Simulink models. My opinion is that Simulink model development should follow a process, and I would like to be a part of the team to develop this process.

   Do you have a recommended modeling development process?
   Do customers follow different processes if it is for production versus internal use only?

4. hoc tieng anh replied on December 19th, 2011 at 04:10 UTC :
   

   Nice information, valuable and excellent design, as share good stuff with good ideas and concepts, lots of great information and inspiration, both of which I need, thanks for all the enthusiasm to offer such helpful information here

Leave a Reply

<pre class="code">
a = magic(3);
sum(a)
</pre>

These postings are the author's and don't necessarily represent the opinions of The MathWorks.