Developer Zone

Advanced Software Development with MATLAB

Don’t Mock Me! 3

Posted by Andy Campbell,

Hi folks, it's been a long time! Thanks for your patience as we have a lot of great incubating topics to discuss on the blog. Today I am excited to introduce David Hruska, who is the lead developer on an exciting new mocking framework that is part of MATLAB as of R2017b, and he is here to explore it a bit with you. Enjoy!

Do you mock when writing code? No, I'm not talking about this kind of mocking. I'm talking about replacing the dependencies of your system-under-test with mock objects to drive the system's behavior a particular way, or verify how it interacts with its collaborators. MATLAB release 2017a includes a mocking framework for just this purpose. Let's look at an example in action to see what this means in practice. Here's the interface for an Iterator class that we'll be using.

classdef Iterator < handle
   properties (Abstract, SetAccess=private)
   methods (Abstract)
      bool = hasNext(iter);

And here's an algorithm we want to test. It determines if a value is present in the iterator.

function bool = hasValue(iter, value)
validateattributes(iter, {'Iterator'}, {'scalar'}, 'hasValue');
validateattributes(value, {'numeric'}, {'nonempty'}, 'hasValue');

bool = false;
while iter.hasNext
   if iter.Value == value
      bool = true;

Constructing a mock

To construct a mock, call the createMock method of the matlab.mock.TestCase class. Typically this would be used inside of a test method in a class which derives from matlab.mock.TestCase. However, for interactive experimentation at the command line, we'll first use the forInteractiveUse static method to obtain a TestCase instance. A call to the createMock method is all that's required to create a mock object. The framework creates a class that implements the necessary abstract properties and methods of the interface and constructs an instance. createMock also returns an associated behavior object for controlling the mock.

testCase = matlab.mock.TestCase.forInteractiveUse;
[mockIterator, behavior] = testCase.createMock(?Iterator);

To make the mock do something useful, we'll need to define its behavior. The behavior specification reads like a sentence:

import matlab.mock.actions.AssignOutputs;
when(get(behavior.Value), then(AssignOutputs(1), then(AssignOutputs(2), then(AssignOutputs(3)))));

Here's what this code does: when the Value property is accessed, first return a value of 1 . The next time the property is accessed, return a value of 2 . Finally, return 3 for any subsequent property accesses.

Let's also set up the hasNext method to return true the first three times it's invoked and false thereafter. Use withAnyInputs to specify that this behavior should be performed regardless of any additional inputs passed to the method.

when(withAnyInputs(behavior.hasNext), then(repeat(3, AssignOutputs(true)), then(AssignOutputs(false))));

With some basic behavior defined, we can now use the mock to test our algorithm.

result = hasValue(mockIterator, 2);
testCase.verifyTrue(result, "Expected hasValue to find 2.");
Interactive verification passed.

The mock iterator was set up to return 1 , then 2 , then 3 ; the hasValue function found the 2 ; and the we've verified this outcome. Great!

The example above uses a technique sometimes referred to as stubbing: we have set up a mock to return pre-defined responses to drive the system in a particular way. We can also spy on how the system interacts with its dependencies. Let's set up a new mock with slightly different behavior and pass it to hasValue.

[mockIterator, behavior] = testCase.createMock(?Iterator);
when(get(behavior.Value), then(AssignOutputs(1)));
when(withAnyInputs(behavior.hasNext), then(repeat(10, AssignOutputs(true)), then(AssignOutputs(false))));

result = hasValue(mockIterator, 1);
testCase.assertTrue(result, "Expected hasValue to find 1.");
Interactive assertion passed.

Here we've defined the mock to always return value 1 when its Value property is accessed. The hasNext method returns true the first 10 times it is called, indicating that there are 10 values in the iterator. It would be reasonable to expect an efficient implementation of hasValue to "short-circuit" once it has found the first value and stop checking the remaining elements in the iterator. Let's see if this is what happened, though. When the mock is constructed, it automatically begins recording information about its interactions. In addition to defining how the mock should act, the behavior also provides access to this spy information. Let's use this information to verify that Value was accessed exactly once, as an efficient implementation would do.

import matlab.mock.constraints.WasAccessed;
testCase.verifyThat(behavior.Value, WasAccessed('WithCount',1), ...
    "Expected Value to be accessed only once.");
Interactive verification failed.

Test Diagnostic:
Expected Value to be accessed only once.

Framework Diagnostic:
WasAccessed failed.
--> Property 'Value' was not accessed the expected number of times.
    Actual property access count:
    Expected property access count:

Specified property access:

The qualification failed because the algorithm accessed the property 10 times. This could be addressed by returning immediately from the function after it finds the first instance of the desired value.

function bool = efficientHasValue(iter, value)
validateattributes(iter, {'Iterator'}, {'scalar'}, 'efficientHasValue');
validateattributes(value, {'numeric'}, {'nonempty'}, 'efficientHasValue');

bool = false;
while iter.hasNext
   if iter.Value == value
      bool = true;
      % Done! We've found the value.

Returning to the original algorithm, let's write one more test. This time let's write a negative test to ensure the proper error checking is performed on the value input to hasValue, rejecting empty values. For this test, we don't plan to even use an Iterator, but we do need to provide a valid instance because the function also validates the iterator input. Therefore, we can simply construct a mock but define no behavior. The mock is then passed to the function being tested.

dummyIterator = testCase.createMock(?Iterator);
testCase.verifyError(@()hasValue(dummyIterator, []), 'MATLAB:hasValue:expectedNonempty');
Interactive verification passed.

Comparison with Hand-Coded Mocks

The tests above could have been performed using hand-written mock subclasses of the Iterator interface. However, this can lead to one of two things: an inventory of many different mocks (stubs, spies, fakes, etc.) or one very complicated mock class that might even be more complicated that the code you're trying to test! For example, we might have been tempted to reuse the hand-written stub or spy for the negative test above which could result in undesirable coupling. By using the mocking framework, we achieve independence of each test.

Also, as your design evolves, your interfaces might change: new abstract properties or methods could be added. Or maybe the access permissions might change. This is particularly likely early in the development process. Hand-written mocks would need to be updated individually to relect the changes to the interface. The mocking framework, on the other hand, automatically implements all abstract members required by the interface with the correct access permissions each time createMock is called.

In summary, compared to hand-coded mocks, the mocking framework provides

  • Isolation and independence of different tests
  • Clear visibility into tests with fewer additional files to look at

What is the cost? The convenience does come at the cost of performance: if you're frequently reusing the exact same mock class, hand-coding will result in faster test execution time. However, the mocking framework overhead is likely small enough to be insignificant in most testing scenarios.

There's a lot more the framework can do, so check out the documentation. Have you used mocking in your testing workflows? Let us know in the comments below.

Get the MATLAB code

Published with MATLAB® R2017a

3 CommentsOldest to Newest

Michael replied on : 1 of 3

Thanks for another nice blog entry 🙂 .Indeed we are using “handwritten”-stubbing for our test environment to prevent the test to stop due to questdlg popping up. However I don’t see how this could be replaced by the mocking framework. What we do is to copy a questdlg.m located inside a package (so that ist not called outside the testevironment by accident) to pwd at test start and delete this questdlg.m again afterwards. Inside the questdlg.m you can define the desired output at teststart (e.g. to return always “Yes”.) Like this we dont have to touch the system under test.

David Hruska replied on : 2 of 3

Hi Michael,

Thanks for the kind words! The mocking framework currently only supports mocking of classes, but mocking of functions is another feature we’ve also been thinking about.

With some refactoring of the source code, you can wrap difficult-to-test functions inside a class to make them more testable. Here’s what I mean:

First, define an interface:

classdef LogicalInputProvider
        choice = obtainInput(prompt);

The system under test then uses this interface (but does not rely on any specific, concrete implementation):

classdef MySystemUnderTestThatRequiresUserInput
        function obj = MySystemUnderTestThatRequiresUserInput(anInputProvider)
            validateattributes(obj, {'LogicalInputProvider'}, {'scalar'})
            obj.InputProvider = anInputProvider.
        function doSomething(obj)
            % Instead of calling questdlg, delegate to the InputProvider
            choice = obj.InputProvider.obtainInput('Would you like to continue?');

You can then implement a QuestionDialogInputProvider which you might typically use:

classdef QuestionDialogInputProvider < LogicalInputProvider
        function choice = obtainInput(prompt)
            choice = questdlg(prompt, 'Dialog', 'No', 'Yes');

You might even add logic to the MySystemUnderTestThatRequiresUserInput to construct a QuestionDialogInputProvider by default if no user-specified LogicalInputProvider is passed in.

With this testing “seam” cut into the system, you could then use the mocking framework to inject a stub LogicalInputProvider for testing purposes.

I recognize this is a pretty big refactoring for a relatively small testing win. But it could provide other benefits as well: perhaps you can also write a LogicalInputProvider that uses the input function for use in environments where it’s not possible to display a modal dialog (e.g., a command-line only interface).

Michael replied on : 3 of 3

Thank you for this detailed explanation of the mocking framework for my use case. Indeed I can use this for another dialog in our environment that is already based on a class. Thank you very much 🙂

Add A Comment

Your email address will not be published. Required fields are marked *

What is 7 + 10 ?

Preview: hide