Hi folks, it's been a long time! Thanks for your patience as we have a lot of great incubating topics to discuss on the blog. Today I am excited to introduce David Hruska, who is the lead developer on an exciting new mocking framework that is part of MATLAB as of R2017b, and he is here to explore it a bit with you. Enjoy!

Do you mock when writing code? No, I'm not talking about this kind of mocking. I'm talking about replacing the dependencies of your system-under-test with mock objects to drive the system's behavior a particular way, or verify how it interacts with its collaborators. MATLAB release 2017a includes a mocking framework for just this purpose. Let's look at an example in action to see what this means in practice. Here's the interface for an Iterator class that we'll be using.

classdef Iterator < handle
   properties (Abstract, SetAccess=private)
      Value;
   end
   methods (Abstract)
      bool = hasNext(iter);
      advance(iter);
   end
end

And here's an algorithm we want to test. It determines if a value is present in the iterator.

function bool = hasValue(iter, value)
validateattributes(iter, {'Iterator'}, {'scalar'}, 'hasValue');
validateattributes(value, {'numeric'}, {'nonempty'}, 'hasValue');

bool = false;
while iter.hasNext
   iter.advance;
   if iter.Value == value
      bool = true;
   end
end
end

Constructing a mock

To construct a mock, call the createMock method of the matlab.mock.TestCase class. Typically this would be used inside of a test method in a class which derives from matlab.mock.TestCase. However, for interactive experimentation at the command line, we'll first use the forInteractiveUse static method to obtain a TestCase instance. A call to the createMock method is all that's required to create a mock object. The framework creates a class that implements the necessary abstract properties and methods of the interface and constructs an instance. createMock also returns an associated behavior object for controlling the mock.

testCase = matlab.mock.TestCase.forInteractiveUse;
[mockIterator, behavior] = testCase.createMock(?Iterator);

To make the mock do something useful, we'll need to define its behavior. The behavior specification reads like a sentence:

import matlab.mock.actions.AssignOutputs;
when(get(behavior.Value), then(AssignOutputs(1), then(AssignOutputs(2), then(AssignOutputs(3)))));

Here's what this code does: when the Value property is accessed, first return a value of 1 . The next time the property is accessed, return a value of 2 . Finally, return 3 for any subsequent property accesses.

Let's also set up the hasNext method to return true the first three times it's invoked and false thereafter. Use withAnyInputs to specify that this behavior should be performed regardless of any additional inputs passed to the method.

when(withAnyInputs(behavior.hasNext), then(repeat(3, AssignOutputs(true)), then(AssignOutputs(false))));

With some basic behavior defined, we can now use the mock to test our algorithm.

result = hasValue(mockIterator, 2);
testCase.verifyTrue(result, "Expected hasValue to find 2.");

Interactive verification passed.

The mock iterator was set up to return 1 , then 2 , then 3 ; the hasValue function found the 2 ; and the we've verified this outcome. Great!

The example above uses a technique sometimes referred to as stubbing: we have set up a mock to return pre-defined responses to drive the system in a particular way. We can also spy on how the system interacts with its dependencies. Let's set up a new mock with slightly different behavior and pass it to hasValue.

[mockIterator, behavior] = testCase.createMock(?Iterator);
when(get(behavior.Value), then(AssignOutputs(1)));
when(withAnyInputs(behavior.hasNext), then(repeat(10, AssignOutputs(true)), then(AssignOutputs(false))));

result = hasValue(mockIterator, 1);
testCase.assertTrue(result, "Expected hasValue to find 1.");

Interactive assertion passed.

Here we've defined the mock to always return value 1 when its Value property is accessed. The hasNext method returns true the first 10 times it is called, indicating that there are 10 values in the iterator. It would be reasonable to expect an efficient implementation of hasValue to "short-circuit" once it has found the first value and stop checking the remaining elements in the iterator. Let's see if this is what happened, though. When the mock is constructed, it automatically begins recording information about its interactions. In addition to defining how the mock should act, the behavior also provides access to this spy information. Let's use this information to verify that Value was accessed exactly once, as an efficient implementation would do.

import matlab.mock.constraints.WasAccessed;
testCase.verifyThat(behavior.Value, WasAccessed('WithCount',1), ...
    "Expected Value to be accessed only once.");

Interactive verification failed.

----------------
Test Diagnostic:
----------------
Expected Value to be accessed only once.

---------------------
Framework Diagnostic:
---------------------
WasAccessed failed.
--> Property 'Value' was not accessed the expected number of times.
    
    Actual property access count:
            10
    Expected property access count:
             1

Specified property access:
    PropertyGetBehavior
        <IteratorMock_1>.Value

The qualification failed because the algorithm accessed the property 10 times. This could be addressed by returning immediately from the function after it finds the first instance of the desired value.

function bool = efficientHasValue(iter, value)
validateattributes(iter, {'Iterator'}, {'scalar'}, 'efficientHasValue');
validateattributes(value, {'numeric'}, {'nonempty'}, 'efficientHasValue');

bool = false;
while iter.hasNext
   iter.advance;
   if iter.Value == value
      bool = true;
      % Done! We've found the value.
      return;
   end
end
end

Returning to the original algorithm, let's write one more test. This time let's write a negative test to ensure the proper error checking is performed on the value input to hasValue, rejecting empty values. For this test, we don't plan to even use an Iterator, but we do need to provide a valid instance because the function also validates the iterator input. Therefore, we can simply construct a mock but define no behavior. The mock is then passed to the function being tested.

dummyIterator = testCase.createMock(?Iterator);
testCase.verifyError(@()hasValue(dummyIterator, []), 'MATLAB:hasValue:expectedNonempty');

Interactive verification passed.

Comparison with Hand-Coded Mocks

The tests above could have been performed using hand-written mock subclasses of the Iterator interface. However, this can lead to one of two things: an inventory of many different mocks (stubs, spies, fakes, etc.) or one very complicated mock class that might even be more complicated that the code you're trying to test! For example, we might have been tempted to reuse the hand-written stub or spy for the negative test above which could result in undesirable coupling. By using the mocking framework, we achieve independence of each test.

Also, as your design evolves, your interfaces might change: new abstract properties or methods could be added. Or maybe the access permissions might change. This is particularly likely early in the development process. Hand-written mocks would need to be updated individually to relect the changes to the interface. The mocking framework, on the other hand, automatically implements all abstract members required by the interface with the correct access permissions each time createMock is called.

In summary, compared to hand-coded mocks, the mocking framework provides

What is the cost? The convenience does come at the cost of performance: if you're frequently reusing the exact same mock class, hand-coding will result in faster test execution time. However, the mocking framework overhead is likely small enough to be insignificant in most testing scenarios.

There's a lot more the framework can do, so check out the documentation. Have you used mocking in your testing workflows? Let us know in the comments below.

Get the MATLAB code

Published with MATLAB® R2017a