MATLAB xUnit: new version available
03-Aug-2016 update: MATLAB xUnit is now in a different location. It is currently maintained by Paul Sexton.
In February 2009 I posted MATLAB xUnit, a unit testing framework, to the MATLAB Central File Exchange. I blogged about it on Feb 3, 2009, and later I wrote an article about it for the IEEE Computing in Science and Engineering journal.
I just posted an update (version 3) to the File Exchange. The update contains some changes that several people have been asking me about, including:
- Running tests stored in packages
- A verbose option for runtests (displays name and running time for each individual test case)
- Test-case files and functions can be named with "test" at the end in addition to the beginning
- Larger default floor tolerance in "almost-equal" assertion functions (makes relative floating-point comparisons a bit more forgiving for two values very close to zero)
- Fixed handling of message strings in assert functions
If you use MATLAB xUnit you might want to grab the new version.
Here's a comparison of the default output and the new verbose output from runtests when running MATLAB xUnit's own test suite.
runtests c:\Work\matlab_xunit\testsStarting test run with 147 test cases. .................... .................... .................... .................... .................... .................... .................... ....... PASSED in 4.208 seconds.
runtests -verbose c:\Work\matlab_xunit\tests
tests
RuntestsTest
test_optionStringsIgnored ............................. passed in 0.179229 seconds
test_noTestCasesFound ................................. passed in 0.030545 seconds
test_packageName ...................................... passed in 0.069988 seconds
test_twoDirnames ...................................... passed in 0.086871 seconds
test_oneDirname ....................................... passed in 0.086698 seconds
test_oneInputArgWithFilter_failing .................... passed in 0.047178 seconds
test_oneInputArgWithFilter_passing .................... passed in 0.075055 seconds
test_testNameThenVerbose .............................. passed in 0.171836 seconds
test_verboseThenTestName .............................. passed in 0.172276 seconds
test_oneInputArg ...................................... passed in 0.066722 seconds
test_Verbose .......................................... passed in 0.098546 seconds
test_noInputArgs ...................................... passed in 0.119439 seconds
RuntestsTest ............................................. passed in 1.231721 seconds
TestCaseTest
testTestError ......................................... passed in 0.069214 seconds
testTestFailure ....................................... passed in 0.072357 seconds
testFixtureCalls ...................................... passed in 0.060515 seconds
testPassingTests ...................................... passed in 0.066578 seconds
testConstructor ....................................... passed in 0.049125 seconds
TestCaseTest ............................................. passed in 0.337303 seconds
TestCaseWithAddPathTest
testRunTestOnPath ..................................... passed in 0.054453 seconds
testPath .............................................. passed in 0.021226 seconds
TestCaseWithAddPathTest .................................. passed in 0.081960 seconds
TestFuncHandleTests
testTeardownFcnButNoSetupFcn .......................... passed in 0.045909 seconds
testFailingTest ....................................... passed in 0.039228 seconds
testFixtureNoTestData ................................. passed in 0.035223 seconds
testTestFixtureError .................................. passed in 0.045860 seconds
testTestFixture ....................................... passed in 0.036676 seconds
testPassingTests ...................................... passed in 0.033212 seconds
testCaseLocation ...................................... passed in 0.029351 seconds
testCaseNames ......................................... passed in 0.028664 seconds
testOutputs ........................................... passed in 0.030963 seconds
testSuiteNameAndLocation .............................. passed in 0.030237 seconds
TestFuncHandleTests ...................................... passed in 0.361527 seconds
TestRunLoggerTest
testFailingTestCase ................................... passed in 0.061540 seconds
testTwoPassingTests ................................... passed in 0.074688 seconds
TestRunLoggerTest ........................................ passed in 0.144972 seconds
TestSuiteTest
test_fromPwd .......................................... passed in 0.230802 seconds
test_fromName_with_dirname ............................ passed in 0.108591 seconds
test_fromName_with_nonmatching_filter_string .......... passed in 0.052603 seconds
test_fromName_with_filter_string ...................... passed in 0.035147 seconds
test_fromName_bogus_name .............................. passed in 0.026466 seconds
test_fromName_subfunctions ............................ passed in 0.032150 seconds
test_fromName_simpleTest .............................. passed in 0.032005 seconds
test_fromName_notTestCaseSubclass ..................... passed in 0.026601 seconds
test_fromName_TestCaseSubclass ........................ passed in 0.042280 seconds
test_fromTestCaseClassName_badclass ................... passed in 0.037137 seconds
test_fromTestCaseClassName ............................ passed in 0.077249 seconds
testNoTestMethods ..................................... passed in 0.060819 seconds
testCurrentDirectory .................................. passed in 0.037616 seconds
testClassNameIn ....................................... passed in 0.050326 seconds
TestSuiteTest ............................................ passed in 0.875839 seconds
ThrowsExceptionTest
testWrongExceptionTest ................................ passed in 0.071667 seconds
testNoExceptionTest ................................... passed in 0.072957 seconds
testPassingTest ....................................... passed in 0.069514 seconds
ThrowsExceptionTest ...................................... passed in 0.228000 seconds
testAssertEqual
testAssertEqualHappyCase .............................. passed in 0.001714 seconds
testAssertEqualWithThreeInputs ........................ passed in 0.001191 seconds
testAssertEqualHappyCaseString ........................ passed in 0.001191 seconds
testAssertEqualHappyCaseMatrix ........................ passed in 0.009695 seconds
testInfAndInf ......................................... passed in 0.000768 seconds
testMinusInfAndMinusInf ............................... passed in 0.000881 seconds
testOppositeSignInfs .................................. passed in 0.005263 seconds
testFiniteAndInf ...................................... passed in 0.002530 seconds
testFiniteAndNaN ...................................... passed in 0.002448 seconds
testInfiniteAndNaN .................................... passed in 0.002447 seconds
testAssertEqualNotEqual ............................... passed in 0.002487 seconds
testAssertEqualSparsity ............................... passed in 0.003061 seconds
testAssertEqualNans ................................... passed in 0.000814 seconds
testAssertEqualClass .................................. passed in 0.003871 seconds
testAssertEqual .......................................... passed in 0.046598 seconds
testAssertExceptionThrown
test_happyCase ........................................ passed in 0.001343 seconds
test_wrongException ................................... passed in 0.002399 seconds
test_noException ...................................... passed in 0.002300 seconds
testAssertExceptionThrown ................................ passed in 0.007276 seconds
testAssertFalse
testAssertFalseHappyCase .............................. passed in 0.001098 seconds
testAssertFalseHappyCaseWithTwoArgs ................... passed in 0.001255 seconds
testAssertFalseFailed ................................. passed in 0.001569 seconds
testAssertFalseNonscalar .............................. passed in 0.001777 seconds
testAssertFalseNonlogical ............................. passed in 0.001815 seconds
testAssertFalse .......................................... passed in 0.009098 seconds
testAssertTrue
testAssertTrueHappyCase ............................... passed in 0.001224 seconds
testAssertTrueHappyCaseWithTwoArgs .................... passed in 0.001010 seconds
testAssertTrueFailed .................................. passed in 0.001635 seconds
testAssertTrueNonscalar ............................... passed in 0.001855 seconds
testAssertTrueNonlogical .............................. passed in 0.001821 seconds
testAssertTrue ........................................... passed in 0.009276 seconds
testContainsRegexp
testOneStringContains ................................. passed in 0.000726 seconds
testOneStringDoesntContain ............................ passed in 0.000651 seconds
testCellArray ......................................... passed in 0.002125 seconds
testContainsRegexp ....................................... passed in 0.004720 seconds
testIsSetUpString
testOneStringIs ....................................... passed in 0.000966 seconds
testOneStringIsNot .................................... passed in 0.000711 seconds
testCellArray ......................................... passed in 0.000957 seconds
testIsSetUpString ........................................ passed in 0.003831 seconds
testIsTearDownString
testOneStringIs ....................................... passed in 0.000860 seconds
testOneStringIsNot .................................... passed in 0.000734 seconds
testCellArray ......................................... passed in 0.000971 seconds
testIsTearDownString ..................................... passed in 0.003920 seconds
testIsTestCaseSubclass
testTestCase .......................................... passed in 0.001117 seconds
testSubclass .......................................... passed in 0.001433 seconds
testNotASubclass ...................................... passed in 0.000704 seconds
testIsTestCaseSubclass ................................... passed in 0.004479 seconds
testIsTestString
testOneStringIs ....................................... passed in 0.001043 seconds
testOneStringIsNot .................................... passed in 0.000722 seconds
testCellArray ......................................... passed in 0.001105 seconds
testIsTestString ......................................... passed in 0.004039 seconds
testRuntestsWithDirectoryName
testDirName ........................................... passed in 0.186239 seconds
testRuntestsWithDirectoryName ............................ passed in 0.189451 seconds
test_TestSuiteInDir
test_constructor ...................................... passed in 0.021166 seconds
test_gatherTestCases .................................. passed in 0.078672 seconds
test_TestSuiteInDir ...................................... passed in 0.107688 seconds
test_arrayToString
test_smallInput ....................................... passed in 0.004122 seconds
test_largeInput ....................................... passed in 0.012194 seconds
test_emptyInput ....................................... passed in 0.002702 seconds
test_arrayToString ....................................... passed in 0.024989 seconds
test_assertElementsAlmostEqual
test_happyCase ........................................ passed in 0.012472 seconds
test_failedAssertion .................................. passed in 0.005771 seconds
test_nonFloatInputs ................................... passed in 0.004278 seconds
test_sizeMismatch ..................................... passed in 0.004909 seconds
test_finiteAndInfinite ................................ passed in 0.004088 seconds
test_infiniteAndInfinite .............................. passed in 0.002276 seconds
test_finiteAndNaN ..................................... passed in 0.004087 seconds
test_nanAndNaN ........................................ passed in 0.001614 seconds
test_plusMinusInfinity ................................ passed in 0.003298 seconds
test_infiniteAndNaN ................................... passed in 0.003342 seconds
test_assertElementsAlmostEqual ........................... passed in 0.048904 seconds
test_assertFilesEqual
test_equal ............................................ passed in 0.011540 seconds
test_differentSize .................................... passed in 0.004074 seconds
test_sameSizeButDifferent ............................. passed in 0.010435 seconds
test_oneFileEmpty ..................................... passed in 0.003349 seconds
test_bothFilesEmpty ................................... passed in 0.002264 seconds
test_cannotReadFirstFile .............................. passed in 0.001903 seconds
test_cannotReadSecondFile ............................. passed in 0.002844 seconds
test_assertFilesEqual .................................... passed in 0.038591 seconds
test_assertVectorsAlmostEqual
test_happyCase ........................................ passed in 0.011034 seconds
test_failedAssertion .................................. passed in 0.004597 seconds
test_failedAssertionWithCustomMessage ................. passed in 0.004030 seconds
test_nonFloatInputs ................................... passed in 0.002827 seconds
test_sizeMismatch ..................................... passed in 0.002661 seconds
test_finiteAndInfinite ................................ passed in 0.003378 seconds
test_infiniteAndInfinite .............................. passed in 0.003427 seconds
test_finiteAndNaN ..................................... passed in 0.003343 seconds
test_NanAndNan ........................................ passed in 0.003311 seconds
test_oppositeSignInfs ................................. passed in 0.003317 seconds
test_assertVectorsAlmostEqual ............................ passed in 0.044592 seconds
test_compareFloats
test_elementwiseRelativeTolerance ..................... passed in 0.004797 seconds
test_elementwiseAbsoluteTolerance ..................... passed in 0.005753 seconds
test_vectorRelativeTolerance .......................... passed in 0.002094 seconds
test_vectorAbsoluteTolerance .......................... passed in 0.001898 seconds
test_NaNs ............................................. passed in 0.004481 seconds
test_Infs ............................................. passed in 0.004494 seconds
test_complexInput ..................................... passed in 0.006239 seconds
test_comparisonTypeSpecified .......................... passed in 0.002089 seconds
test_compareFloats ....................................... passed in 0.034187 seconds
test_comparisonMessage
test_happyCase ........................................ passed in 0.003328 seconds
test_comparisonMessage ................................... passed in 0.004423 seconds
test_packageName
test_happyCase ........................................ passed in 0.038309 seconds
test_badPackageName ................................... passed in 0.001387 seconds
test_packageName ......................................... passed in 0.040861 seconds
test_parseFloatAssertInputs
test_tooFewInputs ..................................... passed in 0.001230 seconds
test_tooManyInputs .................................... passed in 0.001139 seconds
test_twoInputs ........................................ passed in 0.001647 seconds
test_threeInputs ...................................... passed in 0.002824 seconds
test_fourInputs ....................................... passed in 0.006563 seconds
test_fiveInputs ....................................... passed in 0.004510 seconds
test_sixInputs ........................................ passed in 0.003339 seconds
test_twoSingleInputs .................................. passed in 0.002287 seconds
test_twoSingleAndDoubleInputs ......................... passed in 0.002257 seconds
test_parseFloatAssertInputs .............................. passed in 0.028289 seconds
test_stringToCellArray
test_happyCase ........................................ passed in 0.000890 seconds
test_emptyInput ....................................... passed in 0.000639 seconds
test_spacesInFront .................................... passed in 0.000856 seconds
test_spacesAtEnd ...................................... passed in 0.000833 seconds
test_stringToCellArray ................................... passed in 0.004602 seconds
tests ....................................................... passed in 4.180996 seconds
Comments
To leave a comment, please click here to sign in to your MathWorks Account or create a new one.