{"id":9714,"date":"2018-05-11T09:00:57","date_gmt":"2018-05-11T13:00:57","guid":{"rendered":"https:\/\/blogs.mathworks.com\/pick\/?p=9714"},"modified":"2018-05-11T15:46:38","modified_gmt":"2018-05-11T19:46:38","slug":"timing-experiments","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/pick\/2018\/05\/11\/timing-experiments\/","title":{"rendered":"Timing Experiments"},"content":{"rendered":"<div xmlns:mwsh=\"https:\/\/www.mathworks.com\/\/namespace\/mcode\/v1\/syntaxhighlight.dtd\" class=\"content\">\n   <introduction><\/p>\n<p><a href=\"https:\/\/www.mathworks.com\/\/matlabcentral\/profile\/authors\/3208495\">Sean<\/a>&#8216;s pick this week is to revisit <a href=\"https:\/\/blogs.mathworks.com\/pick\/2018\/05\/04\/persistent-data-for-lookup-tables-in-simulink\/\">Richard&#8217;s pick from last week<\/a> and focus on the timing experiments.\n      <\/p>\n<p>   <\/introduction><\/p>\n<p>Today, I&#8217;m going to look at timing MATLAB code using the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/performance-testing-framework.html\">Performance Testing Framework<\/a>.  Last week, Richard, had a few different experiments he wanted to run and time.  Specifically, he also wanted to see how the first run compared to the Nth run.\n   <\/p>\n<p>The performance testing framework makes it very easy to do this.  It leverages the same infrastructure as the unit test framework, but allows you to time parts of a test and run the tests either a fixed number of times or until the time of the Nth run converges.\n   <\/p>\n<p>Here I&#8217;ll take the algorithmic pieces of Richard&#8217;s code and put them into their own tests.  There are two test classes which inherit from <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.perftest.testcase-class.html\">matlab.perftest.TestCase<\/a>.  The classes are both in a +Test package which allows me to automatically create a suite of all tests.\n   <\/p>\n<p>The framework makes it easy to <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/create-basic-parameterized-test.html\">parameterize tests<\/a> so I can exercise both models under both run modes just by passing in the parameters.\n   <\/p>\n<p>Here are the two classes:<\/p>\n<pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">\r\n<span style=\"color: #0000FF\">classdef<\/span> WithWithoutPersistence &lt; matlab.perftest.TestCase\r\n<span style=\"color: #228B22\">% Test with and without persistent data under normal and accelerator<\/span>\r\n\r\n    <span style=\"color: #0000FF\">properties<\/span> (ClassSetupParameter)\r\n        ModelName = {<span style=\"color: #A020F0\">'withoutPersistentLoading'<\/span>, <span style=\"color: #A020F0\">'withPersistentLoading'<\/span>}; <span style=\"color: #228B22\">% Both models<\/span>\r\n        RunMode = {<span style=\"color: #A020F0\">'Normal'<\/span>, <span style=\"color: #A020F0\">'Accelerator'<\/span>}; <span style=\"color: #228B22\">% Both modes<\/span>\r\n    <span style=\"color: #0000FF\">end<\/span>\r\n\r\n    <span style=\"color: #0000FF\">methods<\/span> (TestClassSetup)\r\n        <span style=\"color: #228B22\">% Run before testing starts<\/span>\r\n        <span style=\"color: #0000FF\">function<\/span> startSimulink(~)\r\n            <span style=\"color: #228B22\">% Start Simulink to make sure initial time isn't included.<\/span>\r\n            start_simulink            \r\n        <span style=\"color: #0000FF\">end<\/span>\r\n        <span style=\"color: #0000FF\">function<\/span> loadModel(testCase, ModelName, RunMode)\r\n            <span style=\"color: #228B22\">% Take RunMode to force it to close and reload between modes<\/span>\r\n            load_system(ModelName);\r\n            testCase.addTeardown(@()close_system(ModelName));\r\n        <span style=\"color: #0000FF\">end<\/span>        \r\n    <span style=\"color: #0000FF\">end<\/span>\r\n    \r\n    <span style=\"color: #228B22\">% Test methods<\/span>\r\n    <span style=\"color: #0000FF\">methods<\/span> (Test)       \r\n        <span style=\"color: #0000FF\">function<\/span> testSimulation(~, ModelName, RunMode)\r\n            <span style=\"color: #228B22\">% Simulate<\/span>\r\n            sim(ModelName, <span style=\"color: #A020F0\">'SimulationMode'<\/span>, RunMode);\r\n        <span style=\"color: #0000FF\">end<\/span>\r\n    <span style=\"color: #0000FF\">end<\/span>\r\n<span style=\"color: #0000FF\">end<\/span>\r\n<\/pre>\n<p>And:<\/p>\n<pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">\r\n<span style=\"color: #0000FF\">classdef<\/span> FastRestart &lt; matlab.perftest.TestCase\r\n<span style=\"color: #228B22\">% Test fast restart<\/span>\r\n    \r\n    <span style=\"color: #0000FF\">methods<\/span> (TestClassSetup)\r\n        <span style=\"color: #0000FF\">function<\/span> startSimulink(~)\r\n            <span style=\"color: #228B22\">% Start Simulink to make sure initial time isn't included.<\/span>\r\n            start_simulink\r\n            \r\n            load_system(<span style=\"color: #A020F0\">'withoutPersistentLoading'<\/span>);\r\n        <span style=\"color: #0000FF\">end<\/span>\r\n    <span style=\"color: #0000FF\">end<\/span>\r\n    \r\n    <span style=\"color: #0000FF\">methods<\/span> (TestClassTeardown)\r\n        <span style=\"color: #0000FF\">function<\/span> restoreModel(~)\r\n            <span style=\"color: #228B22\">% Restore state<\/span>\r\n            set_param(<span style=\"color: #A020F0\">'withoutPersistentLoading'<\/span>, <span style=\"color: #A020F0\">'FastRestart'<\/span>, <span style=\"color: #A020F0\">'off'<\/span>);\r\n            close_system(<span style=\"color: #A020F0\">'withoutPersistentLoading'<\/span>);\r\n        <span style=\"color: #0000FF\">end<\/span>\r\n    <span style=\"color: #0000FF\">end<\/span>\r\n    \r\n    <span style=\"color: #0000FF\">methods<\/span> (Test)        \r\n        <span style=\"color: #0000FF\">function<\/span> testFastRestart(~)\r\n            <span style=\"color: #228B22\">% Simulate<\/span>\r\n            sim(<span style=\"color: #A020F0\">'withoutPersistentLoading'<\/span>, <span style=\"color: #A020F0\">'FastRestart'<\/span>, <span style=\"color: #A020F0\">'on'<\/span>);\r\n        <span style=\"color: #0000FF\">end<\/span>\r\n    <span style=\"color: #0000FF\">end<\/span>\r\n<span style=\"color: #0000FF\">end<\/span>\r\n<\/pre>\n<p>From there, we&#8217;ll create the test suite and create a fixed time experiment to run one warmup and 10 samples.<\/p>\n<pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">import <span style=\"color: #A020F0\">matlab.unittest.TestSuite<\/span>\r\nimport <span style=\"color: #A020F0\">matlab.perftest.FixedTimeExperiment<\/span>\r\nsuite = TestSuite.fromPackage(<span style=\"color: #A020F0\">'Test'<\/span>, <span style=\"color: #A020F0\">'IncludingSubpackages'<\/span>, true);\r\nfte = FixedTimeExperiment.withFixedSampleSize(10, <span style=\"color: #A020F0\">'NumWarmups'<\/span>, 1);\r\nresults = run(fte, suite);\r\ntestactivity = vertcat(results.TestActivity);\r\ndisp(testactivity)<\/pre>\n<pre style=\"font-style:oblique\">Running Test.FastRestart\r\n..........\r\n.\r\nDone Test.FastRestart\r\n__________\r\n\r\nRunning Test.WithWithoutPersistence\r\n..........\r\n..........\r\n..........\r\n..........\r\n....\r\nDone Test.WithWithoutPersistence\r\n__________\r\n\r\n                                                   Name                                                   Passed    Failed    Incomplete    MeasuredTime    Objective         Timestamp             Host        Platform           Version                      TestResult                          RunIdentifier            \r\n    __________________________________________________________________________________________________    ______    ______    __________    ____________    _________    ____________________    ___________    ________    _____________________    ________________________________    ____________________________________\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false           3.6662       warmup      10-May-2018 19:50:01    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.022907       sample      10-May-2018 19:50:01    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.023864       sample      10-May-2018 19:50:01    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.034052       sample      10-May-2018 19:50:02    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.023421       sample      10-May-2018 19:50:02    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.035024       sample      10-May-2018 19:50:02    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.024277       sample      10-May-2018 19:50:02    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.021401       sample      10-May-2018 19:50:02    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.021795       sample      10-May-2018 19:50:02    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.021364       sample      10-May-2018 19:50:02    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.FastRestart\/testFastRestart                                                                      true      false       false         0.024193       sample      10-May-2018 19:50:02    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false            4.271       warmup      10-May-2018 19:50:06    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.9064       sample      10-May-2018 19:50:10    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.8215       sample      10-May-2018 19:50:14    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.9881       sample      10-May-2018 19:50:18    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false            3.983       sample      10-May-2018 19:50:22    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.7034       sample      10-May-2018 19:50:26    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.8451       sample      10-May-2018 19:50:30    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.7176       sample      10-May-2018 19:50:33    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.7336       sample      10-May-2018 19:50:37    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.8346       sample      10-May-2018 19:50:41    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation         true      false       false           3.7633       sample      10-May-2018 19:50:45    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.9732       warmup      10-May-2018 19:50:49    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.8535       sample      10-May-2018 19:50:53    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.8244       sample      10-May-2018 19:50:57    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.7287       sample      10-May-2018 19:51:00    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.8755       sample      10-May-2018 19:51:04    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.7872       sample      10-May-2018 19:51:08    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.7933       sample      10-May-2018 19:51:12    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.8545       sample      10-May-2018 19:51:16    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.9182       sample      10-May-2018 19:51:20    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.7648       sample      10-May-2018 19:51:24    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation    true      false       false           3.7802       sample      10-May-2018 19:51:27    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false           3.6732       warmup      10-May-2018 19:51:31    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false          0.18088       sample      10-May-2018 19:51:31    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false          0.07304       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false         0.065155       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false         0.064901       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false          0.06622       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false         0.064989       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false         0.064424       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false         0.064974       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false         0.063272       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation            true      false       false         0.067388       sample      10-May-2018 19:51:32    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false           3.7391       warmup      10-May-2018 19:51:36    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.25133       sample      10-May-2018 19:51:36    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.11322       sample      10-May-2018 19:51:36    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.11479       sample      10-May-2018 19:51:37    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.14222       sample      10-May-2018 19:51:37    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.12787       sample      10-May-2018 19:51:37    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.12439       sample      10-May-2018 19:51:37    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.12707       sample      10-May-2018 19:51:37    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false           0.1336       sample      10-May-2018 19:51:37    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.11642       sample      10-May-2018 19:51:37    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation       true      false       false          0.12538       sample      10-May-2018 19:51:38    ah-sdewolsk     win64      9.4.0.813654 (R2018a)    [1x1 matlab.unittest.TestResult]    cc5b36fc-7c9f-4f09-9054-c60025f31196\r\n<\/pre>\n<p>Now we can either use <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/varfun.html\"><tt>varfun<\/tt><\/a> to analyze the groups or push this to Excel to be stored or pivoted out.\n   <\/p>\n<pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">mt = varfun(@median, testactivity, <span style=\"color: #0000FF\">...<\/span>\r\n    <span style=\"color: #A020F0\">'GroupingVariables'<\/span>, {<span style=\"color: #A020F0\">'Name'<\/span>, <span style=\"color: #A020F0\">'Objective'<\/span>}, <span style=\"color: #0000FF\">...<\/span>\r\n    <span style=\"color: #A020F0\">'InputVariables'<\/span>, {<span style=\"color: #A020F0\">'MeasuredTime'<\/span>})<\/pre>\n<pre style=\"font-style:oblique\">mt =\r\n  10&times;4 table\r\n                                                   Name                                                   Objective    GroupCount    median_MeasuredTime\r\n    __________________________________________________________________________________________________    _________    __________    ___________________\r\n    Test.FastRestart\/testFastRestart                                                                       sample          10             0.023642      \r\n    Test.FastRestart\/testFastRestart                                                                       warmup           1               3.6662      \r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation          sample          10                3.828      \r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Normal]\/testSimulation          warmup           1                4.271      \r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation     sample          10               3.8088      \r\n    Test.WithWithoutPersistence[ModelName=withoutPersistentLoading,RunMode=Accelerator]\/testSimulation     warmup           1               3.9732      \r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation             sample          10             0.065072      \r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Normal]\/testSimulation             warmup           1               3.6732      \r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation        sample          10              0.12623      \r\n    Test.WithWithoutPersistence[ModelName=withPersistentLoading,RunMode=Accelerator]\/testSimulation        warmup           1               3.7391      \r\n<\/pre>\n<p>Or:<\/p>\n<pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">writetable(testactivity, <span style=\"color: #A020F0\">'TestActivity.xlsx'<\/span>)<\/pre>\n<p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/pick\/Sean\/main_timingPDIS\/testactivitypivot.png\"> <\/p>\n<h3>Comments<a name=\"5\"><\/a><\/h3>\n<p>Do you have a use for repetitively timing MATLAB code or Simulink models?<\/p>\n<p>Give it a try and let us know what you think <a href=\"https:\/\/blogs.mathworks.com\/pick\/?p=9714#respond\">here<\/a>.\n   <\/p>\n<p><script language=\"JavaScript\">\n<!--\n\n    function grabCode_6a48d04718574518aa1e3a754edbf772() {\n        \/\/ Remember the title so we can use it in the new page\n        title = document.title;\n\n        \/\/ Break up these strings so that their presence\n        \/\/ in the Javascript doesn't mess up the search for\n        \/\/ the MATLAB code.\n        t1='6a48d04718574518aa1e3a754edbf772 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 6a48d04718574518aa1e3a754edbf772';\n    \n        b=document.getElementsByTagName('body')[0];\n        i1=b.innerHTML.indexOf(t1)+t1.length;\n        i2=b.innerHTML.indexOf(t2);\n \n        code_string = b.innerHTML.substring(i1, i2);\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\n\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \n        \/\/ in the XML parser.\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\n        \/\/ doesn't go ahead and substitute the less-than character. \n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\n\n        author = 'Sean de Wolski';\n        copyright = 'Copyright 2018 The MathWorks, Inc.';\n\n        w = window.open();\n        d = w.document;\n        d.write('\n\n<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add author and copyright lines at the bottom if specified.\r\n        if ((author.length > 0) || (copyright.length > 0)) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (author.length > 0) {\r\n                d.writeln('% _' + author + '_');\r\n            }\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\n\n\\n');\n      \n      d.title = title + ' (MATLAB code)';\n      d.close();\n      }   \n      \n-->\n<\/script><\/p>\n<p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><a href=\"javascript:grabCode_6a48d04718574518aa1e3a754edbf772()\"><span style=\"font-size: x-small;        font-style: italic;\">Get<br \/>\n            the MATLAB code<br \/>\n            <noscript>(requires JavaScript)<\/noscript><\/span><\/a><\/p>\n<p>      Published with MATLAB&reg; R2018a<\/p>\n<\/div>\n<p><!--\n6a48d04718574518aa1e3a754edbf772 ##### SOURCE BEGIN #####\n%% Timing Experiments\n%\n% <https:\/\/www.mathworks.com\/\/matlabcentral\/profile\/authors\/3208495 Sean>'s\n% pick this week is to revisit\n% <https:\/\/blogs.mathworks.com\/pick\/2018\/05\/04\/persistent-data-for-lookup-tables-in-simulink\/\n% Richard's pick from last week> and focus on the timing experiments.\n% \n\n%% \n%\n% Today, I'm going to look at timing MATLAB code using the\n% <https:\/\/www.mathworks.com\/help\/matlab\/performance-testing-framework.html\n% Performance Testing Framework>.  Last week, Richard, had a few different\n% experiments he wanted to run and time.  Specifically, he also wanted to\n% see how the first run compared to the Nth run.  \n%\n% The performance testing framework makes it very easy to do this.  It\n% leverages the same infrastructure as the unit test framework, but allows\n% you to time parts of a test and run the tests either a fixed number of\n% times or until the time of the Nth run converges.\n%\n% Here I'll take the algorithmic pieces of Richard's code and put them into\n% their own tests.  There are two test classes which inherit from\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.perftest.testcase-class.html\n% matlab.perftest.TestCase>.  The classes are both in a +Test package which\n% allows me to automatically create a suite of all tests.\n%\n% The framework makes it easy to\n% <https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/create-basic-parameterized-test.html\n% parameterize tests> so I can exercise both models under both run modes\n% just by passing in the parameters.\n%\n% Here are the two classes:\n%\n% <include>.\/+Test\/WithWithoutPersistence.m<\/include>\n%\n% And:\n%\n% <include>.\/+Test\/FastRestart.m<\/include>\n%\n% From there, we'll create the test suite and create a fixed time experiment\n% to run one warmup and 10 samples.\n\nimport matlab.unittest.TestSuite\nimport matlab.perftest.FixedTimeExperiment\nsuite = TestSuite.fromPackage('Test', 'IncludingSubpackages', true);\nfte = FixedTimeExperiment.withFixedSampleSize(10, 'NumWarmups', 1);\nresults = run(fte, suite);\ntestactivity = vertcat(results.TestActivity);\ndisp(testactivity)\n\n%%\n% Now we can either use\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/varfun.html |varfun|> to\n% analyze the groups or push this to Excel to be stored or pivoted out.\nmt = varfun(@median, testactivity, ...\n    'GroupingVariables', {'Name', 'Objective'}, ...\n    'InputVariables', {'MeasuredTime'})\n\n%%\n% Or:\nwritetable(testactivity, 'TestActivity.xlsx')\n\n%%\n%\n% <<testactivitypivot.png>>\n%\n\n%% Comments\n% Do you have a use for repetitively timing MATLAB code or Simulink models?\n%\n% Give it a try and let us know what you think\n% <https:\/\/blogs.mathworks.com\/pick\/?p=9714#respond here>.\n%\n \n\n##### SOURCE END ##### 6a48d04718574518aa1e3a754edbf772\n--><\/p>\n","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/images\/pick\/Sean\/main_timingPDIS\/testactivitypivot.png\" onError=\"this.style.display ='none';\" \/><\/div>\n<p>Sean&#8216;s pick this week is to revisit Richard&#8217;s pick from last week and focus on the timing experiments.<\/p>\n<p>Today, I&#8217;m going to look at timing MATLAB code using the&#8230; <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/pick\/2018\/05\/11\/timing-experiments\/\">read more >><\/a><\/p>\n","protected":false},"author":87,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[16],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/9714"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/users\/87"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/comments?post=9714"}],"version-history":[{"count":3,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/9714\/revisions"}],"predecessor-version":[{"id":9720,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/9714\/revisions\/9720"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/media?parent=9714"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/categories?post=9714"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/tags?post=9714"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}