{"id":4040,"date":"2025-08-20T14:21:20","date_gmt":"2025-08-20T18:21:20","guid":{"rendered":"https:\/\/blogs.mathworks.com\/developer\/?p=4040"},"modified":"2025-08-20T14:21:20","modified_gmt":"2025-08-20T18:21:20","slug":"tldr-too-long-didnt-run-part-3-incremental-testing-with-the-matlab-build-tool","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/developer\/2025\/08\/20\/tldr-too-long-didnt-run-part-3-incremental-testing-with-the-matlab-build-tool\/","title":{"rendered":"TL;DR: Too Long; Didn\u2019t Run: Part 3 \u2013 Incremental Testing with the MATLAB Build Tool"},"content":{"rendered":"<div class=\"content\">\r\n\r\n<!--introduction-->\r\n\r\n<p><em>Alright! I'm back after a mid-series break to continue our talk about incremental testing.<\/em><\/p>\r\n<p>In parts <a href=\"https:\/\/blogs.mathworks.com\/developer\/2025\/07\/09\/test-impact-analysis-find-tests\/\">1<\/a> and <a href=\"https:\/\/blogs.mathworks.com\/developer\/2025\/07\/23\/test-impact-analysis-test-manager\/\">2<\/a>, we looked at interactive development workflows to help run the right set of tests at the right time. In this post, we'll look at how that same idea extends to automated builds using the MATLAB build tool, and specifically, how incremental testing enables faster local prequalification. The motivation remains the same \u2013 to enable frequent, high-quality integrations verified by automated builds, without incurring the overhead of running the full regression suite after every incremental change.<\/p>\r\n\r\n<p>Running all the tests might be manageable at small scale but it quickly becomes prohibitively expensive for larger projects.<\/p>\r\n<div style=\"border-left: 4px solid #007acc; background-color: #f0f8ff; padding: 12px 16px; margin: 16px 0; font-family: sans-serif;\">This post assumes that you are somewhat familiar with the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/overview-of-matlab-build-tool.html\">MATLAB build tool<\/a>. It provides a uniform and standardized way to define and run MATLAB builds.<\/div>\r\n<!--\/introduction-->\r\n<h3>Contents<\/h3>\r\n<div>\r\n<ul>\r\n \t<li><a href=\"#c1d7c55e-08fe-4d91-a283-e617f1b26733\">What Is Incremental Testing?<\/a><\/li>\r\n \t<li><a href=\"#3a5dd755-05e9-4b4b-b978-c23d6a17fa7c\">Background<\/a><\/li>\r\n \t<li><a href=\"#7ffca756-1a67-45f8-b564-b93ae037796a\">Defining the Build<\/a><\/li>\r\n \t<li><a href=\"#2b1b0609-b83c-454d-b79d-eb8dc4d17c05\">Optimizing Performance with Incremental Build<\/a><\/li>\r\n \t<li><a href=\"#64a78dc2-ff90-47a3-85ed-e66556c0c7a9\">Enabling Incremental Testing<\/a><\/li>\r\n \t<li><a href=\"#6581c44f-ab93-44a3-ad8c-219d842654b7\">A Closer Look at Impact-Based Testing<\/a><\/li>\r\n \t<li><a href=\"#ea885be6-ecda-484d-97b2-5679a814a414\">Interaction Between Incremental Build and Incremental Testing<\/a><\/li>\r\n \t<li><a href=\"#6340e957-281e-41d0-863d-f33c9a483ed6\">Summary<\/a><\/li>\r\n<\/ul>\r\n<\/div>\r\n<h2>What Is Incremental Testing?<a name=\"c1d7c55e-08fe-4d91-a283-e617f1b26733\"><\/a><\/h2>\r\n<p>The MATLAB build tool supports <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/improve-performance-with-incremental-builds.html\">incremental builds<\/a>, which skips unnecessary work by tracking each build task's inputs, outputs, actions and arguments to determine if the task is up-to-date. If nothing has changed since its last successful run, the task is skipped.<\/p>\r\n\r\n<p>Starting in <strong>R2025a<\/strong>, the build tool enhances this capability with incremental testing, available with MATLAB Test. While incremental build determines whether a task should rerun, incremental testing goes further \u2013 it uses test impact analysis to decide which tests need to run. This can significantly optimize test execution and speed up your builds.<\/p>\r\n\r\n<p>With incremental testing enabled, the build tool analyzes the source and test code changes and runs only tests impacted by those changes.<\/p>\r\n\r\n<p>Let's unpack how this works. We'll start with some basics.<\/p>\r\n<h2>Background<a name=\"3a5dd755-05e9-4b4b-b978-c23d6a17fa7c\"><\/a><\/h2>\r\n<p>We'll use the same goal as in <a href=\"https:\/\/blogs.mathworks.com\/developer\/2025\/07\/23\/test-impact-analysis-test-manager\/\">Part 2<\/a> \u2013 enhancing the AND perceptron to introduce a trainable bias term. The exact goal isn't critical; it's just a useful backdrop.<\/p>\r\n<div style=\"border-left: 4px solid #007acc; background-color: #f0f8ff; padding: 12px 16px; margin: 16px 0; font-family: sans-serif;\"><strong>Tip:<\/strong> You can access the supporting files for this post in the <a href=\"https:\/\/github.com\/mathworks\/developer-zone-blog\/tree\/2025-Test-Impact-Analysis-Build-Tool\">Developer Zone blog GitHub repository<\/a>. The <a href=\"https:\/\/github.com\/mathworks\/developer-zone-blog\/tree\/2025-Test-Impact-Analysis-Build-Tool\/before\">\"before\"<\/a> folder contains the original code to reproduce the steps outlined in this blog, while the <a href=\"https:\/\/github.com\/mathworks\/developer-zone-blog\/tree\/2025-Test-Impact-Analysis-Build-Tool\/after\">\"after\"<\/a> folder includes the final version of the code.<\/div>\r\n<h3>Defining the Build<a name=\"7ffca756-1a67-45f8-b564-b93ae037796a\"><\/a><\/h3>\r\n<p>A <tt>buildfile.m<\/tt> at the project root defines your MATLAB build.<\/p>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/buildfile_location.png\" alt=\"\" width=\"438\" height=\"170\" hspace=\"5\" vspace=\"5\" \/>\r\n<p>This build file creates a plan that describes how to build your project \u2013 it defines a set of build tasks and their dependencies to establish a task graph to execute with a single <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/buildtool.html\">buildtool<\/a> command.<\/p>\r\n<pre class=\"language-matlab\"><span class=\"keyword\">function<\/span> plan = buildfile \r\nimport <span class=\"string\">matlab.buildtool.tasks.*\r\n\r\n<\/span>plan = buildplan(localfunctions);\r\n\r\nplan(<span class=\"string\">\"clean\"<\/span>) = CleanTask;\r\nplan(<span class=\"string\">\"test\"<\/span>) = TestTask(<span class=\"string\">\"tests\"<\/span>, SourceFiles=<span class=\"string\">\"toolbox\"<\/span>);\r\n\r\nplan.DefaultTasks=<span class=\"string\">\"test\"<\/span>;\r\n\r\n<span class=\"keyword\">end<\/span>\r\n<\/pre>\r\n<p>In this example, the build file includes:<\/p>\r\n<ul>\r\n \t<li>A <tt>test<\/tt> task to run tests.<\/li>\r\n \t<li>A <tt>clean<\/tt> task to delete task outputs and traces.<\/li>\r\n<\/ul>\r\n<p>These tasks are created using the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.buildtool.tasks-package.html\">built-in task classes<\/a>. Built-in task classes simplify defining common tasks like identifying code issues, building MEX binaries and testing. These classes are designed for reuse and generally support <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/improve-performance-with-incremental-builds.html\">incremental build<\/a> out of the box.<\/p>\r\n\r\n<p>Our <tt>test<\/tt> task uses the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.buildtool.tasks.testtask-class.html\">matlab.buildtool.tasks.TestTask<\/a> class, which runs tests using the MATLAB Unit Testing Framework.<\/p>\r\n\r\n<h3>Optimizing Performance with Incremental Build<a name=\"2b1b0609-b83c-454d-b79d-eb8dc4d17c05\"><\/a><\/h3>\r\n<p>Before diving into incremental testing, let's briefly look at how incremental build works.<\/p>\r\n\r\n<p>When we first clone our repository, the build tool has no task traces.<\/p>\r\n<div style=\"border-left: 4px solid #007acc; background-color: #f0f8ff; padding: 12px 16px; margin: 16px 0; font-family: sans-serif;\">A <b>task trace<\/b> is a MATLAB release-specific record of a task's inputs, outputs, actions and arguments from its last successful run.<\/div>\r\n<\/div>\r\n<p>So, when we run the <tt>test<\/tt> task, it runs because there are no traces yet, and runs <em>all<\/em> the tests defined by the task.<\/p>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/run_with_no_task_trace.png\" alt=\"\" width=\"301\" height=\"64\" hspace=\"5\" vspace=\"5\" \/>\r\n<p>...<\/p>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/test_run_summary_no_task_trace.png\" alt=\"\" width=\"438\" height=\"170\" hspace=\"5\" vspace=\"5\" \/>\r\n<br><br>\r\n<p>This is nice because it gives us confidence that the repository is in a known-good state before making any code changes.<\/p>\r\n\r\n<p>The build tool creates a <tt>.buildtool<\/tt> cache folder in the project root folder to store the task traces.<\/p>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/task_trace_folder.png\" alt=\"\" width=\"549\" height=\"251\" hspace=\"5\" vspace=\"5\" \/>\r\n<br><br>\r\n<p>On subsequent builds, if nothing has changed, the task is skipped.<\/p>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/skipped_task.png\" alt=\"\" width=\"294\" height=\"82\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<br><br>\r\n<p>Now, let's say we modify a source file like <tt>initializeWeights.m<\/tt>, and re-run the build.<\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/sample_change_source_control.png\" alt=\"\" width=\"549\" height=\"251\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<br><br>\r\n<p>Since this file is tracked via the <tt>SourceFiles<\/tt> property, one of the task inputs of the test task that the build tool tracks as part of <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.buildtool.tasks.testtask-class.html#mw_96cca6ff-6c81-4dd9-87d2-5c7df47b475b\">up-to-date check<\/a>, the tool recognizes the changes and re-runs the task.<\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/rerun_after_sample_change.png\" alt=\"\" width=\"410\" height=\"244\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<br><br>\r\n<p>Nice! But do we really need to run all the tests for that single source file change? Wouldn't it be better if the <tt>test<\/tt>\u00a0task ran just the impacted tests. That's exactly what incremental testing in R2025a delivers.<\/p>\r\n<h2>Enabling Incremental Testing<a name=\"64a78dc2-ff90-47a3-85ed-e66556c0c7a9\"><\/a><\/h2>\r\n<p>So, how do we get the incremental testing behavior?<\/p>\r\n\r\n<p>Incremental testing is powered by test impact analysis. To enable it, you can simply run the <tt>test<\/tt> task with the <tt>RunOnlyImpactedTests<\/tt> task argument.<\/p>\r\n<pre class=\"language-matlab\">&gt;&gt; buildtool test(RunOnlyImpactedTests=1)\r\n<\/pre>\r\n<p>Let's look at an example. Say we change <tt>initializeWeights.m<\/tt> and <tt>calculateWeightedSum.m<\/tt> as part of our perceptron enhancement, like we did in Part 2:<\/p>\r\n<b>initializeWeights.m<\/b>\r\n<p><u>Before:<\/u><\/p>\r\n<pre class=\"language-matlab\"><span class=\"keyword\">function <\/span>weights = initializeWeights()\r\n<span class=\"comment\">% Initialize weights randomly for two inputs and one bias\r\n<\/span>weights = rand(1,3) * 0.5;\r\n<span class=\"keyword\">end<\/span>\r\n<\/pre>\r\n<p><u>After:<\/u><\/p>\r\n<pre class=\"language-matlab\"><span class=\"keyword\">function <\/span>[weights, bias] = initializeWeights(numInputs)\r\n<span class=\"comment\">% Initialize weights and bias randomly\r\n<\/span>weights = rand(1, numInputs); <span class=\"comment\">% Random weights for each input\r\n<\/span>bias = rand() - 0.5; <span class=\"comment\">% Bias initialized to a small random value between -0.5 and 0.5\r\n<\/span><span class=\"keyword\">end<\/span>\r\n<\/pre>\r\n<p><b>calculateWeightedSum.m<\/b><\/p>\r\n<p><u>Before:<\/u><\/p>\r\n<pre class=\"language-matlab\"><span class=\"keyword\">function <\/span>output = calculateWeightedSum(weights, inputs)\r\n<span class=\"comment\">% Compute the weighted sum\r\n<\/span>total_input = sum(weights .* [inputs, 1]); \r\n<span class=\"comment\">\r\n% Activation logic\r\n<\/span><span class=\"keyword\">if <\/span>total_input &gt; 0 \r\n<span class=\"comment\">% step function threshold; activate if weighted sum is positive\r\n<\/span>   output = 1;\r\n<span class=\"keyword\">else\r\n<\/span>   output = 0;\r\n<span class=\"keyword\">end\r\n<\/span><span class=\"keyword\">end<\/span>\r\n<\/pre>\r\n<p><u>After:<\/u><\/p>\r\n<pre class=\"language-matlab\"><span class=\"keyword\">function <\/span>output = calculateWeightedSum(weights, bias, inputs)\r\n<span class=\"comment\">% Compute the weighted sum\r\n<\/span>total_input = sum(weights .* inputs) + bias;\r\n\r\n<span class=\"comment\">% Activation logic\r\n<\/span><span class=\"keyword\">if <\/span>total_input &gt; 0\r\n<span class=\"comment\">% step function threshold; activate if weighted sum is positive\r\n<\/span>   output = 1;\r\n<span class=\"keyword\">else\r\n<\/span>   output = 0;\r\n<span class=\"keyword\">end\r\n<\/span><span class=\"keyword\">end<\/span>\r\n<\/pre>\r\n<p>When you set <tt>RunOnlyImpactedTests<\/tt> to <tt>true<\/tt>, the <tt>test<\/tt> task analyzes those changes and runs only the impacted tests. Look at that!<\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/run_impacted_tests.png\" alt=\"\" width=\"712\" height=\"221\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<br><br>\r\n<p>Behind the scenes, the build tool tracks changes to the source and test files of the <tt>test<\/tt> task created using the the built-in <tt>TestTask<\/tt> class (more on this later in the post). When <tt>RunOnlyImpactedTests<\/tt> is true, the <tt>test<\/tt> task doesn't just detect that the files changed \u2013 it determines which tests are impacted by those changes and runs only those.<\/p>\r\n\r\n<p>Increasing the build verbosity to level 3 (<tt>Detailed<\/tt>) or higher reveals which changes are causing the task to rerun. In our example, we see that the <tt>SourceFiles<\/tt> property has changed because <tt>initializeWeights.m<\/tt> and <tt>calculateWeightedSum.m<\/tt> were modified.<\/p>\r\n\r\n<p>As seen in <a href=\"https:\/\/blogs.mathworks.com\/developer\/2025\/07\/23\/test-impact-analysis-test-manager\/\">Part 2<\/a>, changes to just those two files do not meet the goal. The diagnostics will reflect this.<\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/run_impacted_tests_failures.png\" alt=\"\" width=\"585\" height=\"202\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<p>Let's fix the rest of the code base like we did in Part 2 to ensure we have all the necessary changes and rerun the build with incremental testing.<\/p>\r\n\r\n<p><i>After some coding and a cup of coffee...<\/i><\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/rerun_impacted_tests.png\" alt=\"\" width=\"779\" height=\"147\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<br><br>\r\n<p>Success! The build passes and we ran just the related tests.<\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/run_impacted_tests_success.png\" alt=\"\" width=\"483\" height=\"169\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<br><br>\r\n<p>Our toy example had just 5 tests, but real-world projects can have hundreds or even thousands \u2013 so the potential for performance gain is substantial.<\/p>\r\n\r\n<p>To avoid the need to specify the task argument every time, we can configure the test task at plan-creation time in your <tt>buildfile.m<\/tt> to always run only impacted tests by default.<\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/buildfile_roit.png\" alt=\"\" width=\"648\" height=\"239\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<br><br>\r\n<p>Then at run time, we can specify just \"test\".<\/p>\r\n<pre class=\"language-matlab\">&gt;&gt; buildtool test\r\n<\/pre>\r\n<p>You can toggle the <tt>RunOnlyImpactedTests<\/tt> option at run time or create separate test tasks with and without incremental testing to match your workflow preferences.<\/p>\r\n<h2>A Closer Look at Impact-Based Testing<a name=\"6581c44f-ab93-44a3-ad8c-219d842654b7\"><\/a><\/h2>\r\n<p>Impact-based testing involves two key concepts:<\/p>\r\n<div>\r\n<ul>\r\n \t<li><strong>Change Detection<\/strong> refers exclusively to the concern of detecting changes to the task's tracked files<\/li>\r\n \t<li><strong>Impact Analysis<\/strong> refers exclusively to the concern of analyzing the impact of those changes.<\/li>\r\n<\/ul>\r\n<\/div>\r\n<h3>Change Detection<\/h3>\r\n<p>When you enable test impact analysis, the <tt>TestTask<\/tt> detects changes to source files, supporting files and tests as defined by the task's <tt>SourceFiles<\/tt>, <tt>SupportingFiles<\/tt>, and <tt>Tests<\/tt> properties. It also tracks test class folders and test super classes.<\/p>\r\n\r\n<p>But what is the change window? \u2026 <em>It's the changes <strong>since the last successful run<\/strong><\/em>.<\/p>\r\n\r\n<p>A picture may help\u2026 it always goes a long way for me.<\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/change_window.png\" alt=\"\" width=\"568\" height=\"265\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<p>For example, let's suppose you have a successful build, <b>b<sup>1<\/sup><\/b>. After you make a change <b>C1<\/b>, and run the build, the build tool checks for changes in source, supporting, and test files since <b>b<sup>1<\/sup><\/b>. If <b>C1<\/b> causes a regression and <b>b<sup>2<\/sup><\/b> fails, the next build, <b>b<sup>3<\/sup><\/b> still uses <b>b<sup>1<\/sup><\/b> as the reference point \u2013 because it's the last successful run. This ensures the integrity of your commits and code integration.<\/p>\r\n\r\n<p>Only files that are <i>added<\/i> or <i>modified<\/i> are considered for impact analysis; <i>removed<\/i> files are not. If you're using MATLAB projects, removing a file triggers dependency analysis to help you identify which files are affected so you action appropriately and clean up cruft and dead code.<\/p>\r\n\r\n<p>In summary, change detection:<\/p>\r\n<ul>\r\n \t<li>Detects files defined by the <tt>SourceFiles<\/tt>, <tt>SupportingFiles<\/tt> and <tt>Tests<\/tt> properties of the <tt>test<\/tt> task.<\/li>\r\n \t<li>Detects files that are either <i>added<\/i> or <i>modified<\/i>.<\/li>\r\n \t<li>Detects changes since the last successful task run.<\/li>\r\n<\/ul>\r\n<h3>Impact Analysis<\/h3>\r\n<p>Like the <b>Find Tests<\/b> (<a href=\"https:\/\/blogs.mathworks.com\/developer\/2025\/07\/09\/test-impact-analysis-find-tests\/\">Part 1<\/a>) feature in the Editor and <b>Impacted Tests Since Last Commit<\/b> option (<a href=\"https:\/\/blogs.mathworks.com\/developer\/2025\/07\/23\/test-impact-analysis-test-manager\/\">Part 2<\/a>) in the MATLAB test manager app, the TestTask also leverages <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/dependencyanalyzer-app.html#mw_16e23e55-ae1c-4228-8f5f-97273d840a70\">dependency analysis<\/a> to find impacted tests. It leverages it via the <a href=\"https:\/\/www.mathworks.com\/help\/matlab-test\/ref\/matlabtest.selectors.dependson-class.html\">DependsOn<\/a> test selector, which, in R2025a, enhanced the accuracy of test selection with the ability to select individual tests within a test file.<\/p>\r\n<div style=\"border-left: 4px solid #007acc; background-color: #f0f8ff; padding: 12px 16px; margin: 16px 0; font-family: sans-serif;\"><strong>Reminder:<\/strong> Static analysis isn't perfect. It has <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/dependencyanalyzer-app.html#mw_16e23e55-ae1c-4228-8f5f-97273d840a70\">limitations<\/a>. Use incremental testing with vigilance and discipline \u2013 <b>trust but verify<\/b>. Always run the full test suite at key points in your release cycle to ensure complete coverage and confidence.<\/div>\r\n<h2>Interaction Between Incremental Build and Incremental Testing<a name=\"ea885be6-ecda-484d-97b2-5679a814a414\"><\/a><\/h2>\r\n<p>Earlier, I mentioned that if a task's inputs, outputs, actions and arguments haven't changed since the last successful run, the build tool skips the task. The <tt>RunOnlyImpactedTests<\/tt> option is itself considered an input to the task, so incremental build applies to this property as well.<\/p>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"\" src=\"https:\/\/blogs.mathworks.com\/developer\/files\/skipped_task_roit.png\" alt=\"\" width=\"603\" height=\"331\" hspace=\"5\" vspace=\"5\" \/>\r\n\r\n<br><br>\r\n<p>When you rerun the test task with the same <tt>RunOnlyImpactedTests<\/tt> value and the task has no other changes either, the build tool skips the task. Why? 'cuz of Incremental Build!<\/p>\r\n<h3>Quiz Time!<\/h3>\r\n<p>Suppose you change the <tt>TestResults<\/tt> task output property:<\/p>\r\n\r\n<p><em>From:<\/em><\/p>\r\n<pre class=\"language-matlab\">plan(<span class=\"string\">\"test\"<\/span>) = TestTask(<span class=\"string\">\"tests\"<\/span>, SourceFiles = <span class=\"string\">\"toolbox\"<\/span>, TestResults = <span class=\"string\">\"results\/test-results.xml\"<\/span>);\r\n<\/pre>\r\n<p><em>To:<\/em><\/p>\r\n<pre class=\"language-matlab\">plan(<span class=\"string\">\"test\"<\/span>) = TestTask(<span class=\"string\">\"tests\"<\/span>, SourceFiles = <span class=\"string\">\"toolbox\"<\/span>, TestResults = <span class=\"string\">\"results\/test-results.html\"<\/span>);\r\n<\/pre>\r\n<p><em>Question<\/em>: What tests will run if <tt>RunOnlyImpactedTests<\/tt> is set to <tt>true<\/tt>, assuming the above <tt>buildfile.m<\/tt> is the only change made in your project since the last successful run and no other changes?<\/p>\r\n\r\n<p>Let us know what you think in the comments!<\/p>\r\n\r\n<p><b>Extra Credit:<\/b><\/p>\r\n<p>In <tt>buildfile.m<\/tt>, <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.buildtool.task-class.html?searchHighlight=DisableIncremental&amp;s_tid=srchtitle_support_results_2_DisableIncremental\">disable incremental build<\/a>:<\/p>\r\n<pre class=\"language-matlab\">plan(<span class=\"string\">\"test\"<\/span>) = TestTask(<span class=\"string\">\"tests\"<\/span>, SourceFiles = <span class=\"string\">\"toolbox\"<\/span>, TestResults = <span class=\"string\">\"results\/test-results.xml\"<\/span>);\r\nplan(<span class=\"string\">\"test\"<\/span>).DisableIncremental = 1;\r\n<\/pre>\r\n<p>Then run:<\/p>\r\n<pre class=\"language-matlab\">&gt;&gt; buildtool test(RunOnlyImpactedTests=1)\r\n<\/pre>\r\n<p><em>Question:<\/em> What tests do you expect to run? And why? Share your thoughts in the comments.<\/p>\r\n<p>I will share the answers in the comments next week!<\/p>\r\n<h2>Summary<a name=\"6340e957-281e-41d0-863d-f33c9a483ed6\"><\/a><\/h2>\r\n<p>There you have it! Incremental testing with the MATLAB Build Tool and MATLAB Test can greatly improve the efficiency of automated builds, locally and in CI, by running only the tests impacted by recent changes. It complements interactive development and helps reduce test times without sacrificing coverage.<\/p>\r\n\r\n<p>While static dependency analysis has limitations, incremental testing can deliver substantial value with some user awareness and discipline.<\/p>\r\n\r\n<p>This post focused on local builds, but the same benefits apply to CI environment as well \u2013 we'll dive into that in the next part!<\/p>\r\n\r\n<p><i>Would you consider using incremental testing in your workflows? What challenges do you foresee? Are there gaps in change detection or impact analysis that MathWorks could address to help you trust incremental testing more? We'd love to hear your thoughts \u2013 share them in the comments section below!<\/i><\/p>\r\n<script language=\"JavaScript\"> <!-- \r\n    function grabCode_56f6e7b4223646428020c27162ba6f75() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='56f6e7b4223646428020c27162ba6f75 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 56f6e7b4223646428020c27162ba6f75';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2025 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script>\r\n<p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\">\r\n<br>\r\n<a href=\"javascript:grabCode_56f6e7b4223646428020c27162ba6f75()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript>\r\n<\/span><\/a>\r\n<br>\r\n<br>\r\n      Published with MATLAB&reg; R2025a<br>\r\n<\/p>\r\n<\/div>\r\n<!--\r\n56f6e7b4223646428020c27162ba6f75 ##### SOURCE BEGIN #####\r\n%%\r\n% Alright! I'm back after a mid-series break to continue our talk about\r\n% incremental testing.\r\n%\r\n% In parts 1 and 2, we looked at interactive development workflows to help\r\n% run the right set of tests at the right time. In this post, we'll look at\r\n% how that idea extends to automated builds using the MATLAB build tool,\r\n% and specifically, how incremental testing enables faster local\r\n% prequalification. The motivation remains the same \u2013 to enable frequent,\r\n% high-quality integrations verified by automated builds, without incurring\r\n% the overhead of running the full regression suite after every change.\r\n%\r\n% Running all the tests might be manageable at small scale but it quickly\r\n% becomes prohibitively expensive for larger projects.\r\n%\r\n% *Info:* This post assumes that you are somewhat familiar with the\r\n%         <https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/overview-of-matlab-build-tool.html % MATLAB build tool> . If not, you can get an overview here. It\r\n%         provides a uniform and standardized way to define and run\r\n%         MATLAB builds.\r\n%\r\n%% What Is Incremental Testing?\r\n% The MATLAB build tool supports\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/improve-performance-with-incremental-builds.html % incremental builds>, which skips unnecessary work by tracking each build\r\n% task's inputs, outputs, actions and arguments to determine if the task is\r\n% up-to-date. If nothing has changed since its last successful run, the\r\n% task is skipped.\r\n%\r\n% Starting with R2025a, it enhances this capability with incremental\r\n% testing, available with MATLAB Test. While incremental build determines\r\n% whether a task should rerun, incremental testing goes further \u2013 it uses\r\n% test impact analysis to decide which tests need to run. This can\r\n% significantly optimize test execution and speed up your builds\r\n%\r\n% With incremental testing enabled, the build tool analyzes the source and\r\n% test code changes and runs only tests impacted by those changes.\r\n%\r\n% Let's unpack how this works. We'll start with some basics.\r\n%\r\n%% Background\r\n% We'll use the same goal as in Part 2 \u2013 enhancing the AND perceptron to\r\n% introduce a trainable bias term. The exact goal isn't critical; it's just\r\n% a useful backdrop.\r\n%\r\n%%% Defining the Build\r\n% A |buildfile.m| at the project root defines your MATLAB build.\r\n%\r\n% <<buildfile_location.png>>\r\n%\r\n% This build file creates a plan that describes how to build the project \u2013\r\n% it defines a set of build tasks and their dependencies to establish a\r\n% task graph to execute with a single\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/buildtool.html buildtool>\r\n% command.\r\n%\r\n% <include>before\/buildfile.m<\/include>\r\n%\r\n% In our example, the build file includes:\r\n%   * A |test| task to run tests\r\n%   * A |clean| task to delete task outputs and traces\r\n%\r\n% These tasks are created using the\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.buildtool.tasks-package.html % built-in task classes> . Built-in task classes streamline the creation of\r\n% standard tasks like identifying code issues, building MEX binaries and\r\n% testing. The built-in task classes are designed for reuse and generally\r\n% support\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/improve-performance-with-incremental-builds.html % incremental build> out of the box.\r\n%\r\n% Out |test| tasks uses the\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.buildtool.tasks.testtask-class.html % matlab.buildtool.tasks.TestTask> class, which runs tests using the MATLAB\r\n% Unit Testing Framework.\r\n%\r\n%%% Optimizing Performance with Incremental Build\r\n% Before diving into incremental testing, let's briefly look at how\r\n% incremental build works.\r\n%\r\n% When we first clone our repository, the build tool has no task traces.\r\n%\r\n% *Info:* A *task trace* is a MATLAB release-specific record of a task's inputs, outputs,\r\n%         actions and arguments from its last successful run.\r\n%\r\n% So, when we run the |test| task, it runs because there are no traces yet,\r\n% and runs *all* the tests defined by the task.\r\n%\r\n% <<run_with_no_task_trace.png>>\r\n%\r\n% ...\r\n%\r\n% <<test_run_summary_no_task_trace.png>>\r\n%\r\n% This is nice because it gives us confidence that the repository is in a\r\n% known-good state before making any code changes.\r\n%\r\n% The build tool creates a |.buildtool| cache folder in the project root\r\n% folder to store the task traces.\r\n%\r\n% <<task_trace_folder.png>>\r\n%\r\n% On subsequent builds, if nothing has changed, the task is skipped.\r\n%\r\n% <<skipped_task.png>>\r\n%\r\n% Now, let's say we modify a source file like |initializeWeights.m|, and\r\n% re-run the build.\r\n%\r\n% <<sample_change_source_control.png>>\r\n%\r\n% Since this file is tracked via the |SourceFiles| property, one of the\r\n% task inputs of the test task that the build tool tracks as part of\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.buildtool.tasks.testtask-class.html#mw_96cca6ff-6c81-4dd9-87d2-5c7df47b475b % up-to-date check>, the tool recognizes the changes and re-runs the task.\r\n%\r\n% <<rerun_after_sample_change.png>>\r\n%\r\n% Nice! But do we really need to run all the tests for that single source\r\n% file change? Wouldn't it be better if just the impacted tests ran. That's\r\n% exactly what incremental testing in R2025a delivers.\r\n%\r\n%% Enabling Incremental Testing\r\n% So, how do we get the incremental testing behavior?\r\n%\r\n% Incremental testing is powered by test impact analysis. To enable it, you\r\n% can simply run the |test| task with the |RunOnlyImpactedTests| task\r\n% argument.\r\n%\r\n%   >> buildtool test(RunOnlyImpactedTests=1)\r\n%\r\n% Let's look at an example. Say we change |initializeWeights.m| and\r\n% |calculateWeightedSum.m| as part of our perceptron enhancement, like we\r\n% did in Part 2:\r\n%\r\n% *initializeWeights.m*\r\n%\r\n% <html><u>Before:<\/u><\/html>\r\n%\r\n% <include>before\/toolbox\/initializeWeights.m<\/include>\r\n%\r\n% <html><u>After:<\/u><\/html>\r\n%\r\n% <include>after\/toolbox\/initializeWeights.m<\/include>\r\n%\r\n% *calculateWeightedSum.m*\r\n%\r\n% <html><u>Before:<\/u><\/html>\r\n%\r\n% <include>before\/toolbox\/calculateWeightedSum.m<\/include>\r\n%\r\n% <html><u>After:<\/u><\/html>\r\n%\r\n% <include>after\/toolbox\/calculateWeightedSum.m<\/include>\r\n%\r\n% When you set |RunOnlyImpactedTests| to |true|, the |test| task analyzes\r\n% those changes and runs only the impacted tests. Look at that!\r\n%\r\n% <<run_impacted_tests.png>>\r\n%\r\n% Behind the scenes, the build tool tracks changes to the source and test\r\n% files of the |test| task created using the the built-in |TestTask| class\r\n% (more on this later in the post).\r\n%\r\n% When |RunOnlyImpactedTests| is true, the |test| task doesn't just detect\r\n% that the files changed \u2013 it determines which tests are impacted by those\r\n% changes and runs only those.\r\n%\r\n% Increasing the build verbosity to level 3 (Detailed) or higher reveals\r\n% which changes triggered the task execution. In our example, we see that\r\n% the |SourceFiles| property has changed because |initializeWeights.m| and\r\n% |calculateWeightedSum.m| were modified.\r\n%\r\n% As seen in Part 2, changes to just those two files do not meet the goal.\r\n% The diagnostics will reflect this.\r\n%\r\n% <<run_impacted_tests_failures.png>>\r\n%\r\n% Let's fix the rest of the code base like we did in Part 2 to ensure we\r\n% have all the necessary changes and rerun the build with incremental\r\n% testing.\r\n%\r\n% _After some coding and a cup of coffee..._\r\n%\r\n% <<rerun_impacted_tests.png>>\r\n%\r\n% Success! The build passes and we ran just the related tests.\r\n%\r\n% <<run_impacted_tests_success.png>>\r\n%\r\n% Our toy example had just 5 tests, but real-world projects can have\r\n% hundreds or thousands \u2013 so the potential for performance gain is\r\n% substantial.\r\n%\r\n% To avoid the need to specify the task argument every time, we can\r\n% configure the test task in your |buildfile.m| to always run only impacted\r\n% tests by default.\r\n%\r\n% <<buildfile_roit.png>>\r\n%\r\n% Then at run time, we can specify just \"test\".\r\n%\r\n%   >> buildtool test\r\n%\r\n% You can toggle the |RunOnlyImpactedTests| option at run time or create\r\n% separate test tasks with and without incremental testing to match your\r\n% workflow preferences.\r\n%\r\n%% A Closer Look at Impact-Based Testing\r\n%\r\n% Impact-based testing involves two key concepts:\r\n%\r\n% * Change Detection refers exclusively to the concern of detecting changes\r\n% to the task's tracked files\r\n% * Impact Analysis refers exclusively to the concern of analyzing the\r\n% impact of those changes.\r\n%\r\n% *Change Detection*\r\n% When you enable test impact analysis, the |TestTask| detects changes to\r\n% source files, supporting files and tests as defined by the task's\r\n% |SourceFiles|, |SupportingFiles|, and |Tests| properties. It also tracks test\r\n% class folders and test superclasses.\r\n%\r\n% But what is the change window? \u2026 It's changes _since the last successful\r\n% run_.\r\n%\r\n% A picture might help\u2026 it always goes a long way for me.\r\n%\r\n% <<change_window.png>>\r\n%\r\n% For example, let's suppose you have a successful build, *b_1*. After you\r\n% make a change *C1*, and run the build, the build tool checks for changes\r\n% in source, supporting, and test files since *b_1*. If *C1* causes a\r\n% regression and *b_2* fails, the next build, *b_3* still uses *b_1* as the\r\n% reference point - because it's the last successful run. This ensures the\r\n% integrity of your commits and code integration.\r\n%\r\n% Only files that are _added_ or _modified_ are considered  for impact\r\n% analysis; _removed_ files are not. If you're using MATLAB projects,\r\n% removing a file triggers dependency analysis to help you identify which\r\n% files are affected so you action appropriately and clean up cruft and\r\n% dead code.\r\n%\r\n% In summary, change detection:\r\n%   * Detects files defined by the |SourceFiles|, |SupportingFiles| and\r\n%   |Tests| properties of the |test| task.\r\n%   * Detects files that are either _added_ or _modified_.\r\n%   * Detects changes since the last successful task run.\r\n%\r\n% *Impact Analysis*\r\n% Like the *Find Tests* (Part 1) feature in the Editor and *Impacted Tests\r\n% Since Last Commit* option (Part 2) in the MATLAB test manager app, the\r\n% TestTask also uses\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/dependencyanalyzer-app.html#mw_16e23e55-ae1c-4228-8f5f-97273d840a70 % dependency analysis> to find impacted tests. It uses it via the\r\n% <https:\/\/www.mathworks.com\/help\/matlab-test\/ref\/matlabtest.selectors.dependson-class.html % DependsOn> test selector, which, in R2025a, enhanced the accuracy of test\r\n% selection with the ability to select individual tests within a test file.\r\n%\r\n% > *Important Reminder:* Static analysis isn't perfect. It has\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/dependencyanalyzer-app.html#mw_16e23e55-ae1c-4228-8f5f-97273d840a70 % limitations> . Use incremental testing with vigilance and discipline \u2013\r\n% *trust but verify*. Always run the full test suite at key points in your\r\n% release cycle to ensure complete coverage and confidence.\r\n%\r\n%% Interaction Between Incremental Build and Incremental Testing\r\n%\r\n% Earlier, I mentioned that if a task's inputs, outputs, actions and\r\n% arguments haven't changed since the last successful run, the build tool\r\n% skips the task. The |RunOnlyImpactedTests| option is itself considered \r\n% an input to the task, so incremental build applies to this property as well.\r\n%\r\n% <<skipped_task_roit.png>>\r\n%\r\n% When you rerun the test task with the same |RunOnlyImpactedTests| value\r\n% and the task has no other changes either, the build tool skips the task.\r\n% Why? 'cuz of Incremental Build!\r\n%\r\n% *Quiz Time!*\r\n%\r\n% Suppose you change the |TestResults| task output property:\r\n%\r\n% From:\r\n%\r\n%   plan(\"test\") = TestTask(\"tests\", SourceFiles=\"toolbox\", TestResults=\"results\/test-results.xml\");\r\n%\r\n% To:\r\n%\r\n%   plan(\"test\") = TestTask(\"tests\", SourceFiles=\"toolbox\", TestResults=\"results\/test-results.html\");\r\n%\r\n% *Question:* What tests will run if |RunOnlyImpactedTests| is set to\r\n% |true|, assuming the above |buildfile.m| is the only change made in your\r\n% project since the last successful run and no other changes?\r\n%\r\n% Let us know what you think in the comments!\r\n%\r\n% *Bonus Credit*\r\n%\r\n% In |buildfile.m|,\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.buildtool.task-class.html?searchHighlight=DisableIncremental&s_tid=srchtitle_support_results_2_DisableIncremental % disable incremental build>:\r\n%\r\n%   plan(\"test\") = TestTask(\"tests\", SourceFiles=\"toolbox\", TestResults=\"results\/test-results.xml\");\r\n%   plan(\"test\").DisableIncremental = 1;\r\n%\r\n% Then run:\r\n%\r\n%   >> buildtool test(RunOnlyImpactedTests=1)\r\n%\r\n% *Question:* What tests do you expect to run? And why? Share your thoughts\r\n% in the comments.\r\n% \r\n% I will share the answers in the comments next week!\r\n%\r\n%% Summary\r\n% There you have it! Incremental testing with the MATLAB Build Tool and\r\n% MATLAB Test can greatly improve the efficiency of automated builds,\r\n% locally and in CI, by running only the tests impacted by recent changes.\r\n% It complements interactive development and helps reduce test times\r\n% without sacrificing coverage.\r\n%\r\n% While static dependency analysis has limitations, incremental\r\n% testing can deliver substantial value with some user awareness and\r\n% discipline.\r\n%\r\n% This post focused on local builds, but the same benefits\r\n% apply to CI environment as well \u2013 we'll dive into that in the next part!\r\n%\r\n% _Would you consider using incremental testing in your workflows?\r\n% What challenges do you foresee? Are there gaps in change detection or\r\n% impact analysis that MathWorks could address to help you trust\r\n% incremental testing more? We'd love to hear your thoughts \u2013 share them in\r\n% the comments section below!_\r\n\r\n##### SOURCE END ##### 56f6e7b4223646428020c27162ba6f75\r\n-->","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img src=\"https:\/\/blogs.mathworks.com\/developer\/files\/Incremental_Testing_ROIT_3.png\" class=\"img-responsive attachment-post-thumbnail size-post-thumbnail wp-post-image\" alt=\"Incremental testing with MATLAB build tool\" decoding=\"async\" loading=\"lazy\" \/><\/div><!--introduction-->\r\n\r\n<p><em>Alright! I'm back after a mid-series break to continue our talk about incremental testing.<\/em>... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/developer\/2025\/08\/20\/tldr-too-long-didnt-run-part-3-incremental-testing-with-the-matlab-build-tool\/\">read more >><\/a><\/p>","protected":false},"author":222,"featured_media":4235,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[39,4,44,13,20,57,7,1],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/4040"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/users\/222"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/comments?post=4040"}],"version-history":[{"count":51,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/4040\/revisions"}],"predecessor-version":[{"id":4241,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/4040\/revisions\/4241"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/media\/4235"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/media?parent=4040"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/categories?post=4040"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/tags?post=4040"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}