{"id":2964,"date":"2023-03-15T15:54:49","date_gmt":"2023-03-15T19:54:49","guid":{"rendered":"https:\/\/blogs.mathworks.com\/developer\/?p=2964"},"modified":"2023-03-17T12:26:41","modified_gmt":"2023-03-17T16:26:41","slug":"static-analysis-code-checking-and-linting-with-codeissues","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/developer\/2023\/03\/15\/static-analysis-code-checking-and-linting-with-codeissues\/","title":{"rendered":"We&#8217;ve All Got Issues"},"content":{"rendered":"\r\n\r\n<div class=\"content\"><p>Who among us doesn't have issues, amirite? Let's just take a moment and acknowledge this fact and I think we can always be a bit more honest and understanding of all of our unique issues and the various idiosyncrasies we exhibit. While we can all offer understanding and grace to each other, some of the issues we face can be important to address quickly, and some we can perhaps choose to work on when the time is right.<\/p><p>...and so it is with our code. Issues, bugs, and other sub-optimal constructs crop up all the time in any serious codebase. Many of you are already aware of the features of the Code Analyzer in MATLAB that do a great job of alerting you to your issues directly in the MATLAB editor. However, sometimes issues still can creep into a code base through a number of subtle means, even including simply failing to act when a helpful editor is trying to tell me I have a problem in my code.<\/p><p>However, in R2022b there was a great improvement to the programmatic interface for identifying and addressing with these code issues. In fact, this new interface is itself a new function called <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/codeissues.html\">codeIssues<\/a>!<\/p><p>This new API for the code analyzer is in a word - powerful. Let's explore how you can now use this to build quality into your project. Starting with another look at our standard mass-spring-damper mini-codebase. Most recently we talked about this when describing the <a href=\"https:\/\/blogs.mathworks.com\/developer\/2022\/10\/17\/building-blocks-with-buildtool\/\">new MATLAB build tool<\/a>. This code base is pretty simple. It includes a simulator, a function to return some design constants (spring constant, damping coefficient, etc.), and some tests.<\/p><p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"http:\/\/blogs.mathworks.com\/developer\/files\/y2023ToolboxFoldersAndFiles.png\" alt=\"\"> <\/p><p>To get started on a codebase like this, just call codeIssues on the folder you want to analyze:<\/p><p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"http:\/\/blogs.mathworks.com\/developer\/files\/y2023CodeIssuesOutput.png\" alt=\"\"> <\/p><p>You can see that with just a simple call to codeIssues you can quickly get an overview of all the details a static analyzer dreams of. You can easily dig into the files that were analyzed, the configuration, and the very handy table of the issues found, as well as any issues that have been suppressed through suppression pragmas in the editor. If you are in MATLAB you can even click on each issue to get right to it in the MATLAB editor where it can be fixed or suppressed if needed.<\/p><p>Now with this beautiful API at our fingertips, and with the build tool to boot, we can lock down our code in a much more robust, automated way. We can start roughly where we left off at the end of our build tool post with the following buildfile with a mex and a test task:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> plan = buildfile\r\nplan = buildplan(localfunctions);\r\n\r\nplan(<span class=\"string\">\"test\"<\/span>).Dependencies = <span class=\"string\">\"mex\"<\/span>;\r\nplan.DefaultTasks = <span class=\"string\">\"test\"<\/span>;\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> mexTask(~)\r\n<span class=\"comment\">% Compile mex files<\/span>\r\nmex <span class=\"string\">mex\/convec.c<\/span> <span class=\"string\">-outdir<\/span> <span class=\"string\">toolbox\/<\/span>;\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> testTask(~)\r\n<span class=\"comment\">% Run the unit tests<\/span>\r\nresults = runtests;\r\ndisp(results);\r\nassertSuccess(results);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n\r\n<\/pre><p>Let's go ahead and add a \"codeIssues\" task to this build by creating a new local function called <b><tt>codeIssuesTask<\/tt><\/b>:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> codeIssuesTask(~)\r\n<span class=\"comment\">% Get all the issues under toolbox<\/span>\r\nallIssues = codeIssues(<span class=\"string\">\"toolbox\"<\/span>);\r\n\r\n<span class=\"comment\">% Assert that no errors creep into the codebase<\/span>\r\nerrorIdx = allIssues.Issues.Severity == <span class=\"string\">\"error\"<\/span>;\r\nerrors = allIssues.Issues(errorIdx,:);\r\notherIssues = allIssues.Issues(~errorIdx,:);\r\n<span class=\"keyword\">if<\/span> ~isempty(errors)\r\n    disp(<span class=\"string\">\"Found critical errors in code:\"<\/span>);\r\n    disp(errors);\r\n<span class=\"keyword\">else<\/span>\r\n    disp(<span class=\"string\">\"No critical errors found.\"<\/span>);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"comment\">% Display all the other issues<\/span>\r\n<span class=\"keyword\">if<\/span> ~isempty(otherIssues)\r\n    disp(<span class=\"string\">\"Other Issues:\"<\/span>)\r\n    disp(otherIssues);\r\n<span class=\"keyword\">else<\/span>\r\n    disp(<span class=\"string\">\"No other issues found either. (wow, good for you!)\"<\/span>)\r\n<span class=\"keyword\">end<\/span>\r\n\r\nassert(isempty(errors));\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><p>This is quite simple, we just want to find all the issues under the \"toolbox\" folder and throw an assertion error if any of them are of Severity \"error\". This is just about the quickest win you can apply to a code base to build quality into it. This can find syntax and other errors statically, without even writing or running a single test. There really is no reason at all we shouldn't apply this task to every project. It costs virtually nothing and can be remarkably efficient at finding bugs. On that note, let's add it as a default task in our buildfile:<\/p><pre class=\"language-matlab\">\r\nplan.DefaultTasks = [<span class=\"string\">\"codeIssues\"<\/span> <span class=\"string\">\"test\"<\/span>];\r\n\r\n<\/pre><p>...and with that we now have this check built right into our standard development process. To show this let's first put a file with an error into the code base and then we can call the build tool:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> syntaxError\r\n\r\ndisp(<span class=\"string\">\"Forgot the closing parentheses!\"<\/span>\r\n\r\n<\/pre><pre class=\"codeinput\">copyfile <span class=\"string\">.changes\/syntaxError.m<\/span> <span class=\"string\">toolbox\/syntaxError.m<\/span>\r\n<span class=\"keyword\">try<\/span>\r\n    buildtool\r\n<span class=\"keyword\">catch<\/span> ex\r\n    disp(ex.getReport(<span class=\"string\">\"basic\"<\/span>));\r\n<span class=\"keyword\">end<\/span>\r\n<\/pre><pre class=\"codeoutput\">** Starting codeIssues\r\nFailed! Found critical errors in code:\r\n       Location        Severity                                     Description                                      CheckID    LineStart    LineEnd    ColumnStart    ColumnEnd                                             FullFilename        \r\n    _______________    ________    ______________________________________________________________________________    _______    _________    _______    ___________    _________    _______________________________________________________________________________________________\r\n\r\n    \"syntaxError.m\"     error      \"A '(' might be missing a closing ')', causing invalid syntax at end of line.\"    EOLPAR         3           3            5             5        \"\/Users\/acampbel\/Library\/CloudStorage\/OneDrive-MathWorks\/repos\/msd_blog2\/toolbox\/syntaxError.m\"\r\n\r\nOther Issues:\r\n             Location             Severity                     Description                     CheckID    LineStart    LineEnd    ColumnStart    ColumnEnd                                                   FullFilename             \r\n    __________________________    ________    _____________________________________________    _______    _________    _______    ___________    _________    __________________________________________________________________________________________________________\r\n\r\n    \"springMassDamperDesign.m\"    warning     \"Value assigned to variable might be unused.\"     NASGU         4           4            3             3        \"\/Users\/acampbel\/Library\/CloudStorage\/OneDrive-MathWorks\/repos\/msd_blog2\/toolbox\/springMassDamperDesign.m\"\r\n    \"springMassDamperDesign.m\"    warning     \"Value assigned to variable might be unused.\"     NASGU         6           6            3             3        \"\/Users\/acampbel\/Library\/CloudStorage\/OneDrive-MathWorks\/repos\/msd_blog2\/toolbox\/springMassDamperDesign.m\"\r\n\r\n## -----------------------------------------------------------------------------\r\n## Error using assert\r\n## Assertion failed.\r\n## \r\n## Error in buildfile&gt;codeIssuesTask (line 67)\r\n## assert(isempty(errors));\r\n## -----------------------------------------------------------------------------\r\n** Failed codeIssues\r\n\r\nError using buildtool\r\nBuild failed.\r\n<\/pre><p>The build tool has done its part in stopping our development process in its tracks when it sees we have some important syntax errors to address.<\/p><p>Let's remove that bunk error so we can see our \"codeIssues\" task complete successfully. When we do this we also successfully execute the other \"mex\" and \"test\" tasks to complete the whole workflow.<\/p><pre class=\"codeinput\">delete <span class=\"string\">toolbox\/syntaxError.m<\/span>\r\nbuildtool\r\n<\/pre><pre class=\"codeoutput\">** Starting codeIssues\r\nNo critical errors found.\r\nOther Issues:\r\n             Location             Severity                     Description                     CheckID    LineStart    LineEnd    ColumnStart    ColumnEnd                                                   FullFilename             \r\n    __________________________    ________    _____________________________________________    _______    _________    _______    ___________    _________    __________________________________________________________________________________________________________\r\n\r\n    \"springMassDamperDesign.m\"    warning     \"Value assigned to variable might be unused.\"     NASGU         4           4            3             3        \"\/Users\/acampbel\/Library\/CloudStorage\/OneDrive-MathWorks\/repos\/msd_blog2\/toolbox\/springMassDamperDesign.m\"\r\n    \"springMassDamperDesign.m\"    warning     \"Value assigned to variable might be unused.\"     NASGU         6           6            3             3        \"\/Users\/acampbel\/Library\/CloudStorage\/OneDrive-MathWorks\/repos\/msd_blog2\/toolbox\/springMassDamperDesign.m\"\r\n\r\n** Finished codeIssues\r\n\r\n** Starting mex\r\nBuilding with 'Xcode with Clang'.\r\nMEX completed successfully.\r\n** Finished mex\r\n\r\n** Starting test\r\nSetting up ProjectFixture\r\nDone setting up ProjectFixture: Project 'msd' is already loaded. Set up is not required.\r\n__________\r\n\r\nRunning convecTest\r\n.\r\nDone convecTest\r\n__________\r\n\r\nRunning designTest\r\n...\r\nDone designTest\r\n__________\r\n\r\nTearing down ProjectFixture\r\nDone tearing down ProjectFixture: Teardown is not required.\r\n__________\r\n\r\n  1&times;4 TestResult array with properties:\r\n\r\n    Name\r\n    Passed\r\n    Failed\r\n    Incomplete\r\n    Duration\r\n    Details\r\n\r\nTotals:\r\n   4 Passed, 0 Failed, 0 Incomplete.\r\n   0.22349 seconds testing time.\r\n\r\n** Finished test\r\n\r\n<\/pre><p><b>One last thing<\/b><\/p><p>Now we have you all setup nice and cozy with the protection that static analysis gives you. However, while we fail on static analysis errors, I am still uncomfortable with how easy it is to continue to add constructs to my code that result in static analysis warnings, which often point to real problems in your program. We could also fail the build on warnings if we'd like, but I didn't want to start with that idea out of the gate.<\/p><p>It is pretty clear that we want this protection with full-on-bonafide errors, which are almost always bugs. We run into a problem though when a code base already has an inventory of warnings. It would be fantastic to go through that inventory and fix all of those warnings as well. In fact, the new code analysis tooling makes that very easy in many cases! However, you may not be up for this right now. Your code base may be large, and you may want or need to invest a bit more time into this activity. So our first crack at this failed the build only for issues with an \"error\" Severity.<\/p><p>However, if you know me you know I like to sneak one last thing in. What if we accepted all of our current warnings in the codebase, but wanted to lock down our code base such that we are protected from introducing new warnings? To me this sounds like a great idea. We can then ratchet down our warnings by preventing inflow of new warnings and can remove existing warnings over time through interacting with the codebase. How can we do this? We can leverage the power of the codeIssues programmatic API!<\/p><p>We can do this by capturing and saving our existing warnings to a baseline of known issues. As MATLAB tables, theses issues are in a nice representation to save in a *.csv or *.xlsx file. Saving them in this format makes it really easy to tweak them, open them outside of MATLAB, or even remove issues that have been fixed.<\/p><p>To do this we just need to make a couple tweaks to the issues table. We need to overwrite the <b><tt>Location<\/tt><\/b> variable with relative paths to the files, remove the <b><tt>FullFilename<\/tt><\/b> variable, and make a quick datatype tweak to allow for nice CSV'ing. The relative filename adjustment is important because we want to be able to compare these results across different machines and the full path is likely to differ across environments. Such environments include the desktops of individual engineers as well as different build agents in a CI system.<\/p><p>That function looks as follows:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> theTable = preprocessIssues(theTable)\r\n<span class=\"comment\">% Make an issues table conducive for baselining via a few small tweaks<\/span>\r\n\r\n<span class=\"comment\">% Overwrite the location field with relative paths, and remove absolute paths<\/span>\r\nbasePath = string(pwd) + filesep;\r\ntheTable.Location = erase(theTable.FullFilename, basePath);\r\ntheTable.Properties.VariableNames{<span class=\"string\">\"Location\"<\/span>} = <span class=\"string\">'RelativeFilename'<\/span>;\r\ntheTable.FullFilename = [];\r\n\r\n<span class=\"comment\">% Convert the Severity to categorical, which serializes nicely to string<\/span>\r\ntheTable.Severity = categorical(theTable.Severity);\r\n\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><p>...and now with this function we can create a new task in the buildfile to generate a new baseline:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> captureWarningsBaselineTask(~)\r\n<span class=\"comment\">% Captures the current codeIssues warnings and creates a baseline csv file<\/span>\r\nallIssues = codeIssues(<span class=\"string\">\"toolbox\"<\/span>);\r\nwarningIdx = allIssues.Issues.Severity == <span class=\"string\">\"warning\"<\/span>;\r\nwarnings = allIssues.Issues(warningIdx,:);\r\n\r\nwarnings = preprocessIssues(warnings);\r\n<span class=\"keyword\">if<\/span> ~isempty(warnings)\r\n    disp(<span class=\"string\">\"Saving a new \"\"knownIssues.csv\"\" baseline file for \"<\/span> + height(warnings) + <span class=\"string\">\" code warnings\"<\/span>)\r\n    writetable(warnings, <span class=\"string\">\"knownIssues.csv\"<\/span>);\r\n<span class=\"keyword\">else<\/span>\r\n    disp(<span class=\"string\">\"No warnings to create a baseline for\"<\/span>)\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><p>Let's do it!<\/p><pre class=\"codeinput\">buildtool <span class=\"string\">captureWarningsBaseline<\/span>\r\n<\/pre><pre class=\"codeoutput\">** Starting captureWarningsBaseline\r\nSaving a new \"knownIssues.csv\" baseline file for 2 code warnings\r\n** Finished captureWarningsBaseline\r\n\r\n<\/pre><p>Great I now see the csv files. We can take a peek:<\/p><pre class=\"codeinput\">type <span class=\"string\">knownIssues.csv<\/span>\r\n<\/pre><pre class=\"codeoutput\">\r\nRelativeFilename,Severity,Description,CheckID,LineStart,LineEnd,ColumnStart,ColumnEnd\r\ntoolbox\/springMassDamperDesign.m,warning,Value assigned to variable might be unused.,NASGU,4,4,3,3\r\ntoolbox\/springMassDamperDesign.m,warning,Value assigned to variable might be unused.,NASGU,6,6,3,3\r\n<\/pre><p>Beautiful. In this case we just have two minor warnings that I don't want to look into quite yet. However, now we can adjust the \"codeIssues\" task to prevent me from introducing anything new:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> codeIssuesTask(~)\r\n<span class=\"comment\">% Get all the issues under toolbox<\/span>\r\nallIssues = codeIssues(<span class=\"string\">\"toolbox\"<\/span>);\r\n\r\n<span class=\"comment\">% Assert that no errors creep into the codebase<\/span>\r\nerrorIdx = allIssues.Issues.Severity == <span class=\"string\">\"error\"<\/span>;\r\nerrors = allIssues.Issues(errorIdx,:);\r\notherIssues = allIssues.Issues(~errorIdx,:);\r\n<span class=\"keyword\">if<\/span> ~isempty(errors)\r\n    disp(<span class=\"string\">\"Failed! Found critical errors in code:\"<\/span>);\r\n    disp(errors);\r\n<span class=\"keyword\">else<\/span>\r\n    disp(<span class=\"string\">\"No critical errors found.\"<\/span>);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"comment\">% Load the known warnings baseline<\/span>\r\nnewWarnings = [];\r\n<span class=\"keyword\">if<\/span> isfile(<span class=\"string\">\"knownIssues.csv\"<\/span>)\r\n    otherIssues = preprocessIssues(otherIssues);\r\n    \r\n    <span class=\"comment\">% Load the baseline file<\/span>\r\n    opts = detectImportOptions(<span class=\"string\">\"knownIssues.csv\"<\/span>);\r\n    types = varfun(@class, otherIssues,<span class=\"string\">\"OutputFormat\"<\/span>,<span class=\"string\">\"cell\"<\/span>);\r\n    opts.VariableTypes = types;\r\n    knownIssues = readtable(<span class=\"string\">\"knownIssues.csv\"<\/span>,opts);\r\n\r\n    <span class=\"comment\">% Find the new warnings by subtracting the known issues in the baseline<\/span>\r\n    otherIssues = setdiff(otherIssues, knownIssues);\r\n    newWarningIdx = otherIssues.Severity == <span class=\"string\">\"warning\"<\/span>;\r\n    newWarnings = otherIssues(newWarningIdx,:);\r\n    <span class=\"keyword\">if<\/span> ~isempty(newWarnings)\r\n        disp(<span class=\"string\">\"Failed! Found new warnings in code:\"<\/span>);\r\n        disp(newWarnings);\r\n    <span class=\"keyword\">else<\/span>\r\n        disp(<span class=\"string\">\"No new warnings found.\"<\/span>);\r\n    <span class=\"keyword\">end<\/span>\r\n\r\n    otherIssues = [knownIssues; otherIssues(~newWarningIdx,:)];\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"comment\">% Display all the other issues<\/span>\r\n<span class=\"keyword\">if<\/span> ~isempty(otherIssues)\r\n    disp(<span class=\"string\">\"Other Issues:\"<\/span>)\r\n    disp(otherIssues);\r\n<span class=\"keyword\">else<\/span>\r\n    disp(<span class=\"string\">\"No other issues found either. (wow, good for you!)\"<\/span>)\r\n<span class=\"keyword\">end<\/span>\r\n\r\nassert(isempty(errors));\r\nassert(isempty(newWarnings));\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><p>This now loads the issues and does a setdiff to ignore those that are already known and captured in our baseline CSV file. This way, at least from now on I won't introduce any new warnings to the code base. It can only get better from here. Also, if I change some file that has an existing warning, there is a decent chance that my build tooling is going to yell at me because the existing warning is slightly different. For example it might be on a different line due to changes made in the file.<\/p><p>If this happens, great! Make me clean up or suppress the warning while I have the file open and modified. That's a feature not a bug. Worst case scenario, I can always capture a new baseline if I really can't look into it immediately, but I love this approach to help me clean my code through the process.<\/p><p>What does this look like? Let's add a file to the codebase with a new warning:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> codeWarning\r\n\r\nanUnusedVariable = <span class=\"string\">\"unused\"<\/span>;\r\n\r\n<\/pre><p>...and invoke the build:<\/p><pre class=\"codeinput\">copyfile <span class=\"string\">.changes\/codeWarning.m<\/span> <span class=\"string\">toolbox\/codeWarning.m<\/span>\r\n<span class=\"keyword\">try<\/span>\r\n    buildtool\r\n<span class=\"keyword\">catch<\/span> ex\r\n    disp(ex.getReport(<span class=\"string\">\"basic\"<\/span>));\r\n<span class=\"keyword\">end<\/span>\r\n\r\ndelete <span class=\"string\">toolbox\/codeWarning.m<\/span>\r\n<\/pre><pre class=\"codeoutput\">** Starting codeIssues\r\nNo critical errors found.\r\nFailed! Found new warnings in code:\r\n       RelativeFilename        Severity                     Description                     CheckID    LineStart    LineEnd    ColumnStart    ColumnEnd\r\n    _______________________    ________    _____________________________________________    _______    _________    _______    ___________    _________\r\n\r\n    \"toolbox\/codeWarning.m\"    warning     \"Value assigned to variable might be unused.\"     NASGU         3           3            1            16    \r\n\r\nOther Issues:\r\n             RelativeFilename             Severity                     Description                     CheckID    LineStart    LineEnd    ColumnStart    ColumnEnd\r\n    __________________________________    ________    _____________________________________________    _______    _________    _______    ___________    _________\r\n\r\n    \"toolbox\/springMassDamperDesign.m\"    warning     \"Value assigned to variable might be unused.\"     NASGU         4           4            3             3    \r\n    \"toolbox\/springMassDamperDesign.m\"    warning     \"Value assigned to variable might be unused.\"     NASGU         6           6            3             3    \r\n\r\n## -----------------------------------------------------------------------------\r\n## Error using assert\r\n## Assertion failed.\r\n## \r\n## Error in buildfile&gt;codeIssuesTask (line 68)\r\n## assert(isempty(newWarnings));\r\n## -----------------------------------------------------------------------------\r\n** Failed codeIssues\r\n\r\nError using buildtool\r\nBuild failed.\r\n<\/pre><p>Love it! I am now protected from myself. I can leverage this in my standard toolbox development process to help ensure that over time my code only gets better, not worse. You could also imagine tweaking this to fail or otherwise notify when a warning goes away from the known issues so that we have some pressure to help ensure the lockdown gets tighter and tighter as time goes on. For reference, here is the final buildfile used for this workflow discussed today:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> plan = buildfile\r\nplan = buildplan(localfunctions);\r\n\r\nplan(<span class=\"string\">\"test\"<\/span>).Dependencies = <span class=\"string\">\"mex\"<\/span>;\r\nplan.DefaultTasks = [<span class=\"string\">\"codeIssues\"<\/span> <span class=\"string\">\"test\"<\/span>];\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> mexTask(~)\r\n<span class=\"comment\">% Compile mex files<\/span>\r\nmex <span class=\"string\">mex\/convec.c<\/span> <span class=\"string\">-outdir<\/span> <span class=\"string\">toolbox\/<\/span>;\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> testTask(~)\r\n<span class=\"comment\">% Run the unit tests<\/span>\r\nresults = runtests;\r\ndisp(results);\r\nassertSuccess(results);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> codeIssuesTask(~)\r\n<span class=\"comment\">% Get all the issues under toolbox<\/span>\r\nallIssues = codeIssues(<span class=\"string\">\"toolbox\"<\/span>);\r\n\r\n<span class=\"comment\">% Assert that no errors creep into the codebase<\/span>\r\nerrorIdx = allIssues.Issues.Severity == <span class=\"string\">\"error\"<\/span>;\r\nerrors = allIssues.Issues(errorIdx,:);\r\notherIssues = allIssues.Issues(~errorIdx,:);\r\n<span class=\"keyword\">if<\/span> ~isempty(errors)\r\n    disp(<span class=\"string\">\"Failed! Found critical errors in code:\"<\/span>);\r\n    disp(errors);\r\n<span class=\"keyword\">else<\/span>\r\n    disp(<span class=\"string\">\"No critical errors found.\"<\/span>);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n\r\n<span class=\"comment\">% Load the known warnings baseline<\/span>\r\nnewWarnings = [];\r\n<span class=\"keyword\">if<\/span> isfile(<span class=\"string\">\"knownIssues.csv\"<\/span>)\r\n    otherIssues = preprocessIssues(otherIssues);\r\n    \r\n    opts = detectImportOptions(<span class=\"string\">\"knownIssues.csv\"<\/span>);\r\n    types = varfun(@class, otherIssues,<span class=\"string\">\"OutputFormat\"<\/span>,<span class=\"string\">\"cell\"<\/span>);\r\n    opts.VariableTypes = types;\r\n    knownIssues = readtable(<span class=\"string\">\"knownIssues.csv\"<\/span>,opts);\r\n\r\n    otherIssues = setdiff(otherIssues, knownIssues);\r\n    newWarningIdx = otherIssues.Severity == <span class=\"string\">\"warning\"<\/span>;\r\n    newWarnings = otherIssues(newWarningIdx,:);\r\n    <span class=\"keyword\">if<\/span> ~isempty(newWarnings)\r\n        disp(<span class=\"string\">\"Failed! Found new warnings in code:\"<\/span>);\r\n        disp(newWarnings);\r\n    <span class=\"keyword\">else<\/span>\r\n        disp(<span class=\"string\">\"No new warnings found.\"<\/span>);\r\n    <span class=\"keyword\">end<\/span>\r\n\r\n    otherIssues = [knownIssues; otherIssues(~newWarningIdx,:)];\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"comment\">% Display all the other issues<\/span>\r\n<span class=\"keyword\">if<\/span> ~isempty(otherIssues)\r\n    disp(<span class=\"string\">\"Other Issues:\"<\/span>)\r\n    disp(otherIssues);\r\n<span class=\"keyword\">else<\/span>\r\n    disp(<span class=\"string\">\"No other issues found either. (wow, good for you!)\"<\/span>)\r\n<span class=\"keyword\">end<\/span>\r\n\r\nassert(isempty(errors));\r\nassert(isempty(newWarnings));\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> captureWarningsBaselineTask(~)\r\n<span class=\"comment\">% Captures the current codeIssues warnings and creates a baseline csv file<\/span>\r\nallIssues = codeIssues(<span class=\"string\">\"toolbox\"<\/span>);\r\nwarningIdx = allIssues.Issues.Severity == <span class=\"string\">\"warning\"<\/span>;\r\nwarnings = allIssues.Issues(warningIdx,:);\r\n\r\nwarnings = preprocessIssues(warnings);\r\n<span class=\"keyword\">if<\/span> ~isempty(warnings)\r\n    disp(<span class=\"string\">\"Saving a new \"\"knownIssues.csv\"\" baseline file for \"<\/span> + height(warnings) + <span class=\"string\">\" code warnings\"<\/span>)\r\n    writetable(warnings, <span class=\"string\">\"knownIssues.csv\"<\/span>);\r\n<span class=\"keyword\">else<\/span>\r\n    disp(<span class=\"string\">\"No warnings to create a baseline for\"<\/span>)\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> theTable = preprocessIssues(theTable)\r\n<span class=\"comment\">% Make an issues table conducive for baselining via a few small tweaks<\/span>\r\n\r\n<span class=\"comment\">% Overwrite the location field with relative paths, and remove absolute paths<\/span>\r\nbasePath = string(pwd) + filesep;\r\ntheTable.Location = erase(theTable.FullFilename, basePath);\r\ntheTable.Properties.VariableNames{<span class=\"string\">\"Location\"<\/span>} = <span class=\"string\">'RelativeFilename'<\/span>;\r\ntheTable.FullFilename = [];\r\n\r\n<span class=\"comment\">% Convert the Severity to categorical, which serializes nicely to string<\/span>\r\ntheTable.Severity = categorical(theTable.Severity);\r\n\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><p>There you have it, a clean API for MATLAB's code analysis and a standard way to include this in the development process using the build tool. I can practically feel the quality getting higher!<\/p><p>Folks, there is so much to blog about in the next little while. There's more to discuss here on how we can leverage new and improving tools to develop clean build and test pipelines for your MATLAB projects. Also, I am <b>so<\/b> excited for some really fantastic progress we will be able to share shortly. Buckle in, we are gonna be talking about high quality test and automation developments for a bit here on this blog. Chime in with your insights, tools, and workflows you use as you develop your professional MATLAB projects.<\/p><script language=\"JavaScript\"> <!-- \r\n    function grabCode_05bf3646ac79494094e164ceb13bc142() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='05bf3646ac79494094e164ceb13bc142 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 05bf3646ac79494094e164ceb13bc142';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2023 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><br><a href=\"javascript:grabCode_05bf3646ac79494094e164ceb13bc142()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      Published with MATLAB&reg; R2022b<br><\/p><\/div><!--\r\n05bf3646ac79494094e164ceb13bc142 ##### SOURCE BEGIN #####\r\n%% \r\n% Who among us doesn't have issues, amirite? Let's just take a moment and\r\n% acknowledge this fact and I think we can always be a bit more honest and\r\n% understanding of all of our unique issues and the various idiosyncrasies\r\n% we exhibit. While we can all offer understanding and grace to each other,\r\n% some of the issues we face can be important to address quickly, and some\r\n% we can perhaps choose to work on when the time is right.\r\n% \r\n% ...and so it is with our code. Issues, bugs, and other sub-optimal\r\n% constructs crop up all the time in any serious codebase. Many of you are\r\n% already aware of the features of the Code Analyzer in MATLAB that do a\r\n% great job of alerting you to your issues directly in the MATLAB editor.\r\n% However, sometimes issues still can creep into a code base through a\r\n% number of subtle means, even including simply failing to act when a\r\n% helpful editor is trying to tell me I have a problem in my code.\r\n%\r\n% However, in R2022b there was a great improvement to the programmatic\r\n% interface for identifying and addressing with these code issues. In fact,\r\n% this new interface is itself a new function called\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/codeissues.html codeIssues>!\r\n%\r\n% This new API for the code analyzer is in a word - powerful. Let's explore\r\n% how you can now use this to build quality into your project. Starting\r\n% with another look at our standard mass-spring-damper mini-codebase. Most\r\n% recently we talked about this when describing the\r\n% <https:\/\/blogs.mathworks.com\/developer\/2022\/10\/17\/building-blocks-with-buildtool\/\r\n% new MATLAB build tool>. This code base is pretty simple. It includes a\r\n% simulator, a function to return some design constants (spring constant,\r\n% damping coefficient, etc.), and some tests.\r\n% \r\n% <<y2023ToolboxFoldersAndFiles.png>>\r\n% \r\n% To get started on a codebase like this, just call codeIssues on the\r\n% folder you want to analyze:\r\n%\r\n% <<y2023CodeIssuesOutput.png>>\r\n%\r\n% You can see that with just a simple call to codeIssues you can quickly\r\n% get an overview of all the details a static analyzer dreams of. You can\r\n% easily dig into the files that were analyzed, the configuration, and the\r\n% very handy table of the issues found, as well as any issues that have\r\n% been suppressed through suppression pragmas in the editor. If you are in\r\n% MATLAB you can even click on each issue to get right to it in the MATLAB\r\n% editor where it can be fixed or suppressed if needed.\r\n%\r\n% Now with this beautiful API at our fingertips, and with the build tool to\r\n% boot, we can lock down our code in a much more robust, automated way. We\r\n% can start roughly where we left off at the end of our build tool post\r\n% with the following buildfile with a mex and a test task:\r\n%\r\n% <include>.hidden\/starting_build\/buildfile.m<\/include>\r\n%\r\n% Let's go ahead and add a \"codeIssues\" task to this build by creating a new\r\n% local function called *|codeIssuesTask|*:\r\n%\r\n% <include>.hidden\/codesnippets\/codeIssuesTask1.m<\/include>\r\n%\r\n% This is quite simple, we just want to find all the issues under the\r\n% \"toolbox\" folder and throw an assertion error if any of them are of\r\n% Severity \"error\". This is just about the quickest win you can apply to a\r\n% code base to build quality into it. This can find syntax and other errors\r\n% statically, without even writing or running a single test. There really\r\n% is no reason at all we shouldn't apply this task to every project. It\r\n% costs virtually nothing and can be remarkably efficient at finding bugs.\r\n% On that note, let's add it as a default task in our buildfile:\r\n%\r\n% <include>.hidden\/codesnippets\/defaultTasks.m<\/include>\r\n%\r\n% ...and with that we now have this check built right into our standard\r\n% development process. To show this let's first put a file with an error\r\n% into the code base and then we can call the build tool:\r\n%\r\n% <include>.changes\/syntaxError.m<\/include>\r\n% \r\ncopyfile .changes\/syntaxError.m toolbox\/syntaxError.m\r\ntry\r\n    buildtool\r\ncatch ex\r\n    disp(ex.getReport(\"basic\"));\r\nend\r\n\r\n%%\r\n% The build tool has done its part in stopping our development process in\r\n% its tracks when it sees we have some important syntax errors to address.\r\n%\r\n% Let's remove that bunk error so we can see our \"codeIssues\" task complete\r\n% successfully. When we do this we also successfully execute the other\r\n% \"mex\" and \"test\" tasks to complete the whole workflow.\r\ndelete toolbox\/syntaxError.m\r\nbuildtool\r\n\r\n%%\r\n%\r\n% *One last thing* \r\n%\r\n% Now we have you all setup nice and cozy with the protection that static\r\n% analysis gives you. However, while we fail on static analysis errors, I\r\n% am still uncomfortable with how easy it is to continue to add constructs\r\n% to my code that result in static analysis warnings, which often point to\r\n% real problems in your program. We could also fail the build on warnings\r\n% if we'd like, but I didn't want to start with that idea out of the gate.\r\n%\r\n% It is pretty clear that we want this protection with full-on-bonafide\r\n% errors, which are almost always bugs. We run into a problem though when a\r\n% code base already has an inventory of warnings. It would be fantastic to\r\n% go through that inventory and fix all of those warnings as well. In fact,\r\n% the new code analysis tooling makes that very easy in many cases!\r\n% However, you may not be up for this right now. Your code base may be\r\n% large, and you may want or need to invest a bit more time into this\r\n% activity. So our first crack at this failed the build only for issues\r\n% with an \"error\" Severity.\r\n%\r\n% However, if you know me you know I like to sneak one last thing in. What\r\n% if we accepted all of our current warnings in the codebase, but wanted to\r\n% lock down our code base such that we are protected from introducing new\r\n% warnings? To me this sounds like a great idea. We can then ratchet down\r\n% our warnings by preventing inflow of new warnings and can remove existing\r\n% warnings over time through interacting with the codebase. How can we do\r\n% this? We can leverage the power of the codeIssues programmatic API!\r\n%\r\n% We can do this by capturing and saving our existing warnings to a\r\n% baseline of known issues. As MATLAB tables, theses issues are in a nice\r\n% representation to save in a *.csv or *.xlsx file. Saving them in this\r\n% format makes it really easy to tweak them, open them outside of MATLAB,\r\n% or even remove issues that have been fixed.\r\n% \r\n% To do this we just need to make a couple tweaks to the issues table. We\r\n% need to overwrite the *|Location|* variable with relative paths to the\r\n% files, remove the *|FullFilename|* variable, and make a quick datatype\r\n% tweak to allow for nice CSV'ing. The relative filename adjustment is\r\n% important because we want to be able to compare these results across\r\n% different machines and the full path is likely to differ across\r\n% environments. Such environments include the desktops of individual\r\n% engineers as well as different build agents in a CI system. \r\n% \r\n% That function looks as follows:\r\n%\r\n% <include>.hidden\/codesnippets\/preprocessIssues.m<\/include>\r\n%\r\n% ...and now with this function we can create a new task in the buildfile\r\n% to generate a new baseline:\r\n%\r\n% <include>.hidden\/codesnippets\/captureWarningsBaselineTask.m<\/include>\r\n%\r\n% Let's do it!\r\nbuildtool captureWarningsBaseline\r\n\r\n%%\r\n% Great I now see the csv files. We can take a peek:\r\ntype knownIssues.csv\r\n\r\n%%\r\n% Beautiful. In this case we just have two minor warnings that I don't want\r\n% to look into quite yet. However, now we can adjust the \"codeIssues\" task\r\n% to prevent me from introducing anything new:\r\n%\r\n% <include>.hidden\/codesnippets\/codeIssuesTask2.m<\/include>\r\n%\r\n% This now loads the issues and does a setdiff to ignore those that are\r\n% already known and captured in our baseline CSV file. This way, at least\r\n% from now on I won't introduce any new warnings to the code base. It can\r\n% only get better from here. Also, if I change some file that has an\r\n% existing warning, there is a decent chance that my build tooling is going\r\n% to yell at me because the existing warning is slightly different. For\r\n% example it might be on a different line due to changes made in the file.\r\n%\r\n% If this happens, great! Make me clean up or suppress the warning while I\r\n% have the file open and modified. That's a feature not a bug. Worst case\r\n% scenario, I can always capture a new baseline if I really can't look into\r\n% it immediately, but I love this approach to help me clean my code through\r\n% the process.\r\n%\r\n% What does this look like? Let's add a file to the codebase with a new\r\n% warning:\r\n%\r\n% <include>.changes\/codeWarning.m<\/include>\r\n%\r\n% ...and invoke the build:\r\ncopyfile .changes\/codeWarning.m toolbox\/codeWarning.m\r\ntry\r\n    buildtool\r\ncatch ex\r\n    disp(ex.getReport(\"basic\"));\r\nend\r\n\r\ndelete toolbox\/codeWarning.m\r\n\r\n\r\n%%\r\n% Love it! I am now protected from myself. I can leverage this in my\r\n% standard toolbox development process to help ensure that over time my\r\n% code only gets better, not worse. You could also imagine tweaking this to\r\n% fail or otherwise notify when a warning goes away from the known issues\r\n% so that we have some pressure to help ensure the lockdown gets tighter\r\n% and tighter as time goes on. For reference, here is the final buildfile\r\n% used for this workflow discussed today:\r\n%\r\n% <include>buildfile.m<\/include>\r\n%\r\n% There you have it, a clean API for MATLAB's code analysis and a standard way\r\n% to include this in the development process using the build tool. I can\r\n% practically feel the quality getting higher!\r\n%\r\n% Folks, there is so much to blog about in the next little while. There's\r\n% more to discuss here on how we can leverage new and improving tools to\r\n% develop clean build and test pipelines for your MATLAB projects. Also, I\r\n% am *so* excited for some really fantastic progress we will be able to\r\n% share shortly. Buckle in, we are gonna be talking about high quality\r\n% test and automation developments for a bit here on this blog. Chime in\r\n% with your insights, tools, and workflows you use as you develop your\r\n% professional MATLAB projects. \r\n##### SOURCE END ##### 05bf3646ac79494094e164ceb13bc142\r\n-->","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img src=\"https:\/\/blogs.mathworks.com\/developer\/files\/y2023CodeIssuesOutput.png\" class=\"img-responsive attachment-post-thumbnail size-post-thumbnail wp-post-image\" alt=\"\" decoding=\"async\" loading=\"lazy\" \/><\/div><p>\r\n\r\nWho among us doesn't have issues, amirite? Let's just take a moment and acknowledge this fact and I think we can always be a bit more honest and understanding of all of our unique issues and the... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/developer\/2023\/03\/15\/static-analysis-code-checking-and-linting-with-codeissues\/\">read more >><\/a><\/p>","protected":false},"author":90,"featured_media":2973,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[39,42,4,7,33],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/2964"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/users\/90"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/comments?post=2964"}],"version-history":[{"count":9,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/2964\/revisions"}],"predecessor-version":[{"id":2997,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/2964\/revisions\/2997"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/media\/2973"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/media?parent=2964"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/categories?post=2964"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/tags?post=2964"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}