{"id":11696,"date":"2024-10-07T10:31:38","date_gmt":"2024-10-07T14:31:38","guid":{"rendered":"https:\/\/blogs.mathworks.com\/cleve\/?p=11696"},"modified":"2024-10-07T10:31:38","modified_gmt":"2024-10-07T14:31:38","slug":"experience-with-chatbots-generating-matlab","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/cleve\/2024\/10\/07\/experience-with-chatbots-generating-matlab\/","title":{"rendered":"Experience With Chatbots Generating MATLAB"},"content":{"rendered":"<div class=\"content\"><!--introduction-->\r\n<p>A friend is investigating the use of generative AI in his classes. I asked two different popular chatbots to write MATLAB programs for a mathematically nontrivial problem. Both chatbots understood my query and both wrote plausible MATLAB programs, but one of the programs was not correct. My recommendation for coursework: carefully read and test programs produced by generative AI and repair any incorrect ones.<\/p>\r\n<!--\/introduction-->\r\n<h3>Contents<\/h3>\r\n<div>\r\n<ul>\r\n<li>\r\n<a href=\"#0c48695a-3a4f-46a2-9d9c-23506fc3aae1\">Redheffer, Mertens and Ahalt<\/a>\r\n<\/li>\r\n<li>\r\n<a href=\"#40ba61bb-33a4-43e5-9193-7a3889fb91f5\">Program number one<\/a>\r\n<\/li>\r\n<li>\r\n<a href=\"#8d0098c5-f272-4317-8eef-f58166e680af\">Program number two<\/a>\r\n<\/li>\r\n<li>\r\n<a href=\"#a1a49807-f649-4010-af50-85a315d67540\">Program number three<\/a>\r\n<\/li>\r\n<li>\r\n<a href=\"#c38da925-a549-4542-bb35-198572a00c66\">Happy Ending<\/a>\r\n<\/li>\r\n<\/ul>\r\n<\/div>\r\n<h4>Redheffer, Mertens and Ahalt<a name=\"0c48695a-3a4f-46a2-9d9c-23506fc3aae1\"><\/a>\r\n<\/h4>\r\n<p>Frequent readers of this blog know about my recent obsession with Redheffer matrices and the Mertens conjecture. So, when my friend Stan Ahalt visited from North Carolina State a few days ago, and wanted to discuss the use of generative AI in the new Data Science curriculum, I suggested asking chatbots to write MATLAB programs that check the Mertens conjecture.<\/p>\r\n<p>We logged onto the web sites for two different AI companies. Our query was the same at both sites.<\/p>\r\n<pre>Please write a MATLAB program to test the Mertens conjecture.<\/pre>\r\n<p>I was pleasantly surprised when both systems knew about MATLAB and understood the query. One of the systems replied:<\/p>\r\n<pre>Certainly! I'll create a MATLAB program to test the Mertens\r\nconjecture. First, let me briefly explain what the Mertens\r\nconjecture is, and then I'll provide the code.\r\nThe Mertens conjecture states that the Mertens function M(n)\r\nis bounded by &radic;n for all positive integers n. Mathematically,\r\nit can be expressed as:<\/pre>\r\n<pre>|M(n)| &le; &radic;n for all n &gt; 1<\/pre>\r\n<pre>Where M(n) is the Mertens function, defined as the sum of\r\nthe M&ouml;bius function \u03bc(k) for all positive integers k up to n:<\/pre>\r\n<pre>M(n) = \u03a3 \u03bc(k) for k = 1 to n<\/pre>\r\n<pre>Here's a MATLAB program to test this conjecture.<\/pre>\r\n<h4>Program number one<a name=\"40ba61bb-33a4-43e5-9193-7a3889fb91f5\"><\/a>\r\n<\/h4>\r\n<p>The program from one of the web sites begins:<\/p>\r\n<pre>function M = test_Mertens(n)\r\n   % Function to test the Mertens conjecture for a given n\r\n   % Returns true if the conjecture holds for all values up to n, false otherwise<\/pre>\r\n<p>This is followed by code to sum the M&ouml;bius function and verify that <tt>abs(M) &lt; sqrt(n)<\/tt>. The crucial <tt>mobius<\/tt> is a straight-forward implementation of the definition. The code relies on MATLAB intrinsics, <tt>factor<\/tt> and <tt>unique<\/tt>.<\/p>\r\n<pre>function mu = mobius(k)\r\n   % Function to calculate the M&ouml;bius function of k\r\n   % Factorize k\r\n   p = factor(k);\r\n   % Check for square factors\r\n   if length(p) ~= length(unique(p))\r\n      mu = 0;\r\n   else\r\n      mu = (-1)^length(p);\r\n   end\r\nend<\/pre>\r\n<p>I was very pleased. The entire program is only 40 lines long, eminently readable, and correct.<\/p>\r\n<h4>Program number two<a name=\"8d0098c5-f272-4317-8eef-f58166e680af\"><\/a>\r\n<\/h4>\r\n<p>The program from the other web site is more sophisticated. It knows enough about MATLAB to use its cumulative summation.<\/p>\r\n<pre>M = cumsum(mu);<\/pre>\r\n<p>Here is the crucial <tt>mobius<\/tt> function. At first glance, it looks OK. Is it?<\/p>\r\n<pre>function mu = mobius(n)\r\n   % Calculate M&ouml;bius function values up to n\r\n   mu = ones(1, n);\r\n   for i = 2:sqrt(n)\r\n      if mu(i) == 1\r\n         for j = i^2:i:n\r\n            if mod(j, i^2) == 0\r\n               mu(j) = 0;\r\n            else\r\n               mu(j) = -mu(j);\r\n            end\r\n         end\r\n      end\r\n   end\r\nend<\/pre>\r\n<p>As you have guessed, program number two is not correct. Even its own plot reveals that something is amiss.<\/p>\r\n<h4>Program number three<a name=\"a1a49807-f649-4010-af50-85a315d67540\"><\/a>\r\n<\/h4>\r\n<p>Let's return to the second Web site and make a more specific query.<\/p>\r\n<pre>Please write a MATLAB program that uses Redheffer\r\nmatrices to test the Mertens conjecture.<\/pre>\r\n<p>The resulting program generates a Redheffer matrix with<\/p>\r\n<pre>R = zeros(n,n);\r\nfor i = 1:n\r\n   for j = 1:n\r\n      if mod(i, j) == 0\r\n         R(i, j) = 1;\r\n      end\r\n   end\r\nend<\/pre>\r\n<p>It computes the determinant to obtain the Mertens value, <tt>M(n)<\/tt>. It is able to confirm the conjecture for all the <tt>n<\/tt> in the given range.<\/p>\r\n<p>What's wrong now? Look at the plot. The computed <tt>M(n)<\/tt> is always equal to one. Why?<\/p>\r\n<p>\r\n<img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/cleve\/files\/program3.png\" alt=\"\"> <\/p>\r\n<h4>Happy Ending<a name=\"c38da925-a549-4542-bb35-198572a00c66\"><\/a>\r\n<\/h4>\r\n<p>Program number three can be repaired by adding one line to the code that generates <tt>R<\/tt>. What is that line? Respond in the comments.<\/p>\r\n<p>Look at the good side. Instead of just using AI to do homework, we can challenge students to check the results carefully and repair any incorrect programs.<\/p>\r\n<script language=\"JavaScript\"> <!-- \r\n    function grabCode_b5785d3ceb4b4ff38b92ec4f957e13e3() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='b5785d3ceb4b4ff38b92ec4f957e13e3 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' b5785d3ceb4b4ff38b92ec4f957e13e3';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2024 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script>\r\n<p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\">\r\n<br>\r\n<a href=\"javascript:grabCode_b5785d3ceb4b4ff38b92ec4f957e13e3()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript>\r\n<\/span><\/a>\r\n<br>\r\n<br>\r\n      Published with MATLAB&reg; R2024a<br>\r\n<\/p>\r\n<\/div>\r\n<!--\r\nb5785d3ceb4b4ff38b92ec4f957e13e3 ##### SOURCE BEGIN #####\r\n%% Experience With Chatbots Generating MATLAB\r\n% A friend is investigating the use of generative AI in his classes.\r\n% I asked two different popular chatbots to write\r\n% MATLAB programs for a mathematically nontrivial problem.\r\n% Both chatbots understood my query and both wrote plausible\r\n% MATLAB programs, but one of the programs was not correct.\r\n% My recommendation for coursework: carefully read and test \r\n% programs produced by generative AI and repair any incorrect ones.\r\n\r\n%% Redheffer, Mertens and Ahalt\r\n% Frequent readers of this blog know about my recent obsession\r\n% with Redheffer matrices and the Mertens conjecture.\r\n% So, when my friend Stan Ahalt visited from North Carolina State\r\n% a few days ago, and wanted to discuss the use of generative AI in\r\n% the new Data Science curriculum, I suggested asking chatbots\r\n% to write MATLAB programs that check the Mertens conjecture.\r\n\r\n%%\r\n% We logged onto the web sites for two different AI companies.\r\n% Our query was the same at both sites.\r\n% \r\n%  Please write a MATLAB program to test the Mertens conjecture.\r\n%\r\n% I was pleasantly surprised when both systems knew about MATLAB\r\n% and understood the query.\r\n% One of the systems replied:\r\n%\r\n%  Certainly! I'll create a MATLAB program to test the Mertens\r\n%  conjecture. First, let me briefly explain what the Mertens\r\n%  conjecture is, and then I'll provide the code.\r\n%  The Mertens conjecture states that the Mertens function M(n) \r\n%  is bounded by \u221an for all positive integers n. Mathematically, \r\n%  it can be expressed as:\r\n%\r\n%  |M(n)| \u2264 \u221an for all n > 1\r\n%\r\n%  Where M(n) is the Mertens function, defined as the sum of \r\n%  the M\u00f6bius function \u03bc(k) for all positive integers k up to n:\r\n%\r\n%  M(n) = \u03a3 \u03bc(k) for k = 1 to n\r\n%\r\n%  Here's a MATLAB program to test this conjecture.\r\n%\r\n\r\n%% Program number one\r\n% The program from one of the web sites begins:\r\n%\r\n%  function M = test_Mertens(n)\r\n%     % Function to test the Mertens conjecture for a given n\r\n%     % Returns true if the conjecture holds for all values up to n, false otherwise\r\n%\r\n% This is followed by code to sum the M\u00f6bius function and verify that  \r\n% |abs(M) < sqrt(n)|.\r\n% The crucial |mobius| is a straight-forward implementation\r\n% of the definition.   The code relies on MATLAB intrinsics, |factor|\r\n% and |unique|.\r\n%\r\n%  function mu = mobius(k)\r\n%     % Function to calculate the M\u00f6bius function of k\r\n%     % Factorize k\r\n%     p = factor(k);     \r\n%     % Check for square factors\r\n%     if length(p) ~= length(unique(p))\r\n%        mu = 0;\r\n%     else\r\n%        mu = (-1)^length(p);\r\n%     end\r\n%  end\r\n\r\n%%\r\n% I was very pleased.  The entire program is only 40 lines long,\r\n% eminently readable, and correct.\r\n\r\n%% Program number two\r\n% The program from the other web site is more sophisticated.\r\n% It knows enough about MATLAB to use its cumulative summation.\r\n%\r\n%  M = cumsum(mu);\r\n%\r\n% Here is the crucial |mobius| function.  At first glance, it looks OK.\r\n% Is it?\r\n%\r\n%  function mu = mobius(n)\r\n%     % Calculate M\u00f6bius function values up to n\r\n%     mu = ones(1, n);  \r\n%     for i = 2:sqrt(n)\r\n%        if mu(i) == 1\r\n%           for j = i^2:i:n\r\n%              if mod(j, i^2) == 0\r\n%                 mu(j) = 0;\r\n%              else\r\n%                 mu(j) = -mu(j);\r\n%              end\r\n%           end\r\n%        end\r\n%     end\r\n%  end\r\n%\r\n% As you have guessed, program number two is not correct.\r\n% Even its own plot reveals that something is amiss.\r\n\r\n%% Program number three\r\n% Let's return to the second Web site and make a more specific query.\r\n%\r\n%  Please write a MATLAB program that uses Redheffer\r\n%  matrices to test the Mertens conjecture.\r\n%\r\n% The resulting program generates a Redheffer matrix with\r\n%\r\n%  R = zeros(n,n);\r\n%  for i = 1:n\r\n%     for j = 1:n\r\n%        if mod(i, j) == 0\r\n%           R(i, j) = 1;\r\n%        end\r\n%     end\r\n%  end\r\n%\r\n% It computes the determinant to obtain the Mertens value, |M(n)|.\r\n% It is able to confirm the conjecture for all the |n|\r\n% in the given range.\r\n%\r\n% What's wrong now?  Look at the plot.  The computed |M(n)| is always\r\n% equal to one.  Why?\r\n%\r\n% <<program3.png>>\r\n%\r\n\r\n%% Happy Ending\r\n% Program number three can be repaired by adding one line to the code that\r\n% generates |R|.  What is that line?  Respond in the comments.\r\n%\r\n% Look at the good side.  Instead of just using AI\r\n% to do homework, we can\r\n% challenge students to check the results carefully\r\n% and repair any incorrect programs.\r\n##### SOURCE END ##### b5785d3ceb4b4ff38b92ec4f957e13e3\r\n-->\r\n","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img src=\"https:\/\/blogs.mathworks.com\/cleve\/files\/program3.png\" class=\"img-responsive attachment-post-thumbnail size-post-thumbnail wp-post-image\" alt=\"\" decoding=\"async\" loading=\"lazy\" \/><\/div><!--introduction-->\r\n<p>A friend is investigating the use of generative AI in his classes. I asked two different popular chatbots to write MATLAB programs for a mathematically nontrivial problem. Both chatbots understood my query and both wrote plausible MATLAB programs, but one of the programs was not correct. My recommendation for coursework: carefully read and test programs produced by generative AI and repair any incorrect ones.... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/cleve\/2024\/10\/07\/experience-with-chatbots-generating-matlab\/\">read more >><\/a><\/p>","protected":false},"author":78,"featured_media":11687,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[11,4,37],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/11696"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/users\/78"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/comments?post=11696"}],"version-history":[{"count":5,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/11696\/revisions"}],"predecessor-version":[{"id":11753,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/11696\/revisions\/11753"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/media\/11687"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/media?parent=11696"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/categories?post=11696"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/tags?post=11696"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}