{"id":1922,"date":"2016-08-08T12:00:59","date_gmt":"2016-08-08T17:00:59","guid":{"rendered":"https:\/\/blogs.mathworks.com\/cleve\/?p=1922"},"modified":"2016-08-08T16:51:11","modified_gmt":"2016-08-08T21:51:11","slug":"bug-report-revives-interest-in-svd-option-of-eigshow","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/cleve\/2016\/08\/08\/bug-report-revives-interest-in-svd-option-of-eigshow\/","title":{"rendered":"Bug Report Revives Interest in SVD Option of &#8220;Eigshow&#8221;"},"content":{"rendered":"<div class=\"content\"><!--introduction--><\/p>\n<p>A few days ago we received email from Mike Hennessey, a mechanical engineering professor at the University of St. Thomas in St. Paul, Minnesota.  He has been reading my book &#8220;Numerical Computing with MATLAB&#8221; very carefully.  Chapter 7 is about &#8220;Eigenvalues and Singular Values&#8221; and section 10.3 is about one of my all-time favorite MATLAB demos, <tt>eigshow<\/tt>.  Mike discovered an error in my description of the <tt>svd<\/tt> option of <tt>eigshow<\/tt> that has gone unnoticed in the over ten years that the book has been available from both the MathWorks web site and SIAM.<\/p>\n<p><!--\/introduction--><\/p>\n<h3>Contents<\/h3>\n<div>\n<ul>\n<li><a href=\"#2a4ba0ab-3a7a-4cd6-ab65-85242771d750\">Eigshow<\/a><\/li>\n<li><a href=\"#b65b6103-d7d1-47b1-b781-da9543c19429\">NCM<\/a><\/li>\n<li><a href=\"#061440f4-bfa3-442e-8660-505ca498314b\">help eigshow<\/a><\/li>\n<li><a href=\"#62874f67-76f3-4293-966b-ff87e1d55132\">SVD<\/a><\/li>\n<li><a href=\"#97cf8dad-69c0-4058-871a-fef9870c8636\">Example<\/a><\/li>\n<li><a href=\"#fb648022-2387-48e3-9e73-5e1bcc1c3a9f\">Dominant singular vector<\/a><\/li>\n<li><a href=\"#ac4c403f-6a40-4fbe-8aa6-cb90252d2538\">The Correction<\/a><\/li>\n<li><a href=\"#7eaa3496-775f-4ddf-bc82-647d4ad75373\">Reference<\/a><\/li>\n<\/ul>\n<\/div>\n<p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/cleve\/files\/svdshow_animation.gif\" alt=\"\"> <\/p>\n<h4>Eigshow<a name=\"2a4ba0ab-3a7a-4cd6-ab65-85242771d750\"><\/a><\/h4>\n<p>The program <tt>eigshow<\/tt> has been in the MATLAB <tt>demos<\/tt> directory for many years.  I wrote a <a href=\"https:\/\/blogs.mathworks.com\/cleve\/2013\/07\/08\/eigshow-week-1\/\">three-part series of posts<\/a> about <tt>eigshow<\/tt> in this blog three years ago, but I&#8217;m happy to write another post now.<\/p>\n<p>The <tt>svd<\/tt> option of <tt>eigshow<\/tt> invites you to use your mouse to move the green vector <tt>x<\/tt> and make <tt>A*x<\/tt> perpendicular to <tt>A*y<\/tt>.  The animated gif above simulates that motion.  The following figure is the desired final result.<\/p>\n<p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/cleve\/files\/svdshow_final.png\" alt=\"\"> <\/p>\n<p>As you move <tt>x<\/tt>, the vector <tt>y<\/tt> follows along, staying perpendicular to <tt>x<\/tt>.  The two trace out the green unit circle.  The default matrix <tt>A<\/tt> is shown in the title.  (If you run your own <tt>eigshow<\/tt> you can change the matrix by editing the text box in the title.)<\/p>\n<pre class=\"codeinput\">   A = [1 3; 4 2]\/4\r\n<\/pre>\n<pre class=\"codeoutput\">A =\r\n    0.2500    0.7500\r\n    1.0000    0.5000\r\n<\/pre>\n<p>The blue vectors <tt>Ax<\/tt> and <tt>Ay<\/tt> are the images of <tt>x<\/tt> and <tt>y<\/tt> under the multiplicative mapping induced by <tt>A<\/tt>.  As you move <tt>x<\/tt> and <tt>y<\/tt> around the unit circle <tt>Ax<\/tt> and <tt>Ay<\/tt> sweep out the blue ellipse.  And when you stop with <tt>Ax<\/tt> perpendicular to <tt>Ay<\/tt>, they turn out to be the major and minor axes of the ellipse.<\/p>\n<h4>NCM<a name=\"b65b6103-d7d1-47b1-b781-da9543c19429\"><\/a><\/h4>\n<p><i>Numerical Computing with MATLAB<\/i>, which is known to its friends as simply <i>NCM<\/i>, was published over 10 years ago, in 2004.  An electronic edition is available from MathWorks and a print edition from SIAM. In chapter 7, the singular value decomposition, the SVD, of a real matrix, $A$, is defined as the product<\/p>\n<p>$$ A = U \\Sigma V^T$$<\/p>\n<p>In the simplest case where we assume $A$ is square, $U$ and $V$ are orthogonal and $\\Sigma$ is diagonal.<\/p>\n<p>This is illustrated by figure 10.3 in <i>NCM<\/i>, which is the same as the final figure here.  It shows the action with the <tt>svd<\/tt> option on the default matrix.  The explanation is given by the first sentence on page 7.<\/p>\n<p><p style=\"margin-left:3ex;\">\nThe vectors x and y are the columns of U in the SVD, the vectors<br \/>\nAx and Ay are multiples of the columns of V, and the lengths of<br \/>\nthe axes are the singular values.\n<\/p>\n<\/p>\n<p>Sounds OK, right?  This is the sentence that Mike Hennessey questioned. Now that I call your attention to it, you should be able to spot the error right away. I&#8217;ll tell you what it is at the end of this post.<\/p>\n<h4>help eigshow<a name=\"061440f4-bfa3-442e-8660-505ca498314b\"><\/a><\/h4>\n<p>Here is the relevant paragraph from the <tt>help<\/tt> entry for <tt>eigshow<\/tt> . It does not get into the same trouble as <i>NCM<\/i>.<\/p>\n<p><p style=\"margin-left:3ex;\">\nIn svd mode, the mouse moves two perpendicular unit vectors, x and y.<br \/>\nThe resulting A*x and A*y are plotted.  When A*x is perpendicular to<br \/>\nA*y, then x and y are right singular vectors, A*x and A*y are<br \/>\nmultiples of left singular vectors, and the lengths of A*x and A*y<br \/>\nare the corresponding singular values.\n<\/p>\n<\/p>\n<h4>SVD<a name=\"62874f67-76f3-4293-966b-ff87e1d55132\"><\/a><\/h4>\n<p>Here&#8217;s how I often think about the SVD.  Take the definition,<\/p>\n<p>$$A = U \\Sigma V^T$$<\/p>\n<p>Multiply both sides on the right by $V$.<\/p>\n<p>$$A V = U \\Sigma$$<\/p>\n<p>The diagonal matrix $\\Sigma$ is on the right so that the singular values can multiply the columns of $U$.  When we write this out column by column, we have<\/p>\n<p>$$A v_j = \\sigma_j u_j, \\ \\ j = 1,\\ &#8230;, n$$<\/p>\n<p>A little bit more manipulation leads to<\/p>\n<p>$$A^T u_j = \\sigma_j v_j, \\ \\ j = 1,\\ &#8230;, n$$<\/p>\n<h4>Example<a name=\"97cf8dad-69c0-4058-871a-fef9870c8636\"><\/a><\/h4>\n<p>Let&#8217;s see how this works out with the default 2-by-2 matrix.<\/p>\n<pre class=\"codeinput\">   A\r\n<\/pre>\n<pre class=\"codeoutput\">A =\r\n    0.2500    0.7500\r\n    1.0000    0.5000\r\n<\/pre>\n<p>Compute the SVD.<\/p>\n<pre class=\"codeinput\">   [U,S,V] = svd(A)\r\n<\/pre>\n<pre class=\"codeoutput\">U =\r\n   -0.5257   -0.8507\r\n   -0.8507    0.5257\r\nS =\r\n    1.2792         0\r\n         0    0.4886\r\nV =\r\n   -0.7678    0.6407\r\n   -0.6407   -0.7678\r\n<\/pre>\n<p>The singular values are on the diagonal of <tt>S<\/tt>.<\/p>\n<pre class=\"codeinput\">   sigma = diag(S)\r\n<\/pre>\n<pre class=\"codeoutput\">sigma =\r\n    1.2792\r\n    0.4886\r\n<\/pre>\n<p>These are the lengths of the two blue vectors.<\/p>\n<h4>Dominant singular vector<a name=\"fb648022-2387-48e3-9e73-5e1bcc1c3a9f\"><\/a><\/h4>\n<p>It turns out that the components of $v_1$, the right singular vector corresponding to the largest singular value of an (irreducible) matrix with nonnegative elements all have the same sign.  They could be all positive or all negative &#8212; take your pick.  MATLAB happens to always pick negative.  So, both components of V(:,1) are negative.  But we stopped <tt>eigshow<\/tt> with a positive <tt>x<\/tt>.  So we flip the sign of <tt>V(:,1)<\/tt>.<\/p>\n<pre class=\"codeinput\">   x = -V(:,1)\r\n   y = V(:,2)\r\n<\/pre>\n<pre class=\"codeoutput\">x =\r\n    0.7678\r\n    0.6407\r\ny =\r\n    0.6407\r\n   -0.7678\r\n<\/pre>\n<p>These two look like the two green vectors in the final figure. And these look like the two blue vectors.<\/p>\n<pre class=\"codeinput\">   Ax = A*x\r\n   Ay = A*y\r\n<\/pre>\n<pre class=\"codeoutput\">Ax =\r\n    0.6725\r\n    1.0881\r\nAy =\r\n   -0.4156\r\n    0.2569\r\n<\/pre>\n<p>Now check that the SVD relationship works.<\/p>\n<pre class=\"codeinput\">   AV = A*V\r\n   USigma = U*S\r\n<\/pre>\n<pre class=\"codeoutput\">AV =\r\n   -0.6725   -0.4156\r\n   -1.0881    0.2569\r\nUSigma =\r\n   -0.6725   -0.4156\r\n   -1.0881    0.2569\r\n<\/pre>\n<h4>The Correction<a name=\"ac4c403f-6a40-4fbe-8aa6-cb90252d2538\"><\/a><\/h4>\n<p>What was the error in <i>NCM<\/i> that Mike Hennessey reported and that prompted me to write this post?  The matrices U and V should be switched.  The sentence should read<\/p>\n<p><p style=\"margin-left:3ex;\">\nThe vectors x and y are the columns of V in the SVD, the vectors<br \/>\nAx and Ay are multiples of the columns of U, and the lengths of<br \/>\nthe axes are the singular values.\n<\/p>\n<\/p>\n<p>Every time I work with the SVD I have to think carefully about U and V. Which is which?  Maybe writing this post will help.<\/p>\n<h4>Reference<a name=\"7eaa3496-775f-4ddf-bc82-647d4ad75373\"><\/a><\/h4>\n<p>Cleve Moler, <i>Numerical Computering With MATLAB<\/i>, 2004.<\/p>\n<p>MathWorks, <a href=\"https:\/\/www.mathworks.com\/moler\/chapters.html\">&lt;https:\/\/www.mathworks.com\/moler\/chapters.html<\/a>&gt;,<\/p>\n<p>SIAM, <a href=\"http:\/\/epubs.siam.org\/doi\/book\/10.1137\/1.9780898717952\">&lt;http:\/\/epubs.siam.org\/doi\/book\/10.1137\/1.9780898717952<\/a>&gt;,<\/p>\n<p><script language=\"JavaScript\"> <!-- \n    function grabCode_7adce525276f45a9b3447ac7c9071a08() {\n        \/\/ Remember the title so we can use it in the new page\n        title = document.title;\n\n        \/\/ Break up these strings so that their presence\n        \/\/ in the Javascript doesn't mess up the search for\n        \/\/ the MATLAB code.\n        t1='7adce525276f45a9b3447ac7c9071a08 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 7adce525276f45a9b3447ac7c9071a08';\n    \n        b=document.getElementsByTagName('body')[0];\n        i1=b.innerHTML.indexOf(t1)+t1.length;\n        i2=b.innerHTML.indexOf(t2);\n \n        code_string = b.innerHTML.substring(i1, i2);\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\n\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \n        \/\/ in the XML parser.\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\n        \/\/ doesn't go ahead and substitute the less-than character. \n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\n\n        copyright = 'Copyright 2016 The MathWorks, Inc.';\n\n        w = window.open();\n        d = w.document;\n        d.write('\n\n<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\n\n\\n');\n\n        d.title = title + ' (MATLAB code)';\n        d.close();\n    }   \n     --> <\/script><\/p>\n<p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><a href=\"javascript:grabCode_7adce525276f45a9b3447ac7c9071a08()\"><span style=\"font-size: x-small;        font-style: italic;\">Get<br \/>\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><\/p>\n<p>      Published with MATLAB&reg; R2016a<\/p>\n<\/div>\n<p><!--\n7adce525276f45a9b3447ac7c9071a08 ##### SOURCE BEGIN #####\n%% Bug Report Revives Interest in SVD Option of \"Eigshow\"\n% A few days ago we received email from Mike Hennessey, a mechanical\n% engineering professor at the University of St. Thomas in St. Paul,\n% Minnesota.  He has been reading my book \"Numerical Computing\n% with MATLAB\" very carefully.  Chapter 7 is about \"Eigenvalues and\n% Singular Values\" and section 10.3 is about one of my all-time favorite\n% MATLAB demos, |eigshow|.  Mike discovered an error in my description\n% of the |svd| option of |eigshow| that has gone unnoticed in the over\n% ten years that the book has been available from both the MathWorks\n% web site and SIAM.\n\n%%\n% <<svdshow_animation.gif>>\n\n%% Eigshow\n% The program |eigshow| has been in the MATLAB |demos| directory for\n% many years.  I wrote a \n% <https:\/\/blogs.mathworks.com\/cleve\/2013\/07\/08\/eigshow-week-1\/\n% three-part series of posts> about |eigshow| in this blog three years\n% ago, but I'm happy to write another post now.\n\n%%\n% The |svd| option of |eigshow| invites you to use your mouse to move the\n% green vector |x| and make |A*x| perpendicular to |A*y|.  The animated\n% gif above simulates that motion.  The following figure is the desired\n% final result.\n%\n% <<svdshow_final.png>>\n\n%%\n% As you move |x|, the vector |y| follows along, staying perpendicular to\n% |x|.  The two trace out the green unit circle.  The default matrix\n% |A| is shown in the title.  (If you run your own |eigshow| you can\n% change the matrix by editing the text box in the title.)\n\n   A = [1 3; 4 2]\/4\n   \n%%\n% The blue vectors |Ax| and |Ay| are the images of |x| and |y| under the\n% multiplicative mapping induced by |A|.  As you move |x| and |y| around \n% the unit circle |Ax| and |Ay| sweep out the blue ellipse.  And when you \n% stop with |Ax| perpendicular to |Ay|, they turn out to be the major and\n% minor axes of the ellipse.\n\n%% NCM\n% _Numerical Computing with MATLAB_, which is known to its friends as\n% simply _NCM_, was published over 10 years ago, in 2004.  An electronic\n% edition is available from MathWorks and a print edition from SIAM.\n% In chapter 7, the singular value decomposition, the SVD, of a real\n% matrix, $A$, is defined as the product\n%\n% $$ A = U \\Sigma V^T$$\n%\n% In the simplest case where we assume $A$ is square, $U$ and $V$ are\n% orthogonal and $\\Sigma$ is diagonal.\n\n%% \n% This is illustrated by figure 10.3 in _NCM_, which is the same as the\n% final figure here.  It shows the action with the |svd| option on the\n% default matrix.  The explanation is given by the first sentence on\n% page 7.\n%\n% <html>\n% \n\n<p style=\"margin-left:3ex;\">\n% The vectors x and y are the columns of U in the SVD, the vectors\n% Ax and Ay are multiples of the columns of V, and the lengths of\n% the axes are the singular values.\n% <\/p>\n\n\n% <\/html>\n%\n% Sounds OK, right?  This is the sentence that Mike Hennessey questioned.\n% Now that I call your attention to it, you should be able to spot the\n% error right away. I'll tell you what it is at the end of this post.\n\n%% help eigshow\n% Here is the relevant paragraph from the |help| entry for |eigshow| .\n% It does not get into the same trouble as _NCM_.\n%\n% <html>\n% \n\n<p style=\"margin-left:3ex;\">\n% In svd mode, the mouse moves two perpendicular unit vectors, x and y.\n% The resulting A*x and A*y are plotted.  When A*x is perpendicular to\n% A*y, then x and y are right singular vectors, A*x and A*y are\n% multiples of left singular vectors, and the lengths of A*x and A*y\n% are the corresponding singular values.\n% <\/p>\n\n\n% <\/html>\n\n%% SVD\n% Here's how I often think about the SVD.  Take the definition,\n%\n% $$A = U \\Sigma V^T$$\n%\n% Multiply both sides on the right by $V$.\n%\n% $$A V = U \\Sigma$$\n\n%%\n% The diagonal matrix $\\Sigma$ is on the right so that the singular\n% values can multiply the columns of $U$.  When we write this out column by\n% column, we have\n%\n% $$A v_j = \\sigma_j u_j, \\ \\ j = 1,\\ ..., n$$\n%\n% A little bit more manipulation leads to\n%\n% $$A^T u_j = \\sigma_j v_j, \\ \\ j = 1,\\ ..., n$$\n\n%% Example\n% Let's see how this works out with the default 2-by-2 matrix.\n\n   A\n   \n%%\n% Compute the SVD.\n   \n   [U,S,V] = svd(A)\n   \n%%\n% The singular values are on the diagonal of |S|.\n\n   sigma = diag(S)\n   \n%%\n% These are the lengths of the two blue vectors.\n   \n   \n%% Dominant singular vector\n% It turns out that the components of $v_1$, the right singular vector\n% corresponding to the largest singular value of an (irreducible) matrix\n% with nonnegative elements all have the same sign.  They could be all\n% positive or all negative REPLACE_WITH_DASH_DASH take your pick.  MATLAB happens to always\n% pick negative.  So, both components of V(:,1) are negative.  But we\n% stopped |eigshow| with a positive |x|.  So we flip the sign of |V(:,1)|.\n\n   x = -V(:,1)\n   y = V(:,2)\n   \n%%\n% These two look like the two green vectors in the final figure.\n% And these look like the two blue vectors.\n\n   Ax = A*x\n   Ay = A*y\n   \n%%\n% Now check that the SVD relationship works.\n\n   AV = A*V\n   USigma = U*S\n   \n%% The Correction\n% What was the error in _NCM_ that Mike Hennessey reported and that\n% prompted me to write this post?  The matrices U and V should be\n% switched.  The sentence should read\n%\n% <html>\n% \n\n<p style=\"margin-left:3ex;\">\n% The vectors x and y are the columns of V in the SVD, the vectors\n% Ax and Ay are multiples of the columns of U, and the lengths of\n% the axes are the singular values.\n% <\/p>\n\n\n% <\/html>\n\n%%\n% Every time I work with the SVD I have to think carefully about U and V.\n% Which is which?  Maybe writing this post will help.\n\n%% Reference\n% Cleve Moler, _Numerical Computering With MATLAB_, 2004.\n%\n% MathWorks, <https:\/\/www.mathworks.com\/moler\/chapters.html\n% https:\/\/www.mathworks.com\/moler\/chapters.html>,\n%\n% SIAM, <http:\/\/epubs.siam.org\/doi\/book\/10.1137\/1.9780898717952\n% http:\/\/epubs.siam.org\/doi\/book\/10.1137\/1.9780898717952>,\n%\n##### SOURCE END ##### 7adce525276f45a9b3447ac7c9071a08\n--><\/p>\n","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img src=\"https:\/\/blogs.mathworks.com\/cleve\/files\/svdshow_final.png\" class=\"img-responsive attachment-post-thumbnail size-post-thumbnail wp-post-image\" alt=\"\" decoding=\"async\" loading=\"lazy\" \/><\/div>\n<p><!--introduction--><\/p>\n<p>A few days ago we received email from Mike Hennessey, a mechanical engineering professor at the University of St. Thomas in St. Paul, Minnesota.  He has been reading my book &#8220;Numerical Computing with MATLAB&#8221; very carefully.  Chapter 7 is about &#8220;Eigenvalues and Singular Values&#8221; and section 10.3 is about one of my all-time favorite MATLAB demos, <tt>eigshow<\/tt>.  Mike discovered an error in my description of the <tt>svd<\/tt> option of <tt>eigshow<\/tt> that has gone unnoticed in the over ten years that the book has been available from both the MathWorks web site and SIAM&#8230;. <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/cleve\/2016\/08\/08\/bug-report-revives-interest-in-svd-option-of-eigshow\/\">read more >><\/a><\/p>\n","protected":false},"author":78,"featured_media":1924,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[13,6,16,8],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/1922"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/users\/78"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/comments?post=1922"}],"version-history":[{"count":4,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/1922\/revisions"}],"predecessor-version":[{"id":1942,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/1922\/revisions\/1942"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/media\/1924"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/media?parent=1922"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/categories?post=1922"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/tags?post=1922"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}