{"id":737,"date":"2013-08-05T12:00:26","date_gmt":"2013-08-05T17:00:26","guid":{"rendered":"https:\/\/blogs.mathworks.com\/cleve\/?p=737"},"modified":"2013-08-04T15:16:29","modified_gmt":"2013-08-04T20:16:29","slug":"eigshow-week-3","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/cleve\/2013\/08\/05\/eigshow-week-3\/","title":{"rendered":"eigshow, week 3"},"content":{"rendered":"\r\n<div class=\"content\"><!--introduction--><p>An option in <tt>eigshow<\/tt> demonstrates SVD, the matrix singular value decomposition.  The emphasis is on orthogonality.<\/p><!--\/introduction--><h3>Contents<\/h3><div><ul><li><a href=\"#e73c149e-9481-45a4-8e6a-4e57bb4ff79c\">The default matrix<\/a><\/li><li><a href=\"#7c5d6fe9-09cf-4189-bc6b-c4a75d068e3b\">The SVD<\/a><\/li><li><a href=\"#78b63b95-bdc8-43e8-b2cc-f4e332cf6f2f\">Existence<\/a><\/li><li><a href=\"#6bb64804-15aa-455b-ae69-b1cf4d1e8c4d\">A defective matrix<\/a><\/li><li><a href=\"#058e3b63-356f-423d-b5a0-4d0f8298662a\">A singular matrix<\/a><\/li><\/ul><\/div><h4>The default matrix<a name=\"e73c149e-9481-45a4-8e6a-4e57bb4ff79c\"><\/a><\/h4><p>This is the third posting about one of my favorite MATLAB programs, <tt>eigshow<\/tt>. The first two postings were about the <tt>'eig'<\/tt> option for eigenvalues. This posting is about the <tt>'svd'<\/tt> option for the singular value decomposition.<\/p><p>The default matrix, shown in the figure window title, is<\/p><pre class=\"codeinput\"> format <span class=\"string\">rat<\/span>, format <span class=\"string\">compact<\/span>\r\n A = [1 3; 4 2]\/4\r\n<\/pre><pre class=\"codeoutput\">A =\r\n       1\/4            3\/4     \r\n       1              1\/2     \r\n<\/pre><p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/cleve\/svdshowp_w3a.gif\" alt=\"\"> <\/p><p>With the <tt>'svd'<\/tt> option, you have two green vectors, <tt>x<\/tt> and <tt>y<\/tt>, that are perpendicular to each other and that have unit length. They remain perpendicular as you move <tt>x<\/tt> around the unit circle with the mouse, so they always form an orthonormal basis for two-dimensional space.<\/p><p>The two blue vectors, <tt>Ax<\/tt> and <tt>Ay<\/tt>, are images of <tt>x<\/tt> and <tt>y<\/tt> under the mapping induced by the matrix <tt>A<\/tt>.  The angle between them changes and their lengths change as you move around the space.  The instructions at the bottom of the frame say<\/p><p><tt>Make A*x perpendicular to A*y<\/tt><\/p><h4>The SVD<a name=\"7c5d6fe9-09cf-4189-bc6b-c4a75d068e3b\"><\/a><\/h4><p>When you reach the configuration where not only <tt>x<\/tt> is perpendicular to <tt>y<\/tt> but also <tt>Ax<\/tt> is perpendicular to <tt>Ay<\/tt>, you have found the singular value decomposition.  The SVD is usually described as the matrix factorization<\/p><p>$$ A = U \\Sigma V^T $$<\/p><p>with orthogonal $U$ and $V$ and diagonal $\\Sigma$. This can be written<\/p><p>$$ A V = U \\Sigma $$<\/p><p>For each column of $V$ and the corresponding column of $U$,<\/p><p>$$ A v_j = \\sigma_j u_j $$<\/p><p>We can see that when <tt>Ax<\/tt> is perpendicular to <tt>Ay<\/tt>, the vectors <tt>x<\/tt> and <tt>y<\/tt> are the columns of $V$ and the vectors <tt>Ax<\/tt> and <tt>Ay<\/tt> are the columns of $U \\Sigma$.  The blue curve, which is the image of the unit circle under the transformation induced by $A$, is an ellipse.  The vectors <tt>Ax<\/tt> and <tt>Ay<\/tt> are its major and minor axes and the half lengths of these axes are the singular values of the matrix.<\/p><p>For this first matrix it looks like the length of <tt>Ax<\/tt> might be about 1.25 and the length of <tt>Ay<\/tt> might be about 0.50.  We can check this by actually computing the singular values.<\/p><pre class=\"codeinput\"> format <span class=\"string\">short<\/span>\r\n sigma = svd(A)\r\n<\/pre><pre class=\"codeoutput\">sigma =\r\n    1.2792\r\n    0.4886\r\n<\/pre><h4>Existence<a name=\"78b63b95-bdc8-43e8-b2cc-f4e332cf6f2f\"><\/a><\/h4><p>In our previous posts about eigenvalues, we found that real eigenvectors don't always exist.  If the eigenvectors are complex, <tt>eigshow<\/tt> can never find them.  And if the matrix is defective, <tt>eigshow<\/tt> finds only one eigenvector, not the usual two.  The situation is quite different for SVD.  The real singular value decomposition always exists.  It is always possible to rotate <tt>x<\/tt> and <tt>y<\/tt> to a position that makes <tt>Ax<\/tt> orthogonal to <tt>Ay<\/tt>.<\/p><p>In the language of abstract linear algebra, we say that the green vectors are an orthonormal basis for the <i>domain<\/i> of the transformation and, if the blue vectors were scaled to have unit length, they would be an orthonormal basis for the <i>range<\/i> of the transformation.<\/p><h4>A defective matrix<a name=\"6bb64804-15aa-455b-ae69-b1cf4d1e8c4d\"><\/a><\/h4><p>Our final matrix in the eigenvalues post was defective -- it had only one eigenvector.  But its SVD is obtained with no difficulty.<\/p><pre class=\"codeinput\"> format <span class=\"string\">rat<\/span>\r\n A = [6 4; -1 2]\/4\r\n<\/pre><pre class=\"codeoutput\">A =\r\n       3\/2            1       \r\n      -1\/4            1\/2     \r\n<\/pre><p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/cleve\/svdshowp_w3b.gif\" alt=\"\"> <\/p><p>At first it looks like the final blue vectors happen to be parallel to the display axes.  This would say that the matrix $U$ in the SVD is the identity matrix.  But a careful look at the final result shows slight stair-stepping in <tt>Ax<\/tt> and <tt>Ay<\/tt>.  This is confirmed by actually computing the full SVD and observing that $U$ is not quite the identity.<\/p><pre class=\"codeinput\"> format <span class=\"string\">short<\/span>\r\n [U,S,V] = svd(A)\r\n<\/pre><pre class=\"codeoutput\">U =\r\n   -0.9991   -0.0424\r\n   -0.0424    0.9991\r\nS =\r\n    1.8042         0\r\n         0    0.5542\r\nV =\r\n   -0.8247   -0.5655\r\n   -0.5655    0.8247\r\n<\/pre><h4>A singular matrix<a name=\"058e3b63-356f-423d-b5a0-4d0f8298662a\"><\/a><\/h4><p>Our singular example turns out to be a little tricky because the blue ellipse collapses to a one-dimensional line segment.  It is hard to tell when <tt>Ax<\/tt> is perpendicular to <tt>Ay<\/tt>.  It turns out to be when one of them is the zero vector; in other words, when <tt>x<\/tt> or <tt>y<\/tt> is the null vector of <tt>A<\/tt>.  When one of <tt>Ax<\/tt> or <tt>Ay<\/tt> reaches zero, the other reaches its maximum length, the half-length of the line segment. This length is the nonzero singular value.<\/p><pre class=\"codeinput\"> format <span class=\"string\">rat<\/span>\r\n A = [2 4; 2 4]\/4\r\n<\/pre><pre class=\"codeoutput\">A =\r\n       1\/2            1       \r\n       1\/2            1       \r\n<\/pre><p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/cleve\/svdshowp_w3c.gif\" alt=\"\"> <\/p><script language=\"JavaScript\"> <!-- \r\n    function grabCode_0c6e1685d32844a0a3a769473123ef50() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='0c6e1685d32844a0a3a769473123ef50 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 0c6e1685d32844a0a3a769473123ef50';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2013 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><br><a href=\"javascript:grabCode_0c6e1685d32844a0a3a769473123ef50()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      Published with MATLAB&reg; R2013b<br><\/p><p class=\"footer\"><br>\r\n      Published with MATLAB&reg; R2013b<br><\/p><\/div><!--\r\n0c6e1685d32844a0a3a769473123ef50 ##### SOURCE BEGIN #####\r\n%% eigshow, week 3\r\n% An option in |eigshow| demonstrates SVD, the matrix singular value\r\n% decomposition.  The emphasis is on orthogonality.\r\n%\r\n%% The default matrix\r\n% This is the third posting about one of my favorite MATLAB programs, |eigshow|.\r\n% The first two postings were about the |'eig'| option for eigenvalues. \r\n% This posting is about the |'svd'| option for the singular value decomposition.\r\n\r\n%%\r\n% The default matrix, shown in the figure window title, is\r\n\r\n format rat, format compact\r\n A = [1 3; 4 2]\/4\r\n\r\n%%\r\n%\r\n% <<svdshowp_w3a.gif>>\r\n%\r\n\r\n%%\r\n% With the |'svd'| option, you have two green vectors, |x| and |y|,\r\n% that are perpendicular to each other and that have unit length.\r\n% They remain perpendicular as you move |x| around the unit circle\r\n% with the mouse, so they always form an orthonormal basis for\r\n% two-dimensional space.\r\n\r\n%%\r\n% The two blue vectors, |Ax| and |Ay|, are images of |x| and |y| under\r\n% the mapping induced by the matrix |A|.  The angle between them changes\r\n% and their lengths change as you move around the space.  The instructions\r\n% at the bottom of the frame say\r\n%\r\n% |Make A*x perpendicular to A*y|\r\n% \r\n\r\n%% The SVD\r\n% When you reach the configuration where not only |x| is perpendicular to |y|\r\n% but also |Ax| is perpendicular to |Ay|, you have found the singular value\r\n% decomposition.  The SVD is usually described as the matrix factorization\r\n%\r\n% $$ A = U \\Sigma V^T $$\r\n%\r\n% with orthogonal $U$ and $V$ and diagonal $\\Sigma$.\r\n% This can be written\r\n%\r\n% $$ A V = U \\Sigma $$\r\n%\r\n% For each column of $V$ and the corresponding column of $U$,\r\n%\r\n% $$ A v_j = \\sigma_j u_j $$\r\n%\r\n\r\n%%\r\n% We can see that when |Ax| is perpendicular to |Ay|, the vectors |x| and |y|\r\n% are the columns of $V$ and the vectors |Ax| and |Ay| are the columns of\r\n% $U \\Sigma$.  The blue curve, which is the image of the unit circle under\r\n% the transformation induced by $A$, is an ellipse.  The vectors |Ax| and\r\n% |Ay| are its major and minor axes and the half lengths of these axes are\r\n% the singular values of the matrix.\r\n\r\n%%\r\n% For this first matrix it looks like the length of |Ax| might be about 1.25\r\n% and the length of |Ay| might be about 0.50.  We can check this by actually\r\n% computing the singular values.\r\n\r\n format short\r\n sigma = svd(A)\r\n\r\n%% Existence\r\n% In our previous posts about eigenvalues, we found that real eigenvectors \r\n% don't always exist.  If the eigenvectors are complex, |eigshow| can\r\n% never find them.  And if the matrix is defective, |eigshow| finds only one \r\n% eigenvector, not the usual two.  The situation is quite different for\r\n% SVD.  The real singular value decomposition always exists.  It is always\r\n% possible to rotate |x| and |y| to a position that makes |Ax| orthogonal\r\n% to |Ay|.\r\n\r\n%%\r\n% In the language of abstract linear algebra, we say that the green vectors\r\n% are an orthonormal basis for the _domain_ of the transformation\r\n% and, if the blue vectors were scaled to have unit length, they would be\r\n% an orthonormal basis for the _range_ of the transformation.  \r\n\r\n%% A defective matrix\r\n% Our final matrix in the eigenvalues post was defective REPLACE_WITH_DASH_DASH it had only one\r\n% eigenvector.  But its SVD is obtained with no difficulty.\r\n\r\n format rat\r\n A = [6 4; -1 2]\/4\r\n\r\n%%\r\n%\r\n% <<svdshowp_w3b.gif>>\r\n%\r\n\r\n%%\r\n% At first it looks like the final blue vectors happen to be parallel to\r\n% the display axes.  This would say that the matrix $U$ in the SVD is the\r\n% identity matrix.  But a careful look at the final result shows slight\r\n% stair-stepping in |Ax| and |Ay|.  This is confirmed by actually computing\r\n% the full SVD and observing that $U$ is not quite the identity.\r\n\r\n format short\r\n [U,S,V] = svd(A)\r\n\r\n%% A singular matrix\r\n% Our singular example turns out to be a little tricky because the blue\r\n% ellipse collapses to a one-dimensional line segment.  It is hard to\r\n% tell when |Ax| is perpendicular to |Ay|.  It turns out to be when one of\r\n% them is the zero vector; in other words, when |x| or |y| is the null\r\n% vector of |A|.  When one of |Ax| or |Ay| reaches zero, the other reaches\r\n% its maximum length, the half-length of the line segment.\r\n% This length is the nonzero singular value.\r\n\r\n format rat\r\n A = [2 4; 2 4]\/4\r\n\r\n%%\r\n%\r\n% <<svdshowp_w3c.gif>>\r\n%\r\n\r\n##### SOURCE END ##### 0c6e1685d32844a0a3a769473123ef50\r\n-->","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/images\/cleve\/svdshowp_w3a.gif\" onError=\"this.style.display ='none';\" \/><\/div><!--introduction--><p>An option in <tt>eigshow<\/tt> demonstrates SVD, the matrix singular value decomposition.  The emphasis is on orthogonality.... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/cleve\/2013\/08\/05\/eigshow-week-3\/\">read more >><\/a><\/p>","protected":false},"author":78,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[13,6],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/737"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/users\/78"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/comments?post=737"}],"version-history":[{"count":3,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/737\/revisions"}],"predecessor-version":[{"id":740,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/posts\/737\/revisions\/740"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/media?parent=737"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/categories?post=737"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/cleve\/wp-json\/wp\/v2\/tags?post=737"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}