{"id":3230,"date":"2019-02-20T08:27:51","date_gmt":"2019-02-20T13:27:51","guid":{"rendered":"https:\/\/blogs.mathworks.com\/loren\/?p=3230"},"modified":"2019-03-14T19:44:20","modified_gmt":"2019-03-15T00:44:20","slug":"the-missing-link","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/loren\/2019\/02\/20\/the-missing-link\/","title":{"rendered":"The Missing Link"},"content":{"rendered":"<div class=\"content\"><!--introduction--><p>Of course the data we collect is always perfect - NOT!   Maybe yours is different.  What can go wrong?  So many things. Instruments drift, web sites go down, power goes out, ...  So what can you do if you have gaps in your data, and the analysis you want to perform won't tolerate that condition?  You might decide you need to fill in missing values.<\/p><p>We've been working on supplying functionality that makes dealing with missing data easier for a long time, starting with the introduction of <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/nan.html\"><tt>NaN<\/tt><\/a> values right in the beginning. In a floating point array, NaNs act as placeholders.  That's great, but what can you do from there?<\/p><!--\/introduction--><h3>Contents<\/h3><div><ul><li><a href=\"#b5bf554c-dd94-4bd7-9733-66cb4792e254\">Missing Capabilities<\/a><\/li><li><a href=\"#efb3f974-d245-4862-9df5-80f362d9797e\">What Else is Missing?<\/a><\/li><\/ul><\/div><h4>Missing Capabilities<a name=\"b5bf554c-dd94-4bd7-9733-66cb4792e254\"><\/a><\/h4><p>Some functions, or variants of them, work differently your array contains any <tt>NaN<\/tt> values, e.g., <tt><a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/mean.html\">mean<\/a><\/tt>.<\/p><p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"http:\/\/blogs.mathworks.com\/images\/loren\/2019\/meanNaNDoc.png\" alt=\"\"> <\/p><p>We first helped you figure out if you <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/ismissing.html\">have missing values<\/a>. And later added the ability to <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/fillmissing.html\">fill<\/a> and <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/rmmissing.html\">remove missing values<\/a>. More recently, we added the ability to <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/missing.html\">mark<\/a> missing values, even if you don't know the datatype of the array.  This makes it easier to supply <tt>NaN<\/tt>, <tt><a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/nat.html\">NaT<\/a><\/tt> (not a time) values, and similarly for <tt>categorical<\/tt> and <tt>string<\/tt> arrays, without needing to know which one is appropriate - as may happen with different columns in a <tt>table<\/tt>.<\/p><h4>What Else is Missing?<a name=\"efb3f974-d245-4862-9df5-80f362d9797e\"><\/a><\/h4><p>Do you use the functionality to deal with missing values?  If so, tell us how.  If not, please tell us what is missing!<\/p><p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"http:\/\/blogs.mathworks.com\/images\/loren\/2019\/missingLink.jpg\" alt=\"\"> <\/p><p>You can let us know <a href=\"https:\/\/blogs.mathworks.com\/loren\/?p=3230#respond\">here<\/a>.<\/p><script language=\"JavaScript\"> <!-- \r\n    function grabCode_54c2920d37244374bb38a647b2b5cb8a() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='54c2920d37244374bb38a647b2b5cb8a ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 54c2920d37244374bb38a647b2b5cb8a';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2019 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><br><a href=\"javascript:grabCode_54c2920d37244374bb38a647b2b5cb8a()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      Published with MATLAB&reg; R2018b<br><\/p><\/div><!--\r\n54c2920d37244374bb38a647b2b5cb8a ##### SOURCE BEGIN #####\r\n%% The Missing Link\r\n% Of course the data we collect is always perfect - NOT!   Maybe yours is\r\n% different.  What can go wrong?  So many things. Instruments drift, web\r\n% sites go down, power goes out, ...  So what can you do if you have gaps\r\n% in your data, and the analysis you want to perform won't tolerate that\r\n% condition?  You might decide you need to fill in missing values.\r\n%\r\n% We've been working on supplying functionality that makes dealing with\r\n% missing data easier for a long time, starting with the introduction of\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/nan.html |NaN|>\r\n% values right in the beginning. In a floating point array, NaNs act as\r\n% placeholders.  That's great, but what can you do from there?\r\n\r\n%% Missing Capabilities\r\n% Some functions, or variants of them, work differently your array contains\r\n% any |NaN| values, e.g.,\r\n% |<https:\/\/www.mathworks.com\/help\/matlab\/ref\/mean.html mean>|.\r\n%\r\n% <<meanNaNDoc.png>>\r\n%\r\n% We first helped you figure out if you\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/ismissing.html have missing\r\n% values>. And later added the ability to\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/fillmissing.html fill> and\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/rmmissing.html remove missing\r\n% values>. More recently, we added the ability to\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/missing.html mark> missing\r\n% values, even if you don't know the datatype of the array.  This makes it\r\n% easier to supply |NaN|, \r\n% |<https:\/\/www.mathworks.com\/help\/matlab\/ref\/NaT.html NaT>| (not a time)\r\n% values, and similarly for |categorical| and |string| arrays, without needing to know which one is appropriate - as may happen\r\n% with different columns in a |table|.\r\n\r\n%% What Else is Missing?\r\n% Do you use the functionality to deal with missing values?  If so, tell us\r\n% how.  If not, please tell us what is missing! \r\n%\r\n% <<missingLink.jpg>>\r\n%\r\n% You can let us know <https:\/\/blogs.mathworks.com\/loren\/?p=3230#respond\r\n% here>.\r\n##### SOURCE END ##### 54c2920d37244374bb38a647b2b5cb8a\r\n-->","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"http:\/\/blogs.mathworks.com\/images\/loren\/2019\/meanNaNDoc.png\" onError=\"this.style.display ='none';\" \/><\/div><!--introduction--><p>Of course the data we collect is always perfect - NOT!   Maybe yours is different.  What can go wrong?  So many things. Instruments drift, web sites go down, power goes out, ...  So what can you do if you have gaps in your data, and the analysis you want to perform won't tolerate that condition?  You might decide you need to fill in missing values.... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/loren\/2019\/02\/20\/the-missing-link\/\">read more >><\/a><\/p>","protected":false},"author":39,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[78,39,6],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/3230"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/comments?post=3230"}],"version-history":[{"count":7,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/3230\/revisions"}],"predecessor-version":[{"id":3272,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/3230\/revisions\/3272"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/media?parent=3230"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/categories?post=3230"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/tags?post=3230"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}