{"id":3224,"date":"2017-11-08T14:21:52","date_gmt":"2017-11-08T19:21:52","guid":{"rendered":"https:\/\/blogs.mathworks.com\/videos\/?p=3224"},"modified":"2017-11-15T15:37:53","modified_gmt":"2017-11-15T20:37:53","slug":"writing-a-script-to-count-data-in-multiple-text-files","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/videos\/2017\/11\/08\/writing-a-script-to-count-data-in-multiple-text-files\/","title":{"rendered":"Writing a Script to Count Data in Multiple Text Files"},"content":{"rendered":"<p>Here I need to combine the data from multiple text files by grouping and summing by one variable. I also try using a <tt>datastore<\/tt> which I have not used much before.<\/p>\n<p>Features covered in this <a href=\"https:\/\/blogs.mathworks.com\/videos\/2015\/10\/29\/matlab-code-along-videos\/\">code-along<\/a> style video include:<\/p>\n<ul>\n<li><tt>readtable<\/tt>, <tt>fileDatastore<\/tt> with read function parameter<\/li>\n<li><tt>varfun<\/tt><\/li>\n<\/ul>\n<p><div class=\"row\"><div class=\"col-xs-12 containing-block\"><div class=\"bc-outer-container add_margin_20\"><videoplayer><div class=\"video-js-container\"><video data-video-id=\"5639594275001\" data-video-category=\"blog\" data-autostart=\"false\" data-account=\"62009828001\" data-omniture-account=\"mathwgbl\" data-player=\"rJ9XCz2Sx\" data-embed=\"default\" id=\"mathworks-brightcove-player\" class=\"video-js\" controls><\/video><script src=\"\/\/players.brightcove.net\/62009828001\/rJ9XCz2Sx_default\/index.min.js\"><\/script><script>if (typeof(playerLoaded) === 'undefined') {var playerLoaded = false;}(function isVideojsDefined() {if (typeof(videojs) !== 'undefined') {videojs(\"mathworks-brightcove-player\").on('loadedmetadata', function() {playerLoaded = true;});} else {setTimeout(isVideojsDefined, 10);}})();<\/script><\/div><\/videoplayer><\/div><\/div><\/div><\/p>\n<p>Play the video in full screen mode for a better viewing experience.\u00a0<\/p>\n<pre>\r\n\r\n%% Read Redirect Traffic from Server Logs\r\n\r\n%% Parameters\r\nfolderName='C:\\Folder';\r\noutputFileName='Redirects4Weeks.csv';\r\n\r\n%% Get List of Files\r\nfileInfo=dir([folderName filesep '*.csv']);\r\nfileName={fileInfo.name}';\r\nfullFileNames=cellfun(@(x) [folderName filesep x],fileName,'UniformOutput', false);\r\nfileDatesCell=regexp(fileName,'(\\d+-\\d+-\\d+)','tokens');\r\nfileDatesStr=stripCell(fileDatesCell,2);\r\nfileDates=datetime(fileDatesStr);\r\n[fileDates,idx]=sort(fileDates,'descend');\r\nmostRecent4=idx(1:4);\r\n\r\n%% Data Store\r\nfcn=@(x) readtable(x,'ReadVariableNames',false,'Delimiter',',','TextType','string', 'HeaderLines',1);\r\nds = fileDatastore(fullFileNames(mostRecent4),'ReadFcn', fcn);\r\ncellData=readall(ds);\r\ndata=vertcat(cellData{:});\r\ndata.Properties.VariableNames={'Request_URI' 'count'};\r\n\r\n%% Count URLs\r\npivot=varfun(@sum,data,'InputVariables','count','GroupingVariables',{'Request_URI'});\r\npivot.GroupCount=[];\r\npivot.Properties.VariableNames{'sum_count'}='count';\r\npivot=sortrows(pivot,{'count'},{'descend'});\r\n\r\n%% Save\r\nwritetable(pivot,outputFileName);\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<div class=\"thumbnail thumbnail_asset asset_overlay video\"><a href=\"https:\/\/blogs.mathworks.com\/videos\/2017\/11\/08\/writing-a-script-to-count-data-in-multiple-text-files\/?dir=autoplay\"><img decoding=\"async\" src=\"https:\/\/cf-images.us-east-1.prod.boltdns.net\/v1\/static\/62009828001\/11d33a20-f486-4daa-938e-100e85059755\/bbcc4aef-661a-43c3-be65-a5d4dfe36056\/1280x720\/match\/image.jpg\" onError=\"this.style.display ='none';\"\/><\/p>\n<div class=\"overlay_container\">\n      <span class=\"icon-video icon_color_null\"><time class=\"video_length\">37:54<\/time><\/span>\n      <\/div>\n<p>      <\/a><\/div>\n<p>Here I need to combine the data from multiple text files by grouping and summing by one variable. I also try using a datastore which I have not used much before.<br \/>\nFeatures covered in this code-along&#8230; <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/videos\/2017\/11\/08\/writing-a-script-to-count-data-in-multiple-text-files\/\">read more >><\/a><\/p>\n","protected":false},"author":133,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[27,4],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/posts\/3224"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/users\/133"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/comments?post=3224"}],"version-history":[{"count":8,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/posts\/3224\/revisions"}],"predecessor-version":[{"id":3238,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/posts\/3224\/revisions\/3238"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/media?parent=3224"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/categories?post=3224"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/videos\/wp-json\/wp\/v2\/tags?post=3224"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}