Stuart’s MATLAB Videos

Watch and Learn

Writing a Script to Count Data in Multiple Text Files 2

Posted by Stuart McGarrity,

Here I need to combine the data from multiple text files by grouping and summing by one variable. I also try using a datastore which I have not used much before.

Features covered in this code-along style video include:

  • readtable, fileDatastore with read function parameter
  • varfun

Follow me (@stuartmcgarrity) if you want to be notified via Twitter when I post.

Play the video in full screen mode for a better viewing experience. 


%% Read Redirect Traffic from Server Logs

%% Parameters
folderName='C:\Folder';
outputFileName='Redirects4Weeks.csv';

%% Get List of Files
fileInfo=dir([folderName filesep '*.csv']);
fileName={fileInfo.name}';
fullFileNames=cellfun(@(x) [folderName filesep x],fileName,'UniformOutput', false);
fileDatesCell=regexp(fileName,'(\d+-\d+-\d+)','tokens');
fileDatesStr=stripCell(fileDatesCell,2);
fileDates=datetime(fileDatesStr);
[fileDates,idx]=sort(fileDates,'descend');
mostRecent4=idx(1:4);

%% Data Store
fcn=@(x) readtable(x,'ReadVariableNames',false,'Delimiter',',','TextType','string', 'HeaderLines',1);
ds = fileDatastore(fullFileNames(mostRecent4),'ReadFcn', fcn);
cellData=readall(ds);
data=vertcat(cellData{:});
data.Properties.VariableNames={'Request_URI' 'count'};

%% Count URLs
pivot=varfun(@sum,data,'InputVariables','count','GroupingVariables',{'Request_URI'});
pivot.GroupCount=[];
pivot.Properties.VariableNames{'sum_count'}='count';
pivot=sortrows(pivot,{'count'},{'descend'});

%% Save
writetable(pivot,outputFileName);
7 views (last 30 days)  | |

Comments

To leave a comment, please click here to sign in to your MathWorks Account or create a new one.