Stuart’s MATLAB Videos

Watch and Learn

Writing a Script to Count Data in Multiple Text Files 2

Posted by Stuart McGarrity,

Here I need to combine the data from multiple text files by grouping and summing by one variable. I also try using a datastore which I have not used much before.

Features covered in this code-along style video include:

  • readtable, fileDatastore with read function parameter
  • varfun

Follow me (@stuartmcgarrity) if you want to be notified via Twitter when I post.

Play the video in full screen mode for a better viewing experience. 


%% Read Redirect Traffic from Server Logs

%% Parameters
folderName='C:\Folder';
outputFileName='Redirects4Weeks.csv';

%% Get List of Files
fileInfo=dir([folderName filesep '*.csv']);
fileName={fileInfo.name}';
fullFileNames=cellfun(@(x) [folderName filesep x],fileName,'UniformOutput', false);
fileDatesCell=regexp(fileName,'(\d+-\d+-\d+)','tokens');
fileDatesStr=stripCell(fileDatesCell,2);
fileDates=datetime(fileDatesStr);
[fileDates,idx]=sort(fileDates,'descend');
mostRecent4=idx(1:4);

%% Data Store
fcn=@(x) readtable(x,'ReadVariableNames',false,'Delimiter',',','TextType','string', 'HeaderLines',1);
ds = fileDatastore(fullFileNames(mostRecent4),'ReadFcn', fcn);
cellData=readall(ds);
data=vertcat(cellData{:});
data.Properties.VariableNames={'Request_URI' 'count'};

%% Count URLs
pivot=varfun(@sum,data,'InputVariables','count','GroupingVariables',{'Request_URI'});
pivot.GroupCount=[];
pivot.Properties.VariableNames{'sum_count'}='count';
pivot=sortrows(pivot,{'count'},{'descend'});

%% Save
writetable(pivot,outputFileName);

2 CommentsOldest to Newest

Tanya replied on : 1 of 2

Helpful – I like this style of learning. Is there a way we can access this script and input files to practice on our own with it?

HI Tanya,

Here is the script (above), but I think I can’t provide the data as it is somewhat proprietary. You will need to create your own by copying the filenames and format.

Add A Comment

Your email address will not be published. Required fields are marked *

What is 2 + 8 ?

Preview: hide