Stuart’s MATLAB Videos

Watch and Learn

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English version of the page.

Writing a Script to Count Data in Multiple Text Files 2

Posted by Stuart McGarrity,

Here I need to combine the data from multiple text files by grouping and summing by one variable. I also try using a datastore which I have not used much before.

Features covered in this code-along style video include:

  • readtable, fileDatastore with read function parameter
  • varfun

Follow me (@stuartmcgarrity) if you want to be notified via Twitter when I post.

Play the video in full screen mode for a better viewing experience. 


%% Read Redirect Traffic from Server Logs

%% Parameters
folderName='C:\Folder';
outputFileName='Redirects4Weeks.csv';

%% Get List of Files
fileInfo=dir([folderName filesep '*.csv']);
fileName={fileInfo.name}';
fullFileNames=cellfun(@(x) [folderName filesep x],fileName,'UniformOutput', false);
fileDatesCell=regexp(fileName,'(\d+-\d+-\d+)','tokens');
fileDatesStr=stripCell(fileDatesCell,2);
fileDates=datetime(fileDatesStr);
[fileDates,idx]=sort(fileDates,'descend');
mostRecent4=idx(1:4);

%% Data Store
fcn=@(x) readtable(x,'ReadVariableNames',false,'Delimiter',',','TextType','string', 'HeaderLines',1);
ds = fileDatastore(fullFileNames(mostRecent4),'ReadFcn', fcn);
cellData=readall(ds);
data=vertcat(cellData{:});
data.Properties.VariableNames={'Request_URI' 'count'};

%% Count URLs
pivot=varfun(@sum,data,'InputVariables','count','GroupingVariables',{'Request_URI'});
pivot.GroupCount=[];
pivot.Properties.VariableNames{'sum_count'}='count';
pivot=sortrows(pivot,{'count'},{'descend'});

%% Save
writetable(pivot,outputFileName);

2 CommentsOldest to Newest

Tanya replied on : 1 of 2

Helpful – I like this style of learning. Is there a way we can access this script and input files to practice on our own with it?

HI Tanya,

Here is the script (above), but I think I can’t provide the data as it is somewhat proprietary. You will need to create your own by copying the filenames and format.

Add A Comment

Your email address will not be published. Required fields are marked *

Preview: hide