Stuart’s MATLAB Videos

Watch and Learn

Using detectImportOptions with Large Text Files

Posted by Stuart McGarrity,

Yesterday, I was loading a CSV file of about 1 million rows and 300 columns, comprising lots of string variables. It took a while to load, then I remembered I only needed 1 or 2 columns and how detectImportOptions helps you specify which columns to load. It lets you specify the variable names in the header to include, which is much easier than specifying column indices, especially if columns move around.

In fact, detectImportOptions combined with readtable is now my main method of loading subsets of data from large text files. Gone are the days of trying to calculate format strings with textscan. In the past, I even made this submission textscantool on the File Exchange to calculate the format strings for text files with many columns.

Features covered in this code-along style video include:

Follow me (@stuartmcgarrity) if you want to be notified via Twitter when I post.

Play the video in full screen mode for a better viewing experience. 

7 views (last 30 days)  | |

Comments

To leave a comment, please click here to sign in to your MathWorks Account or create a new one.