Predicting Time to Diagnosis for the WiDS Datathon #2
In today’s blog, Grace Woolson will show how you can use MATLAB and machine learning to make meaningful deductions from healthcare data for patients who have been diagnosed with metastatic breast cancer. Over to you Grace!
Introduction
In this blog, I will show how you can use MATLAB for the WiDS Datathon 2024 using the dataset for the WiDS Datathon #2, which runs from April 9th 2024 – June 1st 2024. This challenge tasks participants with creating a model that can predict how long it takes for a patient with metastatic breast cancer to receive a diagnosis based on patient and geographic data. This can help identify relationships between demographics or environmental conditions with the likelihood of getting timely treatment. Please note that this tutorial is based on a subset of the data and there may be slight differences between this dataset and the one you download from Kaggle.
MathWorks is happy to support participants of the Women in Data Science Datathon 2024 by providing complimentary MATLAB licenses, tutorials, workshops, and additional resources. To request complimentary licenses for you and your teammates, go to this MathWorks site, click the “Request Software” button, and fill out the software request form.
This tutorial will walk through the following steps of the model-making process:
- Importing a Tabular Dataset
- Preprocessing the Data
- Exploring Tabular Data
- Choosing and Creating Features
- Training a Machine Learning Model
- Making New Predictions and Exporting Submissions
Import Data
First, make sure the ‘Current Folder’ is the folder where you saved the data. If you have not already done so, you can download the data from Kaggle after you register for the datathon. The data is provided as a .CSV file, so you can use the readtable function to import the whole file as a table.
dataFolder = fullfile(pwd);
trainDataFilename = ‘training.csv’;
allTrainData = readtable(fullfile(dataFolder, trainDataFilename))
I want to see some high-level statistics about the data, so I’ll use the summary function to get an idea of what kind of information we have.
summary(allTrainData)
Take some time to scroll through this summary and see what information or patterns you can learn! Here are some things I notice:
- There are a lot of rows or variables that just say “cell array of character vectors”, which doesn’t tell us much about the data.
- There are a few variables that have a high ‘NumMissing’ value.
- The numeric variables can have dramatically different minimums and maximums.
We can use these observations to make decisions about how we want to explore and preprocess the dataset.
Process and Clean the Data
1. Convert text data to categorical
Text data can be hard for machine learning algorithms to understand, so let’s go through and change each “cell array of character vectors” to a categorical. This will help the algorithm sort the text into different categories instead of understanding it as a series of individual letters.
varTypes = varfun(@class, allTrainData, OutputFormat=“cell”);
catIdx = strcmp(varTypes, “cell”);
varNames = allTrainData.Properties.VariableNames;
catVarNames = varNames(catIdx);
for catNameIdx = 1:length(catVarNames)
allTrainData.(catVarNames{catNameIdx}) = categorical(allTrainData.(catVarNames{catNameIdx}));
end
2. Handle Missing Data
Now I want to handle all that missing data I noticed earlier. I’ll go through each variable and specifically look at variables that are missing data for over half of the rows or observations.
dataSum = summary(allTrainData);
for nameIdx = 1:length(varNames)
varName = varNames{nameIdx};
varNumMissing = dataSum.(varName).NumMissing;
if varNumMissing > (height(allTrainData) / 2)
disp(varName);
disp(varNumMissing);
end
end
Let’s remove those variables entirely, since they might not be too helpful for our algorithm.
allTrainData = removevars(allTrainData, [“patient_race”, “bmi”, “metastatic_first_novel_treatment”, “metastatic_first_novel_treatment_type”])
Now I want to look at each row and remove any that are missing too many values. It’s okay to have a couple of missing data points in your dataset, but if you have too many it could cause your machine learning algorithm to be less accurate. I’ll use the Clean Missing Data live task to remove any rows that are missing 2 or more data points.
% Remove missing data
[cleanTrainData,missingIndices] = rmmissing(allTrainData,“MinNumMissing”,2);
% Display results
figure
% Get locations of missing data
indicesForPlot = ismissing(allTrainData.patient_id);
mask = missingIndices & ~indicesForPlot;
% Plot cleaned data
plot(find(~missingIndices),cleanTrainData.patient_id,“SeriesIndex”,1,“LineWidth”,1.5, …
“DisplayName”,“Cleaned data”)
hold on
% Plot data in rows where other variables contain missing entries
plot(find(mask),allTrainData.patient_id(mask),“x”,“SeriesIndex”,“none”, …
“DisplayName”,“Removed by other variables”)
% Plot removed missing entries
x = repelem(find(indicesForPlot),3);
y = repmat([ylim(gca) missing]’,nnz(indicesForPlot),1);
plot(x,y,“Color”,[145 145 145]/255,“DisplayName”,“Removed missing entries”)
title(“Number of removed missing entries: ” + nnz(indicesForPlot))
hold off
legend
ylabel(“patient_id”,“Interpreter”,“none”)
clear indicesForPlot mask x y
Explore the Data
Now that the data is cleaned up, you should spend some time exploring your data to understand how different variables may interact with each other.
Visual Analysis – Univariate Data
I’ll start by using the kde function to calculate and visualize the kernel density estimate (kde) for individual variables in our dataset. This shows us how the data in that variable is distributed, similar to a histogram, but smooths out the visualization to make it easier to understand the overall distribution and patterns without getting distracted by potential outliers.
I’ll start by visualizing the distribution of patient age to gain a better understanding of the patient data we are working with.
whichColumn = cleanTrainData.patient_age; % Modify this line to explore other variables
[estProbDist,evalPoints] = kde(whichColumn);
plot(evalPoints, estProbDist);
Here we can see that a majority of our patients center around the 60 years old mark, with a few smaller spikes in the 80- and 90-year range. Visualizing your data like this can help you understand where they may be potential gaps in the data or identify patterns in patients who have been diagnosed with metastatic breast cancer.
Visual Analysis – Bivariate Data
You can use the Create Plot live task to create scatter plots of the different variables against how long it took for the patient to receive a diagnosis. Here, I’ve plotted ‘breast_cancer_diagnosis_code’ because I noticed most of the codes tend to skew left, meaning they have earlier diagnoses, but some of the codes, such as 1748, skew to the right, indicating that there may be a relationship between diagnosis code and time to diagnosis.
% Create scatter of selected data
s = scatter(cleanTrainData,“metastatic_diagnosis_period”,“breast_cancer_diagnosis_code”,“DisplayName”,“breast_cancer_diagnosis_code”);
% Add xlabel, ylabel, title, and legend
xlabel(“metastatic_diagnosis_period”)
ylabel(“breast_cancer_diagnosis_code”)
title(“breast_cancer_diagnosis_code vs. metastatic_diagnosis_period”)
legend
Take some time to explore these visualizations on your own! This live task allows you to create a variety of different plots, and you can even add multiple plots to the same axes.
Statistical Analysis
You can also create meaningful deductions or additional data by calculating various statistics from your data. For example, let’s add a column that shows how far the patients age is away from the mean age of all patients.
meanAge = mean(cleanTrainData.patient_age);
yearsFromMeanAge = cleanTrainData.patient_age – meanAge;
cleanTrainData = addvars(cleanTrainData, yearsFromMeanAge, ‘Before’, ‘metastatic_diagnosis_period’)
If you scroll all the way to the right of this table, you’ll see a new column called ‘yearsFromMeanAge’ that contains the data we just created! This is just a simple example, but it should give you an idea of how you can investigate and augment your data.
Feature Engineering
When it comes to machine learning, you don’t have to use all of the data as it is presented to you. Feature Engineering is the process of deciding what data you want to use, creating new data based on the provided data, and transforming the data to be in whatever format or range is suitable for your workflow. You can do this manually, and some of the exploration we just did should influence decisions you make if you want to play around with including or excluding different variables.
For this blog, I’ll use the genrfeatures function to automate this process. I want to use 30 features, so MATLAB will go through and create a set of meaningful features based on our processed dataset. It may keep some data as-is, but will often standardize numeric variables and create new variables by manipulating the provided data.
[T, augTrainData] = genrfeatures(cleanTrainData, “metastatic_diagnosis_period”, 30)
To better understand the generated features, you can use the describe function of the returned FeatureTransformer object, ‘T’.
describe(T)
Train a Machine Learning Model
In this example, I’ll use the fitrauto function to automatically test a variety of regression model types and hyperparameter values and select the best one. I use ASHA optimization, as it tends to find good solutions quickly for data sets with many observations, and I choose to use holdout validation, which uses 20% of the dataset for testing. You should play around with these values to see what improvements you can make.
hypParamOptions.Optimizer = “asha”;
hypParamOptions.Holdout = 0.2;
Mdl = fitrauto(augTrainData, “metastatic_diagnosis_period”, “HyperparameterOptimizationOptions”, hypParamOptions)
Now I have a trained Compact Regression Ensemble model! If you wanted to explore machine learning options interactively, check out the documentation and video for the Regression Learner app, which allows you to rapidly prototype, modify, and explore regression models.
Create Submission
Once you have a model that performs well, it’s time to create a submission for the datathon! As a reminder, you will upload this file to Kaggle to be scored on the leaderboard.
First, import the challenge test dataset:
testDataFilename = ‘test.csv’;
allTestData = readtable(fullfile(dataFolder, testDataFilename))
Then we need to process this dataset in the same way that we did the training data. In this section, I use code instead of the live tasks for simplicity.
% replace cell arrays with categoricals
varTypes = varfun(@class, allTestData, OutputFormat=“cell”);
catIdx = strcmp(varTypes, “cell”);
varNames = allTestData.Properties.VariableNames;
catVarNames = varNames(catIdx);
for catNameIdx = 1:length(catVarNames)
allTestData.(catVarNames{catNameIdx}) = categorical(allTestData.(catVarNames{catNameIdx}));
end
% remove variables with too many missing data points
allTestData = removevars(allTestData, [“patient_race”, “bmi”, “metastatic_first_novel_treatment”, “metastatic_first_novel_treatment_type”]);
% add ‘yearsFromMeanAge’ variable
meanAge = mean(allTestData.patient_age);
yearsFromMeanAge = allTestData.patient_age – meanAge;
allTestData = addvars(allTestData, yearsFromMeanAge);
We also need to use the transform function to create the same features as we created using genrfeatures for the training data.
augTestData = transform(T, allTestData);
Now that the data is in the format our machine learning model expects it to be in, use the predict function to make predictions, and create a table to contain the patient IDs and corresponding predictions.
submissionPreds = predict(Mdl, augTestData);
submissionTable = table(allTestData.patient_id, submissionPreds, VariableNames=[“patient_id”, “metastatic_diagnosis_period”])
Last, export your predictions to a .CSV file, then upload to Kaggle for scoring.
writetable(submissionTable, “Predictions.csv”);
Thank you for following along with this tutorial, and best of luck to all participants. If you have any questions about this tutorial or MATLAB, reach out to us at studentcompetitions@mathworks.com or by tagging gracewoolson in the forum. Keep your eye out for our upcoming livestream on the MATLAB YouTube channel on April 18th, where we will walk through this tutorial and answer any questions you have along the way!
- Category:
- Data Science
Comments
To leave a comment, please click here to sign in to your MathWorks Account or create a new one.