{"id":2014,"date":"2024-02-28T13:50:13","date_gmt":"2024-02-28T18:50:13","guid":{"rendered":"https:\/\/blogs.mathworks.com\/matlab\/?p=2014"},"modified":"2025-09-23T05:15:05","modified_gmt":"2025-09-23T09:15:05","slug":"outlier-detection-and-robust-regression-in-matlab-with-the-fsda-toolbox","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/matlab\/2024\/02\/28\/outlier-detection-and-robust-regression-in-matlab-with-the-fsda-toolbox\/","title":{"rendered":"Outlier detection and Robust Regression in MATLAB with the FSDA Toolbox"},"content":{"rendered":"
In 1857, Scottish physicist James David Forbes published a paper that discussed the relationship between atmospheric pressure and the boiling point of water. To get data, he traveled around the alps measuring the boiling point of water and atmospheric pressure at various altitudes. The idea behind this was that climbers could simply measure the boiling point of water to estimate atmospheric pressure and hence, altitude, instead of trying to use fragile and expensive barometers. <\/span><\/div>
The data he collected are included in the open source MATLAB toolbox <\/span>FSDA (Flexible Statistics Data Analysis)<\/span><\/a>. If you want to run the code examples in this blog post, you'll need to install it from the File Exchange or from MATLAB Add-ons.<\/span><\/div>
Let's take a look at it along with the traditional least squares linear regression fit provided by <\/span>fitlm<\/span><\/a> from MathWorks' Statistics toolbox.<\/span><\/div>
load(<\/span>'forbes.txt'<\/span>); <\/span>% Requires FSDA toolbox to be installed<\/span><\/span><\/div><\/div>
y=forbes(:,2);<\/span><\/span><\/div><\/div>
X=forbes(:,1);<\/span><\/span><\/div><\/div>
X = (X - 32) * 5\/9; <\/span>% Convert to Celsius<\/span><\/span><\/div><\/div>
plot(X,y,<\/span>'o'<\/span>);<\/span><\/span><\/div><\/div>
tradFit = fitlm(X,y); <\/span>% Use Statistics toolbox to fit the data<\/span><\/span><\/div><\/div>
hold <\/span>on<\/span><\/span><\/div><\/div>
plot(X,tradFit.predict(X)); <\/span>% plot the fitted line<\/span><\/span><\/div><\/div>
xlabel(<\/span>'Boiling point'<\/span>, <\/span>'Fontsize'<\/span>,16);<\/span><\/span><\/div><\/div>
ylabel(<\/span>'100 x log(pressure)'<\/span>,<\/span>'Fontsize'<\/span>,16);<\/span><\/span><\/div><\/div>
title(<\/span>\"Forbes data with Linear Regression Fit\"<\/span>)<\/span><\/span><\/div><\/div>
legend({<\/span>\"Points\"<\/span>,<\/span>\"Least squares fit\"<\/span>},Location=<\/span>\"Best\"<\/span>);<\/span><\/span><\/div><\/div>
hold <\/span>off<\/span><\/span><\/div>
<\/div>
<\/div>
<\/div>
<\/div>
<\/div><\/div><\/div><\/div><\/div>
Now, I want you to imagine that Forbes took a student with him whom he allowed to take one additional measurement. Imagine that this student was very careless when he took his measurement. Forbes, however, trusted him completely so we'll add this measurement to the data and see how it affects things.<\/span><\/div>
%% Contaminated data (just 1 outlier)<\/span><\/span><\/div><\/div>
yc = y; <\/span><\/span><\/div><\/div>
yc(end+1) = 140;<\/span><\/span><\/div><\/div>
Xc = X;<\/span><\/span><\/div><\/div>
Xc(end+1) = 110;<\/span><\/span><\/div><\/div>
 <\/div><\/div>
plot(Xc,yc,<\/span>'o'<\/span>);<\/span><\/span><\/div><\/div>
hold <\/span>on <\/span>% Show the outlier in red<\/span><\/span><\/div><\/div>
plot(Xc(end),yc(end),<\/span>'o'<\/span>,<\/span>'MarkerFaceColor'<\/span>,<\/span>'auto'<\/span>,<\/span>'Color'<\/span>,<\/span>'r'<\/span>)<\/span><\/span><\/div><\/div>
contaminatedTradFit = fitlm(Xc,yc);<\/span><\/span><\/div><\/div>
plot(Xc,contaminatedTradFit.predict(Xc)); <\/span>% plot the fitted line<\/span><\/span><\/div><\/div>
xlabel(<\/span>'Boiling point'<\/span>, <\/span>'Fontsize'<\/span>,16);<\/span><\/span><\/div><\/div>
ylabel(<\/span>'100 x log(pressure)'<\/span>,<\/span>'Fontsize'<\/span>,16);<\/span><\/span><\/div><\/div>
legend({<\/span>\"Forbes' points\"<\/span>,<\/span>\"Outlier\"<\/span>,<\/span>\"Least squares fit\"<\/span>},Location=<\/span>\"Best\"<\/span>);<\/span><\/span><\/div><\/div>
title(<\/span>\"Contaminated Forbes data with Linear Regression Fit\"<\/span>)<\/span><\/span><\/div><\/div>
hold <\/span>off<\/span><\/span><\/div>
<\/div>
<\/div>
<\/div>
<\/div>
<\/div><\/div><\/div><\/div><\/div>
That one outlier has changed everything. Standard linear regression is not robust to this type of contamination. <\/span><\/div>

A robust fit using FSDA toolbox<\/span><\/h2>
Enter the <\/span>LXS function<\/span><\/a> from FSDA toolbox which performs a more robust fit. By default, it uses an algorithm called <\/span>Least Median of Squares regression<\/span><\/a> which was first described in a 1984 paper by Peter Rousseeuw, a paper which has been cited over 5,000 times at the time of writing. <\/span><\/div>
[outLXS]=LXS(yc,Xc);<\/span><\/span><\/div>
Total estimated time to complete LMS: 0.20 seconds <\/div><\/div><\/div><\/div><\/div>
Let's get the coefficients of the robust fit and compare it to the traditional least squares fit<\/span><\/div>
b = outLXS.beta; <\/span>% Fit coefficients<\/span><\/span><\/div><\/div>
outliers = outLXS.outliers;<\/span><\/span><\/div><\/div>
plot(Xc,yc,<\/span>'o'<\/span>); <\/span>% Plot data<\/span><\/span><\/div><\/div>
hold <\/span>on <\/span>% Show the outlier in red<\/span><\/span><\/div><\/div>
plot(Xc(end),yc(end),<\/span>'o'<\/span>,<\/span>'MarkerFaceColor'<\/span>,<\/span>'auto'<\/span>,<\/span>'Color'<\/span>,<\/span>'r'<\/span>)<\/span><\/span><\/div><\/div>
plot(Xc,contaminatedTradFit.predict(Xc)); <\/span>% Plot traditional least squares fit<\/span><\/span><\/div><\/div>
plot(Xc,b(1)+b(2)*Xc, <\/span>'r'<\/span>,<\/span>'LineWidth' <\/span>,1); <\/span>% Plot robust fit<\/span><\/span><\/div><\/div>
legend({<\/span>\"Forbes' points\"<\/span>,<\/span>\"Outlier\"<\/span>,<\/span>\"Least squares fit\"<\/span>,<\/span>\"Robust Fit\"<\/span>},Location=<\/span>\"Best\"<\/span>);<\/span><\/span><\/div><\/div>
hold <\/span>off<\/span><\/span><\/div>
<\/div>
<\/div>
<\/div>
<\/div>
<\/div><\/div><\/div><\/div><\/div>
Not only is the fit returned by LXS robust to our deliberate contamination, but it identifies the contamination as an outlier. It also identifies one of the points in the original data as a potential outlier. We can get the indices of the outliers from the structure returned from LXS<\/span><\/div>
outLXS.outliers<\/span><\/span><\/div>
ans = <\/span>1\u00d72<\/span><\/div><\/div>
12 18\r\n<\/div>
<\/mw-icon><\/div>
<\/mw-icon><\/div><\/div><\/div>
<\/div>
<\/div>
<\/div>
<\/div><\/div><\/div><\/div><\/div>
Here's the plot once again, this time with the outliers clearly marked<\/span><\/div>
plot(Xc,yc,<\/span>'o'<\/span>); <\/span>%Plot data<\/span><\/span><\/div><\/div>
hold <\/span>on<\/span><\/span><\/div><\/div>
plot(Xc,contaminatedTradFit.predict(Xc)); <\/span>% Plot traditional least squares fit<\/span><\/span><\/div><\/div>
plot(Xc,b(1)+b(2)*Xc, <\/span>'r'<\/span>,<\/span>'LineWidth' <\/span>,1); <\/span>% Plot robust fit<\/span><\/span><\/div><\/div>
plot(Xc(outliers),yc(outliers),<\/span>'.'<\/span>, MarkerSize=20)<\/span><\/span><\/div><\/div>
legend({<\/span>\"Points\"<\/span>,<\/span>\"Least squares fit\"<\/span>,<\/span>\"Robust Fit\"<\/span>,<\/span>\"Outliers\"<\/span>},Location=<\/span>\"Best\"<\/span>);<\/span><\/span><\/div><\/div>
hold <\/span>off<\/span><\/span><\/div>
<\/div>
<\/div>
<\/div>
<\/div>
<\/div><\/div><\/div><\/div><\/div>
This is an example of robust linear regression which is just one of the areas of statistics and data analysis covered by the FSDA toolbox. For a more in-depth discussion and introduction to a suite of algorithms, refer to the documentation at <\/span>Introduction to robust estimators in linear regression<\/span><\/a>. Further analysis of this dataset is contained in Atkinson and Riani (2000). A comparison of different forms of robust regression estimators is given in the forthcoming book Atkinson <\/span>et<\/span> <\/span>al.<\/span> (2024). <\/span><\/div>

About the FSDA toolbox<\/span><\/h2>
Developed at Universit\u00e0 di Parma and the Joint Research Centre of the European Commission, <\/span>FSDA toolbox<\/span><\/a> is one of the most popular toolboxes in the <\/span>MathWorks File Exchange<\/span><\/a> and <\/span>contains over 300 functions<\/span><\/a> covering areas such as <\/span>Robust Regression Analysis, Robust Multivariate Analysis and Robust Cluster Analysis.<\/span><\/a><\/div>
References<\/span><\/div>
Atkinson A.C. and Riani M. (2000). <\/span>Robust Diagnostic Regression Analysis<\/span>, Springer Verlag, New York.<\/span><\/div>
Atkinson,A.C., Riani,M., Corbellini,A., Perrotta D., and Todorov,V. (2024), <\/span>Applied Robust Statistics through the Monitoring Approach<\/span>, Heidelberg: Springer Nature. <\/span> <\/span><\/div>
Rousseeuw P.J. (1984). Least Median of Squares Regression. <\/span>Journal of the American Statistical Association, <\/span>79:388, 871-880<\/span><\/div>
Forbes, J. (1857). Further experiments and remarks on the measurement of heights and boiling point of water. <\/span>Transactions of the Royal Society of Edinburgh<\/span>, 21, 235-243.<\/span><\/div>\r\n<\/div>