ODE Solver Selection in MATLAB

Today, I'd like to welcome Josh Meyer as this week's guest blogger. Josh works on the Documentation team here at MathWorks, where he writes and maintains some of the MATLAB Mathematics documentation. In this post, Josh provides a bit of advice on how to choose which ODE solver to use. Over to you, Josh...

Contents

Initial Value Problems

There are 7 ordinary differential equation initial value problem solvers in MATLAB:

(note that ode15i is left out of this discussion because it solves its own class of initial value problems: fully implicit ODEs of the form $f(t,y,y') = 0$)

To choose between the solvers, it's first necessary to understand why one solver might be better than another for a given problem.

The ODE solvers in MATLAB all work on initial value problems of the form,

$$y' = f \left( t,y \right)$$

where $y' = dy/dt$. There is also a more general form,

$$ M(t,y) y' = f \left( t,y \right)$$

where $M(t,y)$ is referred to as the mass matrix.

Starting with the initial conditions $y_0$, and a period of time over which the answer is to be obtained $(t_0,t_f)$, the solution is obtained iteratively by using the results of previous steps according to the solver's algorithm. At the first such step, the initial conditions provide the necessary information that allows the integration to proceed. The final result is that the ODE solver returns a vector of time steps $t_0,t_1,...,t_f$ as well as the corresponding solution at each time step $y_0,y_1,...,y_f$.

Theoretically, this numerical solution technique is possible because of the connection between differential equations and integrals provided by the fundamental theorem of calculus:

$$y(t + h) = y(t) + \int_t^{t+h} f\left( s,y(s) \right) ds$$

The problem of calculating $y(t+h)$ becomes a question of how to approximate the integral on the right hand side. This is where different solvers come in. Each different solver evaluates the integral using different numerical techniques, and each solver makes trade-offs between efficiency and accuracy.

Example: Euler's Method

Euler's method is a simple ODE solver, but it provides an illustration of the trade-offs between efficiency and accuracy in an ODE solver algorithm. Suppose you want to solve

$$y' = f(t,y) = 2t$$

over the time span $[0,3]$ using the initial condition $y_0 = 0$. Each step of Euler's method is computed with

$$\begin{array}{cl} y_{n+1} &= y_n + h f \left(t_n,y_n\right)\\ t_{n+1}&= t_n + h\end{array}$$

Using $h=1$, the solution requires just three steps:

$$\begin{array}{cl} y_1 &= y_0 + f\left(t_0,y_0\right)=0\\y_2 &= y_1 + f\left(t_1,y_1 \right)=2\\y_3 &= y_2 + f\left(t_2,y_2 \right)=6 \end{array}$$

... But is it accurate?

Not really. The exact solution to this equation is

$$y(t) = t^2$$

Reducing the step size $h$ can improve the accuracy of the answer a bit, but it also requires more steps to achieve the solution. To see this, the below code solves this problem using Euler's method and compares the answer to the analytic solution for several different values of $h$.

clear, clc
h = 1;
tspan = [0 3];
f = @(t,y) 2*t;
dydt(1) = 0;
t(1) = 0;

y = @(t) t.^2;
x = linspace(0,tspan(end));
figure
plot(x,y(x))
xlabel('t'), ylabel('y(t)')
hold on

while h > 0.1
    for k = 2:tspan(end)/h+1
        dydt(k) = dydt(k-1) + h*f(t(k-1),dydt(k-1));
        t(k) = t(k-1) + h;
    end

    plot(t,dydt,'-o')
    h = h/2;
end
legend('Exact Solution', 'h=1','h=0.5','h=0.25','h=0.125',...
    'Location','NorthWest')
title('Solution of y''=2t using Euler''s method with several step sizes')

Improving on Euler's Method

Using smaller and smaller step sizes turns out to not be a good idea, since the algorithm loses efficiency. For any reasonable problem such a solver would be very slow. Also, Euler's method has a few inherent problems. Since the slope of $y$ is evaluated only once at the beginning of each interval, this solver only produces exact answers for constant functions. There is also no way to estimate the error, so the solver needs to use fixed step sizes.

So, one way to improve on Euler's method is to evaluate $y'$ more often in each step. This provides intermediate slopes that give a better idea of what the function is doing within each interval, allowing the solver to produce exact answers for higher order problems. For example, if you add an evaluation of the slope halfway across each interval to Euler's method, then the result is called the midpoint rule, which produces exact integrations for linear functions:

$$\begin{array}{cl} s_1 &= f(t_n,y_n)\\ s_2 &= f\left( t_n + \frac{h}{2}, y_n + \frac{h}{2}s_1\right)\\y_{n+1} &= y_n + hs_2\\t_{n+1} &= t_n+h\end{array}$$

If you evaluate the slope four times in each interval, you get the classical Runge-Kutta algorithm (a.k.a. RK4), which is a piece of the ode45 algorithm. This algorithm produces exact integrations for cubic functions (and if $f$ is only a function of $t$, then $s_2=s_3$ and this is the same as Simpson's rule for quadrature):

$$\begin{array}{cl} s_1 &= f(t_n,y_n)\\s_2 &= f\left(t_n+\frac{h}{2},y_n+\frac{h}{2}s_1\right)\\s_3 &= f\left(t_n+\frac{h}{2},y_n+\frac{h}{2}s_2\right)\\ s_4 &= f\left(t_n+h,y_n+hs_3\right)\\y_{n+1} &= y_n + \frac{h}{6}\left(s_1+2s_2+2s_3+s_4\right)\\t_{n+1} &= t_n + h\end{array}$$

Runge-Kutta algorithms are all single-step solvers, since each step only depends on the result of the previous step. ode45, ode23, ode23s, ode23t, and ode23tb all employ single-step algorithms. Multi-step algorithms, such as those employed by ode113 and ode15s, use the results of several past steps.

Sophisticated ODE solvers, like the ones in MATLAB, also estimate the error in each step to determine how big the next step size should be. This is another improvement over the fixed step sizes used above, since a solver that does more work per step is able to compensate by taking steps of varying size. The error estimate used to determine the step size is typically obtained by comparing the results of two different methods. MATLAB's ODE solvers follow a naming convention that reveals information about which methods they use. ode45 compares the results of a 4th-order Runge-Kutta method and a 5th-order Runge-Kutta method to determine the error. Similarly, ode23 uses a 2nd-order and 3rd-order Runge-Kutta comparison. So, in general, the smaller the number odeNN, the looser the solver's error tolerance is.

It should be no surprise, then, that ode45 obtains a very accurate answer for the equation we solved before with Euler's method. ode45 is MATLAB's general purpose ODE solver, and it is the first solver you should use for most problems.

y = @(t) t.^2;
x = linspace(0,3);
figure
plot(x,y(x))
xlabel('t'), ylabel('y(t)')
hold on
[t,y] = ode45(@(t,y) 2*t, [0 3], 0);
plot(t,y,'o')
xlabel('t'), ylabel('y(t)')
title('Solution of y''=2t using ode45')

Stiff Differential Equations

For some ODE problems, the step size taken by the solver is forced down to an unreasonably small level in comparison to the interval of integration, even in a region where the solution curve is smooth. These step sizes can be so small that traversing a short time interval might require millions of evaluations. This can lead to the solver failing the integration, but even if it succeeds it will take a very long time to do so.

Equations that cause this behavior in ODE solvers are said to be stiff. This is a nod to the fact that the equations are stubborn and not easily evaluated with numerical techniques. The problem that stiff ODEs pose is that explicit solvers (such as ode45) are untenably slow in achieving a solution. This is why ode45 is classified as a nonstiff solver along with ode23 and ode113. These solvers all struggle to integrate stiff equations.

Equation stiffness resists a precise definition, because there are several factors that cause it. Stiffness results from a combination of the specific equations, the ODE solver being used, the initial conditions, and the error tolerance used by the solver. The following statements about stiffness, attributed to Lambert [6], are exhibited by many examples of stiff ODEs, but counterexamples also exist, so they are not true definitions of stiffness:

A common theme among these statements is that stiffness can result from a difference in scaling somewhere in the problem. This difference in scale (for example, if the Jacobian $J = \partial f_n/\partial y_i$ has a large ratio of negative eigenvalues) constrains the step size that the solver can take in performing the integration. Tiny step sizes become necessary in order to preserve any notion of error tolerance or stability in the solution.

For example, equations describing chemical reactions frequently display stiffness, since it is common for components of the solution to vary on drastically different time scales (reactions occurring at the same time that are both very slow and very fast).

However, there are solvers specifically designed to work on stiff ODEs. Solvers that are designed for stiff problems typically do more work per step, and the pay-off is that they are able to take much larger steps and enjoy improved numerical stability compared to the nonstiff solvers. Stiff solvers are implicit, because the computation of $y$ requires the use of linear algebra to solve systems of linear equations. The Jacobian is used to estimate the local behavior of the ODE as the integration proceeds, so supplying the analytical Jacobian can improve the performance of MATLAB's stiff ODE solvers.

This is just a cursory treatment of stiffness, because it is a complex topic. See Ordinary Differential Equations: Stiffness for a more in-depth look.

To summarize, the nonstiff solvers in MATLAB are:

The stiff solvers are (when ode45 is slow):

It should be noted that nonstiff solvers do work on stiff problems, it is just that they are exceptionally slow. Similarly, solvers designed for stiff problems can work on nonstiff problems, but since they do more work per step they are less efficient than their nonstiff counterparts when that extra work isn't necessary. So equation stiffness is a matter of solver efficiency, and the goal is to strike the right balance between accuracy of the solution and work done in each step by the solver.

Solver Recommendations

The following recommendations are adapted from the MATLAB Mathematics documentation :

For nonstiff problems:

For stiff problems (where ode45 is slow):

Here is a graphic that captures the basic recommendations. In most cases, the only choice in solver you will need to make is to use ode15s instead of ode45.

Example 1: Damped Pendulum

The equation of motion for a damped pendulum is,

$$\ddot{\theta} = -\frac{b}{m}\dot{\theta}-\frac{mg}{L\left(m-2b\right)} \sin \theta$$

where $g$ is the gravitational constant, $m$ the mass of the bob, $L$ the length of the string, and $b$ is a damping coefficient. The goal is to solve for $\theta$, the angle that the pendulum deviates from the vertical, and $\theta '$, the rate at which the angle changes.

Some natural initial conditions would be $\theta_0 = \pi/4$ and $\theta '_0 = 0$, indicating that you lift the pendulum up to a 45 degree angle before letting go, and it has no initial angular velocity. Due to the damping coefficient, you would expect the pendulum to slowly lose momentum and go back down to rest.

The file pendulumODE.m reformulates the problem as a coupled system of first-order ODEs:

$$\begin{array}{cl} y_1' &= y_2\\y_2' &= -\frac{b}{m} y_2 -\frac{mg}{L(m-2b)}sin(y_1)\end{array}$$

then solves using ode45, ode15s, ode23, and ode113. The solutions for $y_1 = \theta$ are plotted, and the file returns the stats for each solver. As is always the case when displaying execution times, "the timings displayed can vary".

function pendulumODE
opts = odeset('stats','on');
tspan = [0 25];
y0 = [pi/4, 0];

disp(' '), disp('Stats for ode45:')
tic, [t1,y1] = ode45(@pendode, tspan, y0, opts);
toc

disp(' '), disp('Stats for ode15s:')
tic, [t2,y2] = ode15s(@pendode, tspan, y0, opts);
toc

disp(' '), disp('Stats for ode23:')
tic, [t3,y3] = ode23(@pendode, tspan, y0, opts);
toc

disp(' '), disp('Stats for ode113:')
tic, [t4,y4] = ode113(@pendode, tspan, y0, opts);
toc

figure
subplot(2,2,1), plot(t1,y1(:,1),'-o'), xlim([0 25]), title('ode45')
subplot(2,2,2), plot(t2,y2(:,1),'-o'), xlim([0 25]), title('ode15s')
subplot(2,2,3), plot(t3,y3(:,1),'-o'), xlim([0 25]), title('ode23')
subplot(2,2,4), plot(t4,y4(:,1),'-o'), xlim([0 25]), title('ode113')

    function dy2dt2 = pendode(t,y)
        g = 9.8; %m/s^2
        m = 1;   % Mass of bob
        L = 2;   % Length of pendulum in meters
        b = 0.2; % Damping coefficient
        dy2dt2 = [y(2); -b/m*y(2)-g/L*sin(y(1))];
    end
end

pendulumODE

 
Stats for ode45:
75 successful steps
0 failed attempts
451 function evaluations
Elapsed time is 0.011970 seconds.
 
Stats for ode15s:
183 successful steps
9 failed attempts
315 function evaluations
1 partial derivatives
31 LU decompositions
311 solutions of linear systems
Elapsed time is 0.081467 seconds.
 
Stats for ode23:
263 successful steps
36 failed attempts
898 function evaluations
Elapsed time is 0.015948 seconds.
 
Stats for ode113:
159 successful steps
3 failed attempts
322 function evaluations
Elapsed time is 0.018056 seconds.

The solvers all perform well, but the damped pendulum is a good example of a nonstiff problem where ode45 performs nicely. In this case ode15s needs to do extra work in order to achieve an inferior solution.

Example 2: van der Pol Oscillator

The van der Pol Oscillator equation becomes stiff in certain intervals when the nonlinear parameter $\mu$ is large:

$$\ddot{y} - \mu \left(1-y^2\right)\dot{y}+y=0$$

The nonlinearity of this equation is contained entirely in the term that involves $\mu$: notice that if $\mu=0$, the equation reduces to that of a simple harmonic oscillator, which has regular periodic behavior.

Attempting to solve this equation using ode45 is met with severe resistance, requiring millions of evaluations and 30+ minutes of execution (I stopped execution after 35 minutes). Since the problem is clearly stiff, this example compares the stiff solvers.

The file vanderpolODE.m finds the solution for $\mu=1000$ using ode15s, ode23s, ode23t, and ode23tb. The function file vdp1000.m ships with MATLAB and encodes this equation as a coupled system of first-order ODEs:

$$\begin{array}{cl}y_1' &= y_2\\y_2' &= \mu \left( 1-y_1^2 \right)y_2 - y_1\end{array}$$

The Jacobian is supplied to assist the solvers, and its use is reflected in the number of partial derivative evaluations.

function vanderpolODE
opts = odeset('stats','on','Jacobian',@J);
tspan = [0 3000];
y0 = [2 0];

disp(' '), disp('Stats for ode15s:')
tic, [t1,y1] = ode15s(@vdp1000, tspan, y0, opts);
toc

disp(' '), disp('Stats for ode23s:')
tic, [t2,y2] = ode23s(@vdp1000, tspan, y0, opts);
toc

disp(' '), disp('Stats for ode23t:')
tic, [t3,y3] = ode23t(@vdp1000, tspan, y0, opts);
toc

disp(' '), disp('Stats for ode23tb:')
tic, [t4,y4] = ode23tb(@vdp1000, tspan, y0, opts);
toc

figure
subplot(2,2,1), plot(t1,y1(:,1),'-o'), ylim([-4 4]), title('ode15s')
subplot(2,2,2), plot(t2,y2(:,1),'-o'), title('ode23s')
subplot(2,2,3), plot(t3,y3(:,1),'-o'), title('ode23t')
subplot(2,2,4), plot(t4,y4(:,1),'-o'), title('ode23tb')

    function dfdy = J(t,y)
        MU = 1000; % Nonlinear coefficient
        dfdy = [         0                  1
            -2*MU*y(1)*y(2)-1    MU*(1-y(1)^2) ];
    end
end

vanderpolODE

 
Stats for ode15s:
591 successful steps
225 failed attempts
1749 function evaluations
45 partial derivatives
289 LU decompositions
1747 solutions of linear systems
Elapsed time is 0.192955 seconds.
 
Stats for ode23s:
741 successful steps
13 failed attempts
2251 function evaluations
741 partial derivatives
754 LU decompositions
2262 solutions of linear systems
Elapsed time is 0.092824 seconds.
 
Stats for ode23t:
776 successful steps
94 failed attempts
2014 function evaluations
36 partial derivatives
294 LU decompositions
2012 solutions of linear systems
Elapsed time is 0.218996 seconds.
 
Stats for ode23tb:
573 successful steps
93 failed attempts
2816 function evaluations
44 partial derivatives
269 LU decompositions
3415 solutions of linear systems
Elapsed time is 0.202237 seconds.

The plots are of the solutions for $y_1$. For this problem, ode23s executes quickest and with the least number of failed steps. The supplied Jacobian greatly assists ode23s in evaluating the partial derivatives in each step. ode23tb also solves the problem with the fewest number of steps, outperforming ode15s. This problem is a good example of a stiff problem with a crude tolerance where ode23s and ode23tb can out perform ode15s. But practically speaking, all of the stiff solvers perform well on this problem and offer significant time savings when compared to ode45.

Summary

Although all of the ODE solvers are capable of working on the same problems, it's recommended that you start with ode45. Then, if the problem exhibits signs of stiffness, ode15s is a good second choice. The other solvers then offer further refinement based on the properties of the specific problem and whether extra information (such as the Jacobian) can be provided.

Comments

Does your work involve the use of MATLAB's ODE solvers? If so, share your experience here.

Further Reading

[1] C. Moler, Ordinary Differential Equations Numerical Computing with MATLAB Electronic edition: The MathWorks, Inc., Natick, MA, 2004

[2] Shampine, L. F. and M. W. Reichelt, The MATLAB ODE Suite SIAM Journal on Scientific Computing, Vol. 18, 1997.

[3] C. Moler, Ordinary Differential Equations: Stiffness Cleve’s Corner: Cleve Moler on Mathematics and Computing, 2014

[4] L. F. Shampine, Numerical Solution of Ordinary Differential Equations, Chapman & Hall, 1994

[5] Shampine, L. F., Gladwell, I. and S. Thompson, Solving ODEs with MATLAB, Cambridge University Press, 2003

[6] J. D. Lambert, Numerical Methods for Ordinary Differential Systems, New York: Wiley, 1992

Copyright 2015 The MathWorks, Inc.

Get the MATLAB code

Published with MATLAB® R2015a