Comments on: HELP – I Got the Wrong Answer for My Optimization!

By: shankar

shankar — Tue, 27 Jul 2010 07:50:54 +0000

Hi james, great information. Thanks. in fact what you suggested is what i kind of did. wrote down the hessian at the initial point, and minimizing it gives us the bifurcation modes. However nonlinear an energy be, usually its hessian’s minimization gives a linear eqn. for bifurcation. Since the bifurcation problem is an eigenvalue, discretizing gives a simple AX=tX matrix eigenvalue problem. Choosing smallest eigenvalue of A, and its corresponding eigenvector, I used that eigenvector multiplied by a (any) small number to perturb the intial point. As soon as it is perturbed, fmincon works very well. Also if the perturbation is not correct direction, ideally it should fall back into the maxima.

By: James

James — Thu, 22 Jul 2010 12:16:49 +0000

Shankar: while algorithms in the optimization toolbox may not be able to descend from a maximum or a saddle point, gradient-free algorithms in the Global Optimization Toolbox certainly can, such as ga or patternsearch. It’s a matter of choosing an algorithm appropriate for your problem.

If you don’t know a lot about what the problem space looks like, and have some indication that multiple stationary points may exist, then a hybrid approach can be helpful, i.e., starting off with a gradient-free method with relatively loose tolerances, and then using the result as a starting point for a gradient-based method to identify a precise solution. The new GlobalSearch and MultiStart algorithms may also be very helpful.

By: shankar

shankar — Thu, 22 Jul 2010 00:46:47 +0000

Thanks alan and james. i understand what you are saying. i am especially surprised as there seems to be no code which can make a point descend from a maxima or even a saddle point. the only way i could get anything out of my matlab code is to perturb the initial point slightly. but then since i am looking at the subsequent evolution of the problem, i fear i might be getting the wrong or right answer depending upon how i perturb the initial point, however so slightly (its a stability/bifurcation problem). strange indeed that numerical guys have not figured out this issue.

By: James Allison

James Allison — Mon, 19 Jul 2010 12:28:25 +0000

Regarding the answer to Ben’s comment: fmincon is a gradient-based algorithm. It’s search direction depends on the gradient of the function at a point, and it stops when first-order conditions are satisfied, i.e., the gradient is zero if no constraints are active. In this case Ben is starting fmincon at a point where the gradient is zero, so of course the algorithm doesn’t move. Another term like this is a ‘stationary point’. This is a property of gradient-based algorithms, and not just MATLAB’s fmincon. It’s expected behavior. In cases like this you might want to try a multi-start approach, or an initial search with a gradient-free algorithm. The new global optimization toolbox can handle problems like this very well (multiple stationary points, noisy responses, etc.).

By: Alan Weiss

Alan Weiss — Mon, 19 Jul 2010 12:18:49 +0000

When your initial point is stationary (i.e., has gradient = 0 or satisfies the KKT conditions), the solver has no way to know whether the point is a local minimum, local maximum, or is simply an indefinite stationary point. The issue is addressed in these sections of the documentation:
http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/br44i40.html#br44j7o
http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/br44i40.html#br44i5e
http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/br44i40.html#brhkghv-65

The final link is particularly relevant, in the it explains the difference between a local and a global solution, and why the initial point is so important.

By: shankar

shankar — Fri, 16 Jul 2010 16:54:57 +0000

Hi, did anyone figure out what Ben asked? Why does matlab get stuck at intitial point for minimizing even cosine. I am having a similiar problem with my objective function getting stuck at initial point. as soon as i perturb the initial point ever so slightly, it evolves very well. but unfortunately i fear that the direction of the perturbation might be influencing the evolution of the solution.

By: Loren

Loren — Wed, 07 Apr 2010 12:46:46 +0000

Lishuqin-

The message is telling you what the issue is (and how to change it). If that doesn’t help, please contact technical support – link at the right.

–Loren

By: lishuqin

lishuqin — Wed, 07 Apr 2010 02:30:56 +0000

ti have the problem:Maximum number of function evaluations exceeded;
increase OPTIONS.MaxFunEvals.
my function as follows:

function f=gear(x)
f=1*0.7408*x(1)^3*x(2)^3*x(3)*[1+(1-x(3))^2+(1-x(3))]+0*[-1.88+4.858/x(2)]
function [g,q]=mycon(x)
g(1)=160057/[x(1)^3*x(2)^3*(1-0.5*x(3))^2*x(3)]^(-0.5)-664;
g(2)=398121*x(1)^(-3)*x(2)^(-2)*(1-0.5*x(3))^(-2)*x(3)^(-1)-480;
g(3)=358848*x(1)^(-3)*x(2)^(-2)*(1-0.5*x(3))^(-2)*x(3)^(-1)-464;
g(4)=-0.48+4.858/x(2);
g(5)=4-x(1);
g(6)=0.25-x(3);
g(7)=x(3)-0.33;
g(8)=x(2)-40;
g(9)=x(1)-10;
g(10)=11-x(2);
q=[];

options=optimset(‘largescale’,'off’);x0=[4,14,0.3];[x,fval]=fmincon(@gear,x0,[ ],[ ],[ ],[ ],[ ],[ ],@mycon,options);please give me help ,thanks.

By: Loren

Loren — Tue, 18 Nov 2008 12:49:58 +0000

Ben-

What you mention is not about the quality of the objective function itself. In fact, when I run the problem in R2008b, I get a warning before I get my answer:

Warning: Trust-region-reflective method does not currently solve this type of problem,
 using active-set (line search) instead.
> In fmincon at 437
Optimization terminated: first-order optimality measure less
 than options.TolFun and maximum constraint violation is less
 than options.TolCon.

which hints at the answer (and the problem as posed, perhaps) needing scrutiny.

–loren

By: Ben

Ben — Tue, 18 Nov 2008 09:17:54 +0000

It’s worth pointing out that fmincon (R2007b) can get stuck at the initial point even for “nice” functions, like

>>fmincon(@cos,0,[],[],[],[],-1,1)

which returns 0 the local maximum. Not that we’d do that in practice though :)