Wow! thanks so much for such a detailed reply- Your examples really help- and I can see where I was going wrong, the biggest problem was using meshgrid correctly for my scenario and getting the syntax right for polyfitn and the backslash.

polyfitn will definitely be the most useful to me to begin with I think- as I want to experiment with fitting to a surface that looks like a peak/impulse, so I’m looking at at least order 2 to get a parabola shape function.

Thanks again.
Gareth

There are two issues here. One is to build the independent variables for the fit itself. (This is the easy part, but even there we will trip over issues to deal with.) The second issue is what model one will choose.

I’ll use the MATLAB standard here to define x and y. So if we have an mxn matrix Z, then we have m points in y, and n in x. We can simply choose to define x and y as living on the integers 1:n and 1:m respectively. Don’t forget that we have a n entire matrix though! The MESHGRID function will allow us to build arrays of he right size and shape for x and y. For example, given a 3×5 array for Z, we might do this:

m = 3;
n = 5;
[ x, y ] = meshgrid(1:n,1:m)

x =
     1     2     3     4     5
     1     2     3     4     5
     1     2     3     4     5

y =
     1     1     1     1     1
     2     2     2     2     2
     3     3     3     3     3

Here x and y are now both mxn matrices, of the same size as Z. How might we build the polynomial model itself? For this, x, suppose we simply wanted a linear model in both x and y? We can use either backslash to do the work, or we might use regression codes like polyfitn or regress from the statistics toolbox. Or, if you have the curvefitting toolbox, it now allows you to do fits in two independent variables. I’ll show you how to do it using some of those alternatives.

Before I do any actual examples, I’ll need some actual data. Ok, made up data.

Z = -3 + x + 2.5*y + randn(m,n)/3
Z =
      0.53798       1.4681       2.0546       3.2694       3.9688
       3.3556       3.7226       5.2381        6.286       6.5197
       5.5198       6.5981       8.0412        8.918       9.6904

surf(x,y,Z)

My model will be of the general form

z = a(1) + a(2)*x + a(3)*y + error

First, solve this with backslash. For a model that is linear in x and y, this is easy.

coef = [ones(numel(x),1),x(:),y(:)] \ Z(:)
coef =
      -3.3027
      0.94052
       2.7469

Not too bad. I had dumped some noise into the surface when I built it, so you should not expect perfect estimates of the numbers.

My own POLYFITN (from the file exchange) also does this nicely enough of course, along with some useful statistics on the model and the parameters.

model = polyfitn([x(:),y(:)],Z(:),'constant x y')
model =
      ModelTerms: [3x2 double]
    Coefficients: [-3.3027 0.94052 2.7469]
    ParameterVar: [0.057808 0.0025134 0.0075402]
    ParameterStd: [0.24043 0.050134 0.086834]
              R2: 0.99121
            RMSE: 0.2456
        VarNames: {'x'  'y'}

Or if you have the statistics toolbox, REGRESS is a very good choice. Here we see 95% confidence intervals returned on the parameter estimates.

[B,BINT] = regress(Z(:),[ones(numel(x),1),x(:),y(:)])
B =
      -3.3027
      0.94052
       2.7469

BINT =
      -3.8266      -2.7789
      0.83129       1.0498
       2.5577       2.9361

Having estimated a linear model so easily, one might choose to expand to higher order models. At some point, you will see numerical conditioning problems. You might be forced to work with orthogonal polynomials, or at the very least, to consider centering and scaling the variables to be lie in the interval [-1,1]. This will greatly reduce the potentiality for singular or nearly singular matrices.

Even so, what happens when we try to estimate a higher order model here? Admittedly, we don’t have much data to estimate that model.

model = polyfitn([x(:),y(:)],Z(:),'constant x y x^2 x*y y^2')
model =
      ModelTerms: [6x2 double]
    Coefficients: [-3.0534 0.99008 2.5181 -0.041565 0.099913 -0.017752]
    ParameterVar: [0.49589 0.076522 0.37543 0.0016708 0.0035087 0.021052]
    ParameterStd: [0.7042 0.27663 0.61272 0.040875 0.059234 0.14509]
              R2: 0.99386
            RMSE: 0.20519
        VarNames: {'x'  'y'}

Yes, we can estimate a fully quadratic model on this data. In fact though, we know the true underlying relationship to have been linear. So we might hope that to see the higher order terms to be statistically insignificant. Look at the ratio of the parameter coefficients to the parameter standard deviations. The absolute value of that ratio should be compared to a student’s t statistic, here with 15-6=9 degrees of freedom.

model.Coefficients./model.ParameterStd
ans =
       -4.336       3.5791       4.1097      -1.0169       1.6867     -0.12235

The critical level for the Student’s t is given here by TINV, from the stats toolbox, or from a set of tables.

tinv(0.975,9)
ans =
       2.2622

So it would appear that the x^2, x*y, and y^2 terms are not significantly different from zero in this model, based on this very limited amount of data. I won’t go any further along these lines. Better is to refer you to any competent book on regression analysis. I like Draper and Smith, as that is what I used many years ago.

Higher order polynomial models are not in general something I recommend though. Linear or quadratic models often are useful. They provide information about the parameters, about the relative importance of the variables in a model. But when people start throwing fifth or tenth order models in several dimensions at their data, I’ll claim this is silly. Many dozens of coefficients will mean nothing to them anyway. Worse, high order polynomial models do terrible things when they are used to extrapolate. They can even do strange and nasty things between your data points. (I call this “intrapolation”.)

In this event, I’ll always recommend a spline type model for your system. For example, my own gridfit from the file exchange is one such member of the general class of spline-like models. Or use the splines toolbox here. These tools do not give you a model that you can simply write down. But why does that matter, since an adequate polynomial model might have had dozens of nonsensical coefficients for the fit?

Finally, it seems that I often get requests from people wanting to know how to find the nonlinear “function” that fits their surface. The nonlinear modeling question is an entirely different one, worth several books worth of writing. Having written virtually a small book here, I’ll stop now.

HTH,
John

However, I am having trouble working out how to find the polynomial fit to a 3D surface, when instead of having 3 vectors for x,y and z I simply have a single matrix, Z, of size [m x n]. The Z values represent the elevations/surface points, obviuosly I can view this with surf(Z) and see my real surface with no problems. I want to find the function Z = f(x,y) that best fits my surface.

I have seen your polyfitn function but I cannot work out how to prepare my matrix Z for input. Do you have any suggestions?

Many thanks,

Gareth

The second scheme is to consider this as a deformation of a domain in three dimensions. To do this, we might perhaps use a finite element code to solve for the deformation of an elastic body in three dimensions. Any single point in the domain then maps to a corresponding point in the range space.

Both of these schemes have their associated merits. A problem with the elastic body solver is what I like to call unwanted baggage in the metaphorical model. Think of it like this. Suppose I were to compress a piece of rubber along one axis. When I do so, my expectation is that the rubber will bulge out into the other dimensions. This is expected behavior for an elastic body. (A parameter in a physical model is Poisson’s ratio, which helps us to quantify how a given material behaves under deformation.) But suppose I were to do a similar operation on a set of data points in some three dimensional domain? Here a simple scaling along one axis need not be accompanied by a bulge in the other directions. It turns out that if you do use an elastic body solver here, then I’ll argue that it probably makes sense to use a Poisson’s ration of zero.

My point is that use of a metaphorical model such as the deformations of an elastic body to interpolate data may make some sense. A cubic spline is my favorite member of the family of metaphorical models, wherein we use a mathematical model for one physical system to predict the behavior or a completely, unrelated physical system. But at the same time any such metaphorical model carries along baggage. That baggage is the set of sometimes strange predictions that we will see if we are less than careful in our usage of a metaphor.

Climbing down now from my metaphorical soapbox…

John

I’d not expect this to be mentioned as an “issue”. A vertical line has an infinite slope. It has no finite value for the slope. Since polyfit estimates a model of the form y = a1*x + a2, what coefficients would you wish polyfit to return? For example,

x = 2 + randn(5,1)*1e-8;
y = randn(5,1);
[x,y]
ans =
          1.99999999813291          0.11393131352081
          2.00000000725791          1.06676821135919
          1.99999999411683        0.0592814605236053
          2.00000002183186        -0.095648405483669
          1.99999999863604        -0.832349463650022
plot(x,y,'o')
axis equal

This is a vertical line, to within any rational limits. The equation of the line is essentially x=2. That is, y is not a function of x at all. So when you try to fit a line using polyfit,

   P = polyfit(x,y,1)
P =
          10177980.6056492         -20355961.1895639

The coefficients are garbage, purely so. But we should expect that! Garbage in, garbage out. When you ask polyfit to fit the model it is designed to fit on this set of data, why would you expect differently?

Having said that, there are several alternatives. One can swap the axes, fitting a model of the form x = b1*y + b2, still using polyfit.

   P = polyfit(y,x,1)
P =
      2.71191329645078e-09           2.0000000038259

Again, as expected. The model suggests that x=2, is independent of the value of y.

You might also choose to use an orthogonal regression code. There should be several such tools to be found by a quick search on the file exchange.

John

This is not mentioned in the polyfit documentation as a known issue.

Can you comment on this?

Thanks,
Jennifer

I’m afraid that I can’t help you too much here. You can look at the MTF for an interpolation method to learn what it will do to the frequencies of interest. See pages 73+ in Vollmerhausen & Driggers, “Analysis of Sampled Imaging Systems” for example. (You can find a large part of this book on Google Books.)

John

I am not sure, if you could have delt with the interpolation before I send you this comment. Even then, i really love to read it, given the site.

Thx a lot

Abiyu

For example, the second derivative of a cubic spline can be no more complex than piecewise linear, so it is quite easy to constrain a spline to be positively or negatively curved over a domain.

Not quite so easy is to constrain a cubic spline to be monotonic, but that too is possible in a linear regression context. Other constraints too are possible. In total, splines can form a splendid way to approximate data where you do not have a mechanistic or physical model for the process, yet you need something that behaves itself.

The next blogs I do will begin to discuss splines, starting with baby steps as piecewise linear interpolants, then next into the realm of cubic Hermite functions and cubic splines. Perhaps in a future blog I could even spend some time on least squares approximation by splines.

John