The MATLAB Blog https://blogs.mathworks.com/matlab Mike Croucher is a Customer Success Engineer for Research and Education who previously spent 20 years working in academic research computing. He writes about MATLAB, MathWorks, and intersections with other technologies for those who are changing the world. Thu, 28 Mar 2024 12:41:23 +0000 en-US hourly 1 https://wordpress.org/?v=6.2.2 The Steve Eddins Interview: 30 years of MathWorking https://blogs.mathworks.com/matlab/2024/03/28/the-steve-eddins-interview-30-years-of-mathworking/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2024/03/28/the-steve-eddins-interview-30-years-of-mathworking/#respond Thu, 28 Mar 2024 12:41:03 +0000 https://blogs.mathworks.com/matlab/?p=2174

This month, Steve Eddins is retiring from MathWorks after 30+ years on the job. When he joined, MathWorks was only 10 years old and had 190 staff. Recently, we just celebrated our 40th anniversary... read more >>

]]>
This month, Steve Eddins is retiring from MathWorks after 30+ years on the job. When he joined, MathWorks was only 10 years old and had 190 staff. Recently, we just celebrated our 40th anniversary and have almost 7,000 staff working all over the world. Steve has seen some massive transformations.
WUJH43Uw.jpg
Long before I joined MathWorks, I knew Steve through his popular blog, Steve on Image Processing with MATLAB which started way back in 2006. He is actually one of the people who inspired me to start blogging back then. As such, it has been a privilege to work alongside him as one of MathWorks’ official bloggers over the last couple of years. Sometimes, it really is a good thing to meet your heroes.
I recently had a chat with Steve, asking him about his time at MathWorks. A write up of this conversation is below.

MathWorks Career

Tell us about your career before MathWorks and how did it lead you here?
Mike, before I dive in, I want to say that I was an avid follower of your work in research software engineering. I always enjoyed your Walking Randomly blog. I am so happy that you have found a place with MathWorks, and especially that you are writing our MATLAB blog!
I was introduced to MATLAB sometime around 1988, when I was pursuing my Ph.D. in the digital signal processing research group at Georgia Tech. When Jim McLellan joined the DSP faculty, he brought with him his enthusiasm for MATLAB, and he convinced some of us grad students to join for a group purchase of MATLAB. I used MATLAB for some of my graduate studies, and there are some MATLAB plots in my dissertation.
6TsfIR3A.jpg
In late 1990, I joined the faculty of the Electrical Engineering and Computer Science department at the University of Illinois at Chicago (UIC). I used MATLAB for image processing research work and for preparing course materials.
By 1993, I was ready to move on from UIC, and I was considering leaving academia. MathWorks came to mind because of my MATLAB experience. I was impressed with how company and MATLAB development leaders Jack Little, Cleve Moler, Loren Shure, and Clay Thompson actively participated in the new Usenet newsgroup, comp.soft-sys.matlab. I met Loren at a signal processing conference that spring, and I sent her my resume.
When Loren told me that no positions were available, I asked to participate in the beta program for the new Image Processing Toolbox. I was a highly motivated beta tester, providing detailed reports, algorithm suggestions, and MATLAB code. Shortly after the beta program ended, I was invited to interview. That went well, and I joined MathWorks in December 1993.
What was your first role here?
I took over development of Image Processing Toolbox. I spent my first MathWorks decade working to make the toolbox faster and more memory efficient and adding the algorithms and visualization tools needed for general research and development. Because the development organization was so small then, I also had an opportunity to do MATLAB work. I created the second-generation MATLAB profiler, optimized multidimensional array processing functions, implemented image and scientific data I/O functions, helped to design image display improvements, and integrated the FFTW library.
What was your final position at MathWorks?
The same as my first position! In the summer of 2020, the first pandemic year, I realized that I really missed doing image processing and toolbox development work. I approached Julianna, the manager of the team responsible today for the Image Processing Toolbox, about coming back to the team. I am grateful to her and to the other development managers who smoothed the way for me to rejoin the Image Processing Toolbox team in February 2021.
For about a dozen years prior to that, I was mostly doing MATLAB development. I managed several MATLAB teams. I was on the language design team, and I was one of a small group of senior designers who reviewed every feature going into MATLAB. I also helped create design standards for MATLAB.
What is it about MathWorks that made you stay so long?
It’s been the same answer throughout my career: I love making tools that make a real difference in the worlds of engineering and science, and I love doing that work with the kind of people who are here at MathWorks.
What advice would you give to a new MathWorker?
Now you’re really making me think hard, Mike. I hesitated for a long time about this, wondering what I could say that could be useful to any new MathWorker, whatever they might be doing and wherever around the world they might be doing it.
It finally dawned on me that MathWorks works hard to give helpful information to new MathWorkers, and the best thing I can do is to add my own perspective to that.
First, look at the company’s mission, which tells you the fundamental things that the company is trying accomplish in the technology, business, human, and social spheres. These have not changed during my 30 years with the company. With my engineering background, I tend to focus on the technology piece: “Our purpose is to change the world by accelerating the pace of discovery, innovation, development, and learning in engineering and science. We work to provide the ultimate computing environment for technical computation, visualization, design, simulation, and implementation. We use this environment to provide innovative solutions in a wide range of application areas.” But all four areas are important.
Your next step is to think about the company’s values. These tell you about the kind of people that MathWorks prefers to hire, and how MathWorks expects them to behave. If you can align your work with the company mission and plans, and if your personal approach to work life and interaction with others is consistent with the company values, then you have a great chance for a long, rewarding MathWorks career.

Blogging

You started the blog in 2006. What led you to do this?
I have always enjoyed taking time to think carefully about something and then write an explanation. Blogging within MathWorks became a thing sometime around 2005, and I actively participated in that. Also, this was soon after the publication of Digital Image Processing Using MATLAB, which I co-authored. So, when Ned Gulley (who writes over at the MATLAB Community blog) started talking about creating technical blogs on mathworks.com, I approached him about it. I want to thank Ned for convincing MathWorks to create this technical blogging space, and I also want to express my gratitude for his encouragement of me, then and in the years since.
What have you got out of blogging over the years?
The biggest thing is a sense of connection with people around the world who are interested in MATLAB and image processing. This connection has been rewarding and motivating for me.
I also find it rewarding to be able to help other people in meaningful ways. It has been a joyful coincidence that some of the things that I personally find interesting have been also interesting and helpful to others. As I have been preparing for my retirement over the last couple of months, I have been delighted to hear from many MathWorkers that my blog helped them to learn about MATLAB and image processing. In some cases, it helped them to find their career here.
Also, the process of working out and writing an explanation of some technical topic almost always teaches me something new about it. So, writing the blog has helped to advance my own knowledge.
What was your most popular blog post?
Popularity is a bit hard for me to measure, as web traffic measurements are affected by search in unexpected and hard-to-interpret ways. If you don’t mind, let me just offer some highlights (from approximately 600 posts over 18 years).
Some fun deep dives:
Best comment thread: Image processing in the movies
Most controversial topics:

Technical Computing

What skills do you have now that you couldn’t imagine having 30 years ago?
Well, that must be, believe it or not, naming things. I actually created a course for teaching software developers how to name API elements expressively, accurately, and effectively. I have taught that course about three dozen times.
Do you have a favourite function that you helped develop?
It’s impossible for me to pick a single favourite (or favorite) function. Here are a few that have special meaning to me.
  • repmat Before repmat came along, MATLAB power users replicated a vector to form a matrix using a using a peculiar-looking indexing expression that was called “Tony’s Trick.”
  • fftshift (support for arbitrary number of dimensions) When MATLAB 5 expanded the MATLAB world from matrices to multidimensional arrays, I worked on adding multidimensional array support to several functions. This experience taught me how to think about and implement generic multidimensional array manipulation, which in turn influenced Image Processing Toolbox designs.
  • imresize I have revisited image resizing several times over the years. The heart of this Image Processing Toolbox function is now doing some heavy lifting in several ways throughout MATLAB, and I think it has influenced implementations elsewhere in the world, such as in deep learning. My colleague Alex and I jokingly ask ourselves whether it is possible to make an entire career about image resizing. I think the answer might be yes.
  • poly2mask I have learned repeatedly, during my career, how difficult computational geometry problems can be when working on a discrete grid. I enjoyed finding creative algorithms to solve this particular problem in a way that is geometrically self-consistent and free from floating-point issues. The resulting function has been a central workhorse in Image Processing Toolbox for years.

What is up next for you?

There are two things coming up next. The first is playing French horn. I have been working intensively to improve my horn playing for about the past eight years, and I will be studying and playing even more after retirement. Today, I play in the Concord Orchestra and the Melrose Symphony, both near Boston, Massachusetts. I will continue to do that and also look for other opportunities to perform and study. I write about studying and playing horn in the Horn Journey blog.
Second, I will continue working with MATLAB and image processing. It will be as a hobbyist, though, and not as my day job. I look forward to continuing to contribute to the MATLAB community. Look for me at MATLAB Answers, the File Exchange, and Discussions. My new MATLAB Central profile is here, and my LinkedIn profile is here. I plan to continue writing about MATLAB and image processing on my new Matrix Values blog.
]]>
https://blogs.mathworks.com/matlab/2024/03/28/the-steve-eddins-interview-30-years-of-mathworking/feed/ 0
MATLAB R2024a has been released: Here are my favourite updates https://blogs.mathworks.com/matlab/2024/03/26/matlab-r2024a-has-been-released-here-are-my-favourite-updates/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2024/03/26/matlab-r2024a-has-been-released-here-are-my-favourite-updates/#comments Tue, 26 Mar 2024 15:30:29 +0000 https://blogs.mathworks.com/matlab/?p=2152

The latest version of MATLAB is now available for download and it's our biggest update yet. I have to tell you, I'm really excited by this one! It has got some features that I've been wanting for for... read more >>

]]>
The latest version of MATLAB is now available for download and it's our biggest update yet. I have to tell you, I'm really excited by this one! It has got some features that I've been wanting for for a long long time. I'll be doing deeper dives into some of my favourite things over the next few weeks but, for now, here's an overview of some of the features that got me excited for R2024a.
These are just a few of my personal highlights out of thousands of updates. The official release highlights page is here but even that is just a subset of the full release notes.
MATLAB
My favourite toolbox updates
1711362016359.gif
  • Simulink 3D animation: This isn't a product I'd used before but just a glance at the video below made me reach out to the team and ask "Tell me everything!"
Simulink3D_2024a.gif
Official release video for R2024a
Untitled Project.gif
]]>
https://blogs.mathworks.com/matlab/2024/03/26/matlab-r2024a-has-been-released-here-are-my-favourite-updates/feed/ 3
Understanding Tolerances in Ordinary Differential Equation Solvers https://blogs.mathworks.com/matlab/2024/03/20/understanding-tolerances-in-ordinary-differential-equation-solvers/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2024/03/20/understanding-tolerances-in-ordinary-differential-equation-solvers/#comments Wed, 20 Mar 2024 12:25:45 +0000 https://blogs.mathworks.com/matlab/?p=2113 This is a guest blog post by Michael Hosea, a numerical analyst at MathWorks. He works on MATLAB Coder and on MATLAB’s ODE and integral solvers.MATLAB's ODE solvers have tolerances that the user can... read more >>

]]>
This is a guest blog post by Michael Hosea, a numerical analyst at MathWorks. He works on MATLAB Coder and on MATLAB’s ODE and integral solvers.
MATLAB's ODE solvers have tolerances that the user can change. Users are often reluctant to set these tolerances, perhaps because they think they are only for power users, and they are reluctant to alter “factory settings.” The sentiment is understandable. Tolerances do occur in many different contexts in MATLAB, and some of them are that sort of thing. For example, you probably won't need or want to change the default tolerance in the rank function. But as we're going to see, that's not the situation with ODE solvers. It may surprise you to hear that we don't actually know how to set your tolerances in an ODE Solver.
Most of what follows will be geared towards understanding how to do exactly that, but the concepts apply much more broadly. The application is direct with integral, ismembertol, and uniquetol, to name a few, despite differences in how tolerances are supplied and used. Even more generally, the principles apply in unit testing when one has exact or reference values to compare with results from software being tested.
Of course we did have reasons for the default tolerances used in the ODE solvers. We wanted the default tolerances to request enough accuracy for plotting, and it probably doesn't take a lot of accuracy to do that. Also, we're using the same defaults for all the solvers. Consequently, they needed to work with low-order and high-order solvers alike. Loose tolerances tend to work well enough with high-order solvers. On the other hand, I don't know if you've ever tried using tight tolerances with low-order solvers, but patience is virtue. Barring enough of that, ctrl-c is your friend. The resulting default tolerances are rather loose. Unfortunately, we don't know the context of the problem you are trying to solve. You might want more accuracy than these default tolerances are requesting, particularly if you are using one of the higher order methods. So let's talk about about the tolerances and how to go about setting them for your purposes.
Fortunately, no great understanding of the software is needed to set tolerances in a reasonable way, only an understanding of the problem you are trying to solve and what these tolerances mean. We should be able to set reasonable tolerances if we can answer the following questions about the problem we are trying to solve:
  1. How small of a value do we consider negligible?
  2. How precise is the data that defines the problem?
  3. What percentage error is small enough to ignore? Or, put differently, how many significant digits of accuracy do we need?
Our ODE solvers allow you to set two different tolerances. One is the absolute tolerance and the other relative tolerance. These two work together. Absolute tolerance is the simpler of the two, so let's start there.

Absolute Tolerance

We say an approximate result y is within an absolute tolerance A of an exact or reference value y0 if
y0-Ayy0+A
Here we assume A0. Subtracting y0 from each term gives us
-Ay-y0A
which we can write succinctly as
|y-y0|A
When y0 is the exact value we are trying to approximate with y, we call the quantity on the left the absolute error.

absoluteerror=|y-y0|

Before we go too far, let's just point out the obvious. We don't usually know y0. If we did, there wouldn't be much point in doing any work to approximate it unless we were just testing the code. Codes like integral and the ODE solvers compute an estimate of the absolute error and use that in place of |y-y0|. Sometimes it is an approximate bound on the error rather than an estimate per se. Exactly how this is done isn't important to our discussion here except to say that if things are going well, this estimate should be about the same size as the actual error.
So, we're done, right? An absolute tolerance should be able to get the job of describing the accuracy we need done no matter what we're doing. Well, yes and no. Suppose you're solving a problem where the correct answer is, say, 10 meters, and you'd like an answer to be correct to within ±0.5mm. So, for that, you'd set your absolute tolerance to A=0.0005m. Notice that the absolute tolerance in a physical context has units, in this case meters because y and y0 are in meters.
So we write some code that defines
AbsoluteTolerance = 0.0005;
and we keep the units in our head. We're used to doing that.
But what if the solution to our problem isn't just a scalar value, rather an array of values representing different kinds of quantities, maybe lengths, areas, volumes, mass, temperature, you name it. Assuming the software allows you to set a different absolute tolerance for each element of a solution array in the first place, you'd need a pretty good idea of the size of the solution element to decide ahead of time what the absolute tolerance on each element should be. Probably we can't just take a number like 0.0005 and think it's going to be appropriate for everything.
Moreover, how did we decide that ±0.5mm was reasonable in the first place? For something 10m long, that's rather precise for many practical purposes, often enough even for something only 1m long, but it isn't for something 0.001m long. In that case it covers a substantial difference percentage-wise. Sometimes it's not so much the absolute error that concerns us, rather the error in a "percentage" sense.
And wouldn't it be convenient to have a way of describing the accuracy we want using a dimensionless tolerance? It would be nice to have something that automatically adjusts itself for different scaling, something that we don't have to change if we merely restate the same problem using different units of measurement. After all, it shouldn't be any harder or easier to compute something to 0.5mm than 0.0005m; that's the same distance.

Relative Tolerance

You're probably familiar with the concept of significant digits or significant figures when reporting data. The basic idea is that if you see the measurement of some kind, say x=1.5mm, it corresponds to an unspecified exact quantity that has (hopefully) been correctly rounded to 1.5mm. This means that
1.45mmxexact<1.55mm
since that is the interval of numbers that rounds to 1.5mm. Expressed in the parlance of absolute error, we're saying that the absolute error
|x-xexact|0.05mm.
That formulation includes the upper bound of 1.55mm, but we needn't quibble about that insofar as tolerances are ultimately used with error estimates, anyway. If we had instead written x=1.500mm, we would be implying that
1.4995mmxexact<1.5005mm,
hence that
|x-xexact|0.0005mm.
Here again we have, without apology, added in the upper end point. In the first case we say that we have 2 significant digits, and the latter 4.
Note that 0.05 can be written as 5×10-2, and 0.0005 as 5×10-4. The exponents seem to indicate the number of significant digits. Unfortunately, the conjecture doesn't hold up. To see this, let's just change units of measurement. That shouldn't change the precision of anything. Switching from millimeters to microns, we have in the first case
1450μxexact<1550μ
and in the second,
1499.5μxexact<1500.5μ .
And again, in the language of absolute error, we have
|x-xexact|50μ
and
|x-xexact|0.5μ,
respectively. The "absolute tolerances" on the right are 5×10+1μ and 5×10-1μ. Now the exponents don't seem to bear any relationship to the number of significant digits.
The problem is clearly scaling. Because absolute error has units of measurement, merely changing units of measurement rescales the data. So how can we express the idea that we want a certain number of digits to be correct? We need something that isn't sensitive to scaling. Enter the idea of relative error:

relativeerror=|y-y0||y0|

All we've done here is divide the absolute errory by |y0|. Obviously we are assuming that y00. More on that later.
Note that relative error is just the percentage difference formula with absolute values (and without the final multiplication by 100% to convert to percent).

If you multiply a relative tolerance by 100%, you get the percentage error that the tolerance allows.

This observation may help later when we discuss how to choose the relative tolerance.
A hopeful sign is that the relative error is now dimensionless because the numerator and denominator have the same units. Obviously any scaling of y cancels out. Let's use R from here on out to denote a relative error tolerance. So we want
|y-y0||y0|R
Clearing the fraction, we get
|y-y0|R|y0|
That's usually how we use relative error in software, since it avoids the division.
Let's try this out on an example. Suppose y0=1234500. Looks like 5 significant digits there. Let's try R=5×10-5. Plugging in, this becomes
|y-1234500|61.725
Which means that
1234500-61.725y1234500+61.725
or
1234438.275y1234561.725
If we were to change the units on y, we would also scale those end points by the same factor as y itself is changed, just as if we had multiplied the inequality through by the scaling factor.
Unfortunately, those end points don't round to 1234500 when expressed with 5 significant digits. The interval is wider than we want it to be if we wanted to guarantee 5 significant digits correct. If, instead, we tighten it up to R=5×10-6and repeat the exercise, we end up with
1234493.8275y1234506.1725
Now it's tighter than we needed, but those endpoints do round to 1234500 written to 5 significant digits. This generalizes:.

A relative tolerance for at least n significant digits is

R=5×10-(n+1)

You can use this relative tolerance when you don't know what y0 is or if you want the same formula to work when y0 varies.
Just as an aside, the interval we would have wanted to see was
1234450y1234550
You might be curious what relative tolerance gives this. If you're not the least bit curious, skip to the next section, because we won't be using this later.
Define a preliminary tolerance R=5×10-(n+1) as above. Then the relative tolerance that gives the loosest bounds that require n correct significant digits is
R10(ceil(c)-c)
where
c=log10(|y0|)
The exponent there is just the fractional part of c. In our example the tolerance works out to be about 8.1 times larger than R. This should be no surprise since we already knew from our experimentation that R was too tight and10R too loose. There's nothing very nice or memorable there, and we can only construct this "ideal" relative tolerance if we know y0, so we will not make further use of this fact, but if you were setting up a unit test comparing software output to a known correct result, you could make use of it.

What if y0=0?

The definition of relative error has a serious problem when the reference value y0 is zero. The requirement
|y-y0||y0|R
becomes impossible to meet. As previously mentioned, we normally avoid the division, anyway, and evaluate
|y-y0|R|y0|
That avoids the division by zero, but it hardly helps very much when y0=0 because the test is only satisfied when y=y0, i.e. when the absolute error is zero.
It might be helpful to think about this as a limiting case. Consider a sequence of problems where the exact solution is not zero but is smaller and smaller in magnitude. Let's say the exact solution is y0=12345×10-K for increasing values of K.You might imagine that we're rewriting the same physical quantity in larger and larger units of measurement as K increases.
K
12345×10-K
10
0.0000012345
20
0.00000000000000012345
30
0.000000000000000000000000012345
40
0.0000000000000000000000000000000000012345
Leading zeros to the right of the decimal point aren't "significant" in the sense of significant digits, but obviously they determine the magnitude of the number. Relative error control is fine for all these values as long as all those leading zeros are computationally free to obtain. That might sound unlikely, but it actually could be the case if this were only a matter of scaling, say when switching units from millimeters to light years.
In the limiting case of y0=0, however, relative error control assumes that all those zeros are "insignificant", computationally free to obtain. Unfortunately, when y0=0 those leading zeros are all the digits that there are!
It still could be the case.that they are free. For example, if everything in the problem is zero, then the intermediate computations are probably going to yield zero at every stage. Or it might happen if there is symmetry that results in perfect cancellation, e.g. integrating an odd function over the interval [-1,1] and exact zero is obtained not because the intermediate calculations are exact, rather because the intermediate results cancel out perfectly, i.e., are equal and opposite, regardless of how inexact they might be.
These are common enough scenarios, but they are not always the case. Generally speaking, it is not easier to compute the ideal result y=0 when y0=0 than it is to compute the ideal result y=1.234567890 when y0=1.234567890. Just as it is usually impractical to require that the absolute error is zero, so it is impractical to impose any relative tolerance when y0=0. As convenient as relative error control is, we have a hole at zero that we need to fill somehow.

Combining Absolute and Relative Error Control

Absolute error is easy to understand but can be difficult to use when the problem contains values of different scales, and we need know something about the magnitude of the solution values before we have computed them. Controlling the error in the relative sense instead rectifies these limitations of absolute error control, but it doesn't work in any practical way when the desired solution happens to be very close to zero.
Fortunately, y0=0 is a value that doesn't have any scaling to worry about. Theoretically, then, absolute error should work fine at and very near zero, while relative error works everywhere else. So let's splice them together. Historically this has been accomplished in more than one way, but more commonly today it is done like this:
|y-y0|max(A,R|y0|)
or in MATLAB code
abs(y - y0) <= max(AbsoluteTolerance,RelativeTolerance*abs(y0))
As previously mentioned, we don't generally know y0, and the left-hand side will be an estimate of the absolute error obtained somehow or other. If we also assume that the value of y that we have computed at least has the right magnitude, we can substitute it for y0 in that expression. So in software the test will take the form
errorEstimate <= max(AbsoluteTolerance,RelativeTolerance*abs(y))
Here errorEstimate is an estimate of the absolute error, or possibly an approximate upper bound on it. Using max, not min, ensures that we choose the least restrictive of the two tolerances. This is important because the hole we are trying to fill occurs because relative error control becomes too restrictive near y0=0.
Perhaps it is interesting to observe where the switchover from using the absolute tolerance to using the relative tolerance occurs. It occurs when AbsoluteTolerance=RelativeToleranceabs(y). In other words, it occurs when
abs(y)=AbsoluteToleranceRelativeTolerance
For example, the default tolerances for the integral function are AbsoluteTolerance = 1e-10 and RelativeTolerance = 1e-6. Consequently when we estimate that a quantity is less than 1e-10/1e-6 = 1e-4 in magnitude, we use AbsoluteTolerance and control the error in the absolute sense, otherwise we use RelativeTolerance and control it in the relative sense.
Not all error control strategies in numerical software have been formulated that way, but they can have similar effects. Often only one tolerance was accepted, and in that case one could adapt the above strategy to
errorEstimate <= tol*max(1,abs(y))
This effectively makes the two tolerances the same, so that absolute error control is used for abs(y)1and relative error control for abs(y)1. Since error control is not an exact affair to begin with, and we have even substituted y for y0, addition can be used instead of the max function:
errorEstimate <= tol*(abs(y) + 1)
If abs(y) is very small, the factor on the right is just a little larger than 1, hence the test is essentially comparing the estimate of the absolute error with the tolerance. That's absolute error control. If, on the other hand, abs(y) is very large, the addition of 1 matters but little, and so the effect is relative error control.

Floating Point Limitations

Floating point arithmetic has limitations that may affect the tolerances we choose, so let's review what they are. MATLAB provides the functions eps, realmin, and realmax that tell you about limits in the floating point type.
Double precision:
eps
2.220446049250313e-16
realmin
2.225073858507201e-308
realmax
1.797693134862316e+308
Single precision
eps("single")
1.1920929e-07
realmin("single")
1.1754944e-38
realmax("single")
3.4028235e+38
We used the eps function above without numeric inputs, but writing just eps with no inputs is the same as writing eps(1), and writing eps("single") is the same as writing eps(single(1)). In what follows we'll stick to the double precision case, but the same principles apply to single precision. The documentation says:
"eps(X) is the positive distance from abs(X) to the next larger in magnitude floating point number of the same precision as X."
Let's unpack this in case you've never thought about it. We use floating point numbers to model the real numbers, but there are infinitely many real values and only a finite number of floating point values. In double precision there are 252evenly spaced numbers in the interval 1x<2. This goes for any consecutive powers of 2, i.e. there are 252 evenly spaced double precision floating point numbers in the interval 2nx<2n+1, so long as -1022n1023. In MATLAB you also have that many in the interval 0x<2-1022. which is the same as 0x<realmin.
In every case, the very next larger floating point number after x0 is x+eps(x), so the spacing between the floating point numbers in the interval 2nx<2n+1 is eps(2n). The value of eps(x) is the same throughout the interval. For example:
n = 3;
x = linspace(2^n,2^(n+1),5)
x = 1×5
8 10 12 14 16
eps(x)
ans = 1×5
10-14 ×
0.1776 0.1776 0.1776 0.1776 0.3553
There are no floating point numbers between, say, 10 and 10+eps(10). If you try to compute the midpoint between a=x and b=x+eps(x), c=(a+b)2, the result will be either c=x or c=x+eps(x).
OK. So what's the point again? We're talking about floating point limitations on tolerances. With extremely tight tolerances, we will, in effect, be asking the code to return one of a small set of discrete values. For example, if we chooose an absolute tolerance of eps when y0=2, then by definition, we are asking for a result y such that
2-epsy2+eps
The floating point number 2-eps is less than 2, but it turns out to be the one floating point number immediately preceding 2. At the upper end point, however, 2+eps=2 in floating point, so the range above is just another way of saying
y{2-eps,2}
Some "range", that! We have allowed for only two possible values. Similarly, for a relative tolerance of eps, we get from applying the definition that
2-2epsy2+2eps
which turns out to be satisfied by just 4 possible values of y:
y{2-2eps,2-eps,2,2+2eps}
The upshot is that we will need to be careful when setting our tolerances so that we are asking for something reasonable.
We will always want to choose Reps. Even in testing scenarios where you have y0 in hand and you're just comparing outputs, you will probably want to use at least a small multiple of eps. The ODE and integral solvers will not allow anything smaller than 100eps.
With absolute tolerances things are more complicated because we must take account of the scale of values. An absolute tolerance can be smaller than eps when the solution values will be much smaller than 1. It can also be larger than 1. Consider a problem where the exact solution is 1 meter and anything smaller than 10 microns is negligible. If you were solving the problem using meters as your unit of measurement, then you'd set your absolute tolerance to 0.00001 meters because that's 10 microns. But now rescale the problem to use microns as your unit of measurement instead of meters. Now the exact solution is 1e6 microns, and your absolute tolerance would be 10 microns. Conversely, if you were to solve the problem in petameters, the exact answer is now 1e-15 petameters, and your absolute tolerance would be 1e-20 petameters. We're just changing units to illustrate the point. Your problem will have probably have units that are natural for the problem, and you'll need to deal with that as it comes. Whereas the value of eps places a natural limit on the relative tolerance, the realmin and realmax values present limitations on what you can use for an absolute tolerance. As with eps and the relative tolerance, you should avoid these extremes. Rescale the problem if you find yourself wanting to set an absolute tolerance anywhere near realmin or realmax.

How to set Tolerances

But how does one go about actually choosing tolerances for a computation that solves some real world problem? To answer that, we just need to answer a few questions.

1. How small of a value do we consider "negligible"?

How small is small enough that it is nothing for all practical intents and purposes? This might be different for different pieces of the solution, but it provides a reasonable value for the absolute tolerance. If the solution to your problem contains a weight for some powder that is 1.23456g, and your scale is only accurate to 0.1g, there's not much you can do with the extra 0.03456g in the result even if those numbers are correct. What we have there is 1.2g plus a negligible quantity. It makes sense to use tolerances that only ask for accuracy to, say, 0.01g. Suppose we did that and got 1.23654 instead. Now 0.00654 of that is "noise", but it's the same 1.2g when rounded to the 0.1g that our scale can actually measure.
Recall that an absolute tolerance is not limited by eps. It has the same units as the solution, and it can be freely set to whatever value makes sense, whether 1e-40 or 1e+40 depends on how the problem is scaled. Sometimes you might even be allowed to make it exactly zero, but that's a bold move unless you know the solution isn't close to zero.

2. How precise is the data that defines the problem?

If there are only a few significant digits of precision in the data that defines the problem to be solved, it's not likely that a more accurate solution will be more useful than one computed to about the same accuracy as the problem itself is specified. Of course, it's not really that simple. It may be that solution curves are very sensitive or very insensitive to variations in different problem data. It usually is not clear how the error in the problem specification manifests in the solution. Nevertheless, we are typically computing an approximate solution to an approximate problem, and that has implications. Given that we're starting with some error, how much are we willing to pay for extra digits of accuracy from the solver? The answer is probably "not a lot", and yet these extra digits will most likely cost us additional time and maybe computing resources to obtain. If the numerical data that defines the problem is only accurate to, say, 5 significant digits, we might reasonably decide to ask the solver for no more than 5 or 6 digits of accuracy when solving it. If 5 digits of accuracy seems OK to us, we could set R=510-6 or R=510-7 for good measure.

3. What percentage error is small enough to ignore? or How many significant digits of accuracy do we need?

If you said that a 0.01% error is small enough to ignore, then a reasonable relative tolerance would be 0.0001, as long as that isn't much beyond the precision of the data that defines the problem. If you prefer to think in terms of significant digits, and need, say, 5 significant digits, then a reasonable relative tolerance might be 5×10-6.
Then combine this with your answer to the previous question. Since there is some hand-waving about how the solution reacts to errors in the problem specification, one can make equally good arguments for picking the larger or the smaller of the two. If accuracy is especially important to you, use the smaller.
This is just a way of setting "reasonable" tolerance values. It provides a baseline of sorts. Nothing prevents you from choosing tighter or looser tolerances, and there can be good reasons to go with tighter or looser tolerances. For one thing, tolerances are based on error estimates, and in some cases not even error estimates for the final solution values. In ODE solvers, for example, they are used "locally", step by step. This can end up limiting the error in the final solution values in different ways to different degrees, so for some "insurance", slightly tighter tolerances can be a good thing. On the other hand, run times tend to go up with tighter tolerances. Memory requirements may also increase. If the solution changes but little and time is of the essence, looser tolerances might be justified, provided you are comfortable with the potential for larger errors. If the computation is occuring in a real-time system, for example, you would want to experiment with tolerances to see how the run-time and solution values react to tighter and looser tolerances.

Example

Let's take a look at a first order system of ordinary differential equations.
ddty=f(t,y),y(0)=y0
This example has been constructed so that we know the exact solution.
function yp = f(t,y,param)
yp = zeros(size(y));
yp(1) = 2*t*(1 - y(2)) + 1;
yp(2) = 2*t*(y(1) - t - param);
end
 
function y = exactSolution(t,param)
y = zeros(2,1);
y(1) = cos(t.*t) + t + param;
y(2) = sin(t.*t) + 1;
end
And let's use this little comparison function I wrote so that we can easily see the accuracy of the results.
function compare(y,yExact)
fprintf(1," y(1) = %14.9f, y(2) = %14.9f\n",y(1),y(2));
fprintf(1,"exact y(1) = %14.9f, exact y(2) = %14.9f\n",yExact(1),yExact(2));
absoluteError = abs(y - yExact);
relativeError = absoluteError./abs(yExact);
fprintf(1,"absolute error y(1) = %5.0e, absolute error y(2) = %5.0e\n",absoluteError(1),absoluteError(2));
fprintf(1,"relative error y(1) = %5.0e, relative error y(2) = %5.0e\n\n",relativeError(1),relativeError(2));
end
Let's define this specific problem using as many digits as double precision allows. This will represent a problem that is specified exactly.
param = 1.234567890123456;
y0 = exactSolution(0,param); % "exact" initial value
D = ode(ODEFcn=@f,Parameters=param,InitialValue=y0);
t1 = 5; % Solve for the solution at this time point for comparisons.
yExact = exactSolution(t1,param); % The exact solution to our differential equation at t1.
% Problem specified to full precision. Using default tolerances.
sol = solve(D,t1);
compare(sol.Solution,yExact);
y(1) = 7.227997543, y(2) = 0.876442186 exact y(1) = 7.225770702, exact y(2) = 0.867648250 absolute error y(1) = 2e-03, absolute error y(2) = 9e-03 relative error y(1) = 3e-04, relative error y(2) = 1e-02
The default tolerances are not tight, and it shows in the accuracy we get back. We got 3 digits correct in y(1) and just 1 digit correct in y(2). Let's suppose we want 8 digits. So we'll set the relative tolerance to 5e-9.
We've just made up this problem, but what amounts to a "negligible amount" in a system like this could in theory be different for y(1) than y(2). A little known fact is that the MATLAB ODE solvers support having different absolute tolerances for different solution components. This problem doesn't really depend on it, but just to show how it's done, assume that for y(1) we think a negligible amount is 1e-10 and for y(2), 1e-11.
% Problem specified to full precision. Asking for about 8 significant
% digits of accuracy.
D.RelativeTolerance = 5e-9;
D.AbsoluteTolerance = [1e-10,1e-11];
sol = solve(D,t1);
compare(sol.Solution,yExact);
y(1) = 7.225770703, y(2) = 0.867648251 exact y(1) = 7.225770702, exact y(2) = 0.867648250 absolute error y(1) = 5e-10, absolute error y(2) = 1e-09 relative error y(1) = 7e-11, relative error y(2) = 1e-09
That's more like it. We got 9 digits correct in y(1) and 8 in y(2). But this is with the problem data at full precision. Your problem may have some measurements that aren't exact. Let's simulate that by tweaking some of the problem data while leaving the exact solution unchanged. Replacing the parameter and initial value with "approximate" values:
% Problem specified with 6 significant digits. Still asking for 8.
% significant digits.
param = 1.23457;
D.InitialValue = [param + 1; 1];
sol = solve(D,t1);
compare(sol.Solution,yExact);
y(1) = 7.225772794, y(2) = 0.867647972 exact y(1) = 7.225770702, exact y(2) = 0.867648250 absolute error y(1) = 2e-06, absolute error y(2) = 3e-07 relative error y(1) = 3e-07, relative error y(2) = 3e-07
Maybe we solved the approximate problem just as accurately as before, but comparing to the "true" solution from the full precision problem, we only got 6 digits correct in y(1) and 5 digits correct in y(2). This is why you might want to let the precision of the data in the problem moderate your tolerance choices. Reducing our request to 6 digits, we get.
% Problem specified with 6 significant digits, now asking for 6
% significant digits.
D.RelativeTolerance = 5e-7;
D.AbsoluteTolerance = 1e-9;
sol = solve(D,t1);
compare(sol.Solution,yExact);
y(1) = 7.225770368, y(2) = 0.867649499 exact y(1) = 7.225770702, exact y(2) = 0.867648250 absolute error y(1) = 3e-07, absolute error y(2) = 1e-06 relative error y(1) = 5e-08, relative error y(2) = 1e-06
This time we got 6 digits correct in y(1). We only got 5 digits correct in y(2), though, so maybe we should have set the relative tolerance at 5e-8. It's not a bad idea to ask for 7 digits when you need 6.

When Failures Occur

A failure related to tolerances usually occurs when the problem is too difficult to solve to the requested accuracy. If that is the case, then one must either use an algorithm that is able to solve that problem more easily, or one must relax the tolerances, or both. Failing that, the problem itself will have to be reformulated.
Sometimes, paradoxically, one must tighten the tolerances to succeed. There is an example of this here:
The tolerances for ode15s are specified in that example because the integration fails with the default tolerances, which are looser.
How is that even possible, you ask? Well, when you're driving a car on a mountain road, you need to keep the car on the road, and that's going to require a practical minimum amount of accuracy. If your tolerances are too loose and you drive off the cliff, then you won't get where you're going. How much accuracy you need in a mathematical problem is less clear than how to keep a car on the road, but the point is that if you get a failure, and if loosening the tolerances doesn't help, try tightening them. What have you got to lose? It might work. Probably the worst that's going to happen is that it fails quicker.
Now get out there and set those tolerances!
]]>
https://blogs.mathworks.com/matlab/2024/03/20/understanding-tolerances-in-ordinary-differential-equation-solvers/feed/ 3
Pi day: Using AI, GPUs and Quantum computing to compute pi https://blogs.mathworks.com/matlab/2024/03/14/pi-day-using-ai-gpus-and-quantum-computing-to-compute-pi/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2024/03/14/pi-day-using-ai-gpus-and-quantum-computing-to-compute-pi/#respond Thu, 14 Mar 2024 10:11:50 +0000 https://blogs.mathworks.com/matlab/?p=2101

14th March is Pi Day, celebrated by geeks everywhere and a great excuse for technical computing bloggers to publish something tenuously related to pi and their favorite technology of choice. This is... read more >>

]]>
14th March is Pi Day, celebrated by geeks everywhere and a great excuse for technical computing bloggers to publish something tenuously related to pi and their favorite technology of choice. This is not the first pi day celebration on a MathWorks blog and it will not be the last. Here are some of our past takes on the subject
This year, over at The Simulink Blog, Guy has been using Simscape Multibody to simulate one way our ancestors could have computed pi -- by rolling a big stone cylinder down a hill.
I, on the other hand, will be at the other end of the time line by making use of Artificial Intelligence, GPUs and Quantum computing. If only I could have fitted in 'Blockchain', I would have won buzzword bingo!

Using AI to compute pi

One method for computing pi uses a Monte Carlo simulation. Its so well known that it's actually used as a demonstration of what a Monte Carlo method is in the wikipedia entry for Monte Carlo Method! For the sake of brevity, I'll leave you to explore the mathematics for yourself. Today, I'm just going to go ahead and use The MATLAB AI Chat Playground and ask it how to code such a simulation in MATLAB.
This is what it came up with, with a small modification by me to show more decimal points:
%Added by Mike to show more decimal points in the result
format long
% Number of random points to generate
numPoints = 1e6;
 
% Generate random points in the range [0,1]
x = rand(numPoints, 1);
y = rand(numPoints, 1);
 
% Count the number of points inside the unit circle
insideCircle = (x.^2 + y.^2) <= 1;
 
% Estimate pi using the ratio of points inside the circle to total points
piEstimate = 4 * sum(insideCircle) / numPoints
piEstimate = 3.139064000000000
One of the issues with Monte Carlo simulations is that they require a huge number of samples to get an accurate answer. Indeed, the 1 million samples suggested by the AI only gives us 1 or 2 correct decimal places for pi. For more accuracy, you need a lot more samples. Not just a few million more but millions of millions more.
On my machine, the above code completes in a fraction of a second: 0.014 seconds to be precise which works out at around 71.4 million samples a second. Sounds pretty fast but it would take almost 4 hours to compute 10^12 samples and, as written, I'd run out of memory if I even tried it.
Starting from the above code and using various parallelisation and optimisation methods, I got close to 700 million samples a second on my 8 core machine and might even be able to squeeze out some more but I'll save that discussion for another time. Today, I'll take another route.

Computing pi on the GPU

Without GPUs, many of the machine learning techniques we use today would not be computationally feasible and technologies like the AI Chat Playground would not exist. I thought it would be fitting, therefore, to show how you'd accelerate the above AI-generated monte carlo code using the very devices that power modern AI.
It often surprises me that many people don't know how extensive the GPU support is in MATLAB. Using Parallel Computing Toolbox, over 1,000 MATLAB functions 'just work' on a GPU when you supply them with a gpuArray. Not only that but MATLAB can automatically compile a subset of the language to run on the GPU using arrayfun. You can even get MATLAB to write CUDA code using GPU Coder.
Today, however, let's keep it simple. Here's the code from AI Chat Playground, modified to run on my NVIDIA GPU and with 100 million samples instead of only 1 million. Other than changing the numPoints line, I only needed to modify the two calls to rand to request that the random numbers, x and y are generated on the GPU. All subsequent computations automatically work on the GPU. More details on how this works in my post How to make a GPU version of this MATLAB program by changing two lines
% This code needs an NVIDIA GPU and Parallel Computing Toolbox
% Number of random points to generate
numPoints = 1e8;
 
% Generate random points in the range [0,1]
x = rand(numPoints, 1, "gpuArray");
y = rand(numPoints, 1, "gpuArray");
 
% Count the number of points inside the unit circle
insideCircle = (x.^2 + y.^2) <= 1;
 
% Estimate pi using the ratio of points inside the circle to total points
piEstimate = 4 * sum(insideCircle) / numPoints
piEstimate = 3.141578440000000
Timing this correctly (A simple tic/toc isn't the way to go with GPU computations, see Measure and Improve GPU Performance for details) resulted in a speed of 0.06 seconds which gives us 1,666 million samples per second. That's a very nice speed up for very little work.
I can't push the above code much further though since I quickly run out of memory on the GPU. One way to proceed is to split the simulation into chunks. I'm also going to use single precision since its faster on my GPU, uses less memory and the drop in precision isn't an issue here. Let's also put the code into a function
function pi = piMonteCarloSplit(N,numChunks)
count = 0;
for i =1:numChunks
x = rand(N/numChunks, 1, "single", "gpuArray");
y = rand(N/numChunks, 1, "single", "gpuArray");
count = count + sum((x.^2 + y.^2)<1);
end
pi = 4*count/N;
end
Time this using 100x more samples than before: 1e10 split into 100 chunks, using a timing technique described at Measure and Improve GPU Performance
dev = gpuDevice();
wait(dev);
tic
piEstimate = piMonteCarloSplit(1e10,100);
wait(dev);
toc
Elapsed time is 0.825407 seconds.
10,000 million samples in 3.123 seconds gives 12,195 million samples per second. Hugely faster than where we started with barely any additional effort at all. Here's the estimate for pi we get now
piEstimate
piEstimate = 3.141574913600000
Even with 10,000 million samples, I only got 4 decimal places correct. There's a reason why this technique is not used if high precision for computing pi is what you need.
Using additional techniques, I eventually squeezed almost 17,000 million samples a second out of my GPU and there could be more performance on the table. I may write this up another time if there is sufficient interest. The point here is that it can be really easy to get started with porting a lot of MATLAB code to the GPU and the performance is pretty good!
For reference, my GPU is an NVIDIA GeForce RTX 3070 and I'm also using it as the display driver.

Calculating pi using a quantum computer

Guy's post took us back to ancient times. I wanted to take a glimpse at the future and look at how we could calculate pi with a quantum computer. I had my first play with quantum computers last year in the post Quantum computing in MATLAB R2023b: On the desktop and in the cloud and so googled how to compute pi using one.
IBM posted the algorithm over at YouTube with an implementation in Qiskit. I reached out to one of our Quantum computing experts at MathWorks, Matt Bowring, and asked him to write a version for MATLAB making use of the MATLAB Support Package for Quantum Computing which you'll need to install if you are following along. Here's his function
function [pie,circ] = estimatePi(N)
% Return pi estimation pie using Quantum Phase Estimation with N qubits
% Inspired by the pi day code at https://github.com/Qiskit/textbook/blob/main/notebooks/ch-demos/piday-code.ipynb
 
% Estimate phase angle theta for U|1> = exp(2*pi*i*theta)|1>, setting U as
% the r1Gate with phase angle 1, so that QPE finds 2*pi*theta = 1.
% Increasing N improves the estimate precision since possible theta values
% are discretized over the 2^N measurable bitstring states.
 
estQubits = 1:N;
phaseQubit = N+1;
 
gates = [
% Initialize bottom qubit to |1>
xGate(phaseQubit)
% Apply QPE using r1Gate with angle 1 by controlling it on the
% estimation qubits
hGate(estQubits)
cr1Gate(estQubits, phaseQubit, 2.^(N-1:-1:0))
inv(qftGate(estQubits))
];
circ = quantumCircuit(gates);
 
sv = simulate(circ);
 
% Use most probable measurement of the estimation qubits to find theta
[bitstrings, probs] = querystates(sv, estQubits);
[~, idx] = max(probs);
theta = bin2dec(bitstrings(idx)) / 2^N;
 
% Estimate pi knowing that 1 = 2*pi*theta
pie = 1 / (2*theta);
end
Simulating this on my PC using a circuit with 8 qubits gets the first 2 digits correct
qPi = estimatePi(8)
qPi = 3.121951219512195
As we increase the number of qubits, we get a better approximation.
qubits = 2:12;
pies = zeros(1,length(qubits));
for n = 1:length(qubits)
pies(n) = estimatePi(qubits(n));
end
 
plot(qubits, pies, 'r-o')
hold on
yline(pi)
legend('estimate', 'true')
xlabel('Number of qubits')
hold off
The function can also return the circuit used in each case which allows us to visualize it. Here's the 4 qubit case
[qPi,circ] = estimatePi(4);
plot(circ)
Today I only simulated these quantum systems but you can also try running them on real quantum hardware too. How to do this using IBM's quantum services is covered in the MATLAB documentation Run Quantum Circuit on Hardware Using IBM Qiskit Runtime Services. My blog post, Quantum computing in MATLAB R2023b: On the desktop and in the cloud discusses how to run on quantum systems via AWS Braket.
So that's it from me and Guy. Whether you calculate your pi's using state of the art computing hardware or a large stone cylinder, happy pi day!
]]>
https://blogs.mathworks.com/matlab/2024/03/14/pi-day-using-ai-gpus-and-quantum-computing-to-compute-pi/feed/ 0
MATLAB extension for Visual Studio Code: Yes, it works with GitHub Copilot https://blogs.mathworks.com/matlab/2024/03/12/matlab-extension-for-visual-studio-code-yes-it-works-with-github-copilot/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2024/03/12/matlab-extension-for-visual-studio-code-yes-it-works-with-github-copilot/#comments Tue, 12 Mar 2024 13:47:15 +0000 https://blogs.mathworks.com/matlab/?p=2074

After publishing my recent blog post about MATLAB code execution in Visual Studio Code, I've had a lot of questions asking about GitHub Copilot support. I've even had fellow MathWorkers telling me... read more >>

]]>
After publishing my recent blog post about MATLAB code execution in Visual Studio Code, I've had a lot of questions asking about GitHub Copilot support. I've even had fellow MathWorkers telling me that this should have been covered in the previous article because "Everyone keeps asking us about Copilot support"
In my defense, I didn't mention GitHub Copilot before because it wasn't part of our extension. MathWorks didn't deliver it so it seemed a little strange to me to 'announce' it. But I hear you...you all want to know about MATLAB and GitHub Copilot from Visual Studio Code.
It works great! Enjoy
coPilot.gif
Visual Studio Code not for you but you'd still like some AI assistance with your MATLAB programming? Try the MATLAB AI Chat Playground.
]]>
https://blogs.mathworks.com/matlab/2024/03/12/matlab-extension-for-visual-studio-code-yes-it-works-with-github-copilot/feed/ 1
MATLAB extension for Visual Studio Code: Now with code execution https://blogs.mathworks.com/matlab/2024/03/05/matlab-extension-for-visual-studio-code-now-with-code-execution/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2024/03/05/matlab-extension-for-visual-studio-code-now-with-code-execution/#comments Tue, 05 Mar 2024 17:18:58 +0000 https://blogs.mathworks.com/matlab/?p=2032

Back in April last year, I announced the MATLAB extension for Visual Studio Code on The MATLAB Blog and everyone at MathWorks was blown away by its popularity. In less than a year there have been... read more >>

]]>
Back in April last year, I announced the MATLAB extension for Visual Studio Code on The MATLAB Blog and everyone at MathWorks was blown away by its popularity. In less than a year there have been over 150,000 installs of the MATLAB extension via the Visual Studio Marketplace, the announcement blog post is one of the most popular blogs across all of MATLAB Central and there has been a lot of active discussion on the extension’s GitHub repository including enhancement requests and bug reports. It’s safe to say that it has been a great success.
If you are new here and are wondering what all the fuss is about with respect to MATLAB and Visual Studio Code, you can check out this YouTube video alongside reading my original blog post announcing the release of the first version. Today’s post will focus on what’s new.

Pull requests are welcome

Split into two repositories,MATLAB-extension-for-vscode and MATLAB-language-server, the project has been available on GitHub since the beginning and community contribution is encouraged. It has been great to see contributions from people outside of MathWorks including Tiago Vilela, who used the MATLAB Language server to provide MATLAB support for the Neovim editor.
We've also had pull requests from Doron Behar and Moeta Yuko fixing various things. Thank you so much to everyone who has contributed so far and we look forward to more collaboration with the community.

Listening to your feedback

Being a developer, product manager or advocate of any popular piece of software can be a wild ride emotionally. On the one hand, you get great feedback like this
Screenshot 2024-02-26 172419.png
Positive with a side order of constructive suggestion, something we can work with. We received many comments like this in social media, via email and in one to one discussions with users. However, we also received a few comments like this.
waste_of_time.png
Ouch! Clearly, we fell way below this user’s expectations and we are sorry that the first release may have come up short for some users. We didn’t want to allow “perfect” to be the enemy of “good” and we felt the first release had a lot of useful functionality, so we released it into the wild. Our goal was to provide a solid foundation that could be built upon quickly in collaboration with the community and it's working! Over 150,000 users so far, contributions from the community and fast iteration on bug fixes. There have been 10 updates since the first release in less than a year. We are really happy with these results.
With that said, the lack of code execution from within Visual Studio Code was a common complaint among users, even among MathWorkers. It was also one of the first things requested on the project's GitHub issues page. Everyone wanted it!

Giving you what you want: MATLAB code execution from Visual Studio Code

So now we've got it. Ensure that you've installed at least version 1.2.0 of the MATLAB extension for Visual Studio Code alongside a copy of MATLAB that's at R2021a or above and you can run MATLAB programs right from Visual Studio Code.
RunVSCode1.gif
As you can see, just push the run button and your code will run in the MATLAB terminal. You can either continue editing or interact with the terminal further, whatever suits you best at the time.
Plots are rendered in a separate window, exactly as if you had run them from MATLAB Desktop.
VSCode_plot.gif
Speaking of MATLAB Desktop, you can access that at any time by typing desktop into the Visual Studio Code MATLAB terminal. It will be connected to the same workspace you are using from Visual Studio Code.

What we haven't given you: Debugging

The addition of code execution to the MATLAB extension for Visual Studio Code makes it much more useful and we hope you'll like it. The next obvious thing to request is debugging support and, if this important to you, please do add your voice to the discussion over at the issues section on GitHub along with everything else that you'd like.
The team are enjoying developing and delivering this functionality to you and we look forward to seeing you play with it, discussing it and letting us know what you want next!
]]>
https://blogs.mathworks.com/matlab/2024/03/05/matlab-extension-for-visual-studio-code-now-with-code-execution/feed/ 11
Outlier detection and Robust Regression in MATLAB with the FSDA Toolbox https://blogs.mathworks.com/matlab/2024/02/28/outlier-detection-and-robust-regression-in-matlab-with-the-fsda-toolbox/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2024/02/28/outlier-detection-and-robust-regression-in-matlab-with-the-fsda-toolbox/#comments Wed, 28 Feb 2024 18:50:13 +0000 https://blogs.mathworks.com/matlab/?p=2014

In 1857, Scottish physicist James David Forbes published a paper that discussed the relationship between atmospheric pressure and the boiling point of water. To get data, he traveled around the alps... read more >>

]]>
In 1857, Scottish physicist James David Forbes published a paper that discussed the relationship between atmospheric pressure and the boiling point of water. To get data, he traveled around the alps measuring the boiling point of water and atmospheric pressure at various altitudes. The idea behind this was that climbers could simply measure the boiling point of water to estimate atmospheric pressure and hence, altitude, instead of trying to use fragile and expensive barometers.
The data he collected are included in the open source MATLAB toolbox FSDA (Flexible Statistics Data Analysis). If you want to run the code examples in this blog post, you'll need to install it from the File Exchange or from MATLAB Add-ons.
Let's take a look at it along with the traditional least squares linear regression fit provided by fitlm from MathWorks' Statistics toolbox.
load('forbes.txt'); % Requires FSDA toolbox to be installed
y=forbes(:,2);
X=forbes(:,1);
X = (X - 32) * 5/9; % Convert to Celsius
plot(X,y,'o');
tradFit = fitlm(X,y); % Use Statistics toolbox to fit the data
hold on
plot(X,tradFit.predict(X)); % plot the fitted line
xlabel('Boiling point', 'Fontsize',16);
ylabel('100 x log(pressure)','Fontsize',16);
title("Forbes data with Linear Regression Fit")
legend({"Points","Least squares fit"},Location="Best");
hold off
Now, I want you to imagine that Forbes took a student with him whom he allowed to take one additional measurement. Imagine that this student was very careless when he took his measurement. Forbes, however, trusted him completely so we'll add this measurement to the data and see how it affects things.
%% Contaminated data (just 1 outlier)
yc = y;
yc(end+1) = 140;
Xc = X;
Xc(end+1) = 110;
 
plot(Xc,yc,'o');
hold on % Show the outlier in red
plot(Xc(end),yc(end),'o','MarkerFaceColor','auto','Color','r')
contaminatedTradFit = fitlm(Xc,yc);
plot(Xc,contaminatedTradFit.predict(Xc)); % plot the fitted line
xlabel('Boiling point', 'Fontsize',16);
ylabel('100 x log(pressure)','Fontsize',16);
legend({"Forbes' points","Outlier","Least squares fit"},Location="Best");
title("Contaminated Forbes data with Linear Regression Fit")
hold off
That one outlier has changed everything. Standard linear regression is not robust to this type of contamination.

A robust fit using FSDA toolbox

Enter the LXS function from FSDA toolbox which performs a more robust fit. By default, it uses an algorithm called Least Median of Squares regression which was first described in a 1984 paper by Peter Rousseeuw, a paper which has been cited over 5,000 times at the time of writing.
[outLXS]=LXS(yc,Xc);
Total estimated time to complete LMS: 0.20 seconds
Let's get the coefficients of the robust fit and compare it to the traditional least squares fit
b = outLXS.beta; % Fit coefficients
outliers = outLXS.outliers;
plot(Xc,yc,'o'); % Plot data
hold on % Show the outlier in red
plot(Xc(end),yc(end),'o','MarkerFaceColor','auto','Color','r')
plot(Xc,contaminatedTradFit.predict(Xc)); % Plot traditional least squares fit
plot(Xc,b(1)+b(2)*Xc, 'r','LineWidth' ,1); % Plot robust fit
legend({"Forbes' points","Outlier","Least squares fit","Robust Fit"},Location="Best");
hold off
Not only is the fit returned by LXS robust to our deliberate contamination, but it identifies the contamination as an outlier. It also identifies one of the points in the original data as a potential outlier. We can get the indices of the outliers from the structure returned from LXS
outLXS.outliers
ans = 1×2
12 18
Here's the plot once again, this time with the outliers clearly marked
plot(Xc,yc,'o'); %Plot data
hold on
plot(Xc,contaminatedTradFit.predict(Xc)); % Plot traditional least squares fit
plot(Xc,b(1)+b(2)*Xc, 'r','LineWidth' ,1); % Plot robust fit
plot(Xc(outliers),yc(outliers),'.', MarkerSize=20)
legend({"Points","Least squares fit","Robust Fit","Outliers"},Location="Best");
hold off
This is an example of robust linear regression which is just one of the areas of statistics and data analysis covered by the FSDA toolbox. For a more in-depth discussion and introduction to a suite of algorithms, refer to the documentation at Introduction to robust estimators in linear regression. Further analysis of this dataset is contained in Atkinson and Riani (2000). A comparison of different forms of robust regression estimators is given in the forthcoming book Atkinson et al. (2024).

About the FSDA toolbox

Developed at Università di Parma and the Joint Research Centre of the European Commission, FSDA toolbox is one of the most popular toolboxes in the MathWorks File Exchange and contains over 300 functions covering areas such as Robust Regression Analysis, Robust Multivariate Analysis and Robust Cluster Analysis.
References
Atkinson A.C. and Riani M. (2000). Robust Diagnostic Regression Analysis, Springer Verlag, New York.
Atkinson,A.C., Riani,M., Corbellini,A., Perrotta D., and Todorov,V. (2024), Applied Robust Statistics through the Monitoring Approach, Heidelberg: Springer Nature. https://github.com/UniprJRC/FigMonitoringBook
Rousseeuw P.J. (1984). Least Median of Squares Regression. Journal of the American Statistical Association, 79:388, 871-880
Forbes, J. (1857). Further experiments and remarks on the measurement of heights and boiling point of water. Transactions of the Royal Society of Edinburgh, 21, 235-243.
]]>
https://blogs.mathworks.com/matlab/2024/02/28/outlier-detection-and-robust-regression-in-matlab-with-the-fsda-toolbox/feed/ 2
Producing animated gifs from MATLAB Flipbook Mini Hack entries https://blogs.mathworks.com/matlab/2024/02/16/producing-animated-gifs-from-matlab-flipbook-mini-hack-entries/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2024/02/16/producing-animated-gifs-from-matlab-flipbook-mini-hack-entries/#respond Fri, 16 Feb 2024 11:43:11 +0000 https://blogs.mathworks.com/matlab/?p=1974

On Valentine's day, the MathWorks linkedIn channel posted this animated gifThe obvious question was duly asked "Where's the MATLAB code for this?" Turns out, I was uniquely placed to answer this as... read more >>

]]>
On Valentine's day, the MathWorks linkedIn channel posted this animated gif
dbc1ee461bf1d391edf5ce670d36353132993b22.gif
The obvious question was duly asked "Where's the MATLAB code for this?" Turns out, I was uniquely placed to answer this as it was one of my entries for the MATLAB Flipbook Mini Hack contest that occurred in late 2023. The full credit, however, has to go to Zhaoxu Liu / slandarer who wrote the original entry called Particle Heart 2. My version of the code was a lightly edited 'remix' that simply made the heart rotate more quickly, causing it to do a full revolution for every loop of the gif. This idea of remixing was a key feature of the contest which helped make it a lot of fun. Some of the best entries got remixed a lot!
I was at dinner at the time and wasn't able to post a full reply to the questions on LinkedIn so I just posted the link to the Minihack entry. Here's the code
function drawframe(f)
global theta1 theta2 theta3 heartHdl star1Hdl star2Hdl hpnts sx1 sx2 sy1 sy2
if f==1
% 所需匿名函数
col1Func=@(n) repmat([255,158,196]./255,[n,1])+repmat([-39,-81,-56]./255,[n,1]).*repmat(rand([n,1]),[1,3]);
col2Func=@(n) repmat([118,156,216]./255,[n,1])+repmat([137,99,39].*.1./255,[n,1]).*repmat(rand([n,1]),[1,3]);
szFunc=@(n) rand([n,1]).*15+8;
hold on
% 计算爱心点位置并绘制爱心
n=120;
x=linspace(-3,3,n);
y=linspace(-3,3,n);
z=linspace(-3,3,n);
[X,Y,Z]=ndgrid(x,y,z);
F=((-(X.^2).*(Z.^3)-(9/80).*(Y.^2).*(Z.^3))+((X.^2)+(9/4).*(Y.^2)+(Z.^2)-1).^3);
FV=isosurface(F,0);
hpnts=FV.vertices;
hpnts=(hpnts-repmat(mean(hpnts),[size(hpnts,1),1])).*repmat([.75,.7,.7],[size(hpnts,1),1]);
hpnts=hpnts+rand(size(hpnts)).*.7;
heartHdl=scatter3(hpnts(:,1),hpnts(:,2),hpnts(:,3),'.','SizeData',5,'CData',col1Func(size(hpnts,1)));
% 计算星星位置并绘制星星
sx1=rand([2e3,1]).*120-60;
sy1=rand([2e3,1]).*120-60;
sz1=ones(size(sx1)).*-30;
star1Hdl=scatter3(sx1,sy1,sz1,'.','SizeData',szFunc(length(sx1)),'CData',col2Func(size(sx1,1)),'LineWidth',1);
sx2=rand([2e3,1]).*120-60;
sy2=rand([2e3,1]).*120-60;
sz2=rand([2e3,1]).*120-20;
star2Hdl=scatter3(sx2,sy2,sz2,'.','SizeData',szFunc(length(sx2)),'CData',[1,1,1]);
% 坐标区域修饰
ax=gca;
ax.XLim=[-30,30];
ax.YLim=[-30,30];
ax.ZLim=[-40,30];
ax.Projection='perspective';
% ax.DataAspectRatio=[1,1,1];
view(-42,14);
ax.Color=[0,0,0];
ax.XColor='none';
ax.YColor='none';
ax.ZColor='none';
set(ax,'LooseInset',[0,0,0,0]);
set(ax,'Position',[-1/5,-1/5,1+2/5,1+2/5])
set(gcf,'Color',[0,0,0]);
% text(0,0,20,'slandarer','Color','w','HorizontalAlignment','center')
% 旋转爱心和星星
theta1=0;theta2=0;theta3=0;
else
theta1=theta1-pi/48;
theta2=theta2-0.003;
theta3=theta3-0.02;
set(heartHdl,'XData',hpnts(:,1).*cos(theta1)-hpnts(:,2).*sin(theta1),...
'YData',hpnts(:,1).*sin(theta1)+hpnts(:,2).*cos(theta1))
set(star1Hdl,'XData',sx1.*cos(theta2)-sy1.*sin(theta2),...
'YData',sx1.*sin(theta2)+sy1.*cos(theta2))
set(star2Hdl,'XData',sx2.*cos(theta3)-sy2.*sin(theta3),...
'YData',sx2.*sin(theta3)+sy2.*cos(theta3))
end
end
It's a function that draws a single frame. The software behind the website then called this function in a loop with the frame number going from 1 to 48 and inserted a delay such that the resulting animation would last 2 seconds before starting again. However, if you just had MATLAB then this function only produces a static frame.
drawframe(1);
How do you use this to produce the animated gif was the follow up question? I reached out to the team who built the Minihack website and asked them how they did it. How can I reproduce the exact same animated gifs using MATLAB and the drawframe() functions found on the website? They sent me the code below
function animateFrames()
animFilename = 'animation.gif'; % Output file name
firstFrame = true;
framesPerSecond = 24;
delayTime = 1/framesPerSecond;
% Create the gif
for frame = 1:48
drawframe(frame);
fig = gcf();
fig.Units = 'pixels';
fig.Position(3:4) = [300,300];
im = getframe(fig);
[A,map] = rgb2ind(im.cdata,256);
if firstFrame
firstFrame = false;
imwrite(A,map,animFilename, LoopCount=Inf, DelayTime=delayTime);
else
imwrite(A,map,animFilename, WriteMode="append", DelayTime=delayTime);
end
end
end
So, create two m-files, one called animateFrames.m and the other called drawframe.m, run animateFrames()
animateFrames()
and after a few seconds, you'll have animation.gif that contains what you are looking for. At this point, I suggest you take a look at the gallery and find something to animate yourself. These flowers, also from Zhaoxu Liu / slandarer, perhaps?
flowers.gif
or how about these lanterns by Tim?
lanterns.gif
Even though the competition only lasted 4 weeks, there are over 600 entries for you to play with. Here is a static view of a few more.
The competition had a few restrictions. For example, all animations had exactly 48 frames that would be turned into a 2 second gif. There were also computational time limits and character limits on the drawframe() function. Now you have the code, you can go crazy and produce animated gifs of whatever complexity you like!
]]>
https://blogs.mathworks.com/matlab/2024/02/16/producing-animated-gifs-from-matlab-flipbook-mini-hack-entries/feed/ 0
Life in the fast lane: Making MATLAB even faster on Apple Silicon with Apple Accelerate https://blogs.mathworks.com/matlab/2023/12/13/life-in-the-fast-lane-making-matlab-even-faster-on-apple-silicon-with-apple-accelerate/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2023/12/13/life-in-the-fast-lane-making-matlab-even-faster-on-apple-silicon-with-apple-accelerate/#comments Wed, 13 Dec 2023 16:39:35 +0000 https://blogs.mathworks.com/matlab/?p=1918

Up to 3.7x faster Matrix-Matrix multiplication, 2x faster LU factorisation, 1.7x faster Cholesky decomposition and the potential for many more speed-ups across different areas of linear algebra in... read more >>

]]>
Up to 3.7x faster Matrix-Matrix multiplication, 2x faster LU factorisation, 1.7x faster Cholesky decomposition and the potential for many more speed-ups across different areas of linear algebra in MATLAB. What if I told you that you could have all of this right now on your Apple Silicon machine? Interested? Read on.

It’s all about the BLAS

BLAS (Basic Linear Algebra Subprograms) is probably the most important software library you’ve never heard of. It takes care of things like multiplying two matrices together or multiplying vectors by a scalar. Operations so simple you may not even expect that something like MATLAB would use a library to perform them.
The reason we use the BLAS library is speed! It’s relatively easy to implement these operations if you don’t care about speed but it is certainly not easy to implement them in such a way that they make maximum use of modern hardware. The difference in speed between a naively written matrix-matrix multiplication routine and a highly optimised one can be thousands of times!
Linear algebra is at the heart of much of modern technical computing so chip manufacturers, such as Intel, AMD and Apple want to ensure that when you do linear algebra on their hardware its as fast as it can possibly be. As such, they all have their own implementations of the BLAS library, exquisitely tuned to perfectly show-off the characteristics of their CPUs.
None of this is news to regular readers of The MATLAB blog of course. I have a minor obsession with the BLAS library, and its big sibling, the LAPACK library; both of which have been covered multiple times in articles such as Trying out AMD’s AOCL and early posts exploring the beta versions of MATLAB on Apple Silicon (here and here). In these articles you can see how MATLAB’s support of BLAS has evolved recently, particularly for Apple Silicon.

You can now use Apple Accelerate as the BLAS for MATLAB on Apple Silicon

MATLAB R2023b, the first general release of MATLAB for Apple Silicon, uses OpenBLAS and, as I reported back in June, performance is pretty good. From the beginning, however, users asked us ‘Why aren’t you using Apple Accelerate; which contains Apples implementation of BLAS?. We had our reasons (see the relevant section in this article) but the situation has since changed and MathWorks recently released an update allowing you to make use of it. It was done so quietly, however, that you almost certainly missed it! So let’s fix that.
You need to be running at least R2023b Update 4. Anything earlier than R2023b isn’t even running natively on Apple Silicon so start there. Once you have R2023b installed, check which update you are running with the version command
version
ans =
'23.2.0.2428915 (R2023b) Update 4'
If you are running on an earlier update, fix that by clicking on Help->Check for Updates
Once that’s done, you can switch from using OpenBLAS to Apple Accelerate by following the instructions at How can I use the BLAS implementations included in Apple Accelerate Framework with MATLAB R2023b Update 4

Performance discussion

At the risk of stating the obvious, if what you’re interested in isn’t related to linear algebra then you are not going to see any performance differences with this change. This is MATLAB though, there’s a lot of linear algebra.
If you want to repeat this benchmarks for yourself, the script I used is on GitHub

Matrix-Matrix multiplication

Matrix-Matrix multiplication is where I saw the most speed-up. This is a BLAS operation that has clearly seen a lot of work by Apple.
Matrix Size
OpenBLAS time (s)
Apple Accelerate time (s)
x Speed-up
1,000
0.0172
0.0046
3.74
5,000
1.1583
0.4171
2.78
10,000
6.8977
3.3186
2.08

LU factorization

LU factorization is a LAPACK operation and we haven’t changed LAPACK libraries here. However, LAPACK often makes use of BLAS so if you accelerate BLAS, you get can faster LAPACK for free.
Matrix Size
OpenBLAS time (s)
Apple Accelerate time (s)
x Speed-up
1,000
0.0124
0.0115
1.08
5,000
0.4345
0.2556
1.7
10,000
3.5928
1.6821
2.14
I also tested Cholesky decomposition and got speed-ups ranging from 1.28x to 1.77x

Eigenvalues

To test eigenvalue computation I used the single output version of eig. e = eig(A). As with other tests, I tried this on a range of random matrices. Things started off well, with nice speed-ups for small matrices but larger matrices saw quite substantial slowdowns on my M2 machine when compared to the OpenBLAS original.
I discussed this with a colleague who has an M1 and he never saw any slowdowns. For his machine, it was always faster to use Apple Accelerate for eig. As such, we suspect that this is an M2-speific issue and will investigate more closely to find out what’s going on here
Matrix Size
OpenBLAS time (s)
Apple Accelerate time (s)
x Speed-up
1,000
0.4245
0.2407
1.76
5,000
22.8654
24.74
0.92
10,000
145.4076
201.57
0.72

Details of test machine

I used an M2-based Macbook Pro that was plugged-in. The output from the cpuinfo command (available on File Exchange) is as follows.
cpuinfo
ans =
structwith fields:
CPUName: 'Apple M2 Pro'
Clock: 'N/A'
Cache: 65536
TotalMemory: 1.7180e+10
NumCPUs: 1
TotalCores: 10
OSType: 'macOS'
OSVersion: '13.3.1'
Other than the issue with eig, which I hope will get fixed at some point, switching to Apple Accelerate seems to be extremely beneficial. If you have an Apple Silicon Mac, give it a try and tell me what you come up with.
]]>
https://blogs.mathworks.com/matlab/2023/12/13/life-in-the-fast-lane-making-matlab-even-faster-on-apple-silicon-with-apple-accelerate/feed/ 10
Creating natural textures with power-law noise: clouds, terrains, and more https://blogs.mathworks.com/matlab/2023/11/29/creating-natural-textures-with-power-law-noise-clouds-terrains-and-more/?s_tid=feedtopost https://blogs.mathworks.com/matlab/2023/11/29/creating-natural-textures-with-power-law-noise-clouds-terrains-and-more/#comments Wed, 29 Nov 2023 15:20:07 +0000 https://blogs.mathworks.com/matlab/?p=1882 Power law noise

Today's guest writer is Adam Danz whose name you might recognize from the MATLAB Central community. Adam is a developer in the MATLAB Graphics and Charting team and the host of the Graphics and... read more >>

]]>
Today's guest writer is Adam Danz whose name you might recognize from the MATLAB Central community. Adam is a developer in the MATLAB Graphics and Charting team and the host of the Graphics and App Building blog. In this article he’ll share insights to creating structured noise and applying it to MATLAB graphics.
Noise is ubiquitous in nature. MATLAB has several tools to remove noise from data and images but in this article I'll review a method of generating noise that is currently being used by participants in the Flipbook Mini Hack contest to generate natural looking clouds, terrains, and textures including these examples below.
CombinedAnimation.gif
From left to right, Rolling fog, Blowin'in the wind, Morning ascent, landScape, from the 2023 MATLAB Flipbook Mini Hack contest.
This type of structured noise has several classifications including 1/f noise, coherent noise, fractal noise, colored noise, pink noise, fBm, and Brownian noise, each with their own definitions. For this article, I will refer to this type of noise as power-law noise, a term that describes the mathematical relationship between a signal's intensity and spatial or temporal frequency. Power-law noise can generate macro-scale variation such as hills and valleys while featuring micro-scale variation that adds texture. This type of noise is often found in natural phenomena from earthquakes to financial markets and it is quite easy to generate it in MATLAB.

Five Simple steps to generate 2D power-law noise in MATLAB

Creating power-law noise is surprisingly simple and requires only three parameters and a few lines of code in MATLAB. I'll walk you through the five steps depicted in the summary image below.
Summary of the five steps to creating 2D power-law noise
1. Random noise. Create a 2D matrix of random values. Some definitions of power-law noise require starting with Gaussian white noise which can be generated using randn() but uniformly distributed random numbers using rand() also produce fine results. The first two parameters are the size of the matrix and the random number generator seed (and generator type). Power-law noise is highly influenced by the initial values, so each seed and matrix size produces different results. The size parameter k must be even.
rng(0,'twister') % Parameter 1. random number generator seed
k = 100; % Parameter 2. size of the square matrix, must be an even scalar
m = randn(k); % k-by-k matrix
2. Fourier transform. Transform the 2D array from spatial to frequency domains using the 2D fast Fourier transform function fft2(). This returns a matrix of frequency coefficients in a pyramid arrangement where coefficients for low frequencies are on the perimeter of the matrix and coefficients for high frequencies are in the middle of the matrix. Step four requires an inversion of this arrangement where coefficients for low frequencies are in the middle and coefficients for high frequencies are on the perimeter of the matrix. Use fftshift() to invert the arrangement.
mf = fftshift(fft2(m));
How do I interpret the 2D frequency coefficients? This paragraph is for curious readers and for my personal challenge to concisely explain it. 2D Fourier transforms decompose an image into a set of planar sinusoidal waves at various frequencies and orientations. The frequency coefficients are complex numbers that provide a set of instructions for how to combine or superimpose the planar waves to recreate the image. Consider the value at mf(u,v). The real and imaginary components of mf(u,v) correspond to the amplitude and phase of the 2D sinusoidal plane that is defined by the frequencies assigned to row u and column v. Since MATLAB's image functions do not support complex values, the magnitudes of the frequency coefficients are shown in the second and fourth panels of the summary figure above. The magnitude of a complex numbers is computed using abs().
3. Create the filter. An important characteristic of power-law noise is the dominance of lower frequencies while still allowing higher frequencies to contribute to the variation. This creates macro patterns in the image such as 2D hills or cloud bellows while maintaining micro variations that add texture. In terms of frequency, sections of an image with fine details have high spatial frequency while smoother sections have low spatial frequency. The goal of the filter is to accentuate lower spatial frequencies in the image to create the macro patterns. Recall from step two that the shifted Fourier transform matrix mf is organized such that coefficients for lower frequency waves are in the center. The 2D filter, therefore, should have an inverse relationship to frequency ($ 1/f$ where $f = \sqrt{f_u^2 + f_v^2}$ for 2D signals) such that greater weight is placed on lower frequencies in the center and less weight on higher frequencies near the perimeter. The amount of emphasis placed on lower frequencies is controlled by the third parameter in this process, the exponent in $1/f^α$ which is typically within the range of 0<α<=3. Larger values of α result in greater influence from low spatial frequencies which result in smoother noise transitions with less detailed texture as shown in the image below. The value of α is used to define several classifications of noise such as white noise (α=0) pink noise (α=1) and Brownian noise (α=2).
Exponent values are shown at the top of each image. Larger exponents result in smoother, lower frequency noise
Every implementation of power-law noise I've seen creates the filter differently but follows the same pattern of weight distribution. This filter is quite robust at returning real number values in step five, though it requires that k is even.
a = 2; % Parameter 3. 1/fᵅ exponent
d = ((1:k)-(k/2)-1).^2;
dd = sqrt(d(:) + d(:)');
filt = dd .^ -a; % frequency filter
filt(isinf(filt))=1; % replace +/-inf at DC or zero-frequency component
4. Filtered frequencies. Multiply the shifted Fourier transform matrix by the filter. Looking at the fourth panel of the 5-step summary image, it is difficult to imagine how this matrix contributes to the final image. We're used to thinking about matrices in the spatial domain where the value at coordinate (u,v) determines the intensity of the pixel at (u,v) of the image. Instead, this panel shows that lower spatial frequencies toward the middle of the matrix have higher coefficients which means that the lower spatial frequencies will have a greater effect on the final results.
ff = mf .* filt;
5. Inverse Fourier transform. We started with a matrix of random values. A Fourier transform reconstructed the random matrix in the frequency domain and the filter accentuated lower frequencies. The final step is to convert the filtered frequencies back to the spatial domain. Shift the frequencies back to their original pyramidal arrangement with coefficients for lower frequencies on the perimeter and coefficients for higher frequencies at the center using ifftshift(). Then apply a 2D inverse Fourier transform using iff2().
b = ifft2(ifftshift(ff));
The result (b) is a matrix of real numbers forming structured noise the same size as the original image. View the results shown in panel five of the 5-step summary image using imagesc(b) and apply a color bar to investigate the image's intensity values.
It is common to scale the noise values to a range of [-1,1] which makes it easier to apply additional customized scaling. To scale the results, use rescale().
bScaled = rescale(b,-1,1);
This pattern of applying a filter to a Fourier transformed image and then inversing the process back to an image is common in image processing and provides a nice model for learning about image filters.

Visualizing the power-law noise matrix.

The array of power-law noise values is commonly applied to color data, alpha value (transparency), or as height values in a 3D surface. The figure below shows the same noise matrix applied to four different visualizations.
Applications of power-law noise in MATLAB graphics
To produces these panels, create the power-law noise matrix (bScaled) following the five steps above but replace the size parameter with k=500. Then continue with the code block below. The second and third panels use a custom colormap terrainmap which is attached to this article (download). Note that the Mapping Toolbox has an improved terrain colormap, demcmap.
% Panel 1
figure()
imagesc(bScaled)
set(gca,'YDir','normal')
colormap(gca,sky(256))
colorbar
%
% Panel 2
figure()
imagesc(bScaled)
set(gca,'YDir','normal')
colormap(gca,terrainmap()) % custom colormap
colorbar
%
% Panel 3
figure()
x = linspace(-4, 4, width(bScaled));
y = linspace(-4, 4, height(bScaled));
surf(x,y,bScaled,EdgeColor='none')
colormap(gca,terrainmap()) % custom colormap
axis equal
%
% Panel 4
figure()
s = surf(zeros(k,k),FaceColor=[.28 .24 .54],EdgeColor='none',...
    AlphaData=bScaled,FaceAlpha='flat');
view(2)

Use light to reveal texture

The texture provided by power-law noise can go unnoticed when there isn't sufficient color variation in the graphics object. The use of lighting in MATLAB reveals the texture by adding detailed shading to objects. The image below shows the same mountain as in the surface plot above. Water level was added as a planar surface with color defined by a second noise matrix. Light position was selected manually using the cameratoolbar tool and its values were extracted and placed in the code below for reproducibility. Camera properties were set to zoom in to the mountain.
The use of lighting accentuates texture
rng(0,'twister')
k = 500;
mountainHeights = makenoise(k, 2); % mountain noise
figure()
x = linspace(-4, 4, k);
y = linspace(-4, 4, k);
mountainHeights = mountainHeights + 0.3; % raise the mountain
surf(x,y,mountainHeights,EdgeColor='none')
colormap(flipud(copper))
hold on
seaShading = makenoise(k, 1.5); % water color noise
nColors = 500; % number of colors to use for the water
seaColors = parula(nColors*3/2);
seaColors(nColors+1:end,:) = []; % Capture the blue-green end of parula
% convert seaShading (-1:1) to colormap indices (1:500)
seaShadingIdx = round((seaShading+1)*(nColors/2-.5)+1);
% Create mxnx3 matrix of CData
cdata = reshape(seaColors(seaShadingIdx,:),nColors,nColors,3);
S=surf(x*3, y*3, 0*seaShading,CData=cdata,EdgeColor='none');
axis equal off
clim([-0.5, 1.5])
light(Position = [-.8 .1 .75]) % add lighting
material dull
camva(10) % camera viewing angle
campos([-14 7 4]) % camera position
camtarget([.4 .8 .4]) % camera target
function noise = makenoise(k, a)
% Create the noise matrix
% k is the matrix size (scalar)
% a is the positive exponent (scalar)
% noise are noise values with range [-1,1]
m = randn(k);
mf = fftshift(fft2(m));
d = ((1:k)-(k/2)-1).^2;
dd = sqrt(d(:) + d(:)');
filt = dd .^ -a;
filt(isinf(filt))=1;
ff = mf .* filt;
b = ifft2(ifftshift(ff));
noise = rescale(b,-1,1);
end

Seamless tiling

Seamless tiling is another neat property of this style of noise. The edges of the matrix wrap seamlessly in a tiled arrangement as shown in the image below using the parula colormap.
figure()
tiledlayout(3, 3, TileSpacing='none', Padding='tight')
for i = 1:9
    nexttile()
    imagesc(bScaled)
    set(gca,'XTickLabel',[],'ytickLabel',[])
end
3x3 tile of the same noise matrix shows circular wrapping around its edges

Working in reverse: classifying noise by estimating the exponent

A 1D vector of power-law noise has the same macro-level hills and micro-level texture as the 2D case. This section below plots 1000 values of pink noise defined by an exponent of α=1.
rng default % Parameter 1. random number generator seed
k = 1000; % Parameter 2. size of the vector, must be even.
m = randn(k,1);
mf = fftshift(fft(m));
a = 1; % Parameter 3. 1/fᵅ exponent
d = sqrt(((1:k).'-(k/2)-1).^2);
filt = d .^ (-a/2); % frequency filter for 1D
filt(isinf(filt))=1;
ff = mf .* filt; % apply the filter
v = ifft(ifftshift(ff)); % power-law noise vector
vScaled = rescale(v,-1,1); % scaled power-law noise [-1,1]
figure()
plot(vScaled)
grid on
title('1D power-law noise')
subtitle('α = 1')
Suppose you start with the noise vector and you want to classify the color of the noise by its exponent value. The exponent can be approximated as the slope of the $1/f^α$ power spectrum in log-log coordinates. Fortunately, MATLAB's Signal Processing toolbox makes this task easy with the pspectrum function.
% Compute the power and frequency of the power-law noise vector
% Requires signal processing toolbox
[power, freq] = pspectrum(vScaled);
%
% Apply the log transform
freqLog = log(freq);
powerLog = log(power);
%
% Remove inf values
infrm = isinf(freqLog) | isinf(powerLog);
freqLog(infrm) = [];
powerLog(infrm) = [];
%
% Fit the relationship between log-power and log-frequency
% Requires the Curve Fitting toolbox
F = fit(freqLog,powerLog,'poly1');
%
% Plot the power density
figure()
plot(freqLog,powerLog)
%
% Add the linear fit
hold on
h = plot(F);
h.AffectAutoLimits = 'off';
grid on
axis tight
%
% Print the slope in the title
coeffs = coeffvalues(F);
xlabel('log frequency')
ylabel('log power spectrum')
title(sprintf('Slope = %.2f', coeffs(1)))
The slope of the linear fit is -1.05 which would correctly classify this signal as pink noise (α=1).

Summary points

  • Power-law noise such as pink and Brownian noise can be used to create 2D and 3D objects with naturally occurring noise variation.
  • Creating power-law noise matrices in MATLAB is quite simple and involves only five steps.
  • The basic principle of power-law noise creation involves filtering a Fourier transformed 2D matrix of random values by weighting the frequency coefficients by the inverse frequency and then applying an inverse Fourier transform.
  • A 2D power-law noise matrix can be applied to images and surfaces as color data, alpha data (transparency), height data, or as intensity values.
  • The use of lighting in MATLAB brings out the texture in surfaces.
  • Several MATLAB users have used this technique to create impressive images in the Flipbook Mini Hack contest which runs until December 3, 2023.

Additional resources

  1. colored_noise is a wonderful tool on the File Exchange by Abhranil Das that creates power-law noise in any number of dimensions (also on git).
  2. Abhranil Das' 2022 thesis on Camouflage Detection & Signal Discrimination (Ch. 5) contains a terrific interpretation of 2D Fourier transform coefficients.
  3. Paul Bourke's 1998 articles on generating noise and applications in graphics is a source of inspiration and creativity.

Did you enjoy this article? Help us create content for future articles by leaving your thoughts below!

  ]]>
https://blogs.mathworks.com/matlab/2023/11/29/creating-natural-textures-with-power-law-noise-clouds-terrains-and-more/feed/ 8