One challenge of teaching Fourier transform concepts is that each concept can be (and is) interpreted and explained in many different ways. It isn't always obvious how the different explanations for the same concepts are connected.
For example, in my last Fourier transform post I talked about aliasing. I said that if you sample a continuous-time cosine at a sampling frequency , then you can't distinguish between a cosine with frequency and a cosine with frequency .
In response, Dave S. wanted to know how this related to what he learned about aliasing: that aliasing is a "problem that occurs when the sampling rate of your discrete signal is not at least twice the [...] highest existing frequency [in the continuous-time signal]."
I know I promised to introduce the discrete Fourier transform next, but I'd like to change my mind and try to answer Dave's question instead. There are two key pieces of the question to address: What is the nature of the "problem," and what is the significance of "twice the highest frequency"?
I thought about drawing some new frequency-domain diagrams showing overlapping triangles like you'd see in Oppenheim and Schafer, but then I thought it might be better to just continue the sampled cosine example from last time.
Let's start with a continuous-time cosine signal at 60 Hz.
f = 60; % Hz tmin = -0.05; tmax = 0.05; t = linspace(tmin, tmax, 400); x_c = cos(2*pi*f * t); plot(t,x_c) xlabel('t (seconds)')
Let's sample with a sampling frequency of 800 Hz.
T = 1/800; nmin = ceil(tmin / T); nmax = floor(tmax / T); n = nmin:nmax; x1 = cos(2*pi*f * n*T); hold on plot(n*T,x1,'.') hold off
The sampling frequency of 800 Hz is well above 120 Hz, which is twice the frequency of the cosine. And you can see that the samples are clearly capturing the oscillation of the continuous-time cosine.
Let's try a lower sampling frequency.
T = 1/400; nmin = ceil(tmin / T); nmax = floor(tmax / T); n = nmin:nmax; x1 = cos(2*pi*f * n*T); plot(t, x_c) hold on plot(n*T, x1, '.') hold off
The samples above are still adequately capturing the shape of the cosine. Now let's drop the sampling frequency down to exactly 120 Hz, twice the frequency of the 60 Hz cosine. (And I'll switch to using circle markers to make the samples easier to see.)
T = 1/120; nmin = ceil(tmin / T); nmax = floor(tmax / T); n = nmin:nmax; x1 = cos(2*pi*f * n*T); plot(t, x_c) hold on plot(n*T, x1, 'o') hold off
See how the samples jump back and forth between 1 and -1? And how they capture only the extremes of each period of the cosine oscillation? This is the significance of "twice the highest frequency of the signal" value for sampling frequency. If you'll allow a "hand-wavy" explanation here, I'll say that this sampling frequency of 120 Hz is just enough to capture the cosine oscillation.
But aliasing is worse that "just" losing information. When we drop the sampling frequency too low, the samples start to look increasingly like they came from a different, lower-frequency signal.
Let's try 70 Hz.
T = 1/70; nmin = ceil(tmin / T); nmax = floor(tmax / T); n = nmin:nmax; x1 = cos(2*pi*f * n*T); plot(t, x_c) hold on plot(n*T, x1, 'o') hold off
The samples above look like they actually could have come from a 10 Hz cosine signal, instead of a 60 Hz cosine signal. Take a look:
T = 1/70; x_c = cos(2*pi*10 * t); nmin = ceil(tmin / T); nmax = floor(tmax / T); n = nmin:nmax; x1 = cos(2*pi*f * n*T); plot(t, x_c) hold on plot(n*T, x1, 'o') hold off
That's the heart of the "problem" of aliasing. Because the sampling frequency was too low, a high-frequency cosine looked like a low-frequency cosine after we sampled it.
Later on in this series I plan to come back again to the concept of aliasing and show some examples of how it looks in an image.
OK, now I'll start working on the upcoming discrete Fourier transform (DFT) post.
Get the MATLAB code
Published with MATLAB® 7.9
22 CommentsOldest to Newest
And, when a signal is contaminated with aliasing, you generally can’t fix it with post processing.
OysterEngineer makes a very good point. Thanks.
Thank you Steve, great post – you nailed it!
I would like to ask you a question about analysing images in raw format. I am just wondering where I should post my question. Could you please let me know?
Tien—This blog isn’t a general help / question forum. I accept only comments that are relevant to the blog posts. You might try posting in the MATLAB newsgroup.
All this DFT stuff has gotten me thinking about discrete convolution. Which Matlab functions implement convolution via FFTs, and which ones use double sums? Thanks.
Jason—I believe there are no MATLAB functions that do FFT-based filtering. There is a Signal Processing Toolbox function called fftfilt that implements overlap-and-add. See my blog post about some of the tradeoffs involved in FFT-based convolution.
Another interesting extension of your example above can be seen when we work with a sine signal. If instead, one creates a sine signal using the same example parameters:
x_s = sin(2*pi*f * t);
and then sample it at exactly 120 Hz, twice the frequency of the 60 Hz sine, we can see that we now have a problem:
T = 1/120; nmin = ceil(tmin / T); nmax = floor(tmax / T); n = nmin:nmax; x1s = sin(2*pi*f * n*T); plot(t, x_s) hold on plot(n*T, x1s, 'o') hold off
In this case one can’t distinguish the sampled signal from zero!
Thanks for the great example.
Karl—Thanks for the example; I like it!
Steve, Could you please elaborate on the point made by Karl. This problem, i.e. zero samples at Fs exactly twice the max. frequency, has haunted me for years. Nyquist th. seems to break at this point. Thanks
Bilal—The Nyquist Sampling Theorem has a strict inequality in it that is critical here. The Nyquist frequency for Karl’s example signal is 60 Hz. The sampling theorem says that the original continuous-time signal is uniquely determined by its samples if the sampling frequency is greater than twice the Nyquist frequency. That condition is not satisfied in Karl’s example, because 120 is not greater than 2*60.
Does that help?
Steve, Really thank you for the response. I truly wish that was the case but nowhere (on the web and classic text books) have I seen strict inequality. Everywhere it says “sampling freq. should be at least twice the max. feq. in the signal”. The “triangle” figure that we often see in text books also does not elaborate.
Bilal—Here’s what the Wikipedia article on the Nyquist-Shannon Sampling Theorem has to say about this:
In essence, the theorem shows that a bandlimited analog signal that has been sampled can be perfectly reconstructed from an infinite sequence of samples if the sampling rate exceeds 2B samples per second, where B is the highest frequency in the original signal. If a signal contains a component at exactly B hertz, then samples spaced at exactly 1/(2B) seconds do not completely determine the signal, Shannon’s statement notwithstanding. This sufficient condition can be weakened, as discussed at Sampling of non-baseband signals below.
More recent statements of the theorem are sometimes careful to exclude the equality condition; that is, the condition is if x(t) contains no frequencies higher than or equal to B; this condition is equivalent to Shannon’s except when the function includes a steady sinusoidal component at exactly frequency B.
The “triangle figures” you mention satisfy the formulation above, because the triangle shape is zero at the Nyquist frequency.
In Discrete-Time Signal Processing (Prentice-Hall 1989 edition, p. 86), Oppenheim and Schafer are careful about this point, but they put the strict inequality in a different place than suggested in the Wikipedia article.
Their formulation allows the signal to contain nonzero energy all the way up to and including the Nyquist frequency, but then they require the sampling frequency to be strictly greater than twice the Nyquist frequency. Either formulation works.
Eq 3.14b is the one I had been looking for. Thanks Steve, I am really grateful to you for this.
Regarding the next Fourier transform topic, I’d like to see more about some of the things that might surprise you when you first try to use and FFT, namely the zero frequency at left, or some practical things like the nuances of windowing.
*** Test of the sampling theorem/theory with signals with “zero bandwidth” ***
There are three test signals with “zero band width” by definition:
* the constant signal,
* the sinusoidal signal and
* the co-sinusoidal signals.
The equations are given below:
Aconst = constant;
A = Amsin(ωt + φ) + 0;
A = Amcos(ωt + φ) + 0;
According to the “classical sampling theorems” these signals should be reconstructed with “zero samples” because they have “zero bandwidth”. It is obvious that when the model applied in the real signals sampling and reconstruction systems the so called “take and memorize the sample until the next samples is taken” is applied:
* in the first case only one sample is needed to reconstruct the signal
* in the second and third cases four samples are needed because there are four parameters to reconstruct and also four samples are guarantying 3dB bandwidth of the sampling and direct reconstruction system.
Obviously refereeing only to the “band width” is wrong. The model should be taken into consideration. Also Nyquist never formulated any “sampling theorem”, etc.……..
Petre—I’m not teaching a formal DSP class here. I’m simply trying to address common points of confusion related to Fourier transforms for those who have not had formal training in DSP. Your thought experiments with “zero-bandwidth” signals simply demonstrate that the Nyquist Sampling Theorem gives a sufficient condition and not a necessary one. As for my use of “Nyquist,” I am following the terminology used in Oppenheim and Schafer, Discrete-Time Signal Processing (1989), which uses the term “Nyquist Sampling Theorem” and gives the following reference: Nyquist, H., “Certain Topics in Telegraph Transmission Theory,” AIEE Transactions, pp. 617–644, 1928.
Petre—Thanks very much for posting the interesting links.
Thanks for posting of my not very well written message and more thanks for your reply.
I like your teaching. Also I think that what you are doing is very useful.
Often the informal teaching is better than the formal.
You are right. There are some pints of confusions in that field.
In order to understand at least partially my position in sampling theorem/theory, please see the papers below.
I apologize for not very clear language and some omissions. May be the papers will be of some use for your and your auditory.
I look forward for more materials from you.
*** Some papers***
 A New Approach to Sampling Sinusoidal and Cosinusoidal – ET 4 CO 198.pmd
 Sampling of the Simplest Signals – 9 CP PE 8.pmd
. “Reevaluation and replacement of basic terms in the sampling theory”,
. New Approach to Comparison of Classical Papers of H. Hyquist, V. Kotelnikov and C. Shannon and Theirs Impact on the Terminology.
 A note about the definitions of “sine/cosine wave”, “sinusoidal/co-sinusoidal signal” and the “simplest band limited signals”.
I started reading your paper , and do not agree with any of your points on the first page. I stopped reading after that. I’m not trying to start a flame war on Steve’s blog, but I just have to call out nonsense when I see it.
-> “Classical sampling theorem is an oversimplified sampling theorem generally stating that two samples are enough to reconstruct “exactly” band limited signals…” — sampling theory says you need an infinite number of samples, not two.
-> “Nyquist frequency […] means several different things.” — It doesn’t. Just because some people misinterpret a phrase doesn’t make it useless.
“the misleading interpretation is one of the proof that the classical SST is not accurate.” — Some people don’t understand the theory, the theory must be wrong!
-> “Delta function – unreal function used to
construct the Dirac comb. No practical value in the
sampling theory […]” — It’s a mathematical concept, central to sampling theory.
-> “Aliasing – a misleading term”, and “The word “alias” has a criminal meaning” — I think you misunderstand the word. It makes perfect sense to use this word for this purpose. If you want to complain about English words used “wrong”, complain about the use of “save” to mean “write a file to disk”. :)
-> “Decimation – incorrect term (nothing to do with the
number 10) […] giving […] cruel historical background of the term)” — You are trying too hard to find things to complain about.
When a theory is as useful as sampling theory, there really is no point in trying to ridicule it. You should put your efforts into improving it, and extending it, rather than trying to prove it wrong.
PS: sampling theory refers to the highest frequency in the signal, not the bandwidth. When Shannon speaks of bandwidth, he refers to the amount of information communicated over a channel. But that’s a whole different story.
Hi Cris Luengo,
Thanks for reading the first page of the most difficult to understand of the papers.
It is really hard to be read and I get already a negative feedback for it.
I. “…flame war on Steve’s blog…”
The war is a usual activity for the human beings. Or not? So – no problem? :-)
II.”… infinite number of samples, not two…”
An infinite number of samples to reconstruct the simplest signals as DC, SS, CS, SS+DC, CS+DC, etc. seems really strange for me.
But may be it is a different perception for the goal of the theoretical concepts.
I my opinion if the theory cannot be validated – it is not a theory.
It is an idea or a supposition or something like that, but not a science and not a theory.
So if “infinite number of samples” is the goal of the “sampling theorem” and if could validate that concept in practice – you are right. :)
III. If you try “Nyquist frequency” with Google and take into consideration the only the first several definition may be you will find the following.
You will agree that these are not the same definitions for the same term.
If you continue the search you will find more interpretations.
And the reason is that Nyquist did not give definition of any particular frequency related to signal sampling, conversion and reconstruction.
“….The Nyquist frequency, named after the Swedish-American engineer Harry Nyquist or the Nyquist–Shannon sampling theorem, is half the sampling frequency of a discrete signal processing system.  It is sometimes known as the folding frequency of a sampling system.…”
“…..In order to recover all Fourier components of a periodic waveform, it is necessary to use a sampling rate nu at least twice the highest waveform frequency. The Nyquist frequency, also called the Nyquist limit, is the highest frequency that can be coded at a given sampling rate in order to be able to fully reconstruct the signal, i.e.,
“…..The Nyquist frequency is the bandwidth of a sampled signal, and is equal to half the sampling frequency of that signal. If the sampled signal should represent a continuous spectral range starting at 0 Hz (which is the most common case for speech recordings), the Nyquist frequency is the highest frequency that the sampled signal can unambiguously represent………..”
“……Nyquist frequency – (telecommunication) twice the maximum frequency occurring in the transmitted signal……….”
The shortest detectable frequency in a time series. All time series are, in practice, recorded at discrete time points (for example, one reading per second). Periodic variations that happen more rapidly than the shortest time interval cannot be detected. With time intervals of size t the Nyquist frequency is t−1 cycles per unit time………”
IV . “….It’s a mathematical concept, central to sampling theory….”
There is no need of that concept in the sampling theory.
The sampling, conversion and reconstruction system is build from ADC, Memory and DAC . To explain it there is no need of Dirac, Dirichet, etc.
The analog signal is converted into “staircase function” not in “comb” and there is no need of any “delta functions” or similar.
V. “Aliasing”, “decimation” – In my opinion there must be more clear and easy to remember terms for both events.
VI. “…. there really is no point in trying to ridicule it. …”
I am unhappy to find so many inaccuracy and misleading concepts in so modern technical field.
I do not see any relation between that terminology and the reality.
Also I do not see any practical relation between Nyquist, Shannon and even Kotelnikov and the systems with ADCD, DAC, Memory, SH etc.
I hope that you will find the answer clear, serious and good enough to be accepted.
Thank you for your time and effords.
I look forward to hear more critics from you and/or from anybody interested in the subject.