This is a very simplified analysis of digital filters, covering mostly the application of the filters, what they do, and how to select one. Members interested in the theory are advised to refer to more advanced books.
Several members have asked about the use of digital filters. With the spreading use of computers to gather radio astronomical data, the interest on digital filters is understandable. This is a complex topic that cannot be covered in a single article but, for the purpose of the radio astronomer, a complete theory is not necessary. Interested members are directed to the many good books on the subject. Consider that the author teaches a course on digital filters and that a quarter is required to cover only the most elementary aspects of the theory.
Today we have two worlds, the analog world in which we live and the parallel universe, the digital world in which computers live. Both have advantages and disadvantages and, since computers are here to stay, we have to understand both.
Signals of any kind have variations. If a signal does not have variations it has very little interest. The rate at which the variations in the signal occur is what we call the frequency. A signal with a single frequency is also of little interest. The signals in radio astronomy are noise, consequently, they have many frequencies.
When we detect a signal, say at 600 Mhz, you have many different frequencies. You amplify this signal and finally you send it through a square law detector, square law to extract the energy content of the signal. At that moment you are interested in the low frequencies that carry the information you desire and the high frequencies are damaging your results. You normally put a little capacitor after the detector, to filter out the high frequencies.
Later, you might put some resistor and capacitor to filter even more your signals before accumulating them in the integrator. In all these steps you are using analog filtering, analog because they are formed by resistors and capacitors and they respond to continuous signals.
Now, imagine that after your detector, you amplify your signal some what and then you use an analog to digital converter so you can put your data into a computer. The advantage is obvious: you can store the data for later use. In the analog world you can also store the data but it is quite complex.
So now, your data is in the computer memory and you have high frequencies you want to remove. High frequencies are those fast variations in your signal as when one measurement is one volt and the next is two volts, or whatever. This is the real realm of the digital filters.
A digital filter then manipulates data inside a computer. A digital filter does not know anything about voltages or resistors or capacitors. The effect is similar but the digital filter does its job inside a computer program.
Let us start by considering you have your data in an array you call D, that you have dimensioned like D(10000), or whatever. The idea is to filter this data. Let us start with a simple example. Imagine your data is formed by values, and the values are only 1, 2, 1, 2, 1, 2, etc. It is clear you have a continuous signal with a large high frequency component. We want to remove the high frequency component.
Let us start with the simplest kind of filter that is only averaging all values around the one we are studying. We use the following formula:
D(n) = [ D(n-1) + D(n) + D(n+1) ] / 3
what we are doing is adding together the value in question, D(n), and the two values close by, D(n-1) and D(n+1). Then we divide the sum by three so the new value of D(n) is an average.
Let us apply this equation to our data. Since the data is repetitive, we have two cases:
D(even) = [1 + 2 + 1] / 3 = 1.33
D(odd) = [2 + 1 + 2] / 3 = 1.66
It is clear what the filter has done. Instead of having a constant signal with a value of 1.5 plus a variations of +/- 0.5, as we had before, now we have the constant signal of 1.5 plus a variation of +/- 0.1666. So, we have not modified the constant signal but we have reduced the high frequency. We have filtered our signal.
It is easy to see how this of filter reduces the high frequencies, the fast variations in the signal; the averaging process reduces the peaks, flatten them up, and at the same time, fills the valleys of the signal.
We want to filter our signal some more, so we apply a more complex averaging, we will average each value with the two values each side of the value we are studying, using the formula:
D(n) = [D(n-2) + D(n-1) + D(n) + D(n+1) + D(n+2)] / 5
This is what is called a filter by five's and the previous one was a filter by threes. Normally this filter is represented simply by [1 1 1 1 1] and the previous one as [1 1 1].
Let us apply this filter to our data; since the data is repetitive, we have again two cases:
D(even) = [1+2+1+2+1]/5 = 1.4
D(odd) = [2+1+2+1+2]/5 = 1.6
what we have now is our constant signal of value 1.5 +/- 0.1 of high frequency. You see we have filtered our signal much more than before. Other effects can be obtained by using more terms in the averaging process, always the same number of terms each side of the value under study, giving always an odd number of terms in your sum.
We will be looking now at another type of filter that is not a simple averaging. In this case we will weight each term by a given value, to change its influence in the value under study. The type of filters we are studying are low pass filters; that is, they reduce the high frequencies and let the low frequencies pass, almost without attenuation.
The filter we will be using this time has the formula:
D(n) = [ D(n-1) + 2 * D(n) + D(n+1) ] / 4
notice that we are using the term n twice, that is why we need to divide into four instead of 3 as before. This filter is known as a [1 2 1] filter.
Let us apply this filter to our data. We have two cases again because of the data being repetitive:
D(even) = [1 + 2 * 2 + 1]/4 = 1.5
D(odd) = [2 + 2 * 1 + 2]/4 = 1.5
so this filter did a perfect job of removing completely the high frequency and leaving only the constant value. Naturally this is a consequence of the data we have. The filter produces a much sharper effect than the previous one.
In this same way we can have many other filters of this type, like [1 3 1] [1 4 1] [2 3 2], etc. In each case, you multiply each term by its coefficient and then divide the sum by the sum of all the coefficients. In the last one you divide by 7.
As a final example let us look for a more complex filter. Let us consider the [1 2 4 2 1] which has a formula
D(n) = [D(n-2) + 2*D(n-1) + 4*D(n) + 2*D(n+1) + D(n)]/10
Applying this filter to our data we obtain the same two cases as before:
D(even) = [1+2*2+4*1+2*2+1]/10 = 1.4
D(odd) = [2+2*1+4*2+2*1+2]/10 = 1.6
which gives us back the continuous signal plus +/- 0.1 of the high frequency signal. In this way, many other filters can be built.
As was mentioned before, the design of the filter; that is, finding out the values of the coefficients that produce the desired result, is quite complex but, with the ideas herein presented, it is possible to obtain results without too much analysis.