DC offset correction

Having issues with the site, hardware, source code, or any other issues?
Post Reply
n_qrtu
Posts: 17
Joined: Fri Feb 12, 2016 3:59 am

DC offset correction

Post by n_qrtu »

Hello,

I am using BladeRF SMALL REV2 to collect GPS samples.

Running the version command I get

bladeRF-cli version: 1.3.1-git-630264c
libbladeRF version: 1.5.1-git-630264c

Firmware version: 1.8.0
FPGA version: 0.3.3

My settings are;
set frequency rx 1575420000
set samplerate rx 4092000
set bandwidth rx 1.5M
set lnagain 6
set rxvga1 28
set rxvga2 27

I am then trying to compensate for the DC offset which is important for the GPS signal because having DC offset reduces the dynamic range of the signal which is very noisy already.

To this end, following the instructions, from the interactive mode I am running;
cal lms
cal dc rx

My questions;

1. When I am running the previous two commands, is it using the
''Revised and MIT-licensed DC calibration routines are now available in host/common/''
which is advertised in the 2016.01-rc1 release? Or I should do something else to be sure I use those routines?

2. I think I am using the latest versions of bladeRF-cli and libbladeRF? If not please correct me.

3. I see that my firmware and FPGA versions are old. Is it important to update them so they work properly with the latest versions of bladeRF-cli and libbladeRF from the 2016.01-rc1 release?

4. When running the calibration commands, is it important that no antenna or generator is attached to the receiver?

5. Any other suggestions to remove properly the DC offset will be very welcome!

6. Just in case; using osmocom_fft and a signal generator to manually correct for the DC offset does not give better results than the automatic one.


Thank you very much for your help!

Best Regards,
Nicolae
jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: DC offset correction

Post by jynik »

Hi there,

I just pushed a fix to the DC cal code to fix issue #453 two days ago. The issue was that after running 'cal dx rx', the nominal value wasn't actually being applied; the last correction values tested were.

Why don't you try this latest commit. If you still see a DC offset that looks high, could you let us know the mean IQ value is over a reasonably large set of samples, in addition to your device parameters?

Below are some answers to your questions and some comments:
n_qrtu wrote: My settings are;
set frequency rx 1575420000
set samplerate rx 4092000
set bandwidth rx 1.5M
set lnagain 6
set rxvga1 28
set rxvga2 27
With your gains that high, the LMS6002D will likely not be able to drive the DC offset to a zero crossing with the limited range of the compensation DACs. Try rxvga2 = 10. If you still need more gain, maybe a nice LNA out front of the bladeRF would be a good idea...
n_qrtu wrote: 1. When I am running the previous two commands, is it using the
''Revised and MIT-licensed DC calibration routines are now available in host/common/''
which is advertised in the 2016.01-rc1 release? Or I should do something else to be sure I use those routines?
Use the code as of this commit or later. You can use the bladeRF-cli as you are doing.
n_qrtu wrote: I think I am using the latest versions of bladeRF-cli and libbladeRF? If not please correct me.
As noted above, the latest from Git master include a fix that I think would be beneficial for you to use.
n_qrtu wrote: 3. I see that my firmware and FPGA versions are old. Is it important to update them so they work properly with the latest versions of bladeRF-cli and libbladeRF from the 2016.01-rc1 release?
I generally recommend keeping these version synced up to the lastest and greatest. While I don't think there are any relevant firmware or FPGA items here, it makes debugging simpler if you're using the same versions as the devs.
n_qrtu wrote: 5. Any other suggestions to remove properly the DC offset will be very welcome!
You could tune to some offset, mix your signal of interest to baseband and filter out the contribution of the DC offset.

If you look at the new DC calibration code for the TX side, you'll see the RX is configured to use a cute trick to perform an Fs/4 mix (no multiplication required! ;) ) and filter.
n_qrtu wrote: 6. Just in case; using osmocom_fft and a signal generator to manually correct for the DC offset does not give better results than the automatic one.
I think this is because of your gains being too high. If they're too high or too low, the amount of offset exceeds the range of the LMS6002D compensation DACs.
n_qrtu
Posts: 17
Joined: Fri Feb 12, 2016 3:59 am

Re: DC offset correction

Post by n_qrtu »

Hello,

Thank you very much for your fast reply!

I just came across your commit with the DC offset about an hour ago, so I started to set manually the I and Q correction values.
And it did the trick, the mean of my samples went down 100 times (so a DC reduction of 20dB).

I am collecting samples as follows:

rx config file=my_samples.sc16q11 format=bin n=245520000

At my chosen sampling rate, this corresponds to about 3000 bits, so I get 10 frames, from which I can extract my data.

I converted the sc16q11 format using the Matlab routines here:
https://github.com/Nuand/bladeRF/blob/m ... _sc16q11.m

Before correcting for the DC, the mean of the samples (after the conversion with load_sc16q11.m) was about 0.2
After the DC correction, this was around 0.002. For a variance of the samples of 0.015.

Yes, I thought about reducing the VGA gain.
While playing with osmocom_fft I have noticed that reducing the gain helps in reducing the DC offset.
I chose it high to have enough signal strength. But now that I understood more how things work, I will start experimenting with it.

My GPS antenna has already an LNA (I use a Bias Tee), but I might use an additional LNA.

For the code version, I will update everything to the current version as you suggested.

Let me know if you need other values from my DC offset correction.

I will let you know how it works when reducing the VGA gain.

Regards,
Nicolae
jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: DC offset correction

Post by jynik »

I forgot to answer your other question -- You absolutely want the antenna disconnected when performing the calibration.
n_qrtu
Posts: 17
Joined: Fri Feb 12, 2016 3:59 am

Re: DC offset correction

Post by n_qrtu »

Thank you!

Just a quick check: for installing the newer version I should not worry about uninstalling the old one, right?
I am following the installation instructions from release PPA.

Some other things I have noticed:

1. I have seen that when running cal dc rx, the correction values for the I/Q components are quite different from the ones I get with osmocom_fft (playing only with the sliders for DC offset). So I was wondering which should I take for reference: the values from cal dc rx or the ones I obtain manually with osmocom_fft?

2. The reverse way: I run cal dc rx, and with the obtained values I go in osmocom_fft and setting the sliders to the values obtained previously with cal dc rx, it does not look that the DC offset spike is reduces, sometimes is even increased.

3. While playing with osmocom_fft, and again only with the sliders for correcting the DC offset, I have noticed that for certain values of I/Q corrections I start seeing a significant spike for the image of the tone. So I was wondering if running cal dc rx can cause IQ imbalance? Would that be possible? (not a desirable thing to happen though.)

4. As far as I understood, the IQ imbalance cannot be corrected automatically. I have the option of correcting it manually with osmocom_fft. But as I mentioned above, I see inconsistencies between the two (osmocom_fft and cal dc rx). So what should I do? Use only cal dc rx for correcting the DC, hope that this will not affect the IQ imbalance, and not worry at all about osmocom_fft? At least this is my intuition for the moment.

Nicolae
jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: DC offset correction

Post by jynik »

n_qrtu wrote: Just a quick check: for installing the newer version I should not worry about uninstalling the old one, right?
I believe you should just be able to do apt-get upgrade or something to that effect; I generally am building from source and don't use the PPA, admittedly.
n_qrtu wrote: 1. I have seen that when running cal dc rx, the correction values for the I/Q components are quite different from the ones I get with osmocom_fft (playing only with the sliders for DC offset). So I was wondering which should I take for reference: the values from cal dc rx or the ones I obtain manually with osmocom_fft?

2. The reverse way: I run cal dc rx, and with the obtained values I go in osmocom_fft and setting the sliders to the values obtained previously with cal dc rx, it does not look that the DC offset spike is reduces, sometimes is even increased.
Are you applying the appropriate scaling factors when converting between the two? Note that gr-osmoodr has a range of [-1.0, 1.0) whereas the libbladeRF DC correction values are integers in [-2048, 2047]. You will need to divide by 2048 when going from bladeRF-cli -> osmocom_fft and divide by 2048 when going the other way.
n_qrtu wrote: 3. While playing with osmocom_fft, and again only with the sliders for correcting the DC offset, I have noticed that for certain values of I/Q corrections I start seeing a significant spike for the image of the tone. So I was wondering if running cal dc rx can cause IQ imbalance? Would that be possible? (not a desirable thing to happen though.)
This sounds strange. I would need to see both frequency and time domain plots to understand this better, along with the gain settings you're using.
n_qrtu wrote: 4. As far as I understood, the IQ imbalance cannot be corrected automatically. I have the option of correcting it manually with osmocom_fft. But as I mentioned above, I see inconsistencies between the two (osmocom_fft and cal dc rx). So what should I do? Use only cal dc rx for correcting the DC, hope that this will not affect the IQ imbalance, and not worry at all about osmocom_fft? At least this is my intuition for the moment.
I think once we get to the bottom of 1, 2, and maybe 3, we'll have a satisfactory setup for you.

Also beware that the gr-osmosdr block as correction modes. Setting this to "Off" zeros the correction values, whereas "Manual" implies they should not be touched (leaving them in the state you set externally). I seem to recall osmocom_fft or osmocom_siggen overwriting my correction values on me, so we'll have to verify that this is not the case (or submit a patch if it is).
n_qrtu
Posts: 17
Joined: Fri Feb 12, 2016 3:59 am

Re: DC offset correction

Post by n_qrtu »

jynik wrote:
n_qrtu wrote: Just a quick check: for installing the newer version I should not worry about uninstalling the old one, right?
I believe you should just be able to do apt-get upgrade or something to that effect; I generally am building from source and don't use the PPA, admittedly.
Thank you for your answers!

I installed the latest version with apt-get upgrade.
Now I have:
bladeRF> version
bladeRF-cli version: 1.3.1-git-1b96a9c
libbladeRF version: 1.5.1-git-1b96a9c
Firmware version: 1.9.0
FPGA version: 0.3.3

However, I do not know how to get the FPGA up to date. Reading the documentation I found that this should be loaded automatically.

jynik wrote:
n_qrtu wrote: 1. I have seen that when running cal dc rx, the correction values for the I/Q components are quite different from the ones I get with osmocom_fft (playing only with the sliders for DC offset). So I was wondering which should I take for reference: the values from cal dc rx or the ones I obtain manually with osmocom_fft?

2. The reverse way: I run cal dc rx, and with the obtained values I go in osmocom_fft and setting the sliders to the values obtained previously with cal dc rx, it does not look that the DC offset spike is reduces, sometimes is even increased.
Are you applying the appropriate scaling factors when converting between the two? Note that gr-osmoodr has a range of [-1.0, 1.0) whereas the libbladeRF DC correction values are integers in [-2048, 2047]. You will need to divide by 2048 when going from bladeRF-cli -> osmocom_fft and divide by 2048 when going the other way.

I did not know about the conversion. When I apply it, the results look consistent.

jynik wrote:
n_qrtu wrote: 3. While playing with osmocom_fft, and again only with the sliders for correcting the DC offset, I have noticed that for certain values of I/Q corrections I start seeing a significant spike for the image of the tone. So I was wondering if running cal dc rx can cause IQ imbalance? Would that be possible? (not a desirable thing to happen though.)
This sounds strange. I would need to see both frequency and time domain plots to understand this better, along with the gain settings you're using.
This is actually happening only when rxvga2 = 27. If I use 21 or lower, this does not happen. I am attaching a screenshot with the settings and the osmocom plots.
Screenshot from 2016-02-19 17_10_30.png
I have tried to use rxvga2 = 10 as you suggested, and indeed the cal dc rx is able to reduce the DC offset. However, my signal is too weak, I am seeing less satellites than before. So I guess I need to try and find the right compromise value (signal strength vs DC offset) of rxvga2 which works for me. For now, I think a value of 21 looks good.
In the end, what I need is a good SNR between my signal and (DC offset + noise).

jynik wrote:
n_qrtu wrote: 4. As far as I understood, the IQ imbalance cannot be corrected automatically. I have the option of correcting it manually with osmocom_fft. But as I mentioned above, I see inconsistencies between the two (osmocom_fft and cal dc rx). So what should I do? Use only cal dc rx for correcting the DC, hope that this will not affect the IQ imbalance, and not worry at all about osmocom_fft? At least this is my intuition for the moment.
I think once we get to the bottom of 1, 2, and maybe 3, we'll have a satisfactory setup for you.

Also beware that the gr-osmosdr block as correction modes. Setting this to "Off" zeros the correction values, whereas "Manual" implies they should not be touched (leaving them in the state you set externally). I seem to recall osmocom_fft or osmocom_siggen overwriting my correction values on me, so we'll have to verify that this is not the case (or submit a patch if it is).
I have noticed that calling
osmocom_fft --dc-offset-mode=0 --iq-balance-mode=0
overwrites the seetings of cal dc rx
But it seems to me that calling just
osmocom_fft
does not overwrite them.

Nicolae
jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: DC offset correction

Post by jynik »

Hi Nicolae,

Sorry for the delayed response.

The latest FPGA images are available on https://nuand.com/fpga.php. This link should explain more about the two ways you can have the RBF file loaded: https://github.com/Nuand/bladeRF/wiki/FPGA-Autoloading

There is definitely set of low gains and high gains in which the LMS6002D's internal DC offset compensation DACs simply cannot apply any more offset. We may eventually add a feature in the FPGA to subtract off the residual counts, but obviously this just limits your dynamic range. Your approach of finding a good balance between the gain and DC offset sounds reasonable to me. You may want to consider trying finding an external LNA if you think you need eve more gain.

Cheers,
Jon
n_qrtu
Posts: 17
Joined: Fri Feb 12, 2016 3:59 am

Re: DC offset correction

Post by n_qrtu »

Hi Jon,

Thank you for your answer.

I have managed to load the latest version of the FPGA, now I have everything up to date.

I have found another active GPS antenna which has a better LNA, with 12 dB more than my current one. I will order it soon and this should compensate for the lower rxvga2 which I have to use.

Quick question: from your experience, these settings:
set lnagain 6
set rxvga1 28
set rxvga2 9

should do well when calling cal dc rx? Or you could recommend another set of values for which the DC offset can be corrected better?

And another small question: I am using the smallest bandwidth possible, i.e., 1.5MHz. I would even go lower if possible (the GPS signal is very low rate) to be sure I am collecting as less noise as possible outside my useful bandwidth.
I am setting:
set samplerate rx 4092000
set bandwidth rx 1.5M

Now: is there a requirement for the proper functioning of bladeRF to set the bandwidth exactly to Nyquist? For my sampling rate I am using a lower bandwidth than required by theory. This is of course fine from a theoretical point of view, but what about bladeRF? Is the board suppose to work fine with these settings?
I just want to make sure that there are no hidden issues here with the hardware that I should be aware of.

Thanks again,
Nicolae
jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: DC offset correction

Post by jynik »

Off the top of my head, those gain settings sound to be within the range where the DC calibration should work well. You can verify this by noting the "Error" values in the DC cal's output, which estimates the residual DC offset in units of counts in the range [-2048, 2047].

Theoretically, you need keep bandwidth <= samplerate. In practical terms, the settings you're using are better, because of the LPF filter roll-off we discussed.

If you need an even lower bandwidth, you can filter digitally.
n_qrtu
Posts: 17
Joined: Fri Feb 12, 2016 3:59 am

Re: DC offset correction

Post by n_qrtu »

The errors values which I see are around 0.1 - 0.5, so cal dc rx seems to work fine for those VGA settings.

I will add an external LNA of 25dB soon, and like this I expect to see/decode satellites even when placing my antenna close to the window. (Right now I need to be on the roof or in an open space to see/decode at least 4 satellites and get my position).

Thank you again for your patience and help!

Nicolae
jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: DC offset correction

Post by jynik »

Hi Nicolae,

That sounds good to me - we can only correct within whole-valued counts. There's not fine-grained controls anywhere in the current system to correct fractions of an LSB.

I'm happy to hear that things are progressing. Do you think you'll be able to share some more info about your project and progress once you get your antenna in? It'd be super exciting to see your methodology and some pictures of your setup!

Thanks for your patience as well -- I know I've been a little slow on forum responses as of late.

Cheers!
Jon
n_qrtu
Posts: 17
Joined: Fri Feb 12, 2016 3:59 am

Re: DC offset correction

Post by n_qrtu »

Hi Jon,

Sure, once I have everything in place I can share my setup / settings / pictures.

I have just ordered today the LNA, it should be shipped within the next few days.

We are using this for a hands-on Software Radio class, and the idea would be to let the students collect the samples and then process them in Matlab to decode the satellites data and get their position.

As a next step I would like to try to make everything from Matlab - setup / calibration / samples acquisition, so everything can be done nicely with scripts from Matlab.

Further on, we are thinking of getting another board and make a Tx / Rx system where we transmit / receive some QAM, etc. We can test this first within one board (internally and then connecting the Tx to Rx - with some appropriate attenuators of course :-) ). Once we get another board I think we should worry about clock synchronization, we have to implement that in Matlab.

Anyway, one thing at a time, for now I would like to have the GPS working robustly.

Cheers,
Nicolae
n_qrtu
Posts: 17
Joined: Fri Feb 12, 2016 3:59 am

Re: DC offset correction

Post by n_qrtu »

Dear Jon,

As promised (a long time ago :-) ), here is the setup for our GPS signal acquisition.
bladeRF_GPS_signal_acquisition_setup_small.JPG
We are using a BiasTee to provide current to the 2 LNAs (one external (cylindrical one) and the another one built within the antenna).
(bladeRF cannot provide enough current for them)

BiasTee:
DSA-546586.pdf
GPS antenna:
http://ch.farnell.com/fr-CH/tallysman-w ... dp/2056650

LNA:
http://ch.farnell.com/fr-CH/tallysman-w ... 7-00001003

Now, the bladeRF settings:

set frequency rx 1575420000
set samplerate rx 4092000
set bandwidth rx 1.5M
set lnagain 6
set rxvga1 28
set rxvga2 9

In order not to drop samples, we collect them in /dev/shm/
From there:
bladeRF-cli -i
bladeRF> cal lms
bladeRF> cal dc rx

One needs to collect at least 2000 bits (a bit more than 6 subframes = 1800 bits). The number of samples is computed as:
2000 bits x 20 CA/bit x 4092 samples/CA = 163680000.
bladeRF> rx config file=my_samples.sc16q11 format=bin n=163680000
Start the signal acquisition:
bladeRF> rx start

We then process the samples in Matlab. (convert them first to .mat format with https://github.com/Nuand/bladeRF/blob/m ... _sc16q11.m)
I cannot make the code public, since the students need to write part of it in the class :-)

In an open space, we can see up to 9-10 satellites. Just putting the antenna outside the window still gets 4-5 satellites.
If we place the antenna by the window but inside the room, we cannot really get/decode any satellites - it looks that the (double window) glass attenuation is too much...

Cheers,
Nicolae
Post Reply