Need some help with headless development.

Discussions related to embedded firmware, driver, and user mode application software development

Moderator: robert.ghilduta

Jaco
Posts: 26
Joined: Wed Jul 30, 2014 2:03 am

Need some help with headless development.

Post by Jaco » Fri Dec 18, 2015 7:42 am

Hello again,

I've been running into some problems getting a headless bladeRF up and running, which I will detail here. The HDL code that runs on the FPGA should do whatever the bladeRF_hosted revision does, except that it passes received samples through a custom block, and this block then repeats the received samples. From looking at the existing bladeRF_hosted code, it seems that the best place to insert the custom block would be between the tx and rx_iq_correction blocks (so, rx_iq_correction->custom_block->tx_iq_correction), which means that the code will ignore the sample and meta FIFOs. I should add that the custom block has been simulated and it's working as intended.

For this to happen, a few things need to be set up with the Si5338 / LMS6002D:
  • TX/RXVGA1/2 gains (LMS)
    LNA gain (LMS)
    TX/RX frequency (LMS)
    TX/RX samplerate (Si5338)
Furthermore, since the system should RX and TX from startup, those modules need to be enabled permanently. Is forcing the rx_enable and tx_enable inputs on the LMS block to 'high' the correct way to do this?

By running the cli in verbose mode, finding the registers to write (along with their respective values, so that it matches the required configuration) is fairly simple, and I've modified the NIOS II code to include a configuration process in bladeRF_nios.c, before the while (run_nios) {...} part begins executing . This is nothing more than calling lms6_write / si5338_write with the respective address / data values from the cli.

The problem that I'm having is that I can't verify if these config values for the LMS/Si5338 actually wrote, since it seems that running the CLI does some sort of initialize through the FX3 that sets the frequency to 1 GHz and the sampling rate to 1 MHz, among other things. Is there any other method to test this? Autoloading the FPGA does not seem to alleviate this (nor does just loading the FPGA through the CLI), so there must be something going wrong in the config process. I don't want to completely remove the capability to retune through the USB interface, and I'm guessing that I'll still need the USB interface to peek registers on the LMS/Si5338.

TL;DR:
My custom block takes RX'd samples from the IQ correction block, does some work and then clocks them out to the TX IQ correction block after a small delay (longest would probably be around 1 ms).
  • 1. What difference does it make between using the TX clock (c4_tx_clock) or the RX clock () as a clock for the custom block, since both signals come from the Si5338 and are configured to the same frequency?
    2. Is setting rx_enable / tx_enable on the LMS block to 'high' the correct way of ensuring that those modules continuously TX/RX?
    3. How do I set config values for the LMS/Si5338 automatically, and how do I confirm that the works? I don't have access to a USB blaster or similar at the moment.
Thanks in advance for any input.

Best regards,
Jaco

Zortz
Posts: 15
Joined: Mon Oct 26, 2015 1:50 am

Re: Need some help with headless development.

Post by Zortz » Fri Dec 18, 2015 11:08 am

Hi!
1. What difference does it make between using the TX clock (c4_tx_clock) or the RX clock () as a clock for the custom block, since both signals come from the Si5338 and are configured to the same frequency?
These clocks are not synchronized, you will have to deal with clock crossing(dual clock FIFO). I could provide an example code on how to do this, if required.
Is setting rx_enable / tx_enable on the LMS block to 'high' the correct way of ensuring that those modules continuously TX/RX?
I have tested this and it will not work. You need to write some register i assume, look here:
https://github.com/Nuand/bladeRF/blob/1 ... /src/lms.c <- lms_enable_rffe
How do I set config values for the LMS/Si5338 automatically, and how do I confirm that the works? I don't have access to a USB blaster or similar at the moment.
No experience here, but maybe you could try adding some delay before your custom write? Make sure it is not a blocking delay.

Jaco
Posts: 26
Joined: Wed Jul 30, 2014 2:03 am

Re: Need some help with headless development.

Post by Jaco » Fri Dec 18, 2015 1:16 pm

Hi, thanks for the input.
Zortz wrote:These clocks are not synchronized, you will have to deal with clock crossing(dual clock FIFO). I could provide an example code on how to do this, if required.
Why are the clocks not synchronized exactly? I know that the digital IQ interface on the LMS handles the TX and RX clocks slightly differently, but according to the Si5338 data sheet, "Upon power up or a soft_reset the Si5338 synchronizes the output clocks" - p28. Not saying you're wrong, just need to understand what's going on. Nonetheless, I haven't worked with dual clock stuff yet, so this will take a couple of hours to figure out / rebuild.
Zortz wrote:I have tested this and it will not work. You need to write some register i assume, look here:
https://github.com/Nuand/bladeRF/blob/1 ... /src/lms.c <- lms_enable_rffe
I've tried writing the enable register for the TX (0x40) / RX (0x70) modules in a similar way that the other config registers are being written, both through the NIOS II code and through the CLI. I'm doing something wrong with the NIOS method because it does not work, as even the register reading after the FPGA loads (part of the default setup) reads out different values than the ones I've written. Using the cli to poke the two addresses mentioned earlier (i.e. write to them) I can enable the TX/RX front ends, I guess that's more or less what lms.c does.
Zortz wrote:No experience here, but maybe you could try adding some delay before your custom write? Make sure it is not a blocking delay.
Adding a long delay either before or after the register writing (>10 sec) didn't do the trick. As I've said earlier, it really seems that the registers aren't writing at all to begin with. I don't know if there's some way of disabling the init_device() routine in bladeRF_private.c (https://github.com/Nuand/bladeRF/blob/1 ... riv.c#L138). I could try modify that particular function to just never load the default cfg and check what the output of <peek lms 0x40/0x70> is, or whatever registers I'm writing to through the NIOS code. I don't know if there's another method to prevent the cli from loading the default values.

Jaco
Posts: 26
Joined: Wed Jul 30, 2014 2:03 am

Re: Need some help with headless development.

Post by Jaco » Sat Dec 19, 2015 5:17 am

An update on the auto-config issue:

I noticed that I did not do anything with the NIOS GPIO state when implementing the auto-config part of the NIOS II code. This means that, whenever the processor is started up, the GPIO would be set to 0x00000000 (32-bit cfg), which the CLI reads through the FX3 and decides that it needs to initialize the LMS. By using the CLI's verbose setting and checking the value returned by nios_config_read() after I've changed to the desired settings, I then added a line with control_reg_write() to the NIOS II code to write the correct value. The CLI detects (after the image is loaded) that the GPIO is not set to default and it doesn't do its default config. This is pretty much exactly what I needed since it auto-loads my own config without removing the ability to set LMS/Si5338 config values over the USB connection.

Next part is to figure out whether synchronizing the LMS TX / RX clocks for read / write is necessary and modifying the existing custom block accordingly. Interestingly enough though, there is one notable difference between compiling the modified bladerf_hosted project using the TX clock as input to the custom block, and using the RX clock in stead. Using the TX clock returns a warning about data bits transferred across asynchronous clock domains, but changing to the RX clock removes this warning. I'm not exactly sure what this means and whether it sheds some light on the clocking problem.

Zortz
Posts: 15
Joined: Mon Oct 26, 2015 1:50 am

Re: Need some help with headless development.

Post by Zortz » Sat Dec 19, 2015 6:48 am

Jaco wrote:An update on the auto-config issue:

I noticed that I did not do anything with the NIOS GPIO state when implementing the auto-config part of the NIOS II code. This means that, whenever the processor is started up, the GPIO would be set to 0x00000000 (32-bit cfg), which the CLI reads through the FX3 and decides that it needs to initialize the LMS. By using the CLI's verbose setting and checking the value returned by nios_config_read() after I've changed to the desired settings, I then added a line with control_reg_write() to the NIOS II code to write the correct value. The CLI detects (after the image is loaded) that the GPIO is not set to default and it doesn't do its default config. This is pretty much exactly what I needed since it auto-loads my own config without removing the ability to set LMS/Si5338 config values over the USB connection.

Next part is to figure out whether synchronizing the LMS TX / RX clocks for read / write is necessary and modifying the existing custom block accordingly. Interestingly enough though, there is one notable difference between compiling the modified bladerf_hosted project using the TX clock as input to the custom block, and using the RX clock in stead. Using the TX clock returns a warning about data bits transferred across asynchronous clock domains, but changing to the RX clock removes this warning. I'm not exactly sure what this means and whether it sheds some light on the clocking problem.
Good to know you got custom configuration working, this information will be useful for others as well.
About the clocking i felt the same way - clocks should be synchronous. As i tried out my custom block it did not work, but started working after i added a dual clock FIFO for synchronization. I did not investigate this issue further, because well..it was working. Maybe you can just use RX clock everywhere and just skip the FIFO, this needs a test.
For quick tests you could replace your custom block with a pass-trough bock and verify that RX=TX.

Jaco
Posts: 26
Joined: Wed Jul 30, 2014 2:03 am

Re: Need some help with headless development.

Post by Jaco » Sat Dec 19, 2015 12:50 pm

Zortz wrote:About the clocking i felt the same way - clocks should be synchronous. As i tried out my custom block it did not work, but started working after i added a dual clock FIFO for synchronization. I did not investigate this issue further, because well..it was working. Maybe you can just use RX clock everywhere and just skip the FIFO, this needs a test.
I still need to work on this, but the custom block is literally just a dual FIFO with a delayed read pointer. From what I've been reading it shouldn't be too difficult to extend this so that it works with two clocks.
Zortz wrote:For quick tests you could replace your custom block with a pass-trough bock and verify that RX=TX.
Just did this, so that there's only an exchange of samples between the two IQ correction blocks, but still nothing. I'm not sure if there's anything that I'm not seeing:
  • ->Set up CLK0,1,2,3 on the Si5338 for a 10 MHz samplerate, make sure all are enabled.
    ->Set up all relevant LMS config (frequency, VGA gains, LNA gain, etc.).
    ->Set up additional LMS config from bladerf_priv.c (power down DC cal comparators, etc.)
    ->Force TX / RX EN high so that the enable signals are not taken from the FX3 GPIF.
    ->Force TX / RX reset low so that the LMS never goes into reset (reset signals no longer come from U_rx_enable_sync / U_tx_enable_sync). Tried both methods; forcing no reset and sourcing reset from U_rx_enable_sync.
    ->Set LMS soft TX / RX EN registers, as well as enable TX / RX RF modules.
Most of these can be tested by peeking registers through the CLI, but I just can't figure out what is missing that's causing the LMS to never TX / RX. I know that my second bladeRF works, since it works exactly as intended with external loopback (and tested with a signal generator / spectrum analyzer).

Zortz
Posts: 15
Joined: Mon Oct 26, 2015 1:50 am

Re: Need some help with headless development.

Post by Zortz » Sun Dec 20, 2015 6:30 am

I think you should start with an GNURadio implementation:
bladePasstrough.png
That way you can be sure that all the configurations are correct with TX and RX both enabled.
I would suggest that you first solve the clocking problems using this flow-graph, note that it does not matter if you are not using the USB sample interface, the purpose of this flow-graph is just to guarantee that all the registers are written correctly- thus you should see TX=RX, when you have no clocking problems.

After that you can slowly migrate from GNURadio writing registers, to your custom NIOS register writing - do it in small steps, so you can find the problem you are having atm.
Attachments

[The extension grc has been deactivated and can no longer be displayed.]


Jaco
Posts: 26
Joined: Wed Jul 30, 2014 2:03 am

Re: Need some help with headless development.

Post by Jaco » Thu Dec 24, 2015 9:12 am

Thanks Zortz, I actually didn't consider using GNUradio for debugging/testing.

Some more progress on this project:

Following Zortz's advice, I've reworked a lot of the debugging methods that I was using before:

[1] Remove custom code on the NIOS II, again enabling the default startup procedure when the CLI opens the device.

[2] Reconfigured the x115 image to add the USB sample interface on RX. This just means that RX'd samples are sent to two places, back to the TX IQ correction block as well as the FIFO writer that writes it back to the PC. The RX'd signal in GNUradio is as expected (shown below). However, there's still something going haywire with the clocking, like you've suggested, because feeding it through for TX doesn't work. I've also tried using the stock x115 image and built a similar configuration in GNUradio than the one you posted, with an added 1000 sample delay (@ 10 MHz samplerate, 100e-6 seconds) before repeating the received signal, but no luck, strangely.

Image

[3] To remedy the clocking issue, I used Altera's FIFO IP generator to create a dual-clock FIFO, where the write operation runs off the RX clock and the read operation off the TX clock. Something like this: (doesn't show the internal working, but it's generated from the IP wizard so it won't be possible to see in any case)

Image

This component is then placed between TX / RX IQ correction blocks (with read / write request signals always high), and works as expected in ModelSim by itself, but unfortunately not in hardware.

[4] Since we know that the x40 TX/RX both work, and from what I've described above, the x115 RX also works, the only thing that remains is the x115 TX, although I'm not quite sure yet how to fix it. I've created a simple source in GNUradio that just streams a 10 kHz sine wave from the x115 to the x40, here's the recorded result:

Image

The odd looking signal between 0 and 100 samples is leakage from the x40 TX, since the TX/RX operations run simultaneously. So, the problem is clearly where the RX'd samples are fed to into the TX chain, which is what I'm working on correcting now.

Season's greetings,
Jaco

Zortz
Posts: 15
Joined: Mon Oct 26, 2015 1:50 am

Re: Need some help with headless development.

Post by Zortz » Mon Dec 28, 2015 7:43 am

[2] ... I've also tried using the stock x115 image and built a similar configuration in GNUradio than the one you posted, with an added 1000 sample delay (@ 10 MHz samplerate, 100e-6 seconds) before repeating the received signal, but no luck, strangely.

So you used the stock FPGA image + the GNURadio graph and it did not work? This is strange indeed, there should be no problems when using the stock FPGA image.I think getting this working should be your first priority, because it just should work.


Now after you get this GNURadio thing working you might dive back into hacking the FPGA code. Here i would suggest using smallest possible steps, because finding bugs is so hard with FPGAs.
For example, here is how i did the passtrough mode:
passtrough.png
I just rerouted the RX fifo and that is it.
For that only 2 modifications were needed (not 100% sure, cannot test this atm - worked before)
1. changed the rx sample clock:

Code: Select all

rx_sample_fifo.rclock <= tx_clock ;
2. changed fifo reader to read from RX fifo:

Code: Select all

fifo_empty          =>  rx_sample_fifo.rempty,
        fifo_usedw          =>  rx_sample_fifo.rused,
        fifo_data           =>  rx_sample_fifo.rdata,
        fifo_read           =>  rx_sample_fifo.rreq,
Hope it helps!

Jaco
Posts: 26
Joined: Wed Jul 30, 2014 2:03 am

Re: Need some help with headless development.

Post by Jaco » Sun Feb 07, 2016 6:06 am

Update on this, managed to get the whole thing working a few weeks ago, and working rather well. Could post some more info if needed.

Thanks Zortz for the assitance!

Post Reply