Overrun when doing sync meta transfers TX and RX

Discussions related to schematic capture, PCB layout, signal integrity, and RF development
Post Reply
Bcd
Posts: 14
Joined: Tue Jun 06, 2017 3:00 am

Re: Overrun when doing sync meta transfers TX and RX

Post by Bcd »

Alright, I did the same tests on a server with an i7 / 8cores. I am using Ubuntu 16.04.3 (kernel 4.10.0-35) on both Laptop and server and the bladerf sources are from mainstream github.

The RX/TX diff timing is also 2.2ms like on the laptop and I needed to set

#define RX_TIME_USEC 2500
#define TX_TIME_USEC 2500

At room temp I get the same reset of the device during transfers which in turn make the app to exit (see below the log). There is actually no time diff above 2.5ms, good server I guess.

This is from the kernel log, the device reset:

[152500.042265] usb 4-2: reset SuperSpeed USB device number 2 using xhci_hcd
[152500.063196] usb 4-2: LPM exit latency is zeroed, disabling LPM.

Just cooling the device makes it run without reset and no overruns.

Basically there is no difference from running the app on Laptop and on the server.

The log below is from a run at room temp:

[VERBOSE @ "backend/usb/libusb.c":593] Using libusb version: 1.0.20.11004
[VERBOSE @ "backend/usb/libusb.c":470] Found a bladeRF (idx=3)
[VERBOSE @ "backend/usb/libusb.c":532] USB port reset succeeded for bladeRF 9eeeeecf610343cba827a9889dcc9c23
[VERBOSE @ "capabilities.c":45] Capability mask before FPGA load: 0x0000000700000000
[VERBOSE @ "backend/usb/usb.c":112] Changing to USB alt setting 0
[VERBOSE @ "backend/usb/usb.c":112] Changing to USB alt setting 1
[VERBOSE @ "backend/usb/usb.c":169] Read FPGA version: 0.6.0
[VERBOSE @ "capabilities.c":96] Capability mask after FPGA load: 0x00000007000003ff
[VERBOSE @ "backend/usb/usb.c":180] Using current packet handler formats
[VERBOSE @ "backend/usb/usb.c":112] Changing to USB alt setting 2
[VERBOSE @ "backend/usb/usb.c":112] Changing to USB alt setting 1
[VERBOSE @ "backend/usb/usb.c":112] Changing to USB alt setting 2
[VERBOSE @ "backend/usb/usb.c":112] Changing to USB alt setting 1
[VERBOSE @ "backend/usb/nios_access.c":298] nios_config_read: Read 0x0003002f
[DEBUG @ "tuning.c":82] Default tuning mode: FPGA
[VERBOSE @ "backend/usb/nios_access.c":298] nios_config_read: Read 0x0003002f
[VERBOSE @ "backend/usb/nios_access.c":745] Clearing RX retune queue.
[VERBOSE @ "backend/usb/nios_access.c":768] RX operation duration: 44
[VERBOSE @ "backend/usb/nios_access.c":745] Clearing TX retune queue.
[VERBOSE @ "backend/usb/nios_access.c":768] TX operation duration: 35
[DEBUG @ "tuning.c":135] Setting RX frequency to 2560000000
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":357] VCOCAP estimate: 42
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1677] ---- Frequency ----
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1678] x : 2
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1679] nint : 133
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1680] nfrac : 2796203
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1681] freqsel : 0x2c
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1682] reference: 38400000
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1683] freq : 2560000001
[VERBOSE @ "backend/usb/nios_access.c":750] nios_retune: module=RX timestamp=0 nint=133 nfrac=2796203
freqsel=0x2c vcocap=0x2a low_band=0 quick_tune=0
[VERBOSE @ "backend/usb/nios_access.c":765] RX retune operation: vcocap=44, duration=2305
[VERBOSE @ "si5338.c":525] Setting integer sample rate: 7680000
[VERBOSE @ "si5338.c":383] Found r value of: 1
[VERBOSE @ "si5338.c":393] MSx a + b/c: 165 + 0/1
[VERBOSE @ "si5338.c":422] MSx a + b/c: 165 + 0/1
[VERBOSE @ "si5338.c":211] MSx P1: 0x00005080 (20608) P2: 0x00000000 (0) P3: 0x00000001 (1)
[VERBOSE @ "si5338.c":347] Calculated multisynth frequency: 7680000 + 0/1
[VERBOSE @ "si5338.c":234] Writing MS1
[VERBOSE @ "backend/usb/nios_access.c":385] nios_si5338_read: Read 0x01 from addr 0x25
[VERBOSE @ "si5338.c":243] Wrote enable register: 0x01
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x01 to addr 0x25
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x80 to addr 0x40
[VERBOSE @ "si5338.c":257] Wrote regs[0]: 0x80
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x50 to addr 0x41
[VERBOSE @ "si5338.c":257] Wrote regs[1]: 0x50
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x42
[VERBOSE @ "si5338.c":257] Wrote regs[2]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x43
[VERBOSE @ "si5338.c":257] Wrote regs[3]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x44
[VERBOSE @ "si5338.c":257] Wrote regs[4]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x45
[VERBOSE @ "si5338.c":257] Wrote regs[5]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x01 to addr 0x46
[VERBOSE @ "si5338.c":257] Wrote regs[6]: 0x01
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x47
[VERBOSE @ "si5338.c":257] Wrote regs[7]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x48
[VERBOSE @ "si5338.c":257] Wrote regs[8]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x49
[VERBOSE @ "si5338.c":257] Wrote regs[9]: 0x00
[VERBOSE @ "si5338.c":272] Wrote r register: 0xc0
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0xc0 to addr 0x20
[VERBOSE @ "si5338.c":542] Set actual integer sample rate: 7680000
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x3e from addr 0x54
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x3e to addr 0x54
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x0c from addr 0x55
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x3e from addr 0x54
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x3e to addr 0x54
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0xe0 from addr 0x75
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0xe0 to addr 0x75
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x68 to addr 0x76
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x00 to addr 0x65
[DEBUG @ "tuning.c":135] Setting TX frequency to 2561920000
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":357] VCOCAP estimate: 42
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1677] ---- Frequency ----
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1678] x : 2
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1679] nint : 133
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1680] nfrac : 3635063
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1681] freqsel : 0x2c
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1682] reference: 38400000
[VERBOSE @ "/home/bogdan/projects/gnuradio/src/bladeRF/fpga_common/src/lms.c":1683] freq : 2561919999
[VERBOSE @ "backend/usb/nios_access.c":750] nios_retune: module=TX timestamp=0 nint=133 nfrac=3635063
freqsel=0x2c vcocap=0x2a low_band=0 quick_tune=0
[VERBOSE @ "backend/usb/nios_access.c":765] TX retune operation: vcocap=44, duration=2097
[VERBOSE @ "si5338.c":525] Setting integer sample rate: 7680000
[VERBOSE @ "si5338.c":383] Found r value of: 1
[VERBOSE @ "si5338.c":393] MSx a + b/c: 165 + 0/1
[VERBOSE @ "si5338.c":422] MSx a + b/c: 165 + 0/1
[VERBOSE @ "si5338.c":211] MSx P1: 0x00005080 (20608) P2: 0x00000000 (0) P3: 0x00000001 (1)
[VERBOSE @ "si5338.c":347] Calculated multisynth frequency: 7680000 + 0/1
[VERBOSE @ "si5338.c":234] Writing MS2
[VERBOSE @ "backend/usb/nios_access.c":385] nios_si5338_read: Read 0x03 from addr 0x26
[VERBOSE @ "si5338.c":243] Wrote enable register: 0x03
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x03 to addr 0x26
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x80 to addr 0x4b
[VERBOSE @ "si5338.c":257] Wrote regs[0]: 0x80
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x50 to addr 0x4c
[VERBOSE @ "si5338.c":257] Wrote regs[1]: 0x50
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x4d
[VERBOSE @ "si5338.c":257] Wrote regs[2]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x4e
[VERBOSE @ "si5338.c":257] Wrote regs[3]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x4f
[VERBOSE @ "si5338.c":257] Wrote regs[4]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x50
[VERBOSE @ "si5338.c":257] Wrote regs[5]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x01 to addr 0x51
[VERBOSE @ "si5338.c":257] Wrote regs[6]: 0x01
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x52
[VERBOSE @ "si5338.c":257] Wrote regs[7]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x53
[VERBOSE @ "si5338.c":257] Wrote regs[8]: 0x00
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0x00 to addr 0x54
[VERBOSE @ "si5338.c":257] Wrote regs[9]: 0x00
[VERBOSE @ "si5338.c":272] Wrote r register: 0xc0
[VERBOSE @ "backend/usb/nios_access.c":397] nios_si5338_write: Wrote 0xc0 to addr 0x21
[VERBOSE @ "si5338.c":542] Set actual integer sample rate: 7680000
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x3e from addr 0x34
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x3e to addr 0x34
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x0c from addr 0x35
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x3e from addr 0x34
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x3e to addr 0x34
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x0f to addr 0x41
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x00 from addr 0x45
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x00 to addr 0x45
[VERBOSE @ "backend/usb/nios_access.c":298] nios_config_read: Read 0x0003002f
[VERBOSE @ "backend/usb/nios_access.c":309] nios_config_write: Wrote 0x0003002f
[VERBOSE @ "backend/usb/nios_access.c":298] nios_config_read: Read 0x0003002f
[VERBOSE @ "backend/usb/nios_access.c":309] nios_config_write: Wrote 0x0003002f
[DEBUG @ "bladerf.c":356] Enable Module: RX - True
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x00 from addr 0x70
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x01 to addr 0x70
[DEBUG @ "bladerf.c":356] Enable Module: TX - True
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x00 from addr 0x40
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x02 to addr 0x40
[DEBUG @ "sync.c":326] sync_rx: Worker is idle. Going to reset buf mgmt.
[DEBUG @ "sync.c":346] sync_rx: Reset buf_mgmt consumer index
[DEBUG @ "sync.c":361] sync_rx: Worker is now running.
[VERBOSE @ "backend/usb/nios_access.c":370] nios_get_timestamp: Read TX timstamp: 0x140728404499152
[DEBUG @ "sync.c":849] sync_tx: Worker is now running.
[DEBUG @ "sync.c":485] Sample discontinuity detected @ buffer 111, message 5: Expected t=92458551, got t=360894007
[DEBUG @ "sync.c":485] Sample discontinuity detected @ buffer 111, message 6: Expected t=360894515, got t=92459059
[DEBUG @ "sync.c":704] Provided timestamp=92460083 is in past: current=360905031
Failed to TX samples: Requested timestamp is in the past
[DEBUG @ "bladerf.c":356] Enable Module: RX - False
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x01 from addr 0x70
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x00 to addr 0x70
[DEBUG @ "bladerf.c":356] Enable Module: TX - False
[VERBOSE @ "backend/usb/nios_access.c":409] nios_lms6_read: Read 0x02 from addr 0x40
[VERBOSE @ "backend/usb/nios_access.c":421] nios_lms6_write: Wrote 0x00 to addr 0x40
[VERBOSE @ "backend/usb/nios_access.c":745] Clearing RX retune queue.
[VERBOSE @ "backend/usb/nios_access.c":768] RX operation duration: 43
[VERBOSE @ "backend/usb/nios_access.c":745] Clearing TX retune queue.
[VERBOSE @ "backend/usb/nios_access.c":768] TX operation duration: 35
Opening and initializing device...

RX frequency: 2560000000 Hz
RX samplerate: 7680000 sps
RX bandwidth: 1500000 Hz
RX LNA Gain: Max
RX VGA1 gain: 20
RX VGA2 gain: 0

TX frequency: 2561920000 Hz
TX samplerate: 7680000 sps
TX bandwidth: 1500000
TX VGA1 gain: -20
TX VGA2 gain: 0

Running...
Current TX timestamp: 17821
RX overrun, count: 8540, timestamp: 92450011
RX overrun, count: 508, timestamp: 360894007
last loop took 0.000987 seconds
Closing the device...
Bcd
Posts: 14
Joined: Tue Jun 06, 2017 3:00 am

Re: Overrun when doing sync meta transfers TX and RX

Post by Bcd »

So far, I did try some more things I had on my list:

- I powered bladeRF with an external power 5V/2A (changed J70) - same result
- changed the USB cable with another one - same result
- I installed the new FPGA 0.7.1 - same result.
Though, I could notice the some differences between 0.6.0 and 0.7.1 although I don't think they are important:

0.6.0
RX operation duration 43
TX operation duration 35

0.7.1
RX operation duration 380
TX operation duration 380

I even installed Quartus and built the FPGA from scratch. I assume that I need the JTAG USB cable in order to try TimeQuest as @bglod mentioned above? Or not?

- reinstalled the FX3 firmware - same result.

In all cases there is always the same temperature dependency as reported above.

Any ideas to try out?
Bcd
Posts: 14
Joined: Tue Jun 06, 2017 3:00 am

Re: Overrun when doing sync meta transfers TX and RX

Post by Bcd »

Ok, yesterday I tried the TimeQuest option in Quartus as @bglod mentioned in a post above.

It uses the Slow 1200mV 85C, Slow 1200mv 0C, Fast 1200mV 0C models.

Although there are several failing paths with a negative slack there is just one that appears only at higher temperature, see below:

+--------------------------------------------------------------------------------+
; Slow 1200mV 85C Model Setup Summary ;
+-------------------------------------------------------+--------+---------------+
; Clock ; Slack ; End Point TNS ;
+-------------------------------------------------------+--------+---------------+
; lms_sclk_pin ; -4.705 ; -9.386 ;
; lms_rx_clock_out ; -3.549 ; -3.549 ;
; U_fx3_pll|altpll_component|auto_generated|pll1|clk[0] ; -0.021 ; -0.021 ;
; fx3_virtual ; 0.229 ; 0.000 ;
; U_pll|altpll_component|auto_generated|pll1|clk[0] ; 0.560 ; 0.000 ;

....

+--------------------------------------------------------------------------------+
; Slow 1200mV 0C Model Setup Summary ;
+-------------------------------------------------------+--------+---------------+
; Clock ; Slack ; End Point TNS ;
+-------------------------------------------------------+--------+---------------+
; lms_sclk_pin ; -4.345 ; -8.646 ;
; lms_rx_clock_out ; -3.629 ; -3.629 ;
; U_fx3_pll|altpll_component|auto_generated|pll1|clk[0] ; 0.031 ; 0.000 ;
; fx3_virtual ; 0.913 ; 0.000 ;
; U_pll|altpll_component|auto_generated|pll1|clk[0] ; 1.678 ; 0.000 ;

And here is the output form the 'worst case scenario' on Slow 1200mV 85C:
-0.021 fx3_ctl[5] fx3_gpif:U_fx3_gpif|can_tx fx3_virtual U_fx3_pll|altpll_component|auto_generated|pll1|clk[0] 10.100 1.428 3.285 1
jbassey
Posts: 1
Joined: Thu Nov 12, 2020 8:25 am

Re: Overrun when doing sync meta transfers TX and RX

Post by jbassey »

Bcd wrote: Wed Dec 13, 2017 9:44 am So I add a modified sync_rxtx.c without prints on the console, for those who want to test:

https://we.tl/Givbyq11vL

and also example_common.h - it defines the samplerate and frequencyes:

https://we.tl/YNyqv0g3QS

and example_common.c

https://we.tl/oYCQfrdyPo
Can I ask how exactly you run the example sync_rx_meta.c example file? The boilerplate.c example seems to work but the sync_rx_meta-c gives errors regarding header files.
Post Reply