Timing out

Having issues with the site, hardware, source code, or any other issues?
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Timing out

Post by blkbx »

I had the board working and showing spectrograms in gnuradio-companion...but now whenever I attempt to run the flowgraph, I just get an error:
bladerf_enable_module has returned with -6
Failed to read samples: File or device I/O failure

Further poking around has revealed that in the bladerf_cli program I can load the FPGA, but whenever I try to read any values from it there is a huge lag (seconds) between issuing the command and getting a response, and doing 'print gpio' just times out all the time now.

Any ideas on why this is happening would be appreciated. Thanks.
bpadalino
Posts: 303
Joined: Mon Mar 04, 2013 4:53 pm

Re: Timing out

Post by bpadalino »

Does this persist even after reboots or USB resets (using the reset button)?
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Re: Timing out

Post by blkbx »

Yes, it does.
bpadalino
Posts: 303
Joined: Mon Mar 04, 2013 4:53 pm

Re: Timing out

Post by bpadalino »

Wow that is interesting. Can you try a different USB port? Did anything else change in the setup?

Does dmesg say anything?
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Re: Timing out

Post by blkbx »

I moved it to a USB 2.0 port (the two USB3 ports are on the same hub, so switching between them doesn't change anything), and now it's responding quickly and printing the output to print gpio. I guess I need to investigate what's going on with the USB3 port. I don't think anything has changed, so I don't know why it would suddenly stop working on that port.
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Re: Timing out

Post by blkbx »

Although I still get the
bladerf_enable_module has returned with -6
Failed to read samples: File or device I/O failure
when I try to run the flowgraph.
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Re: Timing out

Post by blkbx »

So, in an effort to resolve this, I loaded up a fresh install of Ubuntu and followed all the installation instructions again. Still get timeout errors on USB3 when doing "print gpio" and setting anything using the "set" command takes seconds for it to generate feedback.

At this point I can receive samples on USB2 using both the command line program and osmocom source in GRC _immediately_ after a reboot, but then any subsequent attempts fail. USB3 still seems to be a no-go for some reason.

GRC Error on USB2 (although I don't consistently get the get_center_freq error. I do always get the -6 error):
Using nuand LLC bladeRF #2 SN 0000000000000000 FW v0.3 FPGA v0.0
bladerf_enable_module has returned with -6
Traceback (most recent call last):
File "/apps/bladeRF/linux/apps/bin/top_block.py", line 82, in <module>
tb = top_block()
File "/apps/bladeRF/linux/apps/bin/top_block.py", line 51, in __init__
self.osmosdr_source_0.set_center_freq(1000e6, 0)
File "/apps/smokey64/lib/python2.7/dist-packages/osmosdr/osmosdr_swig.py", line 1160, in set_center_freq
return _osmosdr_swig.source_sptr_set_center_freq(self, *args, **kwargs)
RuntimeError: get_center_freq failed to get center frequency, error -3
Failed to read samples: File or device I/O failure
bpadalino
Posts: 303
Joined: Mon Mar 04, 2013 4:53 pm

Re: Timing out

Post by bpadalino »

We made a change to the kernel driver tonight which resolved a timeout issue. Give the new kernel driver a try and see what happens for you.

Essentially what happens in the chain is the host puts together a request to talk to a peripheral down stream (GPIO port of the NIOS, Si5338 I2C port, VCTCXO trim DAC SPI port, or the LMS SPI port) and sends the request to the FX3 on an interface which does a DMA transfer over the built in UART. The UART is connected to the NIOS soft processor in the FPGA which processes the request and sends a result back. Currently this UART is programmed at 115.2kbps. The maximum speed the UART can go is around 4Mbps which we will be setting it to in the future. Since the data is generally long lived and not copious amounts, this type of connection didn't seem to be an issue - unless you only give the kernel driver 1ms to respond, and 4 tries.

The reason this worked beforehand was due to the device initialization setting 0x57 on the GPIO versus 0x51 on the GPIO - where bit 1 and bit 2 control the RX and TX enable portion of the LMS6002D.

The tl;dr version of this - give the new kernel version a shot and please report back. Sorry for the inconvenience.
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Re: Timing out

Post by blkbx »

I updated the kernel driver on my computer and there is no change in behavior.

I also tried it with another computer (which only has USB2) and it acts exactly the same as on my computer (when running on USB2).
I've noticed that when I attempt to run the commands to save rx samples from the command line, it reports a state of running, but receives no samples. When I do a "rx stop" then the device becomes inaccessible until I unplug and replug it into the port (it still shows up under /dev, but running "probe" returns nothing).

Is it possible there's something wrong with my board in particular and not the computers? It seems like if it were the drivers other people would be having similar issues.

This is what dmesg is showing on the new computer:
[259706.920065] usb 2-4: new high-speed USB device number 3 using ehci_hcd
[259707.053066] usb 2-4: New USB device found, idVendor=1d50, idProduct=6066
[259707.053070] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[259707.053072] usb 2-4: Product: bladeRF
[259707.053074] usb 2-4: Manufacturer: Nuand
[259707.067248] nuand_bladerf 2-4:1.0: Nuand bladeRF device is now attached
[259715.038455] nuand_bladerf 2-4:1.0: Failed to read peripheral response
[259715.042572] nuand_bladerf 2-4:1.0: Failed to read peripheral response
[259792.752102] usb_control_msg(ptr=ffff8801bc397ec4) len=4
[259792.752340] done usb_control_msg() = 4
[259809.104094] nuand_bladerf 2-4:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -110, 3 tries left
[259810.104095] nuand_bladerf 2-4:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -110, 2 tries left
[259811.104097] nuand_bladerf 2-4:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -110, 1 tries left
[259963.552265] nuand_bladerf 2-4:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -110, 3 tries left
[259964.552125] nuand_bladerf 2-4:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -110, 2 tries left
[259965.564456] nuand_bladerf 2-4:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -110, 1 tries left
piranha32
Posts: 26
Joined: Fri Mar 01, 2013 6:25 am

Re: Timing out

Post by piranha32 »

blkbx wrote:I updated the kernel driver on my computer and there is no change in behavior.

I also tried it with another computer (which only has USB2) and it acts exactly the same as on my computer (when running on USB2).
I've noticed that when I attempt to run the commands to save rx samples from the command line, it reports a state of running, but receives no samples. When I do a "rx stop" then the device becomes inaccessible until I unplug and replug it into the port (it still shows up under /dev, but running "probe" returns nothing).

Is it possible there's something wrong with my board in particular and not the computers? It seems like if it were the drivers other people would be having similar issues.
I have an identical problems with my board. I managed to get the board to work with osmocom_fft the first time I uploaded the firmware files to FX3 and FPGA, but after reset the board stopped sending samples and it never worked again. The patch fixing the delay problem did not help. Maybe rebooting the computer would help, but I haven't tried it yet.

j.
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Re: Timing out

Post by blkbx »

I finally was able to receive samples using USB2 (USB3 still fails) with the osmocom source block in GRC. It seems to work when setting the DC Offset Mode to Automatic and the Gain Mode to Automatic (at least when I set them to other options, I get the failure to open device error).

On USB3 I still get the same errors, receive failure, and command response lag in the CLI program as in my previous attempts and attempting to run the GRC flowgraph results in dmesg output of:
[37129.556568] nuand_bladerf 4-2:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -71, 3 tries left
[37130.553922] nuand_bladerf 4-2:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -110, 2 tries left
[37131.553704] nuand_bladerf 4-2:1.0: Error in __bladerf_rcv_cmd calling usb_control_msg() with error -110, 1 tries left
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Re: Timing out

Post by blkbx »

I installed the bladeRF on another computer with USB 3.0 and a fresh Ubuntu install. The delay I was experiencing on my other computer when using USB3 where doing the 'print' command in the CLI program would either take many seconds to print a response or would fail entirely is gone (in the sense that now prints immediately). No idea if this is because of the updated software or just different hardware.

However, attempting to get samples over USB3 is still a no go, and the behavior of the bladeRF becoming inaccesible after attempting to receive samples over USB3 is still present. The dmesg output seems a bit different than before.

In one case, running a simple flowgraph in GRC, the output from the flowgrapsh was "Failed to read samples: File or device I/O failure", and the dmesg the output was:

Code: Select all

[ 1176.609598] usb 9-4: new SuperSpeed USB device number 6 using xhci_hcd
[ 1176.627856] usb 9-4: Parent hub missing LPM exit latency info.  Power management will be impacted.
[ 1176.631678] usb 9-4: New USB device found, idVendor=1d50, idProduct=6066
[ 1176.631690] usb 9-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 1176.631698] usb 9-4: Product: bladeRF
[ 1176.631704] usb 9-4: Manufacturer: Nuand
[ 1176.631961] xhci_hcd 0000:02:00.0: xHCI xhci_add_endpoint called with enabled ep ffff8800b9ae06c0
[ 1176.631969] xhci_hcd 0000:02:00.0: xHCI xhci_add_endpoint called with enabled ep ffff88009839c940
[ 1176.646108] nuand_bladerf 9-4:1.0: Nuand bladeRF device is now attached
[ 1196.271983] usb_control_msg(ptr=ffff88008d063eb4) len=4
[ 1196.272824] done usb_control_msg() = 4
[ 1200.201772] usb_control_msg(ptr=ffff88010ed45e84) len=4
[ 1201.198220] done usb_control_msg() = -110
[ 1201.198237] nuand_bladerf 9-4:1.0: Error in __bladerf_snd_cmd calling usb_control_msg() with error -110, 3 tries left
[ 1201.198245] usb_control_msg(ptr=ffff88010ed45e84) len=4
[ 1201.199124] xhci_hcd 0000:02:00.0: URB transfer length is wrong, xHC issue? req. len = 4, act. len = 4294967292
[ 1201.199173] done usb_control_msg() = 0
Another thing I've noticed is that if I enter the CLI program, attempt to 'set GPIO 0x53' then 'print gpio' returns:

Code: Select all

bladeRF> print gpio

  GPIO: 0x00000053

    LMS Enable:         Enabled   
    LMS RX Enable:      Enabled   
    LMS TX Enable:      Disabled  
    TX Band:            Low Band (300M - 1.5GHz)
    RX Band:            Low Band (300M - 1.5GHz)
but then if I exit the program, reenter it and do a 'print gpio' I get:

Code: Select all

bladeRF> print gpio

  GPIO: 0x00000051

    LMS Enable:         Enabled   
    LMS RX Enable:      Disabled  
    LMS TX Enable:      Disabled  
    TX Band:            Low Band (300M - 1.5GHz)
    RX Band:            Low Band (300M - 1.5GHz)
Is that the expected behavior, or should it return what I had previoustly explicitly set the gpio to? If its the latter, it seems as though my commands aren't making it to the board?
blkbx
Posts: 16
Joined: Sat Jul 27, 2013 6:44 pm

Re: Timing out

Post by blkbx »

Aaaaaannnddd immediately after my previous post I try to receive samples and it suddenly starts working.
keyboardgnome
Posts: 2
Joined: Thu Aug 15, 2013 7:52 pm

Re: Timing out

Post by keyboardgnome »

I'm having the same problem. I've recompiled the kernel module; no love there. Rebooted. Tried different USB ports....

Between when you had your last error and for it to work again, what was different?
ekoloski
Posts: 6
Joined: Fri Mar 01, 2013 5:33 pm

Re: Timing out

Post by ekoloski »

I also experienced this issue when running self compiled images, I was able to resolve it by loading one of the precompiled FPGA images from http://nuand.com/fpga/
Post Reply