Metadata & timestamps

Discussions related to embedded firmware, driver, and user mode application software development
Post Reply
jquirke
Posts: 12
Joined: Sat Oct 25, 2014 12:51 am

Metadata & timestamps

Post by jquirke »

Good Evening,

I am trying to get my head around these metadata timestamps for the purpose of building an application in which transmission timing must be tightly synchronised with receive timing.

I see that unlike the USRP/UHD, there is no mechanism to set the timestamp. However, there is a method to read the timestamp, bladerf_get_timestamp(..), which could be useful for me, if I could understand how it worked.

Looking at the "real" implementation of this function, i.e. in usb_get_timestamp, there appears to be a lack of documentation around the specifics of how this really works.

I am hoping some of the FPGA gurus can help me out with these questions I have:

1. Why are there two independent counters for RX and TX?

This seems counter intuitive for the purpose of scheduling. How can I safely establish the relationship between the two counters without timing uncertainty? I can see no way of doing this. I get that the bladerf can run independent sample rates in either direction, so part of the reason for this makes sense, but I still need to establish the relationship when both counters are running.

2. Is the read atomic when the counter is running?

The read in usb_get_timestamp appears to issue two commands to the FPGA to read 32-bit halves of the 64-bit value. When the counter is running, will this result in possible tearing if the MSbs wrap around, or does the FPGA implement a state machine to guarantee an atomic read?

3. Under what circumstances are the counters actually running? The moment I do a bladerf_enable_module()? If this is the case how can I establish RX-TX timing relationship?

4. When am I allowed to read the timestamp? In what states?
jquirke
Posts: 12
Joined: Sat Oct 25, 2014 12:51 am

Re: Metadata & timestamps

Post by jquirke »

OK, so brushing off my VHDL from years ago, I seem to believe I have answered several of these questions.

[1] Because internally counters are in samples, and samplerates differ, two counters are necessary, as explained.

[2] It seems there is a state machine in time_tamer.vhd that snapshots the counter to allow an atomic read.

[3] The counters are running as long as the signals meta_en_tx and meta_en_rx are true; these seem to be tied directly to the GPIO bit 0x16 (BLADERF_GPIO_TIMESTAMP) via the synchronizers.

[4] It seems there is nothing stopping one reading the timestamp counter at any time, whether the results are meaningful depends on the above.

So I suppose if the SI5338 clocks are always running, which I believe they are, the timestamps will be "close" right? But as far as I can tell, you can't atomically program both the TX and RX sample rate clocks together, so there is some indeterminate drift between when those clocks change their sample rate? Will need to double check, but hopefully someone knows off the bat.. Also it appears tx_reset and rx_reset reset the counters, need to find out when these events occur..
bpadalino
Posts: 303
Joined: Mon Mar 04, 2013 4:53 pm

Re: Metadata & timestamps

Post by bpadalino »

Seems like you have most of your questions answered.

Si5338 clocks are always running. If you set the samplerates before setting up your synchronous config or enabling timestamps in general, then there will be at most a single clock delay between the timestamps due to the synchronization flops leading from the NIOS clock domain to each TX or RX clock. The Si5338 also is sourced from the same VCO and will have the same configuration, so there will be no clock drift between them.

If you disable then reenable timestamps, they will both start counting from 0 and be coupled to each other.

Let me know if you have other questions or think this is a bad way to do it.

Brian
dave
Posts: 5
Joined: Thu May 21, 2015 3:36 pm

Re: Metadata & timestamps

Post by dave »

jquirke wrote:The read in usb_get_timestamp appears to issue two commands to the FPGA to read 32-bit halves of the 64-bit value...
I've programmed Atheros 802.11 chips that have a TSF clock that works similarly. As long as you have some practical bounds on how many ticks of the lower half can happen whilst you're reading the upper half, the software solution is easy enough. (Well, it may bother you on an aesthetic level to test for a rare condition in a timing-sensitive place, but it's really OK in the context of everything else that's going on. The time interval of 2^32 samples is short enough that you really do need to code for it, but long enough that the code is easy, i.e., there's no possibility of it wrapping twice while you're trying to figure out whether it wrapped once.)
Dave Aragon
University of Washington, Seattle
Senior Member, IEEE
jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata & timestamps

Post by jynik »

dave wrote: I've programmed Atheros 802.11 chips that have a TSF clock that works similarly. As long as you have some practical bounds on how many ticks of the lower half can happen whilst you're reading the upper half, the software solution is easy enough.
Mind if I ask what particular 802.11 chip that was and what the TSF clock was? Just curious to hear about your adventures, and am not familiar with the "TSF" acronym. :)

Regarding the timestamp counters on the bladeRF, I can confirm that they do register the timestamp upon initiating the read to avoid having to deal with that rollover of one half.

On some related notes...
  • I'm looking to finish up some Doxygen changes today that I hope better clarify metadata and timestamp usage from the host side. Once I've merged that into master, I'd love it if you folks could give it a read and submit feedback or pull requests via the GitHub issue tracker. The more, the merrier on doc review, right? ;)
  • bpadalino and I have been working on re-architecting and re-writing the NIOS II code to be more readable, extensible, and have some improved features. Along the way, we're trying to ensure the UART command packet formats are documented a bit better in the code. I also plan to get rid of some of the silly multiple USB transfers per access of the timestamps and configuration PIOs. It still needs some final work and cleanup, but hopefully by the end of next week I'll be moving it into the official Nuand repo as a development branch for community review and testing. I dropped a .tar.bz2 of the current snapshotin this post and have a dev-nios_refactor branch on my GitHub account.
Cheers,
Jon
Post Reply