Metadata

Discussions related to embedded firmware, driver, and user mode application software development

Moderator: robert.ghilduta

Post Reply
flo003
Posts: 9
Joined: Tue Sep 15, 2015 9:56 am

Metadata

Post by flo003 » Mon Sep 21, 2015 4:09 pm

Hi,

I'm trying to figure out how the metadata works. As seen in the metadata.h :
https://github.com/Nuand/bladeRF/blob/m ... .h#L22-L76
the data from the FPGA to de FX3 is "packed" in messages, these containing a header and 2k -16 bytes of samples.

I'm interested in answering the following questions:
1 - Is the metadata used by the FX3? Is the data provided by the FX3 to the host modified in any way?
2 - Can I capture data with metadata? (using the bladeRF-cli interface or Osmocom? )
3 - I'm interested in using the metadata for my application: I noticed the flags are constant and I'm thinking in replacing them with other useful info for my application. Could this generate any problems?
4 -I found that in the TAMER block inside the FPGA the timestamps are generated but there is one timestamp for one message, I don't understand to which of the samples corresponds.


Thanks in advance.
Florencia

jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata

Post by jynik » Wed Sep 23, 2015 8:14 pm

Hi Florencia,

Here are some comments that should hopefully answer your questions. Let me know if anything is unclear.

1 - Is the metadata used by the FX3? Is the data provided by the FX3 to the host modified in any way?

All sample data is moved Host <--> FPGA through the FX3 via DMA transactions on the GPIF interface. As such, the ARM microprocessor does not "see" or modify any of this data. If you are planning on making modifications to the metadata, the FX3 firmware most likely would not need any changes.

2 - Can I capture data with metadata? (using the bladeRF-cli interface or Osmocom? )

Right now, this metadata really only consists of a timestamp. This is used in the libbladeRF synchronous interface to RX/TX samples at a specific time. On the RX side, this also gives us the ability to detect and report overruns.

Depending on how and what you want to record in terms of both timestamps and data, you might want to write a simple C program to start. This libbladeRF API doc section should help you get started.

3 - I'm interested in using the metadata for my application: I noticed the flags are constant and I'm thinking in replacing them with other useful info for my application. Could this generate any problems?

The flags in the API (e.g., BLADERF_META_FLAG_RX_NOW, BLADERF_META_FLAG_TX_UPDATE_TIMESTAMP) are actually not passed down to the FPGA and are only used in libbladeRF's sync.c.

If I remember correctly, we're not actively using any of the flag bits in the metadata header right now. So, in theory you could customize these now without problem, so long as you keep an eye on whether we make changes there in the future. (I would make note of them in that metadata.h header.)

You may also want to consider using the reserved bytes in the metadata header, and just keep an eye on whether we re-assign that in the future. Currently, there are no plans to make changes to the aforementioned fields anytime soon.

4 -I found that in the TAMER block inside the FPGA the timestamps are generated but there is one timestamp for one message, I don't understand to which of the samples corresponds.

As I alluded to earlier, samples are transferred to/from the host in fixed-size blocks (or "messages" as we say in the metadata.h file). The size of this chunk comes from the DMA transaction sizes, which depends on whether we're using USB 2.0 or USB 3.0.

The timestamp counter value T is associated with the first sample in the message, with the timestamp value being a free-running counter operating at the configured sample rate. The timestamps within the message are guaranteed to be contiguous. Said another way, the 53rd sample in a message will be associated with time T + 53.

Hopefully that clarifies some things!
- Jon

flo003
Posts: 9
Joined: Tue Sep 15, 2015 9:56 am

Re: Metadata

Post by flo003 » Tue Oct 06, 2015 4:48 pm

Hi Jon, thanks for your response.
I still have some doubts. Let me explain what we are thinking in doing so you have a better idea of where I'm aiming to. In my university there's an investigation group that has developed an ISDBT receiver in GNU Radio: https://github.com/git-artes/gr-isdbt. Our goal is to develop three blocks in the Blade's FPGA that are equivalent to the first three blocks in the ISDBT current project (they are part of what its's called the inner receiver) and make it transparent to the following blocks in GNU Radio.

These blocks are in charge of frequency and time correction and also in performing an FFT to the OFDM symbol. These means that the output of our system will have a delimited length (the length of an OFDM symbol) and it is crucial to the application to have a way of identifying them in the FPGA - USB - GNU Radio transmission.
Having these said, we want to add metadata to each message transferred from the FPGA to the host identifying in case there's a new symbol, where it starts. (Bear in mind that an OFDM Symbol is in this case in a worst scenario 8k which means you need 16 and .13 messages to transmit a complete symbol).

1) We need to receive metadata in the PC, by metadata I mean the 16 byte header of each message:

* +-----------------+
* 0x00 | Reserved | 4 bytes
* +-----------------+
* 0x04 | Timestamp | 8 bytes, Little-endian uint64_t
* +-----------------+
* 0x0c | Flags | 4 bytes, Little-endian uint32_t
* +-----------------+

In particular, the flags or the reserved bytes.

We have made some tests, modifying the OSMOCOM Source. We successfully received TIMESTAMPS but in the FLAGS' bytes instead of receiving x"12344321" as we expected, we received (x"000CFFFF").
In fifo_writer.vhd file the 78's line :
meta_fifo_data <= x"FFFFFFFF" & std_logic_vector(timestamp) & x"12344321";
If the metadata is not modified in the FX3 do you have any ideas why are we receiving something different in the FLAGS' bytes?

2) Assuming we could receive metadata not modified with the OSMOCOM Source Block, we are thinking in modifying the FLAGS x"12344321" or RESERVED Bytes x"FFFFFFFF" from the fifo_writer.vhd. Do you use any of these bytes?

3) We would also like to know if we could modify the timestamps i the FPGA. Is these information used in something else than the synchronisation?

Thanks a lot.
Florencia

jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata

Post by jynik » Wed Oct 07, 2015 8:00 am

Hi Florencia,

What have you modified in libbladeRF and/or gr-osmosdr to receive these timestamps? If it's possible for you to share your changes with us, I think we could take a quick look and identify the source of the issue.

(1) After a quick look at how libbladeRF is copying data from the metadata header to the struct bladerf_metadata fields, I think there are a couple bugs and incomplete portions here. Since we do not use those fields, we've overlooked defects here I think. If I am correct, with your changes, I think we can open an issue tracker item to test and fix this.

(2) We are not currently using the flags or reserved words, or have any near-term plans to use them. If and when we change them, we will be sure to highlight this in the release notes so you can be aware of it.

(3) On the receive side, the timestamps are simply applied; they are not used anywhere else. On the transmit side, they are used to trigger an interrupt in the case where scheduled transmissions are being used.

flo003
Posts: 9
Joined: Tue Sep 15, 2015 9:56 am

Re: Metadata

Post by flo003 » Thu Oct 15, 2015 12:55 pm

Hi Jon, thank you for your response.
We have been working some more..

The modification in the OSMOCOM Source block was made in the bladerf_source_c.cc:

Code: Select all

 if (_use_metadata) {
....
    memset(&meta, 0, sizeof(meta));
    meta.flags = BLADERF_META_FLAG_RX_NOW;
    meta_ptr = &meta;
  }
  //
  ///////////////////////////////////////////////////////
  // Set GPIO                                                              //
  // 0x00010257 : Enable timestamp + Count32           //
  ///////////////////////////////////////////////////////
  ret = bladerf_config_gpio_write(_dev.get(), 0x00010257);
  // Chequear state
  if ( ret != 0 ) {
    std::cerr << _pfx << "bladerf_config_gpio_write: "

...
    
With this modification we were able to receive metadata exactly as it was sent from the FPGA. The problem we were having before was on how to interpret the bytes received (as GnuRadio delivers the data in gr-complex).

Do you think there is a better way to receive metadata? A better alternative to the code provided before?
We tried setting the GPIO in the BladeRF-CLI but the bit that enables timestamps was erased as we saw when reading the GPIO status after capturing data with the OSMOCOM with no modification.

Thanks again
Florencia

jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata

Post by jynik » Sun Oct 18, 2015 12:53 pm

Hi Florencia,

Sorry for the delay - I haven't forgot about your questions...just been kept very busy! Thank you for your patience.

First, let me revisit one of your earlier observations:
florencia wrote: We have made some tests, modifying the OSMOCOM Source. We successfully received TIMESTAMPS but in the FLAGS' bytes instead of receiving x"12344321" as we expected, we received (x"000CFFFF").
In fifo_writer.vhd file the 78's line :
meta_fifo_data <= x"FFFFFFFF" & std_logic_vector(timestamp) & x"12344321";
I have confirmed this behavior and we're looking into it. I did so by placing a breaking here and inspecting the header (the first 16 bytes of s->meta.curr_msg). This data was:

Code: Select all

(gdb) x/16bx s->meta.curr_msg
0x7ffff7f8b010: 0x21 0x43 0x34 0x12 0x04 0x00 0x00 0x00
0x7ffff7f8b018: 0x00 0x00 0x00 0x00 0x0c 0x00 0xff 0xff

Which we interpret as....
Reserved  (u32): 0x12344321
Timestamp (u64): 0x0000000000000004
Flags     (u32): 0xffff000c
So yes...something appears to not be right here, which we'll have to figure out and get back to you on.

florencia wrote:We tried setting the GPIO in the BladeRF-CLI but the bit that enables timestamps was erased as we saw when reading the GPIO status after capturing data with the OSMOCOM with no modification.
The default state of the bladeRF is "no timestamps." The reason you see this behavior is that when the RX and/or TX modules are disabled when closing the device, the timestamps are disabled again to place the device back into its default state. This effectively resets the time tamers, allowing them to start back at 0 when you enable and use them again.
florencia wrote: The problem we were having before was on how to interpret the bytes received (as GnuRadio delivers the data in gr-complex).
Do you think there is a better way to receive metadata? A better alternative to the code provided before?
I'm a little unclear about you mean here. If you're modifying gr-osmosdr, you're interacting with libbladeRF. I assume you're modifying libbladeRF to extract the data you'll be placing into the reserved and flags fields?

If you're looking to pass along your custom metadata to other blocks from the gr-osmosdr source, you might want to look at placing this data in stream tags.

jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata

Post by jynik » Sun Oct 18, 2015 1:15 pm

Hi Florencia,

After some more digging, we see that this change seems to indicate that the GPIF code is manipulating some of that metadata on us.

Pending some further discussion next week, I think we're going to back that change out because:
(1) We don't want the GPIF module manipulating metadata.
(2) We are not currently using this TX undderrun notification. However, I'd like to remove this and re-implement it in a manner that doesn't couple RX and TX like this. (i.e., one should be able to determine that there is a TX underrun when running half duplex.)

Whatever is decided, I will keep you informed. For the time being, perhaps you could manually back out this change in your custom FPGA image, or simply avoid using those bits.

flo003
Posts: 9
Joined: Tue Sep 15, 2015 9:56 am

Re: Metadata

Post by flo003 » Tue Oct 20, 2015 12:29 pm

Hi Jon, thanks for your response.
Thanks for the GPIF explanation. We had seen that the GPIF changed some bits of the RESERVED BYTES and we are thinking in not using them for the time being. But it's useful to know in case it's needed we can count on those extra modified bits.

In reference to:
I'm a little unclear about you mean here. If you're modifying gr-osmosdr, you're interacting with libbladeRF. I assume you're modifying libbladeRF to extract the data you'll be placing into the reserved and flags fields?
We only modified the gr-osmosdr. We are obtaining the data from the reserved and flags' fields by receiving a message and inspecting the first bytes. This means the metadata is received as part of the rest of the samples.
After receiving with this block we are planning on implementing a new block on GNURadio to process data depending on the metadata bytes' value in order to handle data to the following blocks on the project.

Thanks again,
Florencia

jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata

Post by jynik » Fri Oct 23, 2015 3:53 pm

Hi Florencia,

I'm glad that clarifies some things. I apologize for accidentally leading you astray earlier.

If possible, we'd love to hear how things progress. Cheers!

Jaco
Posts: 28
Joined: Wed Jul 30, 2014 2:03 am

Re: Metadata

Post by Jaco » Fri Nov 13, 2015 2:51 am

From going over the libbladerf documentation regarding the functioning of the metadata, specifically to schedule TX/RX, it seems that it should be possible to schedule TX/RX routines that are dependent on each other. Am I correct in assuming this? E.g. TX a 2ms pulse every 10ms, and RX for the 8ms downtime.

jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata

Post by jynik » Sun Nov 15, 2015 3:47 pm

Hi Jaco,

I believe it should be possible to have the RX and TX counters synchronized +/- 1 clock cycle.

There is a single timestamp enable bit from the FX3 -> FPGA that is turned on with the first bladerf_sync_config() call that enables timestamps. This implies that you'll want to have the sample rates already setup up before your bladerf_sync_config() calls (for either module), otherwise your counter would have drifted since it was operating at a different rate for some amount of time.

It sounds like you're working on some sort of TDD application? If your uplink and downlink are on the same frequency, I would advise you to ensure the RX and TX frequencies on the bladeRF are tuned at least 1 MHz apart, and digitally mix to the target frequency. I say this because we've found that if the PLLs in the LMS6002D are tuned within 1 MHz of one another, they may begin interfering with one another.

Fortunately, with 28 MHz of RF bandwidth, there's plenty of room to simply mix either RX or TX. :)

Best regards,
Jon

Jaco
Posts: 28
Joined: Wed Jul 30, 2014 2:03 am

Re: Metadata

Post by Jaco » Sun Nov 15, 2015 11:48 pm

Hi Jon, thanks for your reply.
jynik wrote: I believe it should be possible to have the RX and TX counters synchronized +/- 1 clock cycle.
If I remember correctly, the FPGA clock runs at 100 MHz, is that correct?
jynik wrote: There is a single timestamp enable bit from the FX3 -> FPGA that is turned on with the first bladerf_sync_config() call that enables timestamps. This implies that you'll want to have the sample rates already setup up before your bladerf_sync_config() calls (for either module), otherwise your counter would have drifted since it was operating at a different rate for some amount of time.
So, a call to bladerf_sync_config() will start the free-running counter from which the timestamps are derived? Does this mean that the TX and RX counters are inherently offset by the time it takes for the bladerf_sync_config() to complete for each module?
jynik wrote: It sounds like you're working on some sort of TDD application? If your uplink and downlink are on the same frequency, I would advise you to ensure the RX and TX frequencies on the bladeRF are tuned at least 1 MHz apart, and digitally mix to the target frequency. I say this because we've found that if the PLLs in the LMS6002D are tuned within 1 MHz of one another, they may begin interfering with one another.
I'm developing a simple DRFM, but to do so I need a test radar. I have developed one before, but this was before metadata was introduced into libbladeRF, so there were some issues with TX/RX synchronization. What exactly do you mean by the PLLs interfering with one another?

jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata

Post by jynik » Mon Nov 16, 2015 12:43 pm

Hi Jaco,

Hopefully my below comments clarify things for you.

Best regards,
Jon
Jaco wrote:If I remember correctly, the FPGA clock runs at 100 MHz, is that correct?
The clock cycle here is the time stamp clock running at the configured sample rate, not a master FPGA clock.
Jaco wrote:So, a call to bladerf_sync_config() will start the free-running counter from which the timestamps are derived?
Yes. As of FPGA v0.3.x there are separate free-running counters for RX and TX. However, they are both enabled via the same bit for reverse compatibility and for the reason below.
Jaco wrote:Does this mean that the TX and RX counters are inherently offset by the time it takes for the bladerf_sync_config() to complete for each module?
No, because the first bladerf_sync_call() will set the "timestamp enable" bit. Both counters will begin running and should be synchronized within +-1 clock of one another. This offset is fixed and will not change while they are running. It is simply the result of when the enable signal is seen by the time tamers, due to the crossing of clock domains.

Again, it's important to have the samplerates configured prior to calling bladerf_sync_config(). Failing to do so will cause a timestamp slip.

Jaco
Posts: 28
Joined: Wed Jul 30, 2014 2:03 am

Re: Metadata

Post by Jaco » Mon Nov 16, 2015 3:24 pm

Hi Jon,

Thanks, your comments are indeed extremely useful.

So essentially, I don't have to worry about individually configuring module timestamps since they're synchronized within +/- 1 clock cycle through the way they're defined. That also means that it doesn't matter whether I wait for a particular timestamp between RX or TX, since they both are synchronized, making it much simpler to schedule TX and RX bursts based on predetermined time intervals.

I'll update this thread if I come across anything interesting.

Sincerely,
Jaco

jynik
Posts: 455
Joined: Thu Jun 06, 2013 8:15 pm

Re: Metadata

Post by jynik » Wed Nov 18, 2015 8:29 am

Hi Jaco,

It sounds to me like we're on the same page! If you find anything to not be operating as I've noted, let me know and I'll look into it.

One "fine print" detail to be aware of is sample-time ticks between something being scheduled to transmit from the FPGA's FIFO and the sample-time tick at which it actually leaves the LMS6 front end.

This may be totally negligible for your application, but I just wanted to make note of if in case you attempt to do some very tight timing. If so, you could try to experiment with quantifying this by putting the device into one of the RF loopback modes and sending some pulses.

Best of luck! If you're able to share info about your successes on this, I'd love to hear what interesting work you're doing (and I'm sure the community would too)!

Cheers,
Jon

Post Reply