gr-dvbt Issues

[postman]

Hi all,

I have been having issues getting gr-dvbt up and running. I am hoping it is merely a configuration issue, but I have tried a number of permutations of tests and mod/demod parameters to no avail. I am currently using this file (http://www.w6rz.net/adv8dvbt12.ts) from the gr-dvbt project thread(http://nuand.com/forums/viewtopic.php?f=8&t=3499), yet I am unable to play it back after processing it despite it appearing to match the modulation parameters. My test process is as follows, and the flowgraphs in question are provided here:

Mod: http://pastebin.com/S43trdh2
Demod: http://pastebin.com/BGUMYpiw

1. stream video through dvbt_tx_demo.grc to a binary file. Video and binary are read from and written to a ramdisk to reduce latency. samp_rate is set to 12.0642e6, the bitrate of the ts file. The grc flowgraph is stock except for a throttle set to the samp_rate that I inserted in between the file source and the energy_dispersal block so as to keep it from overloading the CPU. The 16QAM constellation and FFT both appear to be valid, with all 16QAM points appearing stable and in range and the FFT centered around 1.2GHz with 8MHz of bandwidth.

2. stream binary through dvbt_rx_demo.grc to a ts file. A ramdisk is again used in this case. Again the samplerate is set to 12.0642e6, and two throttles set are placed on the inputs (File Source and Noise Source) and set to samp_rate. The constellation looks good, with all 16QAM symbols stabilizing between -1 and 1 for both I and Q. However it may be worth noting that the WX GUI FFT Sink coming just after the Multiply Const block shows only the noise floor, with no discernible signals appearing between DC and 1.2GHz (the ranges I manually looked at).

3. play the ts file back over VLC. No video or audio is rendered, and I see the following errors in the log when I run it with the maximum logging verbosity enabled (pastebin here: http://pastebin.com/H10na4wR). As you can see, the TS PIDs are all over the place...

* note my PC is a i7 4770K with 16G/RAM, so there should be plenty of processing power available. Also, I am on Ubuntu 13.10 and gnuradio 3.7.2.1.

I am at a loss for what may be going on. Do you have any ideas? For starters, does my input file and configuration look valid? Please let me know if I can provide any additional information. Thanks for your time!

[drmpeg]

Here are a few tips.

1) The DVB-T sample rate can be calculated with the following equation. sample rate = channel size * 8000000 / 7. So for the usual channel sizes of 6 , 7 and 8 MHz, the sample rate will be:

6 MHz = 6.857142 Msps
7 MHz = 8.000000 Msps
8 MHz = 9.142857 Msps

2) The Rational Resampler block should be deleted (in both TX and RX). It's there for a USRP N210. With bladeRF, it's not required at all.

3) The throttle blocks should be placed in the same spots that the deleted resampling blocks were in. That is, right before the output to the file sink for TX and after the Add block for RX.

4) The Multiply Constant block is not really needed for RX. You can delete it, or set it to 1.

Finally, I have a Transport Stream tool that you can use to check the files from the receiver. It's at:

http://www.w6rz.net/xport.zip

The zip file contains the source for Linux. Compile it with:

gcc -O2 -D_FILE_OFFSET_BITS=64 xport.c - o xport

The syntax is:

xport <-pavtdszgher2> <infile> <program number> <video stream number> <audio stream number>

Since all the test streams have program number 1, the usual command line is:

xport -p <infile> 1 1 1

Give it a try with the test file.

xport -p adv8dvbt12.ts 1 1 1

The -p option supresses the demux. Otherwise xport will extract the elementary streams and place them in the files bits0001.mpv and bits0001.mpa.

Ron

[drmpeg]

I just upgraded to Ubuntu 13.10 today, and a weird bug crept into gr-dvbt. For some reason, this line of C++ code in energy_dispersal_impl.cc compiles differently from gcc 4.7.3 to gcc 4.8.1.

out[count++] = in[index + count] ^ clock_prbs(d_npacks);

Somehow the "count" variable gets hosed up. I have a patch here:

http://www.w6rz.net/energy_dispersal_impl.cc

I'm running the 32-bit version of Ubuntu 13.10, so this issue may only apply to the 32-bit distro.

Ron

[postman]

Hi Ron,

Thanks for the thorough response. I have tried several approaches based on your advice yet have not been successful.

First I made the recommended changes to the flowgraphs (new pastebins here http://pastebin.com/4BfXk82Z and here http://pastebin.com/RSAp1r4Y). Tx and Rx both run, although there are two constellation points that are greater than 1 (is this okay? http://i.imgur.com/mUllRHg.png). xport confirms that the resulting ts file has bogus PIDs - see xport output (https://drive.google.com/file/d/0B9UP0p ... sp=sharing), and the ts file itself (https://drive.google.com/file/d/0B9UP0p ... sp=sharing). The source file only has PIDs 0, 48, 49, 52, and 8191.

After that, I tried modifying the line you indicated in energy_dispersal_impl.cc. I figured this wouldn't do anything since the two lines are syntactically equivalent, however with the change in place the Rx flowgraph crashes almost immediately. Log here (http://pastebin.com/FmLXX5Uu). Constellation and FFT on the Tx side look the same as above. I am on a 64-bit machine and OS for the record.

I am at a loss for what is going on here. Does the ts file from the first case give you any hints as to what may be going on? Can I provide any other information that might be helpful?

[drmpeg]

Your flowgraphs are good. It definitely looks like you're being affected by the "syntax bug". It turns out that the very same bug is also in energy_descramble_impl.cc, so both the transmitter and receiver are affected.

It's pretty easy to test for this syntax issue with some C code.

http://www.w6rz.net/gcctest.c

Code: Select all

/*
gcc 4.8.1 compiler test
*/

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
    static unsigned char    buffer[16384];
    static unsigned char    output[16384];
    int	i, index;

    if (argc != 1) {
        fprintf(stderr, "usage: gcctest\n");
        exit(-1);
    }

    // Generate a pattern
    for (i = 0; i < 16384; i++)
    {
        buffer[i] = i & 0xff;
    }

    // Copy it
    for (i = 0; i < 16384; i++)
    {
        index = 0;
        for (i = 0; i < 16384; i++)
        {
#if 0
            output[index] = buffer[index] ^ 0;
            index++;
#else
            output[index++] = buffer[index] ^ 0;
#endif
        }
    }

    // Test copy
    for (i = 0; i < 16384; i++)
    {
        if (buffer[i] != output[i])
        {
            printf("Mismatch at %d, good = 0x%02x, bad = 0x%02x\n", i, buffer[i], output[i]);
            if (i > 10)
            {
                break;
            }
        }
    }

    return 0;
}

If you compile with -Wall, the compiler complains.

Code: Select all

re@w6rz:~/xfer$ gcc -Wall gcctest.c -o gcctest
gcctest.c: In function ‘main’:
gcctest.c:35:25: warning: operation on ‘index’ may be undefined [-Wsequence-point]
output[index++] = buffer[index] ^ 0;
^

I'm not sure why the changes are causing you problems (especially since the change in energy_dispersal_impl.cc should only affect the transmitter). Sometimes weirdness like that can be solved by a complete re-build. Like so:

Code: Select all

cd gr-dvbt/build
rm -rf *
cmake ../        /* add -DCMAKE_INSTALL_PREFIX= if you've installed GR in /opt */
make
sudo make install

So to recap, both energy_dispersal_impl.cc and energy_descramble_impl.cc need to be changed and the git diff should look like this:

Code: Select all

re@w6rz:~/xfer/gr-dvbt/lib$ git diff energy_dispersal_impl.cc
diff --git a/lib/energy_dispersal_impl.cc b/lib/energy_dispersal_impl.cc
index 1b4257f..990b48b 100644
--- a/lib/energy_dispersal_impl.cc
+++ b/lib/energy_dispersal_impl.cc
@@ -127,7 +127,10 @@ namespace gr {
               out[count++] = sync;
 
               for (int k = 1; k < d_psize; k++)
-                out[count++] = in[index + count] ^ clock_prbs(d_npacks);
+              {
+                out[count] = in[index + count] ^ clock_prbs(d_npacks);
+                count++;
+              }
 
               sync = d_SYNC;
               clock_prbs(d_npacks);
re@w6rz:~/xfer/gr-dvbt/lib$

Code: Select all

re@w6rz:~/xfer/gr-dvbt/lib$ git diff energy_descramble_impl.cc
diff --git a/lib/energy_descramble_impl.cc b/lib/energy_descramble_impl.cc
index e6a7f98..2ef8218 100644
--- a/lib/energy_descramble_impl.cc
+++ b/lib/energy_descramble_impl.cc
@@ -152,7 +152,10 @@ namespace gr {
               // PRBS clocking starts right after NSYNC
 
               for (int k = 1; k < d_bsize; k++)
-                out[count++] = in[d_index + count] ^ clock_prbs(d_nblocks);
+              {
+                out[count] = in[d_index + count] ^ clock_prbs(d_nblocks);
+                count++;
+              }
 
               // For subsequent blocks PRBS is clocked also on SYNC
               // but its output is not used
re@w6rz:~/xfer/gr-dvbt/lib$

Ron

[drmpeg]

I forgot to answer your question about the transmit constellation. The points to the side (at level +/-1) are the modulation of the TPS (Transmission Parameter Signalling) carriers. These carriers contain information about the main signal like constellation, code rates, guard interval and transmission mode. The TPS carriers are modulated with DBPSK.

The points to the side at level +/-1.333 are the continual and scattered pilot carriers. These carriers just contain a known pseudo random sequence for the receiver to acquire synchronization. They are transmitted at a higher power.

The Multiply Const block (with value 0.0022097087) takes care of limiting the IQ samples to less than 1. I'm not sure where that magic number comes from, but it's designed to work with both 8K and 2K modes. In the 2K mode, you can increase that value up to about 0.0055 to get a little more TX power.

Ron

[postman]

This did the trick -- thank you for your help! I'm up and running with gr-dvbt both in file-based simulation and on the bladeRF. The constellation explanation was helpful as well -- I'm going to brush up on the standard with this in mind.

If you don't mind my asking, how did you first identify the compiler-dependent bug in energy_*_init.cc? It seems pretty obscure. Did you have a test that picked it up?

One last question regarding TS stream formatting as it relates to DVB-T. Should the mux rate of the TS stream always be configured to equal the DVB-T sample rate?

[drmpeg]

Awesome! I discovered this issue the other day when I finally upgraded from Ubuntu 13.04 to 13.10. I also updated libboost, so I had to re-build GnuRadio and all my OOT projects including gr-dvbt. For testing, I use this internal loopback flowgraph that sends the decoded stream over UDP to another machine running VLC.

http://www.w6rz.net/dvbt_loop_demo.grc

Be aware that the above flowgraph also tests the CPU performance of the receiver, so it may not work if you don't have enough CPU (and that's why the instrumentation is disabled). If you don't have enough CPU for QAM16, try dropping to QPSK.

I could see that the re-built gr-dvbt was not working, so I used the gbdvb (http://dsplab.diei.unipg.it/software/gbdvb) tools to debug with. After decoding a gr-dvbt transmitter IQ file with gbdvb, I could see that the bytes in the TS packets were off by one. Then I used the gbdvb encoder to check for a mismatch with the gr-dvbt transmitter. The gbdvb encoder can output a file at each stage of the encode, so (to start with) I created an energy dispersal file with that. Then I added a file sink to the dvbt_tx_demo.grc flowgraph after the energy dispersal block and captured that stream. Sure enough, there was a mismatch and that's how I found the bug.

The DVB-T TS rate is a function of sample rate, constellation, code rate, guard interval and number of active carriers. I've wriiten a little utility to calculate the TS rate based on channel size.

https://github.com/drmpeg/dtv-utils/blo ... dvbtrate.c

Typical usage is:

./dvbtrate 6

Ron