# Correlation Estimation Block Oddity **URL:** \ **Category:** GNU Radio\ **Created:** [April 23, 2015, 1:31am UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671 "2015-04-23T01:31:19Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Detlef\_R](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/58f4c7/32.png) [@Detlef\_R](https://www.ruby-forum.com/u/Detlef_R)\ **Post date:** [April 23, 2015, 1:31am UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/1 "2015-04-23T01:31:19Z")
Hello all, I’m using GR 3.7.8. I’m trying to work the new Correlation Estimation block into my design. I’m seeing what looks like unintended behavior for the tag placements. Sometimes the corr\_start tag is placed on the same sample as the corr\_est, time\_est and phase\_est tags, but sometimes the corr\_start tag is placed a sample earlier then those three tags. I’ve attached a screenshot showing this behavior. In the same running flow graph the tag placement will go back and forth between being on the same sample or being off by one sample. As long as the corr\_start tag is always on the correct sample I guess this won’t break anything, but I figured I’d let you all know what I’m seeing. I don’t have the full design working yet so I can’t say myself that the block works. v/r, Rich ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 23, 2015, 5:19am UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/2 "2015-04-23T05:19:15Z")
Short answer: use corr\_est as your tag. The corr\_start tag is undelayed by the matched filter length, and intended for other purposes (data-aided equalizers, etc.). Andy W. can say more, but it’s enough to say the later three tags are the ones that match the peak of the correlation in the output symbol stream. See corr\_est\_cc\_impl.cc lines 206-214. –n On Wed, Apr 22, 2015 at 4:29 PM, Richard B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 23, 2015, 4:25pm UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/3 "2015-04-23T16:25:45Z")
On Wed, Apr 22, 2015 at 11:17 PM, Nick F. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > same sample or being off by one sample. As long as the corr\_start tag is > > always on the correct sample I guess this won’t break anything, but I > > figured I’d let you all know what I’m seeing. > > > > I don’t have the full design working yet so I can’t say myself that the > > block works. > > > > v/r, > > Rich Aside from what Nick said, another issue that we’ve seen in the scheduler is propagation of tags through blocks that have changing sample rates. The clock sync blocks (either the PFB or M&M) for example don’t have a strict N:M input to output ratio, but can have N:M+/-d depending on the state of the signal. Jeff L. and I worked out a mediocre solution to help with this, but it’s not perfect and we can still see the problem occurring on occasion. Generally, this settles down quickly and provides a constant offset of where the tag is relative to where it really should be, and it might move plus or minus 1 sample every now and then. The corr\_est block has the “Tag marking delay” which among other reasons can also help with this problem. The problem that I’m referring to doesn’t seem to have a simple answer, either, and I’ve challenged a few people who are good at this sort of thing to try and sort it out, but we still don’t have a solution. The “good enough” solution that’s in there so far has indeed proved to be good enough for most purposes. Tom ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 24, 2015, 12:40am UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/4 "2015-04-24T00:40:26Z")
I have another question on tag placement for the correlation estimation block. In the screenshot I’ve attached, you’ll see that the corr\_start tag is placed well before the preamble actually starts. If I use the ‘delay tag’ field to move it to the end of the preamble where I need it, it stops delaying, no matter how big I make the delay, before it reaches the end of the preamble. This is shown in the picture. I need this tag to denote the start of the header for the Header Payload Demux block. My question is, given the situation in the screenshot, is there a way I can delay just the tag in grc using a block? I’m not sure how to get the tag positioned where I need it at this point. Help is greatly appreciated v/r, Rich ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 28, 2015, 1:47am UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/5 "2015-04-28T01:47:22Z")
Andy and all, Sorry for the delay in reply, I’ve been working hard to figure things out on my end. I use the polyphase clock recovery block for timing recovery. I have essentially copied the test\_corr\_est.grc example that was included with the block in the examples directory. It seems that this might not be the appropriate way to use this block. Here is why I say this. If you look at the attached screenshot, you will see the correlation output has two peaks, somewhat close in amplitude to each other. The synchronization sequence that was used to generate those peaks was 64 bits long and composed of two 32 bit long PN sequences repeated. Therefore, one would expect the output of the correlation to generate 3 peaks, a center peak that is ~64 units high, and two side peaks spaced 32 samples apart that are ~32 units high. Now, I believe the cause of this to be the use of the modulate vector block to generate the input mask for the correlation estimation block. I have attached a second screenshot of the output of the modulation vector block. You will see in this screen shot a large transient portion that is fed into the correlation estimation block. So, if we agree we should not use the modulate vector block to feed the correlation estimation block with its mask, then the question is how should I? An example what be most helpful. This is difficult to discuss because things get wordy very quickly. Let me know if I can make anything more clear. v/r, Rich On Fri, Apr 24, 2015 at 9:11 AM, Andy W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 24, 2015, 6:13pm UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/6 "2015-04-24T18:13:31Z")
Hi Richard, > estimation block. In the screenshot I’ve attached, you’ll see that the > corr\_start tag is placed well before the preamble actually starts. OK. That’s the first thing you should try to fix. You are crossing the correlation threshold too early. Either a. Get rid of leading 0’s in the correlation sequence that you are using, b. Set the threshold on the corr\_est block to something higher than the default 0.9 (90%), or c. make the correlation sequence “more unique” somehow, e.g. make it longer. As I mentioned in my previous email, the “corr\_start” tags is placed at the length of your matched filter samples before where the correlation peak was detected. You can eyeball how the correlation is going by plotting the Magnitude^2 of the “corr” output on top of the output signal (use a tag\_gate block to block the tags coming out of the “corr” output). The output signal is delayed by the length of the matched filter, so you can see the correlation peak and the “corr\_start” tag line up. See the attached screen shot of an AIS preamble and the scaled magnitude squared of the correlator output. > If I use the ‘delay tag’ field to move it to the end of the preamble > where I need it, it stops delaying, no matter how big I make the > delay, Yup, the code forces a maximum delay. You can delay it to the end of your matched filter and no further: > The code does this because GnuRadio will not let us tag samples outside of the window of samples passed to the current call to work. By setting the bound of the marking delay to the start and end of the matched filter, with other mechanisms in the block, we are guaranteed to be able to mark the sample. > before it reaches the end of the preamble. Well as far as the block is concerned, you are able to mark to the end of the matched filter where the correlation peak occurred (but no further), which should be the end of your preamble. Your problem is that the block declared a correlation too early. > This is shown in the picture. I need this tag to denote the start of > the header for the Header Payload Demux block. Out of curiosity, what’s doing timing recovery, i.e. finding the optimal bit sampling times? I use the tagging delay out of the corr\_est block to mark the center of the first bit in the preamble, to tell downstream bit timing recovery blocks to reset/realign at that point. Regards, Andy ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 28, 2015, 1:53am UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/7 "2015-04-28T01:53:13Z")
I somehow attached the wrong correlation screenshot in my previous post. Here is the correct one. Rich On Mon, Apr 27, 2015 at 4:35 PM, Richard B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 24, 2015, 7:31pm UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/8 "2015-04-24T19:31:44Z")
On Thu, 2015-04-23 at 12:01 -0400, [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Message: 12 > Date: Thu, 23 Apr 2015 10:23:31 -0400 > From: Tom R. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > On Wed, Apr 22, 2015 at 11:17 PM, Nick F. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > > Short answer: use corr\_est as your tag. The corr\_start tag is undelayed by > > the matched filter length, and intended for other purposes (data-aided > > equalizers, etc.). Andy W. can say more, but it’s enough to say the > > later three tags are the ones that match the peak of the correlation in the > > output symbol stream. See corr\_est\_cc\_impl.cc lines 206-214. > > > > –n Ummm, well, to clarify: “corr\_start” is marked at exactly the length of the matched filter before the detected/declared correlation peak. I like to think of this tag as “the preamble I was looking for starts here”. It is useful as an indicator to reset/restart downstream Data-Aided blocks, like an LMS DA equalizer for example. “corr\_est”, “time\_est”, and “phase\_est” are always marked at exactly the user provided mark delay after “corr\_start”: > > (i is the index of the “corr\_start” tag) “corr\_est”, “time\_est”, and “phase\_est” have values that tell you something about the signal at the detected/declared correlation peak (at the length of your matched filter samples after “corr\_start”). As the engineer, you have to decide where you want that information placed relative to corr\_start. Traditionally that is the center of the first symbol in the preamble, to reset/restart timing recovery blocks downstream for example. Since the corr\_est block has no idea if you are FSK or PSK and how much ISI your symbols have, it is up to you as the engineer to set this tag marking delay. 0 is a valid value; negative values and values longer than your matched filter length are not valid. > > > I’ve attached a screenshot showing this behavior. In the same running > > > flow graph the tag placement will go back and forth between being on the > > > same sample or being off by one sample. I can’t honestly see how the corr\_est block could be emitting this directly, unless something internal to the block is trashing the d\_mark\_delay class variable. Some questions for you: 1. Is the direct output of the corr\_est block, or as Tom R. implies, is it after processing by some downstream blocks that decimate? 2. What is the tag marking delay you set on the corr\_est block? 3. What is the samples/symbol you are using and provide as input to the corr\_est block? 4. What is the length of you matched filter (how many samples\taps)? > As long as the corr\_start tag is > > > > always on the correct sample “corr\_start” is always the length of your matched filter samples ahead of the detected/declared correlation peak. Whether or not that is “right” depends on your downstream receiver design and the probability of false alarm with declaring a correlation. > Aside from what Nick said, another issue that we’ve seen in the scheduler > is propagation of tags through blocks that have changing sample rates. The > clock sync blocks (either the PFB or M&M) for example don’t have a strict > N:M input to output ratio, but can have N:M+/-d depending on the state of > the signal. Jeff L. and I worked out a mediocre solution to help with > this, but it’s not perfect and we can still see the problem occurring on > occasion. Generally, this settles down quickly and provides a constant > offset of where the tag is relative to where it really should be, and it > might move plus or minus 1 sample every now and then. IMO, if Richard set the tag marking delay to 0 and is not showing us decimated output, then i think the above factors shouldn’t matter. > The corr\_est block > has the “Tag marking delay” which among other reasons can also help with > this problem. Tag marking delay is specifically there because the block can’t guess about parameters of the demodulation, but a human can easily empirically set the parameter for the target demodulation. > The problem that I’m referring to doesn’t seem to have a simple answer, > either, and I’ve challenged a few people who are good at this sort of thing > to try and sort it out, but we still don’t have a solution. The “good > enough” solution that’s in there so far has indeed proved to be good enough > for most purposes. A little OT, but: The best I could do myself, was to time tag certain critical samples before they go through these variabl-ish rate blocks and then use time tags to sort out things later (e.g. TOA). > Tom Regards, Andy ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 28, 2015, 10:42pm UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/9 "2015-04-28T22:42:54Z")
On Tue, Apr 28, 2015 at 12:48 PM, Richard B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > When you mentioned I should remove leading or trailing zeros from the > Correlation Estimator symbols field, how would you go about doing this if > you used the modulate vector block as done in the example? I’m stuck on > this. The group delay in the pulse shaping filter (in this case, RRC) causes these leading zeros. This is something I’d like to figure out a “proper” solution for, but my quick hack that works pretty well is to duplicate the PN sequence as it goes into modulate\_vector, then take only the second half of the samples of the output for the correlation block. This then “pre-fills” the filter with data that is arguably the right spectral shape, and the resulting second-half vector works much better than the original way. Again, this is a quick hack, but it works well. ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 28, 2015, 9:50pm UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/10 "2015-04-28T21:50:18Z")
Hey Andy, When you mentioned I should remove leading or trailing zeros from the Correlation Estimator symbols field, how would you go about doing this if you used the modulate vector block as done in the example? I’m stuck on this. v/r, Rich On Mon, Apr 27, 2015 at 4:52 PM, Richard B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 29, 2015, 6:11pm UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/11 "2015-04-29T18:11:26Z")
On Wed, Apr 29, 2015 at 8:50 AM, Richard B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Thanks for replying. The question is, how do I do this? We both agree the > zeros need to go. What I’m not sure on is how to do that. I’d rather keep > this contained to GRC, because I don’t like maintaining python files not > connected to grc. Can you explain how you do this or provide an example? Since GRC parameters can be Python expressions, you can use formulas to do this. If ‘seq’ is the original sequence to be modulated, use ‘seq+seq’ in the parameter to the modulate\_vector variable block. If the variable is named ‘mod\_vec’, use: mod\_vec[len(mod\_vec)/2:] …as the taps for the correlation estimator. ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 29, 2015, 7:10pm UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/12 "2015-04-29T19:10:27Z")
On Wed, 2015-04-29 at 09:09 -0700, Johnathan C. wrote: > Since GRC parameters can be Python expressions, you can use formulas > mod\_vec[len(mod\_vec)/2:] > > …as the taps for the correlation estimator. John, Thanks. On a related note, I just tested the PSK modulator and it does generate a long lead-in of zeros. 😕 The GMSK modulator “digital.gmskmod\_bc(sps, pulse\_duration, bt)”, which I usually use, doesn’t do this. I can see some lead-in due to the BT and pulse duration, but I expect that. Scanning the python code under gr-digital, it looks something in the generic\_mod might account for those leading zeros. The GMSK modulator doesn’t use the generic\_mod class. Regards, Andy ---
**Author:** ![reback](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/b5ac83/32.png) [@reback](https://www.ruby-forum.com/u/reback)\ **Post date:** [April 29, 2015, 5:52pm UTC](https://www.ruby-forum.com/t/correlation-estimation-block-oddity/241671/13 "2015-04-29T17:52:08Z")
John, Thanks for replying. The question is, how do I do this? We both agree the zeros need to go. What I’m not sure on is how to do that. I’d rather keep this contained to GRC, because I don’t like maintaining python files not connected to grc. Can you explain how you do this or provide an example? v/r, Rich On Tue, Apr 28, 2015 at 1:41 PM, Johnathan C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)