Segfault

Hey list

I managed to build gnuradio 3.5.1 and (almost) got it working on the
USRP E110. However, when I try to run uhd_rx_cfile.py (with
–samp-rate 10e6 -f 97.7e6) it segfaults. You will find the log below.

The system that I’m running on the USRP has been compiled from scratch
and is using Linux 3.1.10 with the USRP driver from

with uClibc 0.9.32 and AFAICT the latest versions of all the gnuradio
dependencies.

Does anyone have a clue about what I might be doing wrong?

---------- dmesg ----------
usrp_e open called, use_count = 0
rx_dma->ch 0
tx_dma->ch 1
usrp: leaving open
Unable to handle kernel NULL pointer dereference at virtual address
000004fc
pgd = c3440000
[000004fc] *pgd=86c88831, *pte=00000000, *ppte=00000000
Internal error: Oops: 817 [#2]
Modules linked in:
CPU: 0 Tainted: G D (3.1.10 #1)
pc : [] lr : [<401d137c>] psr: 80000013
sp : c4fdbeb0 ip : 00000000 fp : c4fdbef4
r10: 00000000 r9 : c4fda000 r8 : c000d9c4
r7 : dc490ac0 r6 : 00000003 r5 : 00000001 r4 : be9fbfd0
r3 : c4fdbeb8 r2 : c0342bd0 r1 : 000004fc r0 : 00000000
Flags: Nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 10c5387d Table: 83440019 DAC: 00000015
Process python (pid: 1595, stack limit = 0xc4fda2e8)
Stack: (0xc4fdbeb0 to 0xc4fdc000)
bea0: 000004fc 00000001 00000000
0032e9c8
bec0: 00000028 401bb7e4 00329dc0 00000011 42156824 00021a20 401048b4
401d137c
bee0: c6d12228 be9fbfd0 c4fdbf04 c4fdbef8 c01393f8 c0139224 c4fdbf7c
c4fdbf08
bf00: c0090afc c0139378 40908178 003371c8 c03123b0 c4fda000 c4fda000
00000000
bf20: ffffffff 00000000 c4fdbf5c c4fdbf38 c0046f68 c0008c60 ffffffff
df8a801c
bf40: df861140 c6d990c0 c03123b0 c4fda000 c4fdbf74 c4fdbf60 00000003
40307522
bf60: be9fbfd0 dc490ac0 c000d9c4 c4fda000 c4fdbfa4 c4fdbf80 c0090bb0
c0090658
bf80: 003371c8 00000000 be9fbfd0 40307522 00000003 00000036 00000000
c4fdbfa8
bfa0: c000d840 c0090b7c be9fbfd0 40307522 00000003 40307522 be9fbfd0
00000000
bfc0: be9fbfd0 40307522 00000003 00000036 be9fd88c 0032e8f0 4219f660
be9fd500
bfe0: 4219f938 be9fbe98 420ef69c 40180b9c 60000010 00000003 6d726f66
0228745f
Backtrace:
Function entered at [] from []
r5:be9fbfd0 r4:c6d12228
Function entered at [] from []
Function entered at [] from []
r9:c4fda000 r8:c000d9c4 r7:dc490ac0 r6:be9fbfd0 r5:40307522
r4:00000003
Function entered at [] from []
r7:00000036 r6:00000003 r5:40307522 r4:be9fbfd0
Code: e5b3c004 e5900048 e0811105 e2855001 (e780c001)
—[ end trace 689a22803bf4f530 ]—
usrp_e release called
Waiting for DMA to become inactive
Freeing gpio irq’s
Freeing DMA channels
---------- end ----------

cheers

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On Wed, Mar 14, 2012 at 10:42 AM, Stefan O. [email protected] wrote:

Hey list

I managed to build gnuradio 3.5.1 and (almost) got it working on the
USRP E110. However, when I try to run uhd_rx_cfile.py (with
–samp-rate 10e6 -f 97.7e6) it segfaults. You will find the log below.

So there is essentially no chance the E110 can record to Flash at
10Msps.
It probably won’t sustain flash writes at 500ksps. Try a lower sample
rate,
or a small number of samples written to a tmpfs instead.

That said, it shouldn’t segfault, even at high sample rates. Does a
lower
rate work OK?

–n

On Wed, Mar 14, 2012 at 18:48, Nick F. [email protected] wrote:

probably won’t sustain flash writes at 500ksps. Try a lower sample rate, or
a small number of samples written to a tmpfs instead.

That said, it shouldn’t segfault, even at high sample rates. Does a lower
rate work OK?

I am actually writing to RAM so that shouldn’t be the problem. And
yes, it also segfaults at lower sample rates (I tried multiple values
down to 1) and with a small number of samples (also tried just 1
sample).

It would seem that the very idea of reading from the device causes the
segfault.

cheers

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On Wed, Mar 14, 2012 at 10:55 AM, Stefan O. [email protected] wrote:

So there is essentially no chance the E110 can record to Flash at
down to 1) and with a small number of samples (also tried just 1
sample).

It would seem that the very idea of reading from the device causes the
segfault.

What version of UHD are you using? What kernel driver? Have you updated
anything else on the E110 image besides Gnuradio? Did it work for you
before updating Gnuradio?

-n

On Wed, Mar 14, 2012 at 18:58, Nick F. [email protected] wrote:

What version of UHD are you using?

At the moment I am using a git checkout from late February - good
point, I’ll update and report back.

What kernel driver?

The one from

Have you updated anything else on the E110 image besides Gnuradio?

I am not using the shipped E110 image, I compiled the whole operating
system from source

Did it work for you before updating Gnuradio?

Yes, it seems to work when I use the shipped sd cards.

cheers

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On 03/14/2012 01:42 PM, Stefan O. wrote:

dependencies.

Does anyone have a clue about what I might be doing wrong?

Besides completely voiding the warranty :slight_smile: :slight_smile: :slight_smile: ← Note smileys, I
actually enjoy people doing crazy stuff.

When you say Linux 3.1.0, is that mainline?

If you do want to update your kernel to something more recent, try
starting from here:

http://www.sakoman.com/cgi-bin/gitweb.cgi?p=linux-omap-2.6.git;a=shortlog;h=refs/heads/omap-3.2

Finally, there are docs on the web for working out where in the code the
oops really occurs. Can you try working through that procedure and see
if it happens in the driver, or elsewhere.

Philip

Tangential and not directly relevant, but I was testing an embedded ARM
system on Tuesday specifically for SDcard write performance, and on
a Class-10 SDcard was able to sustain 40Mbyte/second on a 32Mbyte
“sprint”–for 100 iterations. That’s unrelated to USRP performance,
but at least on ARM hardware writing to “decent” SD cards, you can get
pretty-good performance.

Just to add more to this discussion in case anyone is curious:

Unfortunately, this doesn’t mean that ARM-based SoCs will follow this
trend. Using the ARM architecture doesn’t guarantee good SD Card
support;
it really comes down to the manufacturer of the chip itself. For the
E1xx
devices, we are reliant on TI’s capability to work well with SD Cards,
since it is the OMAP3 chip acting as the controller (assuming, of
course,
that support in the Linux kernel is solid).

Cheers,
Ben

Just to add more to this discussion in case anyone is curious:

Fair enough. But on my Pandaboard, I get goodish SD card
performance–which uses a TI part. There have been issues with this
(Panda) board
and both SD-card compatibility and performance. My buddie’s Panda-ES
device, for example, gets sucky SD card performance, whereas my
Panda-A3 gets good performance.

On Wed, Mar 14, 2012 at 19:11, Stefan O. [email protected] wrote:

On Wed, Mar 14, 2012 at 18:58, Nick F. [email protected] wrote:

What version of UHD are you using?

At the moment I am using a git checkout from late February - good
point, I’ll update and report back.

I now tried with UHD 3.3.2 as well as the with the latest version from
git, I still get the same segfaults.


Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On Wed, Mar 14, 2012 at 20:51, Philip B. [email protected]
wrote:

Does anyone have a clue about what I might be doing wrong?

Besides completely voiding the warranty :slight_smile: :slight_smile: :slight_smile: ← Note smileys, I
actually enjoy people doing crazy stuff.

:slight_smile:

When you say Linux 3.1.0, is that mainline?

Yep (3.1.10, not 3.1.0, in case that matters)

If you do want to update your kernel to something more recent, try
starting from here:

http://www.sakoman.com/cgi-bin/gitweb.cgi?p=linux-omap-2.6.git;a=shortlog;h=refs/heads/omap-3.2

The kernel that I’m using seems to work just fine on some other
gumstix nodes but sure, I’ll give it a try and see whether anything
changes.

Finally, there are docs on the web for working out where in the code the
oops really occurs. Can you try working through that procedure and see
if it happens in the driver, or elsewhere.

Ok, I’ll see whether I can find it and report back once I have more
information.

cheers

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On Thu, Mar 15, 2012 at 18:02, Stefan O. [email protected] wrote:

Finally, there are docs on the web for working out where in the code the
oops really occurs. Can you try working through that procedure and see
if it happens in the driver, or elsewhere.

Ok, I’ll see whether I can find it and report back once I have more information.

So far, what I’ve found through the magic of strace and printf is that
the offending call seems to be the USRP_E_WRITE_CTL32 ioctl that
happens as a consequence of poke32 (in e100_ctrl.cpp) when called from
e100_ctrl_impl.

That’s still with (vanilla) kernel 3.1.10 though.

cheers

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

So there is essentially no chance the E110 can record to Flash at
10Msps. It probably won’t sustain flash writes at 500ksps. Try a lower
sample rate, or a small number of samples written to a tmpfs instead.

Tangential and not directly relevant, but I was testing an embedded ARM
system on Tuesday specifically for SDcard write performance, and on
a Class-10 SDcard was able to sustain 40Mbyte/second on a 32Mbyte
“sprint”–for 100 iterations. That’s unrelated to USRP performance,
but at least on ARM hardware writing to “decent” SD cards, you can
get pretty-good performance.

On 03/15/2012 02:26 PM, Stefan O. wrote:

happens as a consequence of poke32 (in e100_ctrl.cpp) when called from
e100_ctrl_impl.

That’s still with (vanilla) kernel 3.1.10 though.

Can you look carefully at what data is being passed to the driver? We
may need to range check the data before we use it in the driver.

Philip

On Thu, Mar 15, 2012 at 19:58, Philip B. [email protected]
wrote:

Can you look carefully at what data is being passed to the driver? We
may need to range check the data before we use it in the driver.

I had a look and it what I could find out so far is that offset=1276
and count=1.

cheers

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On Fri, Mar 16, 2012 at 20:36, Philip B. [email protected]
wrote:

From the ioctl number. Maybe work out which line of code is the issue
and look at the source. That would be helpful for us anyway.

e100_ctrl.cpp: e100_ctrl_impl → poke32 → ioctl

I suppose that’s the 32 bit write E100_REG_GLOBAL_RESET ioctl
(USRP_E_WRITE_CTL32)

(also, sorry, I sent the last message from my phone and failed to CC the
list)

cheers

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On Mon, Mar 19, 2012 at 15:11, Philip B. [email protected]
wrote:

This is really running on the e100? No dual core CPU.

On the E110

I’m skiing at the moment, so not much time to think about this :slight_smile:

Heh, enjoy!

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On Tue, Mar 20, 2012 at 14:55, Philip B. [email protected]
wrote:

Hmm, how do I tell?
(also, sorry, I sent the last message from my phone and failed to CC the list)

And uhd_usrp_probe is ok, but one of the other test programs fails?

Oh, interesting. uhd_find_devices works fine but uhd_usrp_probe also
segfaults. I thought that used to run, I’ll try again with different
uhd versions and report back.

I’ll try and reproduce this on a similar kernel next week.

I attached my kernel configuration file, just in case.

Also, I tried running linux 3.2 but so far the USRP_E driver failed to
compile thus the config is still for 3.1.

cheers

Oh, interesting. uhd_find_devices works fine but uhd_usrp_probe also
segfaults. I thought that used to run, I’ll try again with different
uhd versions and report back.

The same thing happens with all versions of UHD that I tried:
uhd_find_devices works but uhd_usrp_probe causes a segfault.

cheers

Stefan O.
Communication and Distributed Systems
Institute of Computer Science and Applied Mathematics
University of Bern

On 03/19/2012 07:28 AM, Stefan O. wrote:

From the ioctl number. Maybe work out which line of code is the issue
and look at the source. That would be helpful for us anyway.

e100_ctrl.cpp: e100_ctrl_impl -> poke32 -> ioctl

I suppose that’s the 32 bit write E100_REG_GLOBAL_RESET ioctl
(USRP_E_WRITE_CTL32)

(also, sorry, I sent the last message from my phone and failed to CC the list)

And uhd_usrp_probe is ok, but one of the other test programs fails? I’ll
try and reproduce this on a similar kernel next week.

Philip

On 04/02/2012 09:27 AM, Stefan O. wrote:

Hey

Just curious, have you had a chance to look at this?

Also, in the meantime I tried using your kernel tree
(e100-3.0-pm-2-fixes-from-review) which seems to have a newer version
of the usrp_e driver, built with the kernel configuration that I took
from a working USRP image. However, when trying to modprobe the usrp-e
module, I get:

That branch is not done yet :slight_smile:

I just updated github to match the current shipping kernel (from last
week) See the e100-3.0-pm-2-bugfixes branch. In particular, I added some
code to do some range checking on the values passed to the ioctl in:

Philip