I’m a puzzled by some UHD behavior and wondering if I need a new host
system. I’m not doing any signal processing in the host – just
transmitting from a buffer and/or receiving from a buffer. Later, we’ll
be wanting to do more processing with two USRP2s both transmitting and
receiving at the same time and may want 25 Msps in some cases.
I’ve managed to get simultaneous receive threads going for each of two
USRP2s, both at 25 Msps (big buffers are good). The GUI still updates,
but is a little sluggish. It’s fragile, though. If I do much at all with
the GUI (sometimes just moving the mouse), recv’s start timing out.
System Monitor shows no memory pressure and both CPUs at 50-60%. FWIW,
I’m running my C++ app via SSH from another machine.
Transmitting is worse. Even a solo transmit thread at 25 Msps (plus a
thread calling recv_async_msg) causes the GUI to be unusable and I get
frequent Us on both the host and the USRP2’s serial port, as well as
tons of EVENT_CODE_SEQ_ERROR and EVENT_CODE_SEQ_ERROR_IN_BURST messages
from recv_async_msg. I can’t tell what the CPU utilization is because
System Monitor stops updating. 6.25 Msps is better, but I still get a
few Us and a bunch of EVENT_CODE_SEQ_ERROR and
EVENT_CODE_SEQ_ERROR_IN_BURST messages. System Monitor manages to update
and shows CPU utilization of 80% and 55%. Again, I’m only transmitting
from a buffer that I fill once up front and send over and over again
with no signal processing.
I’m specifying recv_buff_size=100e6 and send_buff_size=100e6;
net.core.rmem_max and net.core.wmem_max are set to 100000000. I’m
calling uhd::set_thread_priority_safe from each of my threads. I’m using
the latest gnuradio and UHD software/firmware/FPGA (the recent release
with USRP1 support). I have Ubuntu 10.4 with stock ethernet drivers.
I’ve itemized a few details of the hardware setup below. What systems
are you all using for MIMO-like work with USRP2s and UHD? What systems
do you recommend or wish you had? Are there other changes I should make
to my current system?
TIA
-Marc
Dell DCDO tower manufactured in 2007
Dual-core Xeon 5110 @ 1.60 GHz
2 GB RAM
Lots of slots - mix of PCI, PCIe, PCI-X
Ethernet 0: USRP2 with DBSRX
Onboard 10/100/1000 Ethernet.
USRP2 says no flow control and we’re getting another PCI Express card.
Ethernet 1: network connection
D-Link PCI NIC
No flow control
Ethernet 2: USRP2 with XCVR2450 StarTech.com PCI Express 10/100/1000 Ethernet NIC
Ubuntu identifies it as Broadcom NetXtreme BCM5752
USRP2 says it gets WE_TX flow control
Sorry to reply to myself, but we just got the new Ethernet NIC so both USRP2s report WE_TX flow control. Nothing changed.
And I have a correction for the types of ethernet cards. The onboard NIC is the Broadcom. Ethernet 2 (and now 3) appear to be Realtek RTL8111/8168B.
On Sep 3, 2010, at 3:10 PM, Marc E. wrote:
I’ll comment that a 1.6GHz machine that’s getting close to 4 years old
isn’t particularly “beefy” by todays standards. Certainly not
beefy enough to sustain two 25Msps threads without “problems”.
Just the interrupt load alone is rather frightening. The packets used
by the USRP2 aren’t terribly big, something like 300-500 bytes
I think. So, 25e6 / 3e2 = 83Kpackets/second–depending on your NIC,
each of those packets could be creating an interrupt!
Then you have to actually process the darn things – copy up to user
space, “do stuff”, and present the result of that “stuff” either back
out
to the network, or into your software (FFT display, whatever).
To give some perspective. I run my radio astronomy Rx-only “stack” on a
6-core machine doing “stuff” that’s of the same order of
complexity as modern telecom-type signal processing. At 5 or 6MHz
bandwidth, the machine is working really hard. That’s a machine
with an AMD Phenom II X6 1090t, running at 3.2GHz/core with 6GB of fast memory. [And I suspect a flaky power supply, but that’s
a topic for another day ].
–
Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium
On Sep 3, 2010, at 5:57 PM, Marcus D. Leech wrote:
I’ll comment that a 1.6GHz machine that’s getting close to 4 years old
isn’t particularly “beefy” by todays standards. Certainly not
beefy enough to sustain two 25Msps threads without “problems”.
Thanks, Marcus. That was helpful and as I suspected. So over the weekend
I built* a new system that should eliminate the speed concern (for now
;-). It’s an Intel Core i7-980x with 6GB of the “world’s fastest”
XMP-certified RAM. 64-bit Ubuntu screams on it. Running make with
–jobs=8 builds gnu-radio and Ettus UHD rather quickly. My UHD app is
running much better and the GUI is actually usable while I’m doing a
transmit+receive run.
However, the transmit side is still troublesome. Periodically, a burst
of Us comes out of the UART and there are always a lot of
EVENT_CODE_SEQ_ERROR_IN_BURST events – like several thousands per
second. I’ve tried this on two different ethernet NICs identified by
Ubuntu as an Intel 82567V-2 (on the motherboard) and a Realtek
RTL8111/8166B (in a PCIe slot). I have the transmit and receive buffers
set to 500,000,000.
I’m digging into the host and USRP2 source to figure out where
EVENT_CODE_SEQ_ERROR_IN_BURST come from and how transmit flow control
works.
Any pointers would be appreciated.
-Marc
It’s the first time I’ve built a system as I normally leave that to
Apple. It was fun.
I discovered that my event counters were overstating the event counts
(adding each enum’s value instead of 1, D’oh!), but I’m still getting
EVENT_CODE_SEQ_ERROR_IN_BURST events fairly often. I’ve modified the
tx_waveforms.cpp example to count async events. You can get the source
at https://public.me.com/mepard.
It’s showing about a rate of around 1.5 EVENT_CODE_SEQ_ERROR_IN_BURST
events per 100 packets at 25 Msps. Is that rate expected? I think I can
work around it for my current purposes , but I haven’t done it yet.
-Marc
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
].