Gigabit Ethernet cards

I suspect that my GiGE NIC may be dropping packets. It’s an
RTL8168d-type chip, according to “dmesg”.

Are there known-to-be-good-with-GnuRadio NICs for PCI with low-profile
brackets out there? This is for
a 2U server platform.


Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium

On Fri, Jul 2, 2010 at 22:40, Marcus D. Leech [email protected] wrote:

I suspect that my GiGE NIC may be dropping packets. Â It’s an
RTL8168d-type chip, according to “dmesg”.

Are there known-to-be-good-with-GnuRadio NICs for PCI with low-profile
brackets out there? Â This is for
 a 2U server platform.
I have the same problem that some packets from the USRP2 are getting
dropped.

I tested this with a USRP2 with the latest UHD fpga image
(u2_rev3-udp-ise12-20100615.bin) and firmware (txrx_uhd_20100621.bin).
The host computers were running the latest uhd host code
(a34f930a79a0c626706a5f7532d8f692446d3c35). The first test machine is
a Phenom X4 with Windows XP and two cards (i tested both), a Realtek
onboard chipset (RTL8111/8168B PCI Express Gigabit Ethernet
controller) and an Intel PCI Card (82541GI Gigabit Ethernet
Controller). I even tested with a third card (Intel Dual-Port PCI
Express card) but it also had the same problems. Second test machine
is a Core2 Duo running Debian/squeeze and has a Broadcom BCM5787M PCI
Express onboard.

Test was made using “rx_timed_samples --rxrate 25000000 --nsamps
1000000” and “rx_timed_samples --rxrate 62500000 --nsamps 1000000” (on
Windows built as release target) but it often indicates dropped
packets (SX) on both platforms. Capturing with wireshark (or ulogd
with pcap target) and decoding the VRT packets afterwards (using some
python scripts) shows the same result. I tried using the benchmark_rx
example but it fails because even after sending
“STREAM_MODE_STOP_CONTINUOUS”.

I also tried increasing the buffer sized but this didn’t solve the
problem.

Thanks in advance,
Manuel

Manuel,

On Mon, Jul 5, 2010 at 7:33 AM, Manuel F.
[email protected] wrote:

On Fri, Jul 2, 2010 at 22:40, Marcus D. Leech [email protected] wrote:

I suspect that my GiGE NIC may be dropping packets. It’s an
RTL8168d-type chip, according to “dmesg”.

Are there known-to-be-good-with-GnuRadio NICs for PCI with low-profile
brackets out there? This is for
a 2U server platform.
I have the same problem that some packets from the USRP2 are getting dropped.

I can think of a nicer way to print the error messages SX, but dropped
packets are a fact of life; especially running at 25 Msps.

On my linux box, running benchmark at full rate, i usually see a few
dropped packets initially and at the end when it finishes.

AMD Athlon™ II X4 620 Processor
Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

Express onboard.

Test was made using “rx_timed_samples --rxrate 25000000 --nsamps
1000000” and “rx_timed_samples --rxrate 62500000 --nsamps 1000000” (on
Windows built as release target) but it often indicates dropped
packets (SX) on both platforms. Capturing with wireshark (or ulogd
with pcap target) and decoding the VRT packets afterwards (using some
python scripts) shows the same result. I tried using the benchmark_rx
example but it fails because even after sending
“STREAM_MODE_STOP_CONTINUOUS”.

see
http://code.ettus.com/redmine/ettus/projects/public/wiki/U2binaries#UHD-Firmware-and-FPGA-Images

On 07/05/2010 12:33 PM, Josh B. wrote:

I can think of a nicer way to print the error messages SX, but dropped
packets are a fact of life; especially running at 25 Msps.

On my linux box, running benchmark at full rate, i usually see a few
dropped packets initially and at the end when it finishes.

AMD Athlon™ II X4 620 Processor
Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

I see dropped packets at much lower bandwidths, but with more complex
flow-graphs. But the puzzling
thing is the CPU isn’t even close to 100% when I see these.

On my 6-core Phenom II X6 1090T with 8G of 1333 DDR3 with the CPUs
hard-configured to run at 3.2GHz,
I can run usrp2_fft.py at 25Msps, and get the occasional dropped
packet, but I can’t run
more-complex flow-graphs beyond about 8Msps without it getting into
serious droppage, even
without 100% CPU utilization.

This is with an RTL ethernet chip on the mobo. I’ve been told that
Intel e1000e cards do somewhat better–
perhaps deeper FIFOs or something like that.

I know that for the project I just left, I was doing a device driver for
a Xilinx-TEMAC with a proprietary buffer
manager in the FPGA–it had a FIFO depth of only 8, which caused a lot
of heartache for bursts of
closely-spaced packets, and a USRP2 runnning “full bore” would almost
certainly have caused it
really bad heartburn :slight_smile:


Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium

On 07/05/2010 07:33 AM, Manuel F. wrote:

On Fri, Jul 2, 2010 at 22:40, Marcus D. Leech[email protected] wrote:

I suspect that my GiGE NIC may be dropping packets. It’s an
RTL8168d-type chip, according to “dmesg”.

Are there known-to-be-good-with-GnuRadio NICs for PCI with low-profile
brackets out there? This is for
a 2U server platform.
I have the same problem that some packets from the USRP2 are getting dropped.

In general you will want to avoid the realtek chipsets. Though many are
fine, you are more likely to have performance and driver issues.

Test was made using “rx_timed_samples --rxrate 25000000 --nsamps
1000000” and “rx_timed_samples --rxrate 62500000 --nsamps 1000000” (on
Windows built as release target) but it often indicates dropped
packets (SX) on both platforms. Capturing with wireshark (or ulogd
with pcap target) and decoding the VRT packets afterwards (using some
python scripts) shows the same result. I tried using the benchmark_rx
example but it fails because even after sending
“STREAM_MODE_STOP_CONTINUOUS”.

The default buffer size is way too small. You are bound to see dropping
even when the average processing rate can keep up with no problem.

I just pushed code up to the master that requests a sizable buffer (half
a second a full rate) by default, prints a warning when the resized
buffer is too small, and added to the application notes about this:
http://www.ettus.com/uhd_docs/manual/html/usrp2.html#resize-the-send-and-receive-buffers

-Josh

On 07/08/2010 11:50 AM, Lamar Owen wrote:

Since you said PCI, I’m assuming this isn’t a PCI Express box we’re talking about here, since that changes everything. A single lane PCI-e slot is capable of 250MB/s throughput, if the chipset can keep up. Typical server GigE cards for PCI-e can have two or four ports and need a four lane slot; if you have dual PCI-e x16 slots, use the secondary x16 slot in x4 mode.

I have a PCIe x16 slot on the mobo, it’s a GigaByte GA-880GM-UD2H, so I
guess what I really need is a
PCIe GiGE card.


Marcus L.
Principal Investigator
Shirleys Bay Radio Astronomy Consortium

On Friday, July 02, 2010 04:40:42 pm Marcus D. Leech wrote:

I suspect that my GiGE NIC may be dropping packets. It’s an
RTL8168d-type chip, according to “dmesg”.

Are there known-to-be-good-with-GnuRadio NICs for PCI with low-profile
brackets out there? This is for
a 2U server platform.

As I don’t have a USRP2, and probably won’t be getting one any time
soon, I can’t help with that element of the question.

What I can help with is the low-profile GigE business.

If these are 32-bit slots, and if there are other bandwidth hogs on the
same bus (like a VGA card or hard disk controller/HBA), you simply run
out of PCI bandwidth trying to do full-rate GigE (32-bit slots, 33MHz
signalling, 133MB/s absolute max across the shared bus).

My 2U personal server platform is a Supermicro dual Xeon (not the Dell
2850 I’ve mentioned in relation to another project, but this is my own
personal box), which has four complete PCI-X busses, two of which can
run 64-bit cards in 133MHz signaled sockets, giving a max of 1066MB/s
(the actual clock is 133.333333…MHz, thus the data isn’t 1064MB/s),
I have a dual Intel PRO/1000 MT card with deep buffers, low-profile, and
a 64-bit interface, sitting alone on one of the board’s PCI-X busses.
It actually can’t run at 133MHz, but can at 66MHz, but that’s still
533MB/s data rates). Got it on eBay for $39.95, NIB. There’s one on
eBay now for that price; there’s also a single port for $29. Search for
‘low profile PCI-X Intel’ and those two are the only hits, as of 11:45A
EDT, today.

The e1000 driver does have its issues, but it’s been solid for me with
Fedora 13 for a few weeks now, and it was solid with CentOS 5 for months
before that.

Since you said PCI, I’m assuming this isn’t a PCI Express box we’re
talking about here, since that changes everything. A single lane PCI-e
slot is capable of 250MB/s throughput, if the chipset can keep up.
Typical server GigE cards for PCI-e can have two or four ports and need
a four lane slot; if you have dual PCI-e x16 slots, use the secondary
x16 slot in x4 mode.

Here’s the lspci -a output for the above card:
05:01.0 Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet
Controller (Copper) (rev 01)
Subsystem: Intel Corporation PRO/1000 MT Dual Port Server
Adapter
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 96
Memory at e8520000 (64-bit, non-prefetchable) [size=128K]
I/O ports at 5400 [size=64]
Capabilities: [dc] Power Management version 2
Capabilities: [e4] PCI-X non-bridge device
Capabilities: [f0] MSI: Enable- Count=1/1 Maskable- 64bit+
Kernel driver in use: e1000
Kernel modules: e1000

05:01.1 Ethernet controller: Intel Corporation 82546EB Gigabit Ethernet
Controller (Copper) (rev 01)
Subsystem: Intel Corporation PRO/1000 MT Dual Port Server
Adapter
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 97
Memory at e8540000 (64-bit, non-prefetchable) [size=128K]
I/O ports at 5440 [size=64]
Capabilities: [dc] Power Management version 2
Capabilities: [e4] PCI-X non-bridge device
Capabilities: [f0] MSI: Enable- Count=1/1 Maskable- 64bit+
Kernel driver in use: e1000
Kernel modules: e1000

HTH.

On Thursday, July 08, 2010 12:01:53 pm Marcus D. Leech wrote:

I have a PCIe x16 slot on the mobo, it’s a GigaByte GA-880GM-UD2H, so I
guess what I really need is a
PCIe GiGE card.

Broadcom is the biggest player in this market; eBay item # 150455393659
is one such example and only needs a x1 PCI-e slot. The BCM57xx drivers
are in-kernel and fairly robust. This is the chip used by my laptop:

09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5752
Gigabit Ethernet PCI Express (rev 02)
Subsystem: Dell Device 01c8
Flags: bus master, fast devsel, latency 0, IRQ 30
Memory at dcef0000 (64-bit, non-prefetchable) [size=64K]
Expansion ROM at [disabled]
Capabilities: [48] Power Management version 2
Capabilities: [50] Vital Product Data
Capabilities: [58] MSI: Enable+ Count=1/8 Maskable- 64bit+
Capabilities: [d0] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [13c] Virtual Channel
Kernel driver in use: tg3
Kernel modules: tg3

There’s also an HP NC110T on eBay ( item # 140423322343 ). This card
uses the Intel 82572G1 chipset, and it is a server-type adapter. The
eBay one is actually more expensive than one I found online a few
minutes ago NIB, so google for it.

On 09/07/2010, at 1:31, Marcus D. Leech wrote:

On 07/08/2010 11:50 AM, Lamar Owen wrote:

Since you said PCI, I’m assuming this isn’t a PCI Express box we’re talking about here, since that changes everything. A single lane PCI-e slot is capable of 250MB/s throughput, if the chipset can keep up. Typical server GigE cards for PCI-e can have two or four ports and need a four lane slot; if you have dual PCI-e x16 slots, use the secondary x16 slot in x4 mode.

I have a PCIe x16 slot on the mobo, it’s a GigaByte GA-880GM-UD2H, so I
guess what I really need is a
PCIe GiGE card.

I use this
http://www.newegg.com/Product/Product.aspx?Item=N82E16833106033 in my
home (FreeBSD) server.
(When I ordered it it came with full height and LP plates but I didn’t
get it from Newegg so YMMV)

Never used it with a USRP2 though, however Intel em cards are widely
regarded as being very good.

If you really did mean PCI instead of PCI express then I think I had one
of these in an older server…
http://www.newegg.com/Product/Product.aspx?Item=N82E16833106122&Tpk=PWLA8391GTLBLK

It was also an excellent performer.


Daniel O’Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
“The nice thing about standards is that there
are so many of them to choose from.”
– Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C