Solved: Mystery DHCP Requests from Dell R815 Broadcom NeXtreme II NICs

Listen, my friends, and I will tell you a tale of mystery MAC addresses, DCHP, and Broadcom woes.

It all started when we got a new server. A Dell R815, to be exact. The specs looked great, it could hold a ton of RAM for some of our bioinformatics projects, and the cost was low. Everyone was happy.

Then one day someone called and said, hey, there are a ton of DHCP requests and we've traced them back to [room that server is in]. Here's the MAC address: 002xxxxxxxd1.

This seemed weird to me because we had set up static IPs on the server's two live NICs, both on the primary NIC and on the iDRAC enterprise NIC. To my knowledge, there were no NICs set up with DHCP.

Even more weird was the fact that the MAC address 002xxxxxxxd1 did not actually exist. The R815 had five NICs, numbered like this:

002xxxxxxxd0 (NIC 1)
002xxxxxxxd2 (NIC 2)
002xxxxxxxd4 (NIC 3)
002xxxxxxxd6 (NIC 4)
002xxxxxxxd8 (iDRAC)

We pored over the NIC configurations in Windows 2008 R2 server and its virtual switch and virtual machines (since it was running the Hyper-V role). Nothing. Meanwhile, our sniffer showed that we were not dreaming...indeed, DHCP requests were emanating from that MAC address.

We pulled the network cable and the DHCP requests stopped. We pulled out our hair and ceremoniously waved a Windows 98 boot disk over the server -- we keep one around for just these sorts of occasions.

Finally we turned up something...in the iDRAC remote management screen we found a list of MAC addresses which included the mysterious one we were looking for.

I'll cut to the chase. The Broadcom NeXtreme II 5906 NICs have iSCSI capability built-in. The embedded iSCSI HBA has an IPv4 configuration setting that defaults to DHCP. This setting was on by default, and means that DHCP requests were emanating from the server, receiving a 10.11.x.x address.

The DHCP requests cannot be disabled without installing the BroadCom Managed Applications Control Suite and then following the instructions here.

It is very probable that I'm not very bright and other people expect DHCP requests to come from the iSCSI hardware on their NICs by default. Just thought I'd share. And I note that this is only happens on Windows, not Linux on the same hardware.

tl;dr DHCP requests are issued by default on Broadcom NeXtreme II network adapters on Windows servers unless you download a special tweaker to turn them off, and it took me a while to figure this out.

Reference: Broadcom iSCSI offload engine defaults to DHCP enabled (IBM, 2011-03-03)