Today I had the opportunity to update the firmware of the Mellanox ConnectX-3 card, and I thought I'd give a closer look at the process (it should work for previous ConnectX series card models as well).
ConnectX-3 PCIe device is a network card manufactured by Mellanox (now owned by nVidia) and equipped with SFP+ interface (or two) with 10Gbps network performance.
The interface is equipped with inserts that have appropriate sockets adjusted to local network – optical (e.g. LC), copper (e.g. RJ45), and with fixed (copper) cable – so called DAC cables. To update the firmware you need to prepare the mstflint software (https://github.com/Mellanox/mstflint/) and install some libraries in the system. I've done the update in RedHat Enteprise Linux 8.4 operating system (which guarantees that identical steps will work in RockyLinux 8 or CentOS 8). We need to perform all the steps with root user privileges.
System preparation:
# installation of compilation tools dnf group install "Development Tools" -y # installation of required libraries dnf install openssl-devel libibmad-devel zlib-devel wget unzip pciutils -y
The next step is to download the sources and compile the mstflint program – this is a package that allows you to communicate with the ConnectX network card and manage its firmware.
Installing mstflint:
# move to /tmp/ folder to avoid messing up your system cd /tmp # downloading current mstflint sources wget https://github.com/Mellanox/mstflint/archive/refs/heads/master.zip # unpacking and running the preparation script unzip master.zip cd mstflint-master/ ./autogen.sh # configuration, compilation and installation in the system ./configure make make install
Now we need to determine the address of our network card. Just use the "lspci" command, which will display a list of devices. The number on the left is the address we will use to communicate with the card (as in the picture below):
It's worth checking if mstflint is working, to do this we'll ask for information about the current version of the firmware the card is using with the command (type after -d the address that lspci displayed to you):
mstflint -d 01:00.0 q full The result should resemble something like this:
# mstflint -d 01:00.0 q
Image type:
FS2 FW Version: 2.33.5220
FW Release Date: 29.3.2015
MIC Version: 1.5.0
Config Sectors: 1
Product Version: 02.33.52.20
Rom Info: type=PXE version=3.4.467
Device ID: 4099
Description: Node Port1 Port2 Sys image
GUIDs: ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff
MACs: f4521427b0c0 f4521427b0c1
VSD:
PSID: MT_1170110023 As you can see, the communication was successful and we can update the firmware. We should start by downloading the version matching our network card from the manufacturer's website – i.e. https://www.mellanox.com/support/firmware/firmware-downloads.
On the left we look for the model we have (e.g. ConnectX-3) and on the right we choose the type of device (Ethernet in the case of a network card). Looking at the available versions you can see that it is possible to upload a newer one. However, you have to choose (from the OPN table) the version of the device. You may find it out again using the lspci tool and the command (type after -s the address displayed by lspci):
lspci -vv -s 01:00.0 | grep "Part number" -A 3 The result (shown below) indicates the version of the device you have, and it is this version that you should indicate in the OPN column of the firmware page.
[PN] Part number: MCX311A-XCAT
[EC] Engineering changes: A6
[SN] Serial number: MT1402X13666
[V0] Vendor specific: PCIe Gen3 x4 In the next column "PSID" choose the number consistent with that shown by msflint with the argument g full (marked above in blue). On the right in the "Download/Documentation" column you will see a link to the appropriate archive containing the firmware for the device. Download it, unpack and program our card:
# download wget http://www.mellanox.com/downloads/firmware/file.zip # extracting (the extracted file has the extension .bin) unzip file.zip # programming, you must confirm that you want to do it by clicking on the "y" mstflint -d 01:00.0 -i file.bin burn Current FW version on flash: 2.33.5220 (current firmware version) New FW version: 2.42.5000 (upload version) Burn process will not be failsafe. No checks will be performed. ALL flash, including the Invariant Sector will be overwritten. If this process fails, computer may remain in an inoperable state. Do you want to continue ? (y/n) [n] : y Burning FS2 FW image without signatures - OK Restoring signature - OK
After this operation just restart your computer and you are done! We can enjoy the network card software fixes.