New upstream version 17.08

[deb_dpdk.git] / doc / guides / nics / mlx4.rst

diff --git a/doc/guides/nics/mlx4.rst b/doc/guides/nics/mlx4.rst
index f1f26d4..f8885b2 100644 (file)
--- a/doc/guides/nics/mlx4.rst
+++ b/doc/guides/nics/mlx4.rst
@@ -1,5 +1,6 @@
 ..  BSD LICENSE
     Copyright 2012-2015 6WIND S.A.
 ..  BSD LICENSE
     Copyright 2012-2015 6WIND S.A.

+    Copyright 2015 Mellanox

 
     Redistribution and use in source and binary forms, with or without
     modification, are permitted provided that the following conditions
 
     Redistribution and use in source and binary forms, with or without
     modification, are permitted provided that the following conditions


@@ -76,6 +77,7 @@ Compiling librte_pmd_mlx4 causes DPDK to be linked against libibverbs.
 Features
 --------
 
 Features
 --------
 

+- Multi arch support: x86_64 and POWER8.

 - RSS, also known as RCA, is supported. In this mode the number of
   configured RX queues must be a power of two.
 - VLAN filtering is supported.
 - RSS, also known as RCA, is supported. In this mode the number of
   configured RX queues must be a power of two.
 - VLAN filtering is supported.


@@ -87,6 +89,7 @@ Features
 - Inner L3/L4 (IP, TCP and UDP) TX/RX checksum offloading and validation.
 - Outer L3 (IP) TX/RX checksum offloading and validation for VXLAN frames.
 - Secondary process TX is supported.
 - Inner L3/L4 (IP, TCP and UDP) TX/RX checksum offloading and validation.
 - Outer L3 (IP) TX/RX checksum offloading and validation for VXLAN frames.
 - Secondary process TX is supported.

+- RX interrupts.

 
 Limitations
 -----------
 
 Limitations
 -----------


@@ -116,6 +119,14 @@ These options can be modified in the ``.config`` file.
   adds additional run-time checks and debugging messages at the cost of
   lower performance.
 
   adds additional run-time checks and debugging messages at the cost of
   lower performance.
 

+- ``CONFIG_RTE_LIBRTE_MLX4_DEBUG_BROKEN_VERBS`` (default **n**)
+
+  Mellanox OFED versions earlier than 4.2 may return false errors from
+  Verbs object destruction APIs after the device is plugged out.
+  Enabling this option replaces assertion checks that cause the program
+  to abort with harmless debugging messages as a workaround.
+  Relevant only when CONFIG_RTE_LIBRTE_MLX4_DEBUG is enabled.
+

 - ``CONFIG_RTE_LIBRTE_MLX4_SGE_WR_N`` (default **4**)
 
   Number of scatter/gather elements (SGEs) per work request (WR). Lowering
 - ``CONFIG_RTE_LIBRTE_MLX4_SGE_WR_N`` (default **4**)
 
   Number of scatter/gather elements (SGEs) per work request (WR). Lowering


@@ -244,9 +255,8 @@ DPDK and must be installed separately:
 
 Currently supported by DPDK:
 
 
 Currently supported by DPDK:
 

-- Mellanox OFED **4.0-2.0.0.0**.
-- Firmware version **2.40.7000**.
-- Supported architectures:  **x86_64** and **POWER8**.
+- Mellanox OFED **4.1**.
+- Firmware version **2.36.5000** and above.

 
 Getting Mellanox OFED
 ~~~~~~~~~~~~~~~~~~~~~
 
 Getting Mellanox OFED
 ~~~~~~~~~~~~~~~~~~~~~


@@ -273,6 +283,138 @@ Supported NICs
 
 * Mellanox(R) ConnectX(R)-3 Pro 40G MCX354A-FCC_Ax (2*40G)
 
 
 * Mellanox(R) ConnectX(R)-3 Pro 40G MCX354A-FCC_Ax (2*40G)
 

+Quick Start Guide
+-----------------
+
+1. Download latest Mellanox OFED. For more info check the  `prerequisites`_.
+
+2. Install the required libraries and kernel modules either by installing
+   only the required set, or by installing the entire Mellanox OFED:
+
+   For bare metal use:
+
+   .. code-block:: console
+
+        ./mlnxofedinstall
+
+   For SR-IOV hypervisors use:
+
+   .. code-block:: console
+
+        ./mlnxofedinstall --enable-sriov -hypervisor
+
+   For SR-IOV virtual machine use:
+
+   .. code-block:: console
+
+        ./mlnxofedinstall --guest
+
+3. Verify the firmware is the correct one:
+
+   .. code-block:: console
+
+        ibv_devinfo
+
+4. Set all ports links to Ethernet, follow instructions on the screen:
+
+   .. code-block:: console
+
+        connectx_port_config
+
+   Or in the manual way:
+
+   .. code-block:: console
+
+        PCI=<NIC PCI address>
+        echo eth > "/sys/bus/pci/devices/$PCI/mlx4_port0"
+        echo eth > "/sys/bus/pci/devices/$PCI/mlx4_port1"
+
+5. In case of bare metal or hypervisor, configure optimized steering mode
+   by adding the following line to ``/etc/modprobe.d/mlx4_core.conf``:
+
+   .. code-block:: console
+
+        options mlx4_core log_num_mgm_entry_size=-7
+
+   .. note::
+
+        If VLAN filtering is used, set log_num_mgm_entry_size=-1.
+        Performance degradation can occur on this case.
+
+6. Restart the driver:
+
+   .. code-block:: console
+
+        /etc/init.d/openibd restart
+
+   or:
+
+   .. code-block:: console
+
+        service openibd restart
+
+7. Compile DPDK and you are ready to go. See instructions on
+   :ref:`Development Kit Build System <Development_Kit_Build_System>`
+
+Performance tuning
+------------------
+
+1. Verify the optimized steering mode is configured:
+
+  .. code-block:: console
+
+        cat /sys/module/mlx4_core/parameters/log_num_mgm_entry_size
+
+2. Use environment variable MLX4_INLINE_RECV_SIZE=64 to get maximum
+   performance for 64B messages.
+
+3. Use the CPU near local NUMA node to which the PCIe adapter is connected,
+   for better performance. For VMs, verify that the right CPU
+   and NUMA node are pinned according to the above. Run:
+
+   .. code-block:: console
+
+        lstopo-no-graphics
+
+   to identify the NUMA node to which the PCIe adapter is connected.
+
+4. If more than one adapter is used, and root complex capabilities allow
+   to put both adapters on the same NUMA node without PCI bandwidth degradation,
+   it is recommended to locate both adapters on the same NUMA node.
+   This in order to forward packets from one to the other without
+   NUMA performance penalty.
+
+5. Disable pause frames:
+
+   .. code-block:: console
+
+        ethtool -A <netdev> rx off tx off
+
+6. Verify IO non-posted prefetch is disabled by default. This can be checked
+   via the BIOS configuration. Please contact you server provider for more
+   information about the settings.
+
+.. note::
+
+        On some machines, depends on the machine integrator, it is beneficial
+        to set the PCI max read request parameter to 1K. This can be
+        done in the following way:
+
+        To query the read request size use:
+
+        .. code-block:: console
+
+                setpci -s <NIC PCI address> 68.w
+
+        If the output is different than 3XXX, set it by:
+
+        .. code-block:: console
+
+                setpci -s <NIC PCI address> 68.w=3XXX
+
+        The XXX can be different on different systems. Make sure to configure
+        according to the setpci output.
+

 Usage example
 -------------
 
 Usage example
 -------------