MLXCX(4D) Devices MLXCX(4D)

mlxcxMellanox ConnectX-4/5/6 Ethernet controller driver

/dev/net/mlxcx*

The driver is a GLDv3 NIC driver for the ConnectX-4, ConnectX-4 Lx, ConnectX-5 and ConnectX-6 families of ethernet controllers from Mellanox. It supports the Data Link Provider Interface, dlpi(4P).

This driver supports:

At this time, the driver does not support Large Send Offload (LSO), energy efficient Ethernet (EEE), or the use of flow control through hardware pause frames.

The file contains user configurable parameters, including the ability to set the number of rings and groups advertised to MAC, the sizes of rings and groups, and the maximum number of MAC address filters available.

The driver supports the following device properties which may be tuned through its driver.conf file, /kernel/drv/mlxcx.conf. These properties cannot be changed after the driver has been attached.

These properties are not considered stable at this time, and may change.

Minimum: 2 | Maximum: device dependent (up to 255)
The eq_size_shift property determines the number of entries on Event Queues for the device. The number of entries is calculated as (1 << eq_size_shift), so a value of 9 would mean 512 entries are created on each Event Queue. The default value is .
Minimum: 2 | Maximum: device dependent (up to 255)
The cq_size_shift property determines the number of entries on Completion Queues for the device. The number of entries is calculated as (1 << cq_size_shift), so a value of 9 would mean 512 entries are created on each Event Queue. The default value is device dependent, 10 for devices with maximum supported speed of 10Gb/s or less and 12 for devices with higher supported speeds. This should be kept very close to the value set for rq_size_shift and sq_size_shift.
Minimum: 2 | Maximum: device dependent (up to 255)
The rq_size_shift property determines the number of descriptors on Receive Queues for the device. The number of descriptors is calculated as (1 << rq_size_shift), so a value of 9 would mean 512 descriptors are created on each Receive Queue. This sets the number of packets on RX rings advertised to MAC. The default value is device dependent, 10 for devices with maximum supported speed of 10Gb/s or less and 12 for devices with higher supported speeds.
Minimum: 2 | Maximum: device dependent (up to 255)
The sq_size_shift property determines the number of descriptors on Send Queues for the device. The number of descriptors is calculated as (1 << sq_size_shift), so a value of 9 would mean 512 descriptors are created on each Send Queue. This sets the number of packets on RX rings advertised to MAC. The default value is device dependent, for devices with maximum supported speed of 10Gb/s or less and for devices with higher supported speeds. Note that large packets often occupy more than one descriptor slot on the SQ, so it is sometimes a good idea to increase this if using a large MTU.
Minimum: 1 | Maximum: device dependent
The tx_ngroups property determines the number of TX groups advertised to MAC. The default value is 1.
Minimum: 1 | Maximum: device dependent
The tx_nrings_per_group property determines the number of rings in each TX group advertised to MAC. The default value is .
Minimum: 1 | Maximum: device dependent
The rx_ngroups_large property determines the number of "large" RX groups advertised to MAC. The size of "large" RX groups is set by the rx_nrings_per_large_group property. The default value is 2.
Minimum: 1 | Maximum: device dependent
The rx_nrings_per_large_group property determines the number of rings in each "large" RX group advertised to MAC. The number of such groups is determined by the rx_ngroups_large property. The default value is 16.
Minimum: 1 | Maximum: device dependent
The rx_ngroups_small property determines the number of "small" RX groups advertised to MAC. The size of "small" RX groups is set by the rx_nrings_per_small_group property. It is recommended to use many small groups when using a large number of VNICs on top of the NIC (e.g. on a system with many zones). The default value is 256.
Minimum: 1 | Maximum: device dependent
The rx_nrings_per_small_group property determines the number of rings in each "small" RX group advertised to MAC. The number of such groups is determined by the rx_ngroups_small property. The default value is 4.
Minimum: 4 | Maximum: device dependent
The ftbl_root_size_shift property determines the number of flow table entries on the root flow table, and therefore how many MAC addresses can be filtered into groups across the entire NIC. The number of flow entries is calculated as (1 << ftbl_root_size_shift), so a value of 9 would mean 512 entries are created in the root flow table. The default value is 12.
Minimum: 1 | Maximum: 65535
The cqemod_period_usec property determines the maximum delay after a completion event has occurred before an event queue entry (and thus an interrupt) is generated. The delay is measured in microseconds. The default value is .
Minimum: 1 | Maximum: 65535
The cqemod_count property determines the maximum number of completion events that can have occurred before an event queue entry (and thus an interrupt) is generated. The default value is .
Minimum: 1 | Maximum: 65535
The intrmod_period_usec property determines the maximum delay after an event queue entry has been generated before an interrupt is raised. The delay is measured in microseconds. The default value is 10.
Minimum: 1 | Maximum: 65535
The tx_bind_threshold property determines the minimum number of bytes in a packet before the driver uses ddi_dma_addr_bind_handle(9F) to bind the packet memory for DMA, rather than copying the memory as it does for small packets. DMA binds are expensive and involve taking locks in the PCI nexus driver, so it is seldom worth using them for small packets. The default value is .
Minimum: 16 | Maximum:
The rx_limit_per_completion property determines the maximum number of packets that will be processed on a given completion ring during a single interrupt. This is done to try and guarantee some amount of liveness in the system. The default value is 256.
Minimum: | Maximum:
The rx_p50_loan_min_size property determines the minimum size of packet buffers allowed to be loaned to MAC when the ring has reached >=50% of its buffers already on loan. Packet buffers larger than this value will be copied. At <50% of ring buffers on loan, all buffers will be loaned. At >=75% of buffers on loan, all packets will be copied instead to ensure ring availability. The default value is 256.

/kernel/drv/amd64/mlxcx
Device driver (x86)
/kernel/drv/mlxcx.conf
Driver configuration file containing user-configurable options

dlpi(4P), driver.conf(5), dladm(8), snoop(8)

August 27, 2020 OmniOS