IPORT(9) Kernel Concepts IPORT(9)

iport, iportmap, phymap, tgtmap —
SCSI Device Management Concepts

The iport, iportmap, phymap, and tgtmap abstractions enable host bus adapter (HBA) drivers to represent the devices that they are responsible for enumerating, as well as the relationships between these devices. These interfaces simplify device drivers by taking care of the creation and destruction of device nodes in the devices tree for enumerated devices as well as performing some amount of hysteresis.

These abstractions are used in tandem with SCSI complex addressing. A device driver that uses these interfaces generally passes both the SCSI_HBA_HBA flag and the SCSI_HBA_ADDR_COMPLEX in the hba_flags argument to scsi_hba_attach_setup(9F).

The iport, or initiator port, abstracts a collection of attached devices. One way to view an iport is that each iport maps to a phy on the HBA. A phy refers to a physical connector between the HBA and devices. A phy may be made up of individual lanes. A lane is connected to a device, for example a disk driver. Multiple lanes maybe plugged into the same device, for example, an expander. When a phy connects to a device with a single lane, this is often called a narrow phy. When a phy connects to a device with multiple lanes, this is often called a wide phy.

Consider a device that has two physical ports, and thus two phys. Each phy has four lanes, thus we describe the phy as having a mask of 0xf. Each bit in the mask corresponds to a specific lane. In this example, each phy would be represented in the system by an iport and may enumerate a different device for each lane of the phy. If an expander is attached to one or more of the lanes of a phy, then additional devices will be enumerated under the expander and be added to that phy's iport.

Another example to consider is when each lane of a phy is directly connected to a single disk through a passive backplane. In this case, each lane may represent its own iport, since the management of each is independent, basically there are many devices each with a mask of 0x1.

iports do not need to map to a physical phy. Some HBAs support a combination of both physical and virtual devices. In that case, the driver may create two different iports, one for the physical devices and one for the virtual devices.

One property of iports is that they're attached separately from the main device and therefore have their own scsi_hba_tran(9S) structure. As a result, that means that a driver can provide different entry points for each iport, especially if they represent different classes of resources, for example one iport for all physical devices and one for all virtual devices. This allows for a driver to return different capabilities, among other behaviors and entry points, for these different iports. One specific case of this is that while physical devices may provide a means to get to a SCSI WWN, virtual devices may not have a WWN and instead must use a different addressing format.

iports are considered children of the device driver that attach them, but they are bound to the same driver. This means that when an iport is created, the attach(9E) and probe(9E) entry points of the parent driver (usually indicated by passing a dev_info structure) will be called. Similarly, when an iport is removed from the system, then the driver's detach(9E) entry point will be called. A driver can determine whether an iport is being attached or not by calling the scsi_hba_iport_unit_address(9F) function. The value will return NULL if the attaching device represents the driver.

To manage iports, drivers have two different options. If the set of iport an HBA supports are static, then they should use the scsi_hba_iport_register(9F) function to register an iport.

If instead, the set of iports are dynamic and map to the coming and going of phys discovered by the driver (or some other dynamic source), then the driver should use the iportmap set of functions. See the section phymap and iportmap for more information.

The target map represents a set of devices that have been enumerated under an iport. Each device is represented by a string, which is an address of some kind. Usually a physical device's WWN is used.

By using a target map, the operating system will take responsibility for notifying the driver when devices have come and gone from a target map, once it has settled, and it will also take responsibility for having device nodes come and go, meaning that the device driver does not need to know anything about the devices tree or worry about other parts of being a nexus driver.

Target maps come in two forms which change how the HBA driver is responsible for reporting changes:

  1. Full-set
  2. Per-address

In the full-set mode, the driver always reports the full set of current devices that it sees. When the driver finishes the report, the operating system will inform the driver of addresses that were added and addresses that were removed. These addresses correspond to newly found devices and recently removed devices, respectively. The full-set mode allows for a simpler device driver, particularly if addition and removal notifications may be dropped by the hardware.

When using the per-address mode of a target map, the HBA driver is responsible for indicating which addresses have come and gone from the system.

In either mode, the driver will receive two callbacks, if they have been registered when the target map was created. The first callback fires before a target driver like sd, ses, etc. is attached. The second callback fires after the corresponding driver has been attached. These allow the HBA driver to perform any operations that are required on the devices.

Each target map has two different sets of devices that it manages in this form. The devices are separated into the following groups:

  1. SCSI Devices
  2. SMP (SCSI Management Protocol) devices

All SATA, SCSI, SAS, SES, etc. devices all are considered part of the first category.

Target maps can be created and destroyed with the scsi_hba_tgtmap_create(9F) and scsi_hba_tgtmap_destroy(9F) functions.

The following functions are used to manage target maps operating in full-set mode:

The following functions are used to manage target maps operating in per-address mode:

The phymap and iportmap are often used together to represent complex SAS topologies. The phymap provides a way to see what phys have been grouped together under the same SAS port. The SAS port is represented by the “local” and “remote” WWNs. When additional phys come online, if they end up referring to the same WWNs, then they'll map to the same port.

The iportmap is used to maintain a dynamic set of iports related to a device. The iports are each identified by an address, which is generally a unit address string. For example, when a new phy is added to the phymap which represents a new SAS port being used, then a corresponding iport will be created and associated with that entry from the phymap. Once the iport has been created, a normal target map can be created on top of it to handle detected SCSI and SMP devices.

Both the phymap and iportmap operate in a similar fashion to the per-address mode of a tgtmap. Entries can be added and removed through direct functions. The phymap provides callbacks similar to the tgtmap; however, the iportmap does not. This is because when an iport is added or removed, a new node is added to the devices tree and the driver's attach(9E) entry point is called with a new dev_info_t structure representing the iport.

During the phymap callback, the HBA driver should create a new iport with the unit address passed in from the callback function. This relationship is important when taking advantage of the ability to map between an iport and the set of phys that it represents.

The following functions are used to manage iportmaps:

The following functions are used to manage phymaps:

Traditionally, SCSI devices were represented by a simple structure, the scsi_address(9S). This represented devices by a simple target and lun number. While this interface is useful for simple devices and traditional parallel SCSI devices, it is not as useful for SAS-era devices where the SCSI bus is now a fabric. A driver may opt into such a complex addressing mode by setting the SCSI_HBA_ADDR_COMPLEX flag.

When this flag is set, the HBA driver must treat the SCSI address as an opaque structure. Once in this mode, the driver may get and set a private data structure on the SCSI device. This is facilitated by the scsi_device_hba_private_set(9F) and scsi_device_hba_private_get(9F) functions. In addition, the system provides a means to map between the scsi_address(9S) structure and the corresponding scsi_device(9S) structure. This is performed by the scsi_device_unit_address(9F) function.

attach(9E), detach(9E), sas_phymap_create(9F), sas_phymap_destroy(9F), sas_phymap_phy_add(9F), sas_phymap_phy_rem(9F), scsi_device_hba_private_get(9F), scsi_device_hba_private_set(9F), scsi_device_unit_address(9F), scsi_hba_attach_setup(9F), scsi_hba_iport_register(9F), scsi_hba_iport_unit_address(9F), scsi_hba_iportmap_create(9F), scsi_hba_iportmap_destroy(9F), scsi_hba_iportmap_iport_add(9F), scsi_hba_iportmap_iport_remove(9F), scsi_hba_tgtmap_create(9F), scsi_hba_tgtmap_destroy(9F), scsi_hba_tgtmap_set_add(9F), scsi_hba_tgtmap_set_begin(9F), scsi_hba_tgtmap_set_end(9F), scsi_hba_tgtmap_set_flush(9F), scsi_hba_tgtmap_tgt_add(9F), scsi_hba_tgtmap_tgt_remove(9F), scsi_address(9S), scsi_device(9S), scsi_hba_tran(9S)
April 18, 2017 OmniOS