|OVERLAY(7)||Standards, Environments, and Macros||OVERLAY(7)|
overlay — Overlay
Overlay devices are a GLDv3 device that allows users to create overlay networks that can be used to form the basis of network virtualization and software defined networking. Overlay networks allow a single physical network, often called an underlay network, to provide the means for creating multiple logical, isolated, and discrete layer two and layer three networks on top of it.
Overlay devices are administered through dladm(8). Overlay devices themselves cannot be plumbed up with IP, vnd, or any other protocol. Instead, like an etherstub, they allow for VNICs to be created on top of them. Like an etherstub, an overlay device acts as a local switch; however, when it encounters a non-local destination address, it instead looks up where it should send the packet, encapsulates it, and sends it out another interface in the system.
A single overlay device encapsulates the logic to answer two different, but related, questions:
Each of these questions is answered by a plugin. The first question is answered by what's called an encapsulation plugin. The second question is answered by what's called a search plugin. Packets are encapsulated and decapsulated using the encapsulation plugin by the kernel. The search plugins are all user land plugins that are consumed by the varpd service whose FMRI is svc:/network/varpd:default. This separation allows for the kernel to be responsible for the data path, while having the search plugins in userland allows the system to provide a much more expressive interface.
Overlay devices come in two different flavors, one where all packets are always sent to a single address, the other, where the destination of a packet varies based on the target MAC address of the packet. This information is maintained in a target table, which is independent and unique to each overlay device. We call the plugins that send traffic to a single location, for example a single unicast or multicast IP address, a point to point overlay and the overlay devices that can send traffic to different locations based on the MAC address of that packet a dynamic overlay. The plugin type is determined based on the type of the search plugin. These are all fully listed in the section Plugins and their Properties.
Both encapsulation and search plugins define the kinds of destinations that they know how to support. An encapsulation plugin always has a single destination type that's determined based on how the encapsulation is defined. A search plugin, on the other hand, can support multiple combinations of destinations. A search plugin must support the destination type of the encapsulation device. The destination may require any of the following three pieces of information, depending on the encapsulation plugin:
The list of destination types that are supported by both the search and encapsulation plugins is listed in the section Plugins and their Properties.
The varpd service, mentioned above, is responsible for providing the virtual ARP daemon. Its responsibility is conceptually similar to ARP. It runs all instances of search plugins in the system and is responsible for answering the kernel's ARP-like questions for where packets should be sent.
The varpd service, svc:/network/varpd:default, must be enabled for overlay devices to function. If it is disabled while there are active devices, then most overlay devices will not function correctly and likely will end up dropping traffic.
Properties fall into three categories in the system:
Each property in the system has the following attributes, which mirror the traditional dladm(8) link properties:
For example, the property vxlan/listen_ip is associated with the vxlan encapsulation module.
The following sections describe both the modules and the properties that exist for each module, noting their name, type, permissions, whether or not they are required, and if there is a default value. In addition, the effects of each property will be described.
The vxlan module requires both an IP address and port to address it. It has a 24-bit virtual network ID space, allowing for virtual network identifiers that range from 0 - 16777215.
The vxlan module has the following properties:
The vxlan/listen_ip and vxlan/listen_port properties determine how the system will accept VXLAN encapsulated packets for this interface. It does not determine the interface that packets will be sent out over. Multiple overlays that all use VXLAN can share the same IP and port combination, as the virtual network identifier can be used to tell the different overlays apart.
Because search plugins may support multiple destinations, they may have more properties listed than necessarily show up for a given overlay. For example, the direct plugin supports destinations that are identified by both an IP address and a port, or just an IP address. In cases where the device is created over an overlay that only uses an IP address for its destination, then it will not have the direct/dest_port property.
The direct plugin has the following properties:
The files plugin has the following property:
Each overlay has the following properties which are used to give additional information about the system. None of these properties may be specified as part of a dladm create-overlay, instead they come from other arguments or from internal parts of the system.
To modify the mtu property, use dladm set-linkprop.
Overlay devices are wired into FMA, the illumos fault management architecture, and generates error reports depending on the search plugin in use. Due to limitations in FMA today, when a single overlay enters a degraded state, meaning that it cannot properly perform look ups or another error occurred, then it degrades the overall overlay pseudo-device driver.
For more fine-grained information about which overlay is actually in a degraded state, one should run dladm show-overlay -f. In addition, for each overlay in a degraded state a more useful diagnostic message is provided which describes the reason that caused this overlay to enter into a degraded state.
The overlay driver is self-healing. If the problem corrects itself on its own, it will clear the fault on the corresponding device.
|March 30, 2022||OmniOS|