Designing Real-Time Ethernet Industrial Control

Ethernet was not originally conceived for real-time applications. For this reason, the emergence of Ethernet as a replacement for the traditional industrial field bus may be somewhat surprising. However, using the right design steps and components it is possible to translate the low costs, simplicity and field-proven open standardisation offered by Ethernet into real-time industrial-control applications.

 

Redundant-ring topology

The typical Ethernet topology is the star network, where a multi-port switch provides point-to-point links to other nodes. Unlike typical Ethernet networks, the control layer of an industrial network is usually based on a redundant-ring topology, as shown in Figure 1. This format is favoured to ease the logistics of cable installation.

The basic building block of a ring network is the three-port switch, such as the KSZ8893MQL from Micrel. The Ethernet ring usually requires management to ensure that one of the links acts as a protection mechanism, hence the term redundant-ring network.

 

 

Deterministic real-time performance

Of course, the redundant-ring topology is not without its disadvantages. Network latency is increased as a result of data passing through each node in the ring before reaching its destination. All Ethernet switches today employ a store-and-forward architecture, which stores the complete packet and processes prior to forwarding. This results in non-deterministic latency behaviour. Table 1 shows the latency measurements of the KSZ8995MAI industrial five-port switch, operating at 100Mbit/s over 1mof CAT5 cable. The total latency is derived from two components: packet size and the internal forwarding delay. It can be calculated using the following formula:

 

Table 1 shows a constant and small forwarding delay for Micrel’s KSZ8995MAI switch, independent of packet size. Therefore, fixing the size of the packets in a network results in a constant switch latency, and minimising the packet size reduces the overall switch latency.

 

 

To reduce latency jitter, the Ethernet Powerlink Standardisation Group (EPSG) recommends using 100Base-TX/FX Ethernet repeater hubs. These repeater hubs deploy a cut-through architecture to significantly reduce latency. This is achieved by forwarding incoming packets before the complete packet is received. This results in lower latency than store-and-forward switches, regardless of packet size. However, one limitation of repeater hubs is the fact that they remain scarce compared to some common switches.

The scarcity of repeater hubsmeans that they are often implemented using FPGAs and external Ethernet PHYs, increasing cost, board-space requirements and time tomarket. However,Micrel’s KSZ8893MQLI three-port switch and KSZ8842-xxxxI two-port Ethernet controllers integrate a unique low-latency repeatermode. This repeater-mode provides a maximum port-to-port latency of 310nS and total deviation of just 40nS.

Another approach that can produce deterministic delivery of packets uses higher-level protocols such as IEEE 1588. Figure 2 shows a typical hardware implementation of IEEE 1588 functionality. IEEE 1588 utilises User Datagram Protocol (UDP) packets over IP on the Ethernet network to provide synchronisation down to a microsecond. This meets the stringent real-time requirements formotion-control applications. ProfiNet, Ethernet/IP and EtherCAT industrial Ethernet groups have all adopted this standard for network synchronisation.

 

 

MII or RMII?

Media Independent Interface (MII) is the standard Ethernet MAC/PHY interface for implementing a solution using FPGAs, network processors, Ethernet switches or PHYs. This interface, as defined in IEEE 802.3u, comprises 16 pins for data and control signals.

To reduce the pin count for designs using multi-port FPGAs, ASICs or processors, the Reduced MII (RMII™) specification was developed. RMII provides independent, 2-bit-wide transmit and receive paths. These are synchronised to a common 50MHz reference clock, reducing the interface’s pin count to eight. RMII seems to offer a number of design advantages. However, when creating real-time applications, there are several other factors to consider when selecting between MII or RMII.

What is frequently not appreciated is that, architecturally, RMII is not a replacement for MII but an additional reconciliation layer on either side of the MII. Moving from the recovered incoming line clock to the RMII reference clock requires a FIFO to tolerate differences in frequency. Ultimately this translates into increased latency jitter, which is undesirable in real-time networks.

To predict the typical latency jitter that will be introduced into a MII or RMII network requires analysis of the basic building blocks of the PHY. A simple timing model example is shown in Figure 3. It is also important to note that this analysis is independent of the implementation and ignores any variations due to process and temperature.

 

 

In the transmit direction, data is synchronised to the local 25MHz oscillator and typically adds a fixed delay of around 10T or 80nS (where T is the transmit period for a 125MHz PHY). Unlike the transmit direction, the receive direction incurs a variable delay due to the clock recovery alignment. This delay is typically 20T or 160nS, and is caused by the alignment of the generated 25MHz MII RX clock with respect to the recovered 125MHz line clock. This results in an offset due to one of the five possible phases; 0nS, 8nS, 16nS, 24nS or 32nS. PLL recovered clock jitter will also add around 10nS to the total receive-path latency. The KSZ8041NLI single-port PHY from Micrel removes these delays by ensuring a fixed phase for the recovered clock.

In a real-time network, variable delays are far more critical than fixed delays, as they cannot be compensated.

Using an MII interface, an Ethernet PHY will typically exhibit a round-trip delay of around 240nS to 282nS, thus adding up to 42nS to the overall network latency jitter. However, interfacing to the Ethernet PHY via RMII adds latency as a result of the reconciliation layer (see Figure 4).

With the RMII interface, like the MII interface, the PHY uses REF_CLK as the network clock in the transmit direction. This minimises any added delay. This is not the case in the receive direction as the reconciliation layer needs to account for the differences between the local clock, REF_CLK, and recovered clock, RX_CLK.

Implementation of the RMII interface will typically be achieved using a FIFO. The RMII specification advises on a FIFO size of 20 bits with transfer of the recovered data onto RXD[1:0] when the FIFO is half full. Such a design consequently adds up to 20 x 10nS, or 200nS, of latency jitter to the receive path, on top of MII latency.

Many vendors design their silicon to operate in a far more robust timing environment than the 100ppm or 0.01% specified by the IEEE. Cisco Systems’ equipment is designed to cope with frequency errors of up to 0.1%, which is not uncommon to encounter in today’s networks. Based on the formula below, this will increase the required FIFO size to 27 bits. The maximum RMII latency jitter would increase as a result to 27 x 10nS, or 270nS.

Another PHY/MAC interface standard has recently become popular amongst applications using processors, FPGAs and Ethernet transceivers: The Serial Media Independent Interface (SMII) offers a reduced pin requirement, like RMII, but requires an elastic buffer to retime the incoming recovered line clock onto the synchronous reference clock. Similar to RMII, this results in increased latency jitter.

 

 

Conclusion

It is widely agreed that Ethernet will eventually replace the traditional field bus for device-level industrial communications. This will not happen overnight so, in the short term, it will operate alongside field bus to provide customers with the most efficient, lowest-cost solution. Designing Ethernet to provide the necessary deterministic real-time performance is demanding. However, by making the right architectural decisions and careful implementation these goals can be successfully met. For example, Micrel’s latest single-port fast Ethernet PHY, KSZ8041, offers the choice of SMII, RMII or MII interface, in addition to LinkMD™ cable diagnostics.

 

Micrel / Design Note