Frame Preemption functional description (using ethtool and tc)

Frame preemption is defined in IEEE 802.1Qbu and 802.3br and allows low priority frames to be suspended by time-critical frames of higher priority.

This section describes how to enable frame preeemption using the tools ethtool and tc. This is the standard way of enabling the frame preemption.

This is working fine only for the following platforms:

  • lan9645x

To enable the frame preemption the port the following command can be run

$ ethtool --set-mm lan2 tx-enabled on verify-enabled on

To see if the frame preemption is enabled, the following command can be used:

$ ethtool --show-mm lan2

Where the mm stands for MAC-merge

More information regarding ethtool can be found here

Now that the frame preemption is enabled, we need to decide which frames can be preempted. To do that we can use the tc command to set which queues contain preemptable traffic.

For example the frames in queue 2 to be preemptable the following command can be used:

$ tc qdisc add dev lan2 root mqprio num_tc 8 hw 1 fp E E P E E E E E

Where:

  • E stands for express traffic

  • P stands for preemptable traffic

More information regarding the tc command can be found here

Before frames can be preempted, they must be assigned to the queues for which frame preemption is enabled.

A natural way to do this is the pcp value of VLAN tagged frames. It is important to enable trust for pcp on the ingress ports, otherwise this value is ignored in the queue classification.

$ dcb apptrust set dev lan0 order pcp

1. Frame Preemption with Time-Aware Scheduling (TAS)

Combining frame preemption with TAS is only supported on lan9645x.

Frame preemption and the Time-Aware Shaper (taprio, see Time-Aware Scheduling) work well together.

With TAS alone, the hardware must insert a guard band before every gate close, sized for a maximum-length frame (~12.3 µs at 1 Gbps), so that no frame is still being transmitted when the window closes. On short cycles this guard band wastes a large part of the window.

When the queue is preemptible, the hardware instead preempts the in-flight frame at the gate boundary and resumes it later, so the guard band collapses from a full max-frame time to the much smaller 64B fragment completion time. This both reclaims cycle bandwidth and keeps the latency of the protected (express) window tight, without having to cap the best-effort frame size.

1.1. Configuration

The preemptible-queue mask (fp) is given directly on the taprio qdisc — a port has a single root qdisc, so there is no separate mqprio qdisc to install. The full sequence on the egress port is:

# Trust PCP so pcp -> traffic-class -> queue is 1:1
$ dcb apptrust set dev lan1 order pcp

# Enable MAC-merge / frame preemption on the egress port
$ ethtool --set-mm lan1 tx-enabled on verify-enabled on

# TAS schedule with inline fp. Q2 is preemptible.
$ tc qdisc replace dev lan1 root taprio \
    num_tc 8 queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 map 0 1 2 3 4 5 6 7 \
    base-time 0 cycle-time 50000 \
    sched-entry S 0x02 20000 \
    sched-entry S 0xFD 30000 \
    fp E E P E E E E E \
    flags 0x2

The fp parameter takes one token per traffic class, E for express and P for preemptible, exactly as for mqprio. When a gate operation closes a preemptible queue, the driver programs it as a Set-And-Hold-MAC gate operation, so an in-flight preemptible frame is preempted at the boundary and the express window is protected; the gate operation that re-opens the preemptible queue releases the hold again.

verify-enabled on requires the link partner to participate in the 802.3br verification handshake before preemption becomes active. Against a non-participating partner or a test generator, use verify-enabled off to force preemption active.

1.2. Verifying operation

$ ethtool --show-mm lan1                                    # expect: TX active: on
$ ethtool -S lan1 | grep -E 'tx_merge_frag|rx_assembly_ok'  # increment once preemption runs

To change the schedule or the preemptible set at runtime you replace the full schedule including the preemptible mask

$ tc qdisc replace dev lan1 root taprio \
    num_tc 8 queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 map 0 1 2 3 4 5 6 7 \
    base-time 0 cycle-time 50000 \
    sched-entry S 0x01 20000 \
    sched-entry S 0xFe 30000 \
    fp E P E E E E E E \
    flags 0x2