Sparx5 and LAN969x Watchdog

Sparx5 and LAN969x each have a hardware watchdog that Linux exposes as /dev/watchdog0, using the standard kernel watchdog API. This page shows how to use it from an application, and what to configure to get it working on your board.

1. What it does

  • If the watchdog is not fed in time, it resets the CPU subsystem (VCore) - the hardware watchdog itself does not touch the switch core.

  • Before that reset happens, a pretimeout warning fires first. Linux uses it to reset the switch core (mainly its PLL and SERDES settings), so the bootloader can bring the network back up after the reboot instead of finding the switch in whatever state it was left in.

  • If nothing feeds the watchdog after the pretimeout warning, the CPU reset follows.

2. SoC resources

SoC Base address Clock IRQ

Sparx5

0x600106000

ahb_clk, 250 MHz

GIC SPI 5

LAN969x

0xe0090000

fabric_clk, 250 MHz

GIC SPI 36

3. Timeout limits

The requested timeout is split into two equal halves: the pretimeout warning comes first, the reset follows one more half-timeout later. Above roughly 8.6 seconds, the kernel feeds the hardware in the background so the pretimeout still arrives close to that mark, while the full requested timeout is honored for detecting a hung system.

Requested timeout Pretimeout warning Reset at Notes

1-2 s

1 s

2.15 s

3-4 s

2 s

4.29 s

5-8 s

4 s

8.59 s

9 s and above

8 s

requested + 8.59 s

kernel-assisted feeding above the hardware limit

Feed the watchdog more often than the pretimeout warning interval shown above, or the pretimeout fires (and the switch core resets) even if a later keepalive would otherwise have prevented the final reset.

4. Kernel configuration

  • CONFIG_WATCHDOG=y

  • CONFIG_DW_WATCHDOG=y

  • CONFIG_WATCHDOG_SYSFS=y

  • CONFIG_WATCHDOG_PRETIMEOUT_GOV=y

  • CONFIG_WATCHDOG_PRETIMEOUT_GOV_NOOP=y

  • CONFIG_WATCHDOG_PRETIMEOUT_DEFAULT_GOV_NOOP=y

These are already set in microchip_standalone_defconfig.

5. Device tree configuration

Both SoCs use the same watchdog binding:

watchdog: watchdog@600106000 {
	compatible = "microchip,sparx5-wdt", "snps,dw-wdt";
	reg = <0x6 0x00106000 0x1000>;
	interrupts = <GIC_SPI 5 IRQ_TYPE_EDGE_RISING>;
	clocks = <&ahb_clk>;
};

The interrupt trigger type must be IRQ_TYPE_EDGE_RISING.

To get the switch-core reset described above, the switch node also needs the same interrupt, named "wdt", added to its own interrupts and interrupt-names. On Sparx5:

switch: switch@600000000 {
	...
	interrupt-names = "xtr", "fdma", "ptp", "wdt";
	interrupts = <GIC_SPI 30 IRQ_TYPE_LEVEL_HIGH>,
		     <GIC_SPI 25 IRQ_TYPE_LEVEL_HIGH>,
		     <GIC_SPI 27 IRQ_TYPE_LEVEL_HIGH>,
		     <GIC_SPI  5 IRQ_TYPE_EDGE_RISING>;
	...
};

On LAN969x, the switch node has two more interrupts (ptp-ext, oam-vop) ahead of wdt:

switch: switch@e0000000 {
	...
	interrupt-names = "xtr", "fdma", "ptp", "ptp-ext", "oam-vop", "wdt";
	interrupts = <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>,
		     <GIC_SPI 88 IRQ_TYPE_LEVEL_HIGH>,
		     <GIC_SPI  9 IRQ_TYPE_LEVEL_HIGH>,
		     <GIC_SPI  8 IRQ_TYPE_LEVEL_HIGH>,
		     <GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH>,
		     <GIC_SPI 36 IRQ_TYPE_EDGE_RISING>;
	...
};

Both of these are already set up in the shipped device trees. Without the "wdt" entry, a watchdog reset still recovers the VCore, but leaves the switch core in whatever state it was in, which can prevent netboot on the following boot.

6. UserSpace API

The watchdog is controlled through /dev/watchdog0, using the generic kernel watchdog API: https://docs.kernel.org/watchdog/watchdog-api.html.

6.1. ioctls

ioctl Effect

WDIOC_SETTIMEOUT

Set the timeout in seconds.

WDIOC_GETTIMEOUT

Read back the configured timeout.

WDIOC_SETPRETIMEOUT

Any non-zero value enables the pretimeout warning (the default); 0 disables it.

WDIOC_GETPRETIMEOUT

Read back the pretimeout, in seconds.

WDIOC_GETTIMELEFT

Seconds until reset.

Opening /dev/watchdog0 arms the watchdog. From then on, write to the file descriptor (or use WDIOC_KEEPALIVE) at least as often as the pretimeout interval to keep the board up. Writing the byte V before closing performs a magic close, so the kernel keeps feeding the watchdog on your behalf after you close the file; closing without it leaves the watchdog armed and unfed.

The configured timeout and pretimeout stay in effect after the device is closed, so the next application to open /dev/watchdog0 inherits them.

6.2. sysfs

Path Content

/sys/class/watchdog/watchdog0/identity

Synopsys DesignWare Watchdog

/sys/class/watchdog/watchdog0/state

active or inactive

/sys/class/watchdog/watchdog0/timeout

Current timeout in seconds, read-only

/sys/class/watchdog/watchdog0/pretimeout

Current pretimeout in seconds

/sys/class/watchdog/watchdog0/timeleft

Seconds left before reset

/sys/class/watchdog/watchdog0/options

Feature bitmask

/sys/class/watchdog/watchdog0/bootstatus

Always reads 0

There is no module parameter or writable sysfs attribute for the timeout; WDIOC_SETTIMEOUT on an open file descriptor is the only way to change it from userspace.

7. Example: userspace watchdog control

The following shows a demonstration of how to control the watchdog from userspace via a small utility called wdtest.c.

You can set the timeout and the keepalive interval, which shows how to do this in your own application.

The program runs for a duration provided via a command line argument, so it is possible to see the watchdog in action.

There are a few more options that can be used for further testing, but they will not be described here.

// SPDX-License-Identifier: GPL-2.0+
/* Feed /dev/watchdog0 for a while, then stop and let it reset the board. */

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <time.h>
#include <linux/watchdog.h>

static void usage(const char *me)
{
	fprintf(stderr, "usage: %s [-d dev] [-t secs] [-k secs] [-p] [-e] runtime\n", me);
	fprintf(stderr, "  runtime  keep it alive this many seconds before stopping\n");
	fprintf(stderr, "  -d dev   watchdog device, default /dev/watchdog0\n");
	fprintf(stderr, "  -t secs  set the timeout, default leaves it alone\n");
	fprintf(stderr, "  -k secs  keepalive interval, default 5\n");
	fprintf(stderr, "  -p       restore interrupt mode if it was cleared\n");
	fprintf(stderr, "  -e       clear the pretimeout once it fires, then keep it alive\n");
	exit(1);
}

/* seconds since the reference taken at t0 */
static int elapsed(const struct timespec *t0)
{
	struct timespec now;

	clock_gettime(CLOCK_MONOTONIC, &now);
	return (int)(now.tv_sec - t0->tv_sec);
}

int main(int argc, char **argv)
{
	const char *dev = "/dev/watchdog0";
	int timeout = 0, runtime = 0, pretimeout = 0, keepalive = 5;
	int escape = 0, escaped = 0;
	int fd, val, pre, opt;
	struct timespec t0;

	while ((opt = getopt(argc, argv, "d:t:k:pe")) != -1) {
		switch (opt) {
		case 'd': dev = optarg; break;
		case 't': timeout = atoi(optarg); break;
		case 'k': keepalive = atoi(optarg); break;
		case 'p': pretimeout = 1; break;
		case 'e': escape = 1; break;
		default: usage(argv[0]);
		}
	}

	/* the runtime is the only positional argument */
	if (optind != argc - 1)
		usage(argv[0]);

	runtime = atoi(argv[optind]);
	if (runtime < 1 || keepalive < 1)
		usage(argv[0]);

	/* opening the device arms the watchdog and counts as the first ping */
	fd = open(dev, O_WRONLY);
	if (fd < 0) {
		perror(dev);
		return 1;
	}

	/* any non-zero pretimeout puts dw_wdt in interrupt mode, the value is a flag */
	if (pretimeout) {
		val = 1;
		if (ioctl(fd, WDIOC_SETPRETIMEOUT, &val))
			perror("WDIOC_SETPRETIMEOUT");
	}

	if (timeout) {
		val = timeout;
		if (ioctl(fd, WDIOC_SETTIMEOUT, &val))
			perror("WDIOC_SETTIMEOUT");
	}

	if (ioctl(fd, WDIOC_GETTIMEOUT, &timeout))
		timeout = 0;
	if (ioctl(fd, WDIOC_GETPRETIMEOUT, &pre))
		pre = 0;

	printf("timeout %d s, pretimeout %d s, keepalive every %d s, kept alive for %d s\n",
	       timeout, pre, keepalive, runtime);
	fflush(stdout);

	clock_gettime(CLOCK_MONOTONIC, &t0);

	/* keepalive phase, the board must stay up throughout */
	while (elapsed(&t0) < runtime) {
		sleep(keepalive);
		if (write(fd, "1", 1) != 1)
			perror("keepalive");
		if (ioctl(fd, WDIOC_GETTIMELEFT, &val))
			val = -1;
		printf("fed at %3d s, timeleft %d s\n", elapsed(&t0), val);
		fflush(stdout);
	}

	printf("stopped keepalive at %d s, expecting a reset within %d s\n",
	       elapsed(&t0), timeout + pre);
	fflush(stdout);

	/* hold the fd open without pinging, a close would let the kernel take over */
	clock_gettime(CLOCK_MONOTONIC, &t0);
	for (;;) {
		sleep(1);
		if (ioctl(fd, WDIOC_GETTIMELEFT, &val))
			val = -1;

		/* timeleft drops below the pretimeout once the interrupt has latched */
		if (escape && !escaped && pre > 0 && val >= 0 && val < pre) {
			int zero = 0;

			if (ioctl(fd, WDIOC_SETPRETIMEOUT, &zero))
				perror("WDIOC_SETPRETIMEOUT");
			printf("WDIOC_SETPRETIMEOUT 0 at %d s, feeding from now\n",
			       elapsed(&t0));
			escaped = 1;
		}

		if (escaped && write(fd, "1", 1) != 1)
			perror("keepalive");

		printf("%s %3d s, timeleft %d s\n",
		       escaped ? "fed, ignored" : "unfed for", elapsed(&t0), val);
		fflush(stdout);
	}
	return 0;
}