Runtime Self-Tests and Diagnostic Coverage Measurement

A safety mechanism is only worth its claimed coverage if it is still working when the demand arrives. The comparator in a lockstep pair, the plausibility check on a sensor reading, the watchdog’s window comparator, and the CRC on a flash image are all logic, and logic can fail. A fault that disables a checker while the checker is idle is invisible until the day it is needed, at which point the system discovers both the original fault and the dead checker at once. Runtime self-tests exist to close that window, and diagnostic coverage is the number that says how much of it they close.

Latent faults and the arithmetic of exposure

The exposure of a latent fault is the product of its undetected residence time and the demand rate of the function it guards. In low demand mode, a diagnostic that runs once per proof-test interval leaves a fault undetected for up to that interval, and the resulting contribution to the probability of failure on demand is roughly the diagnostic’s own failure rate multiplied by half the test interval, in the standard approximation that treats detection time as uniformly distributed. Halving the test interval halves that contribution, which is why test intervals are a design parameter and not a maintenance convenience.

In continuous mode the same reasoning applies with the interval expressed in hours, and the contribution is the diagnostic failure rate multiplied by the mean time to detect. A diagnostic that runs every ten milliseconds contributes a factor of ten less than one that runs every hundred milliseconds, provided the detection latency is dominated by the interval rather than by the diagnostic’s own execution.

The catch is that the self-test itself consumes the resource it is testing. A full memory test that reads every location takes time proportional to the size of the memory, and while it runs, either the safety function is suspended or the test is interleaved with it. A 1 MiB flash region read at tens of megabytes per second takes tens of milliseconds; a full march test over the same region takes several passes and proportionally longer. The design question is not whether to test but how to partition the test so that no part of the logic goes untested for longer than the assumed detection time.

Partitioned testing and the test slot

The practical pattern is to divide the hardware into blocks and test one block per cycle of a supervisory schedule, with the schedule arranged so that every block is visited within the required interval. The schedule is a table of test slots, each with a duration and a set of blocks, and the total cycle time is the sum. If the required interval is 100 milliseconds and there are twenty blocks, each block gets a 5 millisecond slot on average, and the design must ensure that no block’s slot is skipped under any operating condition.

Skipping is the failure mode that matters. A test that is deferred when the system is busy is a test that is never run on the busiest system, which is exactly the one where faults matter most. The scheduler therefore has to treat test slots as reserved capacity rather than as background work, and the acceptance argument has to show that the worst-case load still leaves the slots intact. This is the same reasoning as the reserved-bandwidth argument for a periodic task in Worst-Case Execution Time Analysis and Its Uses, applied to a task whose output is a diagnostic verdict.

Testing in parallel with the function is possible for some blocks and not others. A memory region that the function is not currently using can be tested while the function runs from a different region, provided the test’s accesses do not contend for the bus in a way that changes the function’s timing. A register file in active use cannot be tested that way, so its test has to either run at a point where the contents are saved elsewhere or use a technique that does not disturb the live values, such as reading through a shadow copy and comparing. Designs that claim full coverage of a register file without either saving or shadowing are usually claiming coverage of the logic that writes it, not of the storage itself.

How coverage is actually measured

Diagnostic coverage is not derived from the test’s description; it is measured, and the measurement method is fault injection. A fault model is chosen, typically single stuck-at faults on gate-level nets for digital logic or a bit-flip model for memory, and each fault is injected into a simulation or into a hardware model while the diagnostic runs. A fault is covered if the diagnostic reports a failure, and the coverage is the fraction of injected faults that are detected, often weighted by the fault’s classification.

The weighting is where the number becomes contestable. A raw stuck-at coverage counts every net equally, which overweights logic that has many nets and underweights blocks with few. ISO 26262-5 defines the classifications that matter for the analysis: single-point faults, which alone cause the violation of a safety goal; residual faults, which are part of a multiple-point failure whose other parts are not detected; and latent faults, which are not detected by a mechanism the driver would notice. The diagnostic coverage figure for a mechanism is computed against the relevant population, so a mechanism that covers 99 percent of all nets but misses the few nets in the safety-critical path has a poor coverage figure for the purpose it was designed for.

The distinction between the two coverage concepts in ISO 26262 is worth stating precisely, because they are frequently conflated. Diagnostic coverage is the fraction of a defined fault population that a mechanism detects. Single-point fault metric and latent fault metric are ratios computed over the safety-relevant faults of the hardware element, with targets that depend on the integrity level the element carries, which is itself derived from the hazard rather than chosen, as Deriving a Safety Integrity Level from Risk Parameters describes. A mechanism can have high diagnostic coverage and still leave the metric below target if the fault population it does not cover is the dangerous one, which is why the metrics are computed per element and not per mechanism.

Quantity	Population	What it answers
Diagnostic coverage	Faults the mechanism could see	How good is this mechanism
Single-point fault metric	Safety-relevant faults of the element	How much of the dangerous population is covered at all
Latent fault metric	Faults in mechanisms themselves	How likely is a mechanism to be dead when needed

Self-test of the checker, not just of the checked

The tests described so far check the functional hardware. The mechanisms that check it need their own tests, and those tests must exercise the detection path rather than the data path. A CRC unit is tested by feeding it a known message and verifying the expected signature; a comparator is tested by forcing a mismatch; a window watchdog is tested by refreshing outside the window and confirming that the reset occurs.

/* A self-test that exercises the detection path. The watchdog is expected
   to reset the device, so the test records its intent in non-volatile
   memory first and the startup code checks whether the reset arrived. */
#define ST_NV_MAGIC 0x5A17C0DEu

int watchdog_detection_self_test(void)
{
    nv_write(WDT_TEST_MARK, ST_NV_MAGIC);

    /* Refresh before the window opens. On a healthy device this resets
       the core and the next boot observes the mark plus a reset cause
       of watchdog, which the startup path turns into a pass. */
    wdt_write_key(WDT_REFRESH_KEY);

    /* Reaching this line means the device accepted an illegal refresh. */
    nv_write(WDT_TEST_MARK, 0u);
    return -1;
}

The pattern generalises: a checker’s self-test must be able to distinguish “the checker did not fire” from “the condition was never generated”, and the only way to distinguish them is to record the intent before generating the condition and confirm the outcome after the reset. Tests that generate the condition and then read a status bit are weaker, because the status bit is part of the same checker that may be dead.

There is a second-order problem here that the standards name explicitly. A self-test that runs at startup and fails produces a fault at a moment when the system has not yet begun to operate, so the response can be simple: refuse to start. A self-test that runs during operation and fails must decide what to do while the plant is running, and the decision may be harder than the test. A failed flash CRC during a mission usually means “enter the safe state”, but a failed test of a non-critical diagnostic may only warrant a maintenance flag. The classification of each test’s failure into a response is part of the safety concept, not of the test code, and a test whose failure has no defined response is a test that will eventually be ignored.

The limits of what a test can establish

A self-test proves that the tested logic produced the expected result under the tested conditions. It does not prove the absence of faults that the test pattern does not excite, and it does not prove that the diagnostic will behave correctly in a situation the test did not model. Coverage measurement addresses the first limitation through the fault model, and the model’s adequacy is the weak link: stuck-at faults do not model bridging, crosstalk, or the analog behaviour of a marginal timing path, and a design can have 99 percent stuck-at coverage and still fail in the field on a fault the model never represented.

The second limitation is structural. A test that runs periodically has a window, and the window is a design parameter that the safety analysis must use rather than a detail the test code can choose. A mechanism whose detection time exceeds the fault tolerant time interval of the process it guards is not a mechanism for that process, however high its coverage, because the harm occurs before the detection. That comparison, between the diagnostic’s detection time and the process’s tolerance, is the constraint that ties the test schedule to the physical system, and it is the reason the interval appears in the safety requirements rather than in the test specification. The same comparison appears in the watchdog design, where the window’s lower bound is set by the fastest legitimate pass through the supervised loop, as discussed in Windowed Watchdogs and the Program-Flow Contract.