DO-178C Objectives, Design Assurance Levels, and Structural Coverage

The airborne software standard does not prescribe a development process. It states objectives, and for each objective it states, per software level, whether the objective must be satisfied, must be satisfied with independence, or need not be satisfied at all. A project is compliant when it can show that every applicable objective was met and that the evidence exists. The distinction matters because it makes the standard adaptable: the process can be waterfall, incremental, or iterative, and the tooling can be anything, provided the objectives are addressed and the evidence is produced. What the standard does constrain heavily is verification, and the most cited of those constraints is structural coverage.

From failure condition to software level

The software level is not chosen for the software. It is inherited from the hazard assessment of the aircraft function that the software implements. The system safety assessment classifies each failure condition by its effect on the aircraft and its occupants: catastrophic, hazardous, major, minor, or no effect. Catastrophic means the failure condition would prevent continued safe flight and landing, with multiple fatalities. Hazardous means a large reduction in safety margins or the ability of the crew to cope with adverse conditions. The levels run down through major, which is a significant reduction in margins, to minor, which is a nuisance but within the crew’s capacity, and to no effect.

The mapping to software levels is fixed: catastrophic yields Level A, hazardous Level B, major Level C, minor Level D, and no effect Level E. The mapping is the standard’s way of making the software’s rigour a function of the consequence rather than of the code’s size or complexity. A small routine that contributes to a catastrophic failure condition carries Level A obligations; a large routine that can only cause a nuisance carries Level D.

The inheritance has a subtlety worth stating, because it drives architecture. A function whose failure is catastrophic may be implemented as two dissimilar channels, each of which alone cannot cause the catastrophic condition. In that case the failure condition for each channel is less severe, and the software level can be lower for each channel than for the function as a whole. The argument is a failure-probability argument, and it is only valid if the channels are genuinely independent, which brings in the common-cause reasoning of Common-Cause Failure and What Design Diversity Actually Buys. A software level reduction claimed on the basis of two identical channels is not supportable, because identical software fails identically.

What the objective tables contain

DO-178C organises its objectives into ten tables, lettered A-1 through A-10. Each table covers an activity area: planning, development standards, verification of requirements, verification of design, verification of code, testing of the integrated system, and the lifecycle data that must be produced. Each row names an objective and gives the level-specific status. A row’s status at Level A can be “with independence”, at Level C “without independence”, and at Level D “not required”.

The independence column is the part most projects underestimate. Independence means the person or team that verifies an item did not produce it, and the Annex A tables mark each objective as required with independence, required without independence, or not required, with the level deciding which. The number of objectives carrying the independence marker falls as the level drops, so the heaviest burden sits at Level A. The obligation is organisational as much as technical, and it is why a small team cannot claim Level A compliance without either growing or engaging an external verifier.

The lifecycle data is the other half of the obligation. The standard names the documents and records that must exist: the plan for software aspects of certification, the software development and verification plans, the requirements standards, design and code standards, the trace data linking high-level requirements to low-level requirements to source code to test cases, and the verification results. The trace data is what makes the objectives checkable, because it lets a reviewer follow a requirement to the code that implements it and to the test that exercises it, and follow a line of code back to a requirement or a justification for its existence.

The tables are not a checklist of documents to be produced and filed. Each objective has an activity behind it, and the activity has a purpose: to show that the requirements are correct and complete with respect to the system, that the design implements the requirements, that the code implements the design, and that the tests exercise the code against the requirements. A project that produces the documents without the activities will fail an audit on the first traceability question that goes to a specific line of code.

The structural coverage ladder

Structural coverage analysis asks a different question from requirements-based testing. Requirements-based tests ask whether the software does what it is supposed to do. Coverage analysis asks whether the tests exercised the code, so that the absence of a failure in testing can be taken as evidence about the code rather than as evidence about the tests.

The ladder has three rungs, assigned by level. At Level C, statement coverage is required: every statement in the code must have been executed by at least one test. At Level B, decision coverage: every branch decision must have taken both outcomes. At Level A, modified condition/decision coverage: every condition within a decision must have been shown to independently affect the decision’s outcome, and every entry and exit point must have been invoked.

The definitions carry the weight. A decision is a boolean expression that controls the flow of the program, such as the expression in an if or a loop condition. A condition is an operand within that decision that evaluates to a boolean without intermediate operators. Decision coverage requires the whole decision to be true at least once and false at least once. Condition coverage, which the standard does not require on its own, requires each condition to be true at least once and false at least once. Modified condition/decision coverage requires more than both: for each condition, there must exist a pair of test cases in which that condition’s value differs, every other condition in the decision is held fixed, and the decision’s outcome differs.

The consequence for test design is a bound on the minimum number of cases. A decision with n independent conditions requires at least n+1 test cases under MC/DC, because each condition needs a pair and one case can serve as the reference for several pairs. For a decision with four conditions, that is five cases rather than the sixteen a truth-table approach would enumerate, which is why MC/DC is described as a linear-cost criterion. The bound is a lower bound only: real decisions with coupled conditions, where changing one condition forces another to change, need more cases or cannot be covered at all, and the standard requires that such cases be justified.

/* A decision with three conditions. MC/DC needs at least four cases:
   one where each condition alone flips the outcome while the other two
   are held fixed. */
if (armed && (pressure > limit || override))
    trip();

/* Case 1: armed=1, pressure above limit, override=0  -> true
   Case 2: armed=0, pressure above limit, override=0  -> false  (armed flips)
   Case 3: armed=1, pressure below limit, override=0  -> false  (pressure flips)
   Case 4: armed=1, pressure below limit, override=1  -> true   (override flips)

   The fourth case also shows why short-circuit evaluation matters: with
   && evaluated left to right, override is only reached when armed is true
   and pressure is below the limit, so no test where armed is false can
   demonstrate override's independence. */

That last comment points at the analysis’s central practical difficulty. Compilers and short-circuit evaluation mean that not every condition is evaluated in every path, so a condition’s influence on the outcome is not a pure function of its value. The coverage tool must account for the actual evaluation semantics of the generated code, not the source-level expression, and a tool that reports coverage from source alone can report conditions as covered when the generated code never evaluates them. This is one reason the standard’s objectives are applied to the source code as verified against the object code, and why the compiler’s behaviour is part of the qualification argument.

When coverage cannot be achieved

The standard does not require one hundred percent coverage as an end in itself. It requires the analysis, and it requires that any code not covered be explained. The explanations fall into a few recognised categories.

Dead code is code that cannot be reached by any input, usually the residue of a defensive check for a condition the design excludes. It is removed, or it is justified and retained with an argument that the condition cannot occur.

Deactivated code is code that is present in the build but not intended to execute in the delivered configuration, such as a test mode or a function enabled only for a different variant. It must be shown that the deactivation mechanism cannot fail to deactivate it, which usually means the mechanism is a build-time exclusion rather than a run-time flag, because a run-time flag is itself code that can fail.

Defensive code is code that checks for conditions the requirements say cannot happen. It may be justified on the grounds that its failure would be detected by the check itself, but the argument has to be made, and the coverage tool will still report it as uncovered unless the tests can force the condition.

The residual category is the interesting one: code that is reachable, not defensive, and not exercised by any requirements-based test. That combination means either the requirements are incomplete or the code does something the requirements do not describe, and both are findings. The standard’s discipline is that a coverage gap of this kind cannot be closed by writing a test that merely executes the code; the test must be derived from a requirement, so the gap forces a requirements change and then a test.

Tools, qualifiers, and the substitution question

The verification chain depends on tools, and the standard’s answer is tool qualification. A tool whose output is used to justify the absence of a verification activity, or whose output could insert an error into the software without being detected, must be qualified to the level of the software it supports. A compiler at Level A is in the second category, because a miscompilation inserts an error, so the compiler must be qualified or its output must be verified by an independent means.

DO-330 defines the qualification process, and its central idea is that a tool is qualified against its intended use rather than in general. A coverage tool that reports which conditions were exercised is in the first category if its report is used in place of manual analysis, so it needs qualification at the level of the software whose coverage it measures. A requirements management tool that stores trace links but whose output is reviewed by a person is not in either category, because the review is the verification activity and the tool merely organises the evidence.

The substitution argument is what keeps the standard from freezing technology. DO-331 allows model-based development to replace some textual lifecycle data with models, provided the objectives are still met and the model’s verification is defined. DO-333 allows formal methods to substitute for some verification objectives, provided the formal analysis is shown to be sound with respect to the requirements and its results are traceable. In both cases the substitution is not a relaxation; it is a redirection of evidence, and the project has to argue that the evidence is at least as strong as what the objective originally demanded.

Where the standard stops

DO-178C governs the software’s development and verification. It does not govern the system’s safety assessment, the hardware’s integrity, or the operational procedures. A software level of A says the software must be developed to the highest rigour; it does not say the aircraft is safe, because the hazard analysis that produced the level is a separate argument made elsewhere and can itself be wrong. The standard’s own supporting document is explicit that a compliant software process is necessary but not sufficient for airworthiness.

Nor does the standard claim that its verification finds all defects. Structural coverage measures test adequacy against the code’s structure, and a requirement that is wrong will be implemented and tested and covered, with every objective satisfied and the aircraft still unsafe. This is why the requirements objectives sit at the top of the tables and carry independence at the higher levels: the standard’s authors understood that coverage analysis cannot detect a specification error, and placed the burden where it belongs. The equivalent problem in the automotive world, where the metrics are computed differently but the structural limit is the same, is discussed in Runtime Self-Tests and Diagnostic Coverage Measurement: a measurement of how thoroughly something was checked says nothing about whether the thing being checked was right.