An acceptance flight is not the same as acceptance evidence. A system may appear to complete a mission while the parties remain unable to answer which software version flew, which parameters were active, which anomalies occurred, and whether the retained logs support the observed result. The evidence matrix closes that gap by placing every acceptance-critical requirement beside its method, configuration, threshold, artifact, and disposition.
This article presents a vendor-neutral planning workflow. It does not replace the applicable flight authorization, airworthiness process, safety case, quality system, or contract.
The minimum matrix
| Field | What to record | Why it matters |
|---|---|---|
| Requirement ID | Stable identifier and exact requirement text | Prevents interpretation drift |
| Criticality | Release-blocking, conditional, or informational | Focuses decision authority |
| Verification method | Inspection, analysis, demonstration, or test | Defines the proof route |
| Configuration | Aircraft, payload, software, parameters, GCS, accessories | Makes the result reproducible |
| Preconditions | Site, weather rule, battery state, personnel, approvals | Prevents invalid comparisons |
| Procedure | Approved step or test-case reference | Controls execution |
| Measurement | Source, unit, sampling, calculation | Identifies the authoritative data |
| Acceptance rule | Pass, fail, tolerance, or conditional rule | Removes after-the-fact negotiation |
| Artifact | Log, photograph, report, checklist, signature | Preserves the evidence |
| Disposition | Pass, fail, waived, deferred, retest | Records the release decision |
The matrix is not a summary prepared after the test. It is the control document that connects requirements, execution, and decision authority before the test begins.
1. Freeze the configuration
Acceptance evidence is meaningful only when it refers to an identified configuration. Record the aircraft serial or asset identifier, propulsion and payload configuration, autopilot and ground-control versions, parameter file hash or controlled export, radio and network setup, battery identifiers, relevant firmware, map or mission file, and installed accessories.
If a parameter changes during troubleshooting, do not silently continue the same test case. Record the change, decide whether the earlier evidence remains valid, and establish the configuration for the retest. Configuration discipline is not paperwork added to flight operations; it is what allows the buyer to know what was accepted.
2. Run a test-readiness review
Before launch, the test director should confirm that the requirement baseline and test procedure are approved, personnel roles are assigned, the operating approval and site conditions are valid, instrumentation and data capture are ready, emergency and lost-link responses are briefed, and stop criteria are understood.
A concise readiness checklist should include:
- approved test objective and requirement IDs;
- test configuration and known deviations;
- airspace, site, and operating authorization status;
- weather observation and decision rule;
- crew roles and communication channel;
- data-storage capacity and time synchronization;
- emergency, abort, and recovery actions;
- witness and signature requirements.
The readiness review should be proportionate to risk. Its purpose is to prevent a technically successful flight from becoming unusable evidence because prerequisites were not controlled.
3. Distinguish observation from measurement
“The video looked clear” is an observation. A controlled result identifies the source file, sensor configuration, target or scene, distance and lighting assumptions, processing applied, and acceptance threshold. The same distinction applies to endurance, position accuracy, communications, payload release, route completion, and operator workload.
For each measured result, identify the authoritative source. A ground-control display, onboard log, external tracker, and observer note may disagree. The test plan should state which source governs each requirement and how discrepancies will be investigated.
4. Control anomalies and retests
Do not force every event into pass or fail. Use an anomaly register that captures the time, test step, observed behavior, safety effect, affected requirement, initial containment, evidence references, owner, and disposition. Then decide whether the anomaly is a defect, test limitation, environmental interruption, documentation error, or configuration issue.
A retest must answer three questions: what changed, which prior evidence is invalidated, and what scope must be repeated? Repeating only the visible failed step may be insufficient if the corrective action affects interfaces or earlier assumptions.
| Disposition | Appropriate use | Required control |
|---|---|---|
| Pass | Threshold met with valid evidence | Signed result and artifact link |
| Conditional pass | Limited open item does not block defined use | Owner, due date, operating constraint |
| Fail | Threshold not met or evidence invalid | Defect record and release block |
| Deferred | Test cannot validly be completed | Reason, prerequisites, new date |
| Waiver/deviation | Authorized acceptance of a known departure | Named authority and scope |
5. Build the release pack
The release pack should allow a qualified reviewer who did not attend the flight to understand what was tested and why the decision was made. Include the signed matrix, approved procedures, configuration baseline, readiness record, environmental record, raw and processed data, anomaly register, corrective actions, retest evidence, open-item list, operating limitations, and acceptance certificate or decision memo.
Official autopilot documentation is useful when defining data capture and configuration exports. See the PX4 flight log documentation and the ArduPilot log analysis documentation. Use current project documentation for the selected stack and verify the actual installed version; documentation alone does not prove that logging was enabled or complete during a specific test.
6. Use evidence quality gates
Before accepting a result, apply five gates:
- Identity: Does the evidence identify the tested configuration?
- Integrity: Is the artifact complete, readable, and protected from accidental replacement?
- Traceability: Does it point to the exact requirement and test step?
- Interpretability: Can a reviewer understand the measurement and calculation?
- Authority: Is the disposition signed by the person authorized to make it?
If any gate fails, the system may still have performed well, but the acceptance record is incomplete.
Practical closeout test
Give the completed evidence pack to a reviewer who was not present. Ask that person to reproduce the pass/fail table, identify the accepted configuration, and list all open limitations without interviewing the test team. If the reviewer cannot do so, the pack is not yet a release record.
Operator takeaway
The acceptance matrix converts a flight event into an accountable decision. It protects both buyer and supplier by making the conditions, proof, and disposition visible before disagreement begins. The best matrix is not the most elaborate one. It is the one that preserves a complete line from requirement to authorized release.
