Test data can effectively support design decisions only if it corresponds to clear questions. The assumptions you want to verify and the quantities you need to collect should be stated before testing.

First, convert design problems into hypotheses.

Before conducting the experiment, it is important to clarify what we hope to determine from the data, such as whether a certain structural change leads to specific responses. The clearer the assumptions, the easier it is to determine the quantities to be measured and the methods to be used. This also helps to avoid having to search for data to support a conclusion after the experiment has been completed.

Separate raw data from processed data.

Data processing steps such as filtering, converting, and smoothing can affect how the data is presented. The report should clearly distinguish between the original data and the processed results, and explain the methods used. Maintaining this distinction allows others to review the conclusions, and also makes it easier to reuse existing data when analytical methods change.

When organizing the results, it is important to keep track of the experimental conditions and any anomalies, and to distinguish between direct measurements and calculated indicators. Before comparing data from different batches, it is necessary to check that the testing conditions are consistent.

Is there a basis for the differences found?

If there are differences between the two groups of results, it may be due to design changes, or it could be due to variations in conditions and measurements. When comparing results, it is important to first ensure that all common conditions are consistent, and to identify which factors have been controlled. If necessary, repeated experiments or further checks can be conducted to reduce uncertainty, rather than simply comparing the average values.

Improvements should be validated in the next round of testing.

Design modifications should specify which findings are targeted and what changes are expected. In the next round of verification, it is important to see if the expected outcomes are achieved, and to note any new issues that arise. This way, the data can support continuous iterations, rather than being limited to a single test report.

Process example: Using control conditions to explain the effects of modifications.

It is assumed that we want to compare two different structural options. The experiment design should strive to standardize other key conditions, and any factors that cannot be standardized should be recorded. After identifying any differences, it is necessary to determine whether these differences are causally related to the design changes. Only then can a decision be made on whether additional tests are needed. Including both data supporting and opposing the original hypothesis in the report helps the team to more objectively determine the next steps to take.

Common question: Is a result that does not meet expectations considered a failure?

As long as the conditions and records are reliable, deviations from expectations can still provide valuable information about the design. It is important to identify areas where the model or assumptions need adjustment, rather than just retaining the results that perform best.

Key points for verification during implementation

• Hypotheses should be clear.
• Data processing should be transparent.
• Conditions must be comparable.
• The effects of modifications have been reviewed.

The improvement plan should be able to trace back to specific findings, and its effectiveness should be confirmed through subsequent verifications. This creates a coherent process for design iteration.