Science Shorts 2: The Scientific Method

What is science?

Science is a way of seeking to explain the world around us, either by simply observing the world or by experimentally manipulating it in some way. Science proceeds by posing familiar questions: What? Where? When? How? and Why? Observations, estimates, and patterns are three different kinds of factual claims that describe what occurred, where it occurred or when it occurred. Answers to how and why questions are factual claims of the fourth kind, namely causal hypotheses.

Scientific hypotheses

What makes a causal hypothesis “scientific”? For both scientists and the courts,Footnote 1 scientific hypotheses are those that can be empirically tested.

Empirically testable hypotheses satisfy two conditions. First, the hypothesis must be refutable.Footnote 2 A refutable hypothesis is one for which there exists the logical possibility of observations that would be considered inconsistent with the hypothesis and hence, would lead us to conclude the hypothesis is false.

Second, empirical testability requires that contradictory observations be not only logically possible but capable of being collected in practice.Footnote 3

The question of whether an hypothesis is scientific is a different question than whether it is true. For example, the claim that God made life on earth could be true. But whatever its truth, this factual claim (of the fourth kind) is not scientific because there are no observations that are, even in principle, inconsistent with it: it fails to satisfy the criterion of refutability.

Testing scientific hypotheses: The scientific method

Testing a scientific hypothesis proceeds by three basic steps. The first involves formulation of a clear, unambiguous scientific hypothesis. The second step involves the design and prosecution of a study in which the hypothesis under investigation generates at least one prediction.

Predictions are simply the study results one expects if the hypothesis under investigation is true.

Third, scientists compare the study results to those predicted. If study results match the predictions sufficiently well, then the hypothesis is supported — the results are consistent with the hypothesis being true. On the other hand, if they do not match, the hypothesis is not supported — the results are consistent with the hypothesis being false.

We can think of scientific hypotheses and predictions as a type of IF-THEN statement: IF the hypothesis is true THEN the study results should match those predicted. (Fig. 1).

Fig. 1. A simple example of the scientific method.

  • Fig. 1 - Text version

    Fig. 1. A simple example of the scientific method. We want to figure out why a light doesn’t work. One hypothesis is that the bulb is burnt out. To test this hypothesis, there are several different experiments that could be conducted, each of which yields a specific prediction. For example, in experiment 2, if the cause of the non-functioning light is a burnt-out bulb (the causal hypothesis), then the light should work when the old bulb is replaced (prediction). On the other hand, if it still doesn’t work, then the correct explanation is unlikely to be a burnt-out bulb — perhaps the light is unplugged!

We tend to think of science as being concerned only with the physical or natural world. But much of the social sciences is concerned with understanding the reasons why people behave (or misbehave) as they do. Explanations for individual or group behavior can be formulated as scientific hypotheses and tested accordingly.

Hypothesis testing as refutation

Suppose observed results match predictions. Can we conclude that the hypothesis is true? No, for a simple reason: there are always alternative explanations (hypotheses) for observed results. For example, although the light working after the bulb is replaced is consistent with the hypothesis of a burnt-out bulb, it is also consistent with the hypothesis that the power to the house was off and was restored between the time the old bulb was removed and the new one installed. So it is entirely possible that observed results match predictions yet the hypothesis is nonetheless false.

Scientific hypotheses cannot be proven because for any set of results, there are always alternate hypotheses that generate the same predictions, and scientists cannot test all possible hypotheses. This means that scientific hypotheses that scientists accept as “facts” are simply those that have been subjected to the most rigorous and exhausting testing and have failed to be refuted. In the words of the late paleontologist Stephen J. Gould:

“In science, ‘fact’ can only mean ‘confirmed to such a degree that it would be perverse to withhold provisional assent.’”Footnote 4

Note the “provisional assent” here: even scientific hypotheses for which there is compelling supporting evidence may turn out to be false, or at least incomplete. In science, there are no absolute truths.

Science for decision-making

All decisions are based on (usually implicit) causal hypotheses that connect the decision with desired or undesired outcomes. It is these underlying hypotheses that give rise to the predicted effects of alternative decisions.

For decision-makers, explicit enumeration of all underlying causal hypotheses is important because it:

  • clearly identifies the relevant science, that is, science that provides evidence concerning the truth, or otherwise, of at least one underlying hypothesis. This dramatically increases the efficiency of evidence gathering as well as clearly proscribing irrelevant science.
  • reduces the risk of “unanticipated” consequences of policy decisions. Theories of change explicitly identify the underlying causal pathways — simply collections of hypotheses — that link candidate decisions with desired and undesired outcomes. Explicit consideration of these causal pathways can bring to light potential effects of candidate decisions that might not otherwise have been considered, reducing the risk of unanticipated consequences.Footnote 5