Measure impact as though the answer really mattered

Despite the REF’s huge bureaucratic burden, impact is still not taken seriously. Kathryn Oliver outlines what a true effort to understand the effects of research might look like.

Almost everyone agrees that bibliometrics are a bad way to measure the impact of research on society. Such measures are said to discriminate against early-career researchers, interdisciplinary researchers and possibly women, and in any case they are not good indicators for wider societal impact. But there is not much consensus on how impact should be evaluated.

Perhaps the best way to think about this is to admit that measurement is a political choice, always contingent on practical and logistic concerns. Choosing metrics such as citations, for example, tells us about what we value—in this case, time and convenience.