London School of Economics Impact Blog (LSE) – ‘For many years, academia has relied on citation count as the main way to measure the impact or importance of research, informing metrics such as the Impact Factor and the h-index. But how well do these metrics actually align with researchers’ subjective evaluation of impact and significance? Rachel Borchardt and Matthew R. Hartings report on a study that compares researchers’ perceptions of significance, importance, and what is highly cited with actual citation data. The results reveal a strikingly large discrepancy between perceptions of impact and the metric we currently use to measure it.

Academia, we have a problem. What began as an attempt to quantify research quality has gotten away from us and taken on a life of its own. This problem isn’t particularly new; it has been widely recognised by scholars and researchers and, as a result, is being talked about more openly. The problem comes down to defining and measuring impact. A simple description for impactful research is research that gets used. Some research has the ability to transform society through groundbreaking discovery, to impact social policy and government regulation through eye-opening analysis, and the ability to engage public attention with research that’s relevant to their lives, environment, or wellbeing. Loosely bound together, we tend to refer to this kind of research as “high-impact”, and it has become the focus for many universities, research centres, and administrators as they compete for grant funding, to attract the best and brightest students, and for prestige and rankings. However, designating research as high-impact is not as straightforward as it may seem. For many years, academia has relied on citation count as the main way in which we measure impact or importance of research. As a result, citation count is one of the primary metrics used when evaluating researchers. Citation counts also form the basis for other metrics, most notably Clarivate’s Impact Factor as well as the h-index, which respectively evaluate journal quality/prestige and researcher renown…”