From Data to Policy Options: A Six-Step Route
Move from an observed pattern to feasible policy choices without overstating causation.
GRS Editorial · GPGRead guideGRS editorial · Data & indicators
Designing Research Indicators without Misleading Rankings
Choose meaningful measures, document their limits and compare institutions responsibly.
Data & indicators
A practical starting point.
Choose meaningful measures, document their limits and compare institutions responsibly. This guide offers a starting method for teams developing research and institutional decisions. Adapt it to the question, context and relevant scientific, ethics and data requirements.
Step 01
Write the construct first
An indicator is a proxy for something that matters. Define the underlying concept before choosing a number. For research quality, publication volume may describe activity but does not alone capture rigour, relevance or impact. Use a small set of measures when one cannot represent the whole construct.
Step 02
Check the data generating process
Document the source, unit of analysis, reporting period, missingness and incentives that shape the data. A change in a count may reflect a new reporting rule rather than a real change in performance. Disaggregate where it improves fairness and confidentiality permits.
Step 03
Compare like with like
Institutions differ in mission, scale, population and resources. Choose peer groups and normalisation rules before seeing results. Show ranges or uncertainty where appropriate. If comparability is too weak, use descriptive profiles and case learning rather than a league table.
Step 04
Build a review cycle
Test whether users interpret the measure as intended and whether it creates perverse incentives. Assign responsibility for corrections and version control. An indicator is useful when it improves a decision and remains open to revision, not when it merely fills a dashboard.
Illustrative application
What this could look like.
Consider comparing research centres by annual publications. Before any ranking, examine centre size, disciplinary norms, collaborative authorship and the quality of metadata. Add measures such as methods transparency or policy uptake only where they can be assessed consistently. A narrative profile may reveal a small centre’s valuable specialised work that a simple volume score would miss.
Put it into practice
A short working checklist.
- Write the precise decision or research question this work should support.
- Record the sources, methods, stakeholders and permissions needed before proceeding.
- Agree who reviews the output and how limitations will be communicated.
- Set a date to revisit the decision when evidence or conditions change.
GRS Group editorial guide · Published 24 September 2026 · Informational guidance, not an empirical study or formal professional advice.
Related research
Continue with the centre and knowledge model.
Global Institutional Excellence Research Center Research & Knowledge All guidesFurther reading