Benchmarks in QRiH
In a benchmark a comparison is made between the research unit to be evaluated and a comparable other unit, national or international. A benchmark is at the one hand useful for the research unit to be evaluated: how do we compare with another relevant unit? On the other hand a benchmark helps evaluators to evaluate.
A good, transparent benchmark is a comparison on characteristics with level playing fields. This is difficult to achieve in practice, firstly because research units can vary considerably in scope, mission and research program. Secondly, because evaluation procedures and criteria often show large differences.
There are two ways to work with benchmarks. In a limited number of domains in which a lot is published in international journals, it is possible to use production and citation indicators (for the usability per domain see the Domain Profiles). A second way is a more qualitative form of benchmarking which can be applied to all domains: a comparison is made based on research programs and objectives. Both forms of benchmarking can be part of the SWOT analysis, which also aims to establish the positioning of the research unit in the research environment.
For the benchmark, indicate which relevant research unit (or research units) is used for the comparison, and in which way a comparison can be made:
- The relevance of the chosen research unit (or units) becomes clear from the own narrative, in particular the mission and ambition.
- Besides indicating similarities, it is also possible to indicate the differences.
- If possible, compare the research units on the basis of the relevant indicators (section 2 of the format). The indicators will usually be of a qualitative nature. Consider, for example, a comparison of the used research methods, the type of partners outside the academy, the type of activities to share knowledge, or the focus on personal or social grants.
- If available, results from recent research evaluations of the other units can be mentioned.
Comparisons can also be made on other grounds, for example on the basis of the QRiH instrument:
- The profile of the research unit can be compared with the (national) domain profiles (see Domain profiles).
- The interdisciplinary character can be underlined by highlighting the diversity of publications in authorized channels of different domains.
- The publication profile of the unit can be translated into specialism, disciplinary and multidisciplinary orientation (see the webpage production profiles of institutes).
In general, the LAP advises to be cautious with quantitative benchmarks. In the humanities it is in most cases not possible to draw up quantitative benchmarks that are based on generic domain-specific characteristics, such as citation analyzes based on bibliometric data from Web of Science, Scopus or Google Scholar (see web pages Bibliometrics, Domain Profiles).
If desired, it is possible to make use of the QRiH lists of journals and publishers or other international quality systems, such as VABB-SHW, CRISTiN or ERIH Plus (see webpage Other Initiatives). The LAP also notes that these lists are much more extensive than the QRiH lists and are therefore less selective.