Keywords
assessment bias
higher education policy
research evaluation
How to Cite
Abstract
The standing of universities is increasingly tied to assessments of their research prowess. Research assessment exercises have the potential to provide participants in Higher Education (HE) systems with objective assessments of research quality. However, if the evaluations of research quality produced by research assessment exercises are biased, this may undermine their contribution to a more efficient allocation of scarce resources, both within and between universities. If the bias takes the form of some Units of Assessment (UOA) receiving unjustifiably favourable assessments, submissions to these UOAs may be viewed more favourably by university managers when it comes to deciding where to put their scarce research funding. As well as distorting the internal allocation of resources, a research assessment process which lacks objectivity may alter a university’s reputational capital, which in turn may affect its ability to attract and retain students and staff. This paper provides fresh evidence based on decomposition techniques on the merits of claims of bias in research assessment exercises using data on submissions to the Business and Management Studies (BMS) and Economics and Econometrics (E&E) UOAs at the 2014 REF.
References
Banal-Estañol, A., Jofre-Bonet, M., Iori, G., Maynou, L., Tumminello, M., & Vassallo, P. (2023). Performance-based research funding: Evidence from the largest natural experiment worldwide. Research Policy, 52(6), 104780. https://doi.org/10.1016/j.respol.2023.104780
Bence, V., & Oppenheim, C. (2005). The Evolution of the UK’s Research Assessment Exercise: Publications, Performance and Perceptions. Journal of Educational Administration and History, 37(2), 137-155. https://doi.org/10.1080/00220620500211189
Biagioli, M., Kenney, M., Martin, B. R., & Walsh, J. P. (2019). Academic misconduct, misrepresentation and gaming: A reassessment. Research Policy, 48(2), 401-413. https://doi.org/10.1016/j.respol.2018.10.025
Blinder, A. S. (1973). Wage discrimination: Reduced form and structural estimates. Journal of Human Resources, 8(4), 436–455. https://doi.org/10.2307/144855
Bolton, P., & Lewis, J. (2025). Universities grip financial crisis – but at what cost to the nation? (Research Briefing CBP10037). House of Commons Library.
Checchi, D., Malgarini, M., & Sarlo, S. (2019). Do performance-based research funding systems affect research production and impact? Higher Education Quarterly, 73(1), 45-69. https://doi.org/10.1111/hequ.12185
Checchi, D., Mazzotta, I., Momigliano, S., & Olivanti, F. (2020). Convergence or polarisation? The impact of research assessment exercises in the Italian case. Scientometrics, 124(2), 1439-1455. https://doi.org/10.1007/s11192-020-03517-2
Clerides, S., Pashardes, P., & Polycarpou, A. (2011). Peer review vs metric-based assessment: Testing for bias in the RAE rating of UK economics departments. Economica, 78(311), 565–583. https://doi.org/10.1111/j.1468-0335.2009.00837.x
Coupé, T. (2003). Revealed performances: Worldwide rankings of economists and economics departments, 1990–2000. Journal of the European Economic Association, 1(6), 1309–1345. https://doi.org/10.1162/154247603322752557
Daymont, T. N., & Andrisani, P. J. (1984). Job Preferences, College Major, and the Gender Gap in Earnings. The Journal of Human Resources, 19(3), 408. https://doi.org/10.2307/145880
Deem, R. (1998). ‘New managerialism’ and higher education: The management of performances and cultures in universities in the United Kingdom. International Studies in Sociology of Education, 8(1), 47-70. https://doi.org/10.1080/0962021980020014
Doyle, J. R., Arthurs, A. J., Green, R. H., McAulay, L., Pitt, M. R., Bottomley, P. A., & Evans, W. (1996). The judge, the model of the judge, and the model of the judged as judge: Analyses of the UK 1992 research assessment exercise data for business and management studies. Omega, 24(1), 13–28. https://doi.org/10.1016/0305-0483(95)00044-5
Erickson, M., Hanna, P., & Walker, C. (2021). The UK higher education senior management survey: A statactivist response to managerialist governance. Studies in Higher Education, 46(11), 2134–2151. https://doi.org/10.1080/03075079.2020.1712693
Groot, T., & García-Valderrama, T. (2006). Research quality and efficiency. Research Policy, 35(9), 1362-1376. https://doi.org/10.1016/j.respol.2006.07.002
HESA. (2025). Higher education student statistics: UK, 2023/24 – Student numbers and characteristics (Statistical Bulletin SB271). Higher Education Statistics Agency.
Johnes, J., Taylor, J., & Francis, B. (1993). The research performance of UK universities: A statistical analysis of the results of the 1989 Research Selectivity Exercise. Journal of the Royal Statistical Society, Series A, 156(2), 271–286. https://doi.org/10.2307/2982732
Johnston, J., Reeves, A., & Talbot, S. (2014). Has economics become an elite subject for elite UK universities?. Oxford Review of Education, 40(5), 590-609. https://doi.org/10.1080/03054985.2014.959912
Johnston, J., & Reeves, A. (2017). Assessing research performance in UK universities using the case of the economics and econometrics unit of assessment in the 1992–2014 research evaluation exercises. Research Evaluation, 26(1), 28–40. https://doi.org/10.1093/reseval/rvw021
Johnston, J., & Reeves, A. (2018). An investigation into the role played by research assessment in the socio-geographic fragmentation of undergraduate economics education in the UK. Higher Education, 76(4), 589-614. https://doi.org/10.1007/s10734-017-0227-9
Kelly, A., Morris, H., & Harvey, C. (2009). Modelling the outcome of the UK business and management studies RAE 2008 with reference to the ABS journal quality guide. Unpublished manuscript.
Marginson, S. (2017). Elite, mass, and high-participation higher education. In J. C. Shin & P. Teixeira (Eds.), Encyclopedia of international higher education systems and institutions. Springer. https://doi.org/10.1007/978-94-017-9553-1_50-1
Martin, B. R. (2011). The Research Excellence Framework and the ‘impact agenda’: Are we creating a Frankenstein monster? Research Evaluation, 20(3), 247–254. https://doi.org/10.3152/095820211x13118583635693
McKay, S. (2012). Social policy excellence – peer review or metrics? Analyzing the 2008 Research Assessment Exercise in social work and social policy and administration. Social Policy & Administration, 46(5), 526–543. https://doi.org/10.1111/j.1467-9515.2011.00824.x
Mingers, J., Watson, K., & Scaparra, M. P. (2012). Estimating Business and Management journal quality from the 2008 Research Assessment Exercise in the UK. Information Processing & Management, 48(6), 1078-1093. https://doi.org/10.1016/j.ipm.2012.01.008
Neumark, D. (1988). Employers’ discriminatory behavior and the estimation of wage discrimination. Journal of Human Resources, 23(3), 279–295. https://doi.org/10.2307/145830
Oaxaca, R. (1973). Male-female wage differentials in urban labour markets. International Economic Review, 14(3), 693–709. https://doi.org/10.2307/2525981
Oaxaca, R. L., & Ransom, M. R. (1988). Searching for the effect of unionism on the wages of union and nonunion workers. Journal of Labor Research, 9(2), 139-148. https://doi.org/10.1007/bf02685237
Oaxaca, R. L., & Ransom, M. R. (1994). On discrimination and the decomposition of wage differentials. Journal of Econometrics, 61(1), 5-21. https://doi.org/10.1016/0304-4076(94)90074-4
Oaxaca, R. L., & Ransom, M. (1998). Calculation of approximate variances for wage decomposition differentials. Journal of Economic and Social Measurement, 24(1), 55-61. https://doi.org/10.3233/jem-1998-0143
Oaxaca, R. L., & Ransom, M. R. (1994). On discrimination and the decomposition of wage differentials. Journal of Econometrics, 61(1), 5-21.
Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences, 5(2), 187–195. https://doi.org/10.1017/s0140525x00011183
Robbins, L. (1963). Higher education: Report of the Committee on Higher Education. HMSO.
Roberts, G. (2003). Review of research assessment: A report to the funding bodies (Report 2003/22). HEFCE.
Rowlinson, M., Harvey, C., Kelly, A., Morris, H., & Todeva, E. (2015). Accounting for research quality: Research audits and the journal rankings debate. Critical Perspectives on Accounting, 26, 2–22. https://doi.org/10.1016/j.cpa.2013.05.012
Sharp, S., & Coleman, S. (2005). Ratings in the Research Assessment Exercise 2001: The patterns of university status and panel membership. Higher Education Quarterly, 59(2), 153–171. https://doi.org/10.1111/j.1468-2273.2005.00288.x
Spence, C. (2019). ‘Judgement’ versus ‘metrics’ in higher education management. Higher Education, 77(5), 761–775. https://doi.org/10.1007/s10734-018-0300-z
Tapper, T., & Salter, B. (2003). Interpreting the process of change in higher education: The case of the research assessment exercises. Higher Education Quarterly, 57(1), 4-23. https://doi.org/10.1111/1468-2273.00232
Taylor, J. (1995). A statistical analysis of the 1992 Research Assessment Exercise. Journal of the Royal Statistical Society, Series A, 158(2), 241–261. https://doi.org/10.2307/2983291
Taylor, J. (2011). The assessment of research quality in UK universities: Peer review or metrics. British Journal of Management, 22(2), 202–217. https://doi.org/10.1111/j.1467-8551.2010.00722.x
Tomlinson, M., & Watermeyer, R. (2022). When masses meet markets: Credentialism and commodification in twenty-first century higher education. Discourse: Studies in the Cultural Politics of Education, 43(2), 173–187. https://doi.org/10.1080/01596306.2020.1814996
Trow, M. (1973). Problems in the transition from elite to mass higher education. Carnegie Commission on Higher Education.
Universities UK. (2025). Universities grip financial crisis – but at what cost to the nation? https://www.universitiesuk.ac.uk/what-we-do/creating-voice-our-members/media-releases/universities-grip-financial-crisis-what
Willmott, H. (2011). Journal list fetishism and the perversion of scholarship: Reactivity and the ABS list. Organization, 18(4), 429–442. https://doi.org/10.1177/1350508411403532
Willson, R., Stewart-Robertson, O., Julien, H., & Given, L. M. (2026). Academic casualisation and precarity: a scoping review. Higher Education Research & Development, 45(4), 1165-1187. https://doi.org/10.1080/07294360.2025.2573983
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright (c) 2026 James Johnston