Skip to content
Educational Management and Policy
e‑ISSN 3143-4541 Open Access · CC BY 4.0
Vol. 1 · Issue 1 · 2026 May 21, 2026 Articles

Assessment bias in the 2014 UK Research Excellence Framework: Evidence from economics and business management

JJ
James Johnston Corresponding University of the West of Scotland James.Johnston@uws.ac.uk United Kingdom
Pages43-68 PublishedMay 21, 2026 LicenseOpen Access
PDF
EMP
VOL 1 · ISS 1 · 2026 EMP Journal View issue

Abstract

The standing of universities is increasingly tied to assessments of their research prowess. Research assessment exercises have the potential to provide participants in Higher Education (HE) systems with objective assessments of research quality. However, if the evaluations of research quality produced by research assessment exercises are biased, this may undermine their contribution to a more efficient allocation of scarce resources, both within and between universities. If the bias takes the form of some Units of Assessment (UOA) receiving unjustifiably favourable assessments, submissions to these UOAs may be viewed more favourably by university managers when it comes to deciding where to put their scarce research funding. As well as distorting the internal allocation of resources, a research assessment process which lacks objectivity may alter a university’s reputational capital, which in turn may affect its ability to attract and retain students and staff. This paper provides fresh evidence based on decomposition techniques on the merits of claims of bias in research assessment exercises using data on submissions to the Business and Management Studies (BMS) and Economics and Econometrics (E&E) UOAs at the 2014 REF.

Keywords

#Research assessment #assessment bias #higher education policy #research evaluation

References

  1. Banal-Estañol, A., Jofre-Bonet, M., Iori, G., Maynou, L., Tumminello, M., & Vassallo, P. (2023). Performance-based research funding: Evidence from the largest natural experiment worldwide. Research Policy, 52(6), 104780. https://doi.org/10.1016/j.respol.2023.104780 DOI: https://doi.org/10.1016/j.respol.2023.104780
  2. Bence, V., & Oppenheim, C. (2005). The Evolution of the UK’s Research Assessment Exercise: Publications, Performance and Perceptions. Journal of Educational Administration and History, 37(2), 137-155. https://doi.org/10.1080/00220620500211189 DOI: https://doi.org/10.1080/00220620500211189
  3. Biagioli, M., Kenney, M., Martin, B. R., & Walsh, J. P. (2019). Academic misconduct, misrepresentation and gaming: A reassessment. Research Policy, 48(2), 401-413. https://doi.org/10.1016/j.respol.2018.10.025 DOI: https://doi.org/10.1016/j.respol.2018.10.025
  4. Blinder, A. S. (1973). Wage discrimination: Reduced form and structural estimates. Journal of Human Resources, 8(4), 436–455. https://doi.org/10.2307/144855 DOI: https://doi.org/10.2307/144855
  5. Bolton, P., & Lewis, J. (2025). Universities grip financial crisis – but at what cost to the nation? (Research Briefing CBP10037). House of Commons Library.
  6. Checchi, D., Malgarini, M., & Sarlo, S. (2019). Do performance-based research funding systems affect research production and impact? Higher Education Quarterly, 73(1), 45-69. https://doi.org/10.1111/hequ.12185 DOI: https://doi.org/10.1111/hequ.12185
  7. Checchi, D., Mazzotta, I., Momigliano, S., & Olivanti, F. (2020). Convergence or polarisation? The impact of research assessment exercises in the Italian case. Scientometrics, 124(2), 1439-1455. https://doi.org/10.1007/s11192-020-03517-2 DOI: https://doi.org/10.1007/s11192-020-03517-2
  8. Clerides, S., Pashardes, P., & Polycarpou, A. (2011). Peer review vs metric-based assessment: Testing for bias in the RAE rating of UK economics departments. Economica, 78(311), 565–583. https://doi.org/10.1111/j.1468-0335.2009.00837.x DOI: https://doi.org/10.1111/j.1468-0335.2009.00837.x
  9. Coupé, T. (2003). Revealed performances: Worldwide rankings of economists and economics departments, 1990–2000. Journal of the European Economic Association, 1(6), 1309–1345. https://doi.org/10.1162/154247603322752557 DOI: https://doi.org/10.1162/154247603322752557
  10. Daymont, T. N., & Andrisani, P. J. (1984). Job Preferences, College Major, and the Gender Gap in Earnings. The Journal of Human Resources, 19(3), 408. https://doi.org/10.2307/145880 DOI: https://doi.org/10.2307/145880
  11. Deem, R. (1998). ‘New managerialism’ and higher education: The management of performances and cultures in universities in the United Kingdom. International Studies in Sociology of Education, 8(1), 47-70. https://doi.org/10.1080/0962021980020014 DOI: https://doi.org/10.1080/0962021980020014
  12. Doyle, J. R., Arthurs, A. J., Green, R. H., McAulay, L., Pitt, M. R., Bottomley, P. A., & Evans, W. (1996). The judge, the model of the judge, and the model of the judged as judge: Analyses of the UK 1992 research assessment exercise data for business and management studies. Omega, 24(1), 13–28. https://doi.org/10.1016/0305-0483(95)00044-5 DOI: https://doi.org/10.1016/0305-0483(95)00044-5
  13. Erickson, M., Hanna, P., & Walker, C. (2021). The UK higher education senior management survey: A statactivist response to managerialist governance. Studies in Higher Education, 46(11), 2134–2151. https://doi.org/10.1080/03075079.2020.1712693 DOI: https://doi.org/10.1080/03075079.2020.1712693
  14. Groot, T., & García-Valderrama, T. (2006). Research quality and efficiency. Research Policy, 35(9), 1362-1376. https://doi.org/10.1016/j.respol.2006.07.002 DOI: https://doi.org/10.1016/j.respol.2006.07.002
  15. HESA. (2025). Higher education student statistics: UK, 2023/24 – Student numbers and characteristics (Statistical Bulletin SB271). Higher Education Statistics Agency.
  16. Johnes, J., Taylor, J., & Francis, B. (1993). The research performance of UK universities: A statistical analysis of the results of the 1989 Research Selectivity Exercise. Journal of the Royal Statistical Society, Series A, 156(2), 271–286. https://doi.org/10.2307/2982732 DOI: https://doi.org/10.2307/2982732
  17. Johnston, J., Reeves, A., & Talbot, S. (2014). Has economics become an elite subject for elite UK universities?. Oxford Review of Education, 40(5), 590-609. https://doi.org/10.1080/03054985.2014.959912 DOI: https://doi.org/10.1080/03054985.2014.959912
  18. Johnston, J., & Reeves, A. (2017). Assessing research performance in UK universities using the case of the economics and econometrics unit of assessment in the 1992–2014 research evaluation exercises. Research Evaluation, 26(1), 28–40. https://doi.org/10.1093/reseval/rvw021 DOI: https://doi.org/10.1093/reseval/rvw021
  19. Johnston, J., & Reeves, A. (2018). An investigation into the role played by research assessment in the socio-geographic fragmentation of undergraduate economics education in the UK. Higher Education, 76(4), 589-614. https://doi.org/10.1007/s10734-017-0227-9 DOI: https://doi.org/10.1007/s10734-017-0227-9
  20. Kelly, A., Morris, H., & Harvey, C. (2009). Modelling the outcome of the UK business and management studies RAE 2008 with reference to the ABS journal quality guide. Unpublished manuscript.
  21. Marginson, S. (2017). Elite, mass, and high-participation higher education. In J. C. Shin & P. Teixeira (Eds.), Encyclopedia of international higher education systems and institutions. Springer. https://doi.org/10.1007/978-94-017-9553-1_50-1 DOI: https://doi.org/10.1007/978-94-017-9553-1_50-1
  22. Martin, B. R. (2011). The Research Excellence Framework and the ‘impact agenda’: Are we creating a Frankenstein monster? Research Evaluation, 20(3), 247–254. https://doi.org/10.3152/095820211x13118583635693 DOI: https://doi.org/10.3152/095820211X13118583635693
  23. McKay, S. (2012). Social policy excellence – peer review or metrics? Analyzing the 2008 Research Assessment Exercise in social work and social policy and administration. Social Policy & Administration, 46(5), 526–543. https://doi.org/10.1111/j.1467-9515.2011.00824.x DOI: https://doi.org/10.1111/j.1467-9515.2011.00824.x
  24. Mingers, J., Watson, K., & Scaparra, M. P. (2012). Estimating Business and Management journal quality from the 2008 Research Assessment Exercise in the UK. Information Processing & Management, 48(6), 1078-1093. https://doi.org/10.1016/j.ipm.2012.01.008 DOI: https://doi.org/10.1016/j.ipm.2012.01.008
  25. Neumark, D. (1988). Employers’ discriminatory behavior and the estimation of wage discrimination. Journal of Human Resources, 23(3), 279–295. https://doi.org/10.2307/145830 DOI: https://doi.org/10.2307/145830
  26. Oaxaca, R. (1973). Male-female wage differentials in urban labour markets. International Economic Review, 14(3), 693–709. https://doi.org/10.2307/2525981 DOI: https://doi.org/10.2307/2525981
  27. Oaxaca, R. L., & Ransom, M. R. (1988). Searching for the effect of unionism on the wages of union and nonunion workers. Journal of Labor Research, 9(2), 139-148. https://doi.org/10.1007/bf02685237 DOI: https://doi.org/10.1007/BF02685237
  28. Oaxaca, R. L., & Ransom, M. R. (1994). On discrimination and the decomposition of wage differentials. Journal of Econometrics, 61(1), 5-21. https://doi.org/10.1016/0304-4076(94)90074-4
  29. Oaxaca, R. L., & Ransom, M. (1998). Calculation of approximate variances for wage decomposition differentials. Journal of Economic and Social Measurement, 24(1), 55-61. https://doi.org/10.3233/jem-1998-0143 DOI: https://doi.org/10.3233/JEM-1998-0143
  30. Oaxaca, R. L., & Ransom, M. R. (1994). On discrimination and the decomposition of wage differentials. Journal of Econometrics, 61(1), 5-21. DOI: https://doi.org/10.1016/0304-4076(94)90074-4
  31. Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences, 5(2), 187–195. https://doi.org/10.1017/s0140525x00011183 DOI: https://doi.org/10.1017/S0140525X00011183
  32. Robbins, L. (1963). Higher education: Report of the Committee on Higher Education. HMSO.
  33. Roberts, G. (2003). Review of research assessment: A report to the funding bodies (Report 2003/22). HEFCE.
  34. Rowlinson, M., Harvey, C., Kelly, A., Morris, H., & Todeva, E. (2015). Accounting for research quality: Research audits and the journal rankings debate. Critical Perspectives on Accounting, 26, 2–22. https://doi.org/10.1016/j.cpa.2013.05.012 DOI: https://doi.org/10.1016/j.cpa.2013.05.012
  35. Sharp, S., & Coleman, S. (2005). Ratings in the Research Assessment Exercise 2001: The patterns of university status and panel membership. Higher Education Quarterly, 59(2), 153–171. https://doi.org/10.1111/j.1468-2273.2005.00288.x DOI: https://doi.org/10.1111/j.1468-2273.2005.00288.x
  36. Spence, C. (2019). ‘Judgement’ versus ‘metrics’ in higher education management. Higher Education, 77(5), 761–775. https://doi.org/10.1007/s10734-018-0300-z DOI: https://doi.org/10.1007/s10734-018-0300-z
  37. Tapper, T., & Salter, B. (2003). Interpreting the process of change in higher education: The case of the research assessment exercises. Higher Education Quarterly, 57(1), 4-23. https://doi.org/10.1111/1468-2273.00232 DOI: https://doi.org/10.1111/1468-2273.00232
  38. Taylor, J. (1995). A statistical analysis of the 1992 Research Assessment Exercise. Journal of the Royal Statistical Society, Series A, 158(2), 241–261. https://doi.org/10.2307/2983291 DOI: https://doi.org/10.2307/2983291
  39. Taylor, J. (2011). The assessment of research quality in UK universities: Peer review or metrics. British Journal of Management, 22(2), 202–217. https://doi.org/10.1111/j.1467-8551.2010.00722.x DOI: https://doi.org/10.1111/j.1467-8551.2010.00722.x
  40. Tomlinson, M., & Watermeyer, R. (2022). When masses meet markets: Credentialism and commodification in twenty-first century higher education. Discourse: Studies in the Cultural Politics of Education, 43(2), 173–187. https://doi.org/10.1080/01596306.2020.1814996 DOI: https://doi.org/10.1080/01596306.2020.1814996
  41. Trow, M. (1973). Problems in the transition from elite to mass higher education. Carnegie Commission on Higher Education.
  42. Universities UK. (2025). Universities grip financial crisis – but at what cost to the nation? https://www.universitiesuk.ac.uk/what-we-do/creating-voice-our-members/media-releases/universities-grip-financial-crisis-what
  43. Willmott, H. (2011). Journal list fetishism and the perversion of scholarship: Reactivity and the ABS list. Organization, 18(4), 429–442. https://doi.org/10.1177/1350508411403532 DOI: https://doi.org/10.1177/1350508411403532
  44. Willson, R., Stewart-Robertson, O., Julien, H., & Given, L. M. (2026). Academic casualisation and precarity: a scoping review. Higher Education Research & Development, 45(4), 1165-1187. https://doi.org/10.1080/07294360.2025.2573983 DOI: https://doi.org/10.1080/07294360.2025.2573983

License

CCBY 4.0
Creative Commons Attribution 4.0 International

This work is openly licensed — share and adapt freely with attribution to the authors and the journal. View license terms ↗

Copyright (c) 2026 James Johnston

§ 06 — Related

Similar articles in this journal

Related peer-reviewed studies published in this journal.
View all issues

Teacher leadership and communities of practice: Female teachers’ experiences in Pakistan

Marie Clarke, Deirdre Raftery, Magdelina Kitanova, Ruth Ferris, Ellen Regan, Thanh Nguyen Thao Tran, Angeliki Lima, Catherine Nowlan-Roebuck

This small-scale qualitative study examines the lived experiences and enactment of female teacher leadership in communities of practice in four schools in Pakistan. The…

#Teacher leadership #communities of practice #Global South #women teachers
p. 4-26
10.70116/31434541158