In 2023 I received a National Science Centre grant to carry out a study on evaluation of research impact. Below you can find more information on the grant in English and in Polish.
The grant will be carried out at SWPS University in Warsaw, Poland where I am based, in collaboration with Nordic Institute for Studies in Innovation, Research and Education – NIFU and The Robert K. Merton Center for Science Studies (RMZ) at Humboldt University, Berlin.
Abstract (ENG)
Evaluation of research impact and academic discourse – a comparative approach (Poland, UK, Norway)
2023-25
The object of the study is a new academic evaluative practice – the evaluation of “research impact”. The study offers an interdisciplinary perspective on the topic, focusing on the emergent genre of academic text created in the evaluative context – “impact case study”. The genre is used as a lens through which related social practices in academia can be studied. The project draws on methods and theories of Linguistics, Sociology and Philosophy.
The genre of impact case study was created in the context of the British Research Excellence Framework (REF) – a periodic research evaluation exercise in the UK. The introduction of the criterion of ‘extra-academic impact’ – the influence of research on the non-academic world (industry, education, policy, culture etc.) is a major change in the area of research evaluation. The practice of impact evaluation gives rise to shifts in the organization of academic life, hierarchies of power, every-day scholarly practice and communication patterns.
This British model of impact evaluation, first used in REF 2014, has been closely observed and emulated in other evaluation systems. From 2016 an approach modeled on the REF is used in the evaluations of disciplines in Norway. The new model of evaluation of scientific units in Poland, scheduled for 2022 also includes an impact element. The implementation of an analogous evaluation exercise in three different academic systems and as part of different evaluation models allows for comparison in terms of policy-making patterns and consequences of the exercise for the academic community.Impact evaluation has been the object of research among scholars from the disciplines of Sociology, Education, Management. However, an important aspect which has been glossed over is the role of language in shifts related to research evaluation. This study puts forward a novel discursive perspective on the topic, addressing the gap in understanding the role of discourse in the construction of a shared notion of ‘impact’.
The study takes a comparative perspective looking at discursive practices related to impact evaluation in three countries: UK, Norway and Poland. The research design is interdisciplinary and aims to offer an complete description of a new phenomenon within academia.
The research questions are:
1) what are the main features of the genre of impact case study?
2) how do its expressions differ in different national contexts?
3) what is the discursive process through which the genre emerges?
4) what is the place of the genre in the academic discourse community? which procedures uphold it? what practices does it orchestrate?
5) how do academics incorporate the notion of impact into their own understanding of academic excellence in a process of subjectivation?
Three types of data will be collected in the three studied countries: 1) impact case studies submitted to the most recent evaluation, 2) interviews with key actors in the process of impact evaluation (policy-makers, authors of case studies, academic managers, possibly evaluators), 3) questionnaire on administrative procedures distributed across HE institutions.
The project focuses on the dimension of discourse and uses of methods developed within of linguistic pragmatics and Foucauldian discourse analysis. A fine-grained analysis of text and talk will be carried out with methods developed in the linguistic tradition. This includes genre analysis (Swales, Bhatia, Bazerman, Maingueneau) and elements of narrative analysis. Academics’ positioning towards the notion of ‘impact’ in interviews will be analysed with methods proper to positioning theory (Goffman, Davies and Harré, Angermuller). An interpretation of the role of impact case studies in the academic community will build on sociological/philosophical theories of discourse (Foucault, Rose, Angermuller). The use of Foucauldian discourse analysis and governmentality theory will allow for general (meta level) conclusions on the effect of impact evaluation on hierarchies of power and academics’ performance of ‘self’.
The study will contribute to specialised fields of research: Genre Studies, Higher Education Studies, Evaluation and Valuation Studies, Governmentality Studies. The project is qualitative but findings will be underpinned with elements of quantitative analysis: i) corpus-linguistic analysis of impact case studies and ii) survey data from academic institutions.
The grant will be carried out at SWPS University in Warsaw, Poland where I am based, in collaboration with Nordic Institute for Studies in Innovation, Research and Education – NIFU and The Robert K. Merton Center for Science Studies (RMZ) at Humboldt University, Berlin.