Last year, a set of Swedish funding agencies proposed a new model for quality-based allocation of government funds for strategic profile areas at the nation’s higher-education institutions (HEIs). Now, the agencies have published detailed guidance and criteria for the assessment of research quality.
Since 2008, the Swedish government has factored in the quality of research while allocating a portion of its research funds to HEIs. Quality has been assessed primarily on the basis of indicators such as bibliometrics and external funding, which the government now acknowledges are limiting. With this in mind, last year, the government directed four national funding agencies* to develop a new model that involves an expert assessment instead of indicators. Notably, the government is aiming to provide increased funding for certain areas that have the potential to foster research of the highest international quality.
Following consultation with representatives of the HEIs, the agencies published a report last year that outlines the contours of the new model. Although the government’s original intention was to implement the model in 2023, it concurred with the agencies’ view that developing the model comprehensively required more time. In response to a new directive in this regard, the agencies have now published a follow-up report that clarifies the grounds and criteria for assessing the quality of strategic profile areas. It also describes the nature of the application that HEIs are expected to submit to avail of the increased funding.
Strategic profile areas
The profile areas will be defined by the HEIs themselves in a bottom-up process, and it is up to each institution to decide how many profile areas to put forward for quality assessment. The areas can encompass new or emerging research as well as established fields and disciplinary as well as inter/transdisciplinary research. The areas can have a fundamental or applied focus or direct themselves to solving specific challenges.
This flexibility naturally lends itself to a diversity of areas being proposed. What should be common to all is the potential to achieving research of the highest international quality. The HEIs are expected to work actively towards enhancing research quality and ensuring the sustainability of the areas in terms of competence, finance and access to adequate facilities. In addition, research within the profile areas should encompass collaboration with relevant actors, which is expected to not only boost the quality and relevance of the research but also foster a better understanding of societal challenges or contribute to meeting them.
How the assessment is expected to work
Panel and proposals
Each HEI will need to first send in a short note that outlines the area(s) it wishes to profile (title, 0.5 p description and keywords), which will facilitate the selection of assessment-panel members. The institutions can also put forward the names of international, independent experts. Following this, each HEI will be asked to send a complete, approximately 30-page proposal that will be the subject of a comprehensive assessment.
The agencies propose a gender-balanced and geographically diverse, 10-member international panel with individuals who have experience of such areas as research strategy and evaluation, organisation and leadership, etc. During the assessment, the panel will be able to draw on reviews from two-to-three subject-matter experts per profile area. The agencies will provide the panel with an overview of the Swedish higher-education landscape, including statistics on research funding, personnel, etc., for each institution.
Assessment criteria
Each proposal will be assessed on the basis of three criteria: 1) scientific quality, 2) adequate conditions for achieving quality and 3) quality in the context of collaboration with relevant societal actors. For each criterion, the agencies’ report proposes one overarching question and several guiding or indicative questions. Each criterion will be graded by a three-point scale.
Scientific quality
Here, the main question relates to whether research within a given profile area is of high quality and demonstrates potential for achieving the highest quality. The guiding questions are related to the potential for ground-breaking research (including a sex and gender perspective) and the feasibility of carrying it out (including the right competence). The HEIs will be asked to address this criterion by describing aspects that are along similar lines as typical research proposals, including sections on the goal and objectives, state of the research field, description of the research, etc., as well as a list of up to 30 key publications and a justification of their significance.
Adequate conditions for achieving quality
The panel members will look at the extent to which the conditions are right for achieving quality with respect to a given profile area. This includes the engagement and organisational capacity of the HEI, an assurance regarding the short-term and long-term availability of skills and competences, and access to suitable facilities or infrastructure. The HEIs will provide relevant information that will help the panel in assessing this criterion (e.g., projected budget, numbers of staff that will work within a given profile area, the provision of an attractive work environment, a plan for strategic recruitment, etc.).
Quality in the context of collaboration with relevant societal actors
The main question that will guide the assessment in this regard is whether adequate strategies and processes are in place to foster quality development in collaboration with societal actors. Aspects to be considered include not only whether the collaboration leads to an improvement in research quality and relevance, but also whether the collaboration is planned such that it can address societal challenges or add value in another way. To justify how a given area meets this criterion, the HEIs will need to describe, for example, planned collaborative activities, how the research is informed by or considers the needs of societal stakeholders and how knowledge generated via engagement will be put to use.
Time frame
Realistically speaking, the report suggests, the disbursement of additional state funds to the HEIs would take place in the year 2025, which implies that the government should have the results of, and recommendations from, the assessment process in early 2024. Working backwards, the HEIs would need to submit their proposals in early 2023 and thus need to prepare for them already this year, with the recruitment of panel members also likely to begin this year.
Opportunities and challenges
In both this report and the one from last year, the agencies note several possible risks with the new model. One is that the process will favour the profiling of areas that were already strong at the expense of new or emerging ones. However, as the report notes, there is nothing that points in this direction at the outset. Another risk is that the model will perpetuate previously witnessed gender imbalances in funding, not least because women-dominated research fields have few professors, whereas men-dominated fields have more professors. This is a harder risk to mitigate given its roots in entrenched patterns and practices, but it is clear that the HEIs will need to be mindful of it.
To develop this new model for quality-based allocation of funds to profile areas, the agencies have drawn on approaches and experiences of research-quality assessments in other countries, including the well-known Research Excellence Framework (REF) in the UK. However, according to the report, none of the approaches is directly applicable for the type of assessment that is implied by the Swedish government’s proposal. Given the lack of a clear precedent, the model presents a leap and its consequences are thus difficult to predict. Nevertheless, the reports point to its grounding in a well-thought-out process that has involved sustained consultation with the representatives of the HEIs. Certainly, the move away from indices to expert assessment is to be welcomed.
Ninad Bondre (Research Coordinator)
* The agencies include the Swedish Research Council (that coordinated the model development), Formas, Forte and Vinnova.