Best practices

Changes in scientific assessment systems

An ever-increasing number of voices are advocating changes in order to improve systems for assessing scientific activity. They defend the development and application of criteria that attach value to different qualities and contributions in research, not solely to the visibility of studies or the bibliometric data of scientific publications.


Factsheet
 

1. Context

This last decade has seen an intensification of debate, across the international scientific community, around the ways in which science is funded and practiced. Exchanges consider how scientific activity can attain greater openness and inclusion and achieve a higher impact. These debates are accompanied by critical reflections regarding research assessment processes and criteria.

2. Debate

The use of bibliometric indicators based on journal impact factors has spread to every branch of knowledge and practically all areas of scientific activity. Assessment systems have progressed from being based on evaluations by experts to depending almost exclusively on these metrics.

The consequences of this abusive use of bibliometric indicators are pernicious on an individual and a collective scale. Scientific activity is transformed into a spiral, geared towards accumulating more and more publications and increasing the number of citations. Researchers concentrate on publication as their main goal and neglect other activities that are not as profitable for their career. The pressure to publish increases the number of irrelevant studies and encourages the proliferation of poor practices and unethical behaviours. The governance and the strategic lines of universities and research centres are disrupted. The diversity of research agendas is undermined, while lines that are slow to produce results, uncertain, complex, or costly, are abandoned in favour of issues that are more fashionable, with sure and immediate returns. 

In response to this situation, numerous initiatives and declarations have emerged on a worldwide scale – such as the San Francisco Declaration on Research Assessment (DORA), in 2013, or the Leiden Manifesto, in 2015 – which advocate changes in assessment systems and have received the adhesion of numerous institutions from around the world. In recent years, countless organisations in different academic fields and regions have kicked off a process of reform in their assessment processes and are trialling new practices, such as the use of narrative formats for curriculums, the evaluation of actions linked to the advancing of open science, references to multidisciplinarity, peer assessment, and attention to relevance and the local and social impact of contributions.

3. Conclusions

The endless stream of initiatives that have been developed in recent years suggests that research assessment has entered a new era. Practices based purely on quantitative valuations and on the exclusive use of journal impact factors are being dropped, and progress is being made towards new assessment models that strive to be more plural and to take into consideration other factors too, such as the value, quality and impact – scientific but also social and local – of contributions and the accessibility of results.

Classification

Tags

Subject areas

Related content

Article

The STEM field is failing to attract female talent

In Spain, only 16% of STEM professionals are women. We analyse this gender gap.

Article

Evolution of science and technology in Portugal and Spain

The European Union set a target for the business sector to invest 2% of GDP in R&D. How is the convergence of Spain and Portugal towards this goal progressing?

Activity

Discussion "Research and Innovation in Spain and Portugal"

What are the most decisive factors in a country’s research and innovation systems? You can see the video of the discussion we organised to debate this issue here.

Best practices

The use of public engagement for technological innovation

What is society’s opinion regarding the possible impacts of science and technology? Establishing citizen participation mechanisms is necessary to generate confidence and detect points for improvement.

Infodata

Expenditure on R&D by sectors

In 2019, the percentage of total public spending assigned to R&D was 1.24% in Spain and 0.82% in Portugal, both below the EU-27 average.

You may also find interesting

Uses, skills and attitude in digital technology fields. Does a gender gap exist?

Article

Uses, skills and attitude in digital technology fields. Does a gender gap exist?


Science

Boys consider that they are more skilled than girls in digital technology and communication fields, whereas girls feel better prepared in ethical and security-related aspects.

Desertified Spain

Article

Desertified Spain


Science

Some 17.5% of rural territory is exposed simultaneously to high rates of depopulation and of aridity. This study analyses how some factors for mitigating depopulation can have undesired effects, of both a social and an environmental nature.

Inequalities in covid-19 inequalities research: Who had the capacity to respond?

Article

Inequalities in covid-19 inequalities research: Who had the capacity to respond?


Science

Did inequality exist in the research into the inequalities of covid-19? We analyse it in this comparative study focusing on research production, distribution and collaborations between countries.