Best practices

Changes in scientific assessment systems

An ever-increasing number of voices are advocating changes in order to improve systems for assessing scientific activity. They defend the development and application of criteria that attach value to different qualities and contributions in research, not solely to the visibility of studies or the bibliometric data of scientific publications.


Factsheet
 

1. Context

This last decade has seen an intensification of debate, across the international scientific community, around the ways in which science is funded and practiced. Exchanges consider how scientific activity can attain greater openness and inclusion and achieve a higher impact. These debates are accompanied by critical reflections regarding research assessment processes and criteria.

2. Debate

The use of bibliometric indicators based on journal impact factors has spread to every branch of knowledge and practically all areas of scientific activity. Assessment systems have progressed from being based on evaluations by experts to depending almost exclusively on these metrics.

The consequences of this abusive use of bibliometric indicators are pernicious on an individual and a collective scale. Scientific activity is transformed into a spiral, geared towards accumulating more and more publications and increasing the number of citations. Researchers concentrate on publication as their main goal and neglect other activities that are not as profitable for their career. The pressure to publish increases the number of irrelevant studies and encourages the proliferation of poor practices and unethical behaviours. The governance and the strategic lines of universities and research centres are disrupted. The diversity of research agendas is undermined, while lines that are slow to produce results, uncertain, complex, or costly, are abandoned in favour of issues that are more fashionable, with sure and immediate returns. 

In response to this situation, numerous initiatives and declarations have emerged on a worldwide scale – such as the San Francisco Declaration on Research Assessment (DORA), in 2013, or the Leiden Manifesto, in 2015 – which advocate changes in assessment systems and have received the adhesion of numerous institutions from around the world. In recent years, countless organisations in different academic fields and regions have kicked off a process of reform in their assessment processes and are trialling new practices, such as the use of narrative formats for curriculums, the evaluation of actions linked to the advancing of open science, references to multidisciplinarity, peer assessment, and attention to relevance and the local and social impact of contributions.

3. Conclusions

The endless stream of initiatives that have been developed in recent years suggests that research assessment has entered a new era. Practices based purely on quantitative valuations and on the exclusive use of journal impact factors are being dropped, and progress is being made towards new assessment models that strive to be more plural and to take into consideration other factors too, such as the value, quality and impact – scientific but also social and local – of contributions and the accessibility of results.

Classification

Tags

Subject areas

Related content

Article

The STEM field is failing to attract female talent

In Spain, only 16% of STEM professionals are women. We analyse this gender gap.

Article

Evolution of science and technology in Portugal and Spain

The European Union set a target for the business sector to invest 2% of GDP in R&D. How is the convergence of Spain and Portugal towards this goal progressing?

Activity

Discussion "Research and Innovation in Spain and Portugal"

What are the most decisive factors in a country’s research and innovation systems? You can see the video of the discussion we organised to debate this issue here.

Best practices

The use of public engagement for technological innovation

What is society’s opinion regarding the possible impacts of science and technology? Establishing citizen participation mechanisms is necessary to generate confidence and detect points for improvement.

Infodata

Expenditure on R&D by sectors

In 2019, the percentage of total public spending assigned to R&D was 1.24% in Spain and 0.82% in Portugal, both below the EU-27 average.

You may also find interesting

Gambling addiction in adolescence: betting, technologies and drug use

Article

Gambling addiction in adolescence: betting, technologies and drug use


Science

According to this study, the most popular form of gambling among teenagers is sports betting, and there is a higher level of online gambling addiction among boys.

From Moderate to Hyperconnected Users: Six Smartphone Use Profiles and Their Impact on Personal Well-being

Article

From Moderate to Hyperconnected Users: Six Smartphone Use Profiles and Their Impact on Personal Well-being


Science

What use are young Spaniards making of mobile phones? This study indicates that 19% are hyperconnected, showing difficulties in controlling the use of this technology.

Can algorithms expose tax fraud?

Article

Can algorithms expose tax fraud?


Science

Things are becoming increasingly difficult for tax evaders, with the new big data and artificial intelligence techniques that detect hidden wealth, the abuse of aggressive tax engineering and money laundering.