In UNESCO’s 2021 Recommendation on Open Science, there is a call for increased open science monitoring. This is to ensure that all the principles, aims, and initiatives behind open science are achieving what they set out to do, namely, ensure science is accessible to all and transparent.

Here, we will outline open science monitoring, how monitors work, examples of them, and why it is important.

What is open science monitoring?

Open science (OS) monitors are diverse infrastructures that measure the success and spread of open practices. They can monitor anything, including academic articles, data, software, and publishers. Often, they are developed for countries or institutions, and how they work reflects the creator’s aims.

They are not assessment tools, instead providing information and data that can be evaluated by people to understand trends.

The 2023 UNESCO Open Science Outlook describes that OS monitors are needed to “gauge its impact, identify challenges, and lay groundwork for future progress”. They provide some suggestions of how they should work:

• Measuring how transparent infrastructures and metadata are.
• Not reducing OS to a few standardised metrics in a top-down approach, by instead embracing variability.
• Not overestimating mainstream practices or disregarding less diffused practices.
• Looking for major monitoring gaps.

UNESCO also suggests using people-centred monitoring, which would involve moving away from data and focusing on “the process of doing science and the people who are doing, engaging with, and/or benefitting from science”. They suggest this could help improve the understanding of the impact of OS on the Sustainable Development Goals, which we explored in out article We Need Open Access to Tackle Climate Change.

How open science monitors work

Many open science monitors will look at the total output of a form of scholarly research, like articles, and break down the percentages of open and closed content. One that looks at articles in this way would be known as an open access monitor.

OS monitors can be broader and may include the aggregation of different monitors to create a more comprehensive picture of the openness of scholarly outputs. This requires some standardisation, which requires decisions from the developer on how to organise and input information, which we will provide examples of below.

This approach looks specifically at the outputs, not at their usage. Monitors can also look at the multiplicity of usages and outcomes to monitor the different trajectories of OS. This reflects the ‘people-centred’ approach recommended by UNESCO and helps connect the data on what open research is and what its implications are.

Examples of monitors

There are various open science monitors around the world, with many being developed in Europe. Here are two examples that exhibit what monitors can achieve and the decisions that are made during the development process.

Open Science Monitor

For example, the Open Science Monitor is developed by the European Union to gain insights into the ongoing development of OS practices. It provides data on various indicators like Open Access publications, policies of journals and funders, open code, citizen science projects, and data.

In 2019, a report was created after a broad analysis using data from the Monitor. All trends clearly indicated progress, showcasing that OS is a wide-ranging set of trends, not just Open Access publication. Further, it showed when academics engage with open science, it is not “an ideological decision but a rational decision based on inherent advantages”.

The monitor successfully explores the diversity of OS and begins exploring the motivations behind those practising it. It demonstrates that monitors can focus both on the quantitative, looking at statistics of how open something is, and the qualitative, the success and impact of the openness.

French Open Science Monitor

The French Open Science Monitor (BSO) measures the progress of OS in France through indicators of the openness of scientific publications, clinical trials, observational studies, doctoral theses, research data, and research software.

The Monitor is transparent about how it aggregates data, which provides insight into how monitors are developed:

• The monitor is made up of publications with at least one author affiliated with a French institution.
• Publications must have a Crossref DOI or a HAL (French national open archive) identifier.
• Automatic methods systematically harvest access pages of DOIs to identify the French affiliations.
• Then, the publications are completed using data from PubMed and HAL.
• Finally, Unpaywall (open database of open scholarly articles) or HAL is used to determine whether something is open.

Metadata are highly important for this process, as they enable automatic processes to analyse and aggregate open research material. Further, Unpaywall and HAL are used, showing the interconnectedness of open infrastructures and how they support each other.

If you want to learn more about open science in France, read our article on the subject.

Why open science monitoring is important

Open science monitoring ensures that, as science undergoes a transformation in communication, new practices and technologies achieve the intended goals, namely, openness.

Moreover, science operates in a global network featuring academics, institutions, infrastructure and service providers, and policy-making groups. Therefore, it must be placed within this context of collaboration around the globe.

UNESCO’s 2021 Recommendation suggests regularly monitoring OS across the research lifecycle using a range of approaches. UNESCO cites three major reasons:

• Science is a global good, so we must ensure no one is left behind. This means recognising variations between disciplines, languages, and formats.
• Being conscious of the ‘streetlight’ effect. This refers to how major indicators can draw or distract attention and resources, therefore risking over-standardisation.
• Ensuring there are no major gaps in our awareness of the effects of openness, especially relating to society.

In short, if OS is about benefitting everyone, then developing fair and inclusive strategies to monitor the overall progress and impact of OS is key.

Learn more about open science

Open science monitoring emerges out of the diversity inherent to OS.

Overall, OS aims to make scientific research transparent and more collaborative. As an umbrella term, advocates for OS approach it from many different aspects.

We’re dedicated to giving you all the information you need to understand OS. Click here for our article, All You Need to Know About Open Access, which covers a range of topics that can help boost your understanding and also keep you up to date.