Advancing Transparency and Reproducibility with Open Methods

Open Access makes research freely and easily available to all. However, to fully satisfy the ideology of open science, this increased visibility must extend beyond the research article alone.

Open methods, sometimes used interchangeably with open methodology, is a component of open science which pertains to increased transparency across the entire research process.

In this article, we explore open methods and outline their growing importance, along with some of the initiatives helping researchers enhance the reproducibility and transparency of their work.

The importance of methods

A comprehensive and clear methods section is an integral component of any research article.

A key pillar of scientific credibility, methods should clearly outline how researchers have arrived at their findings, in turn allowing others to understand, evaluate, and more importantly reproduce their research.

To achieve reproducibility means providing an in-depth account of your materials and research process, so readers know exactly how results were obtained.

Shrinking methods and the reproducibility crisis

Methods sections are not a novel concept, nor is their importance newly understood. They have always been important to ensuring that studies can be repeated, and their findings can be verified and built upon through further study.

However, methods sections have become shorter and less detailed due to restrictions on the number of words or pages that can be included within articles. In some cases, this extends to restraints on the number of tables and figures used in an article or even the number of sources that can be cited.

Exceeding these limits can be accompanied with charges. Therefore, the incentive for researchers to detail their methods in its entirety is diminished. This contributes to growing concerns within the scholarly community that research is facing a reproducibility crisis.

Previous surveys have found that more than 70% of researchers found themselves unable to repeat other researchers work, demonstrating how undetailed, missing, or inaccessible methods can limit the extent which existing research can be used as a foundation for scientific progress.

Open Access and open methods

Open Access publishers such as MDPI typically operate online. This removes page restrictions and encourages authors to be as detailed as possible in their methods.

Nonetheless, even with the benefits of Open Access publishing, incomplete methods prevent research being completely reproducible. These gaps have led to the rise of open methods.

What are open methods?

Ensuring open methods in research requires making the protocols, procedures, and materials used in a study fully transparent and publicly available.

Open methods serve to outline research processes in as much granular detail as possible, working to bolster the reproducibility of research.

Often surpassing the requirements of traditional methods sections, open methods feature various additional components:

  • Experimental protocols
  • Specific materials, reagents, and instruments used
  • Information on equipment settings and calibration
  • Survey instruments or questionnaires
  • Descriptions of qualitative data collection procedures
  • Details of data analysis techniques
  • Lab notebooks or digital records

Why do we need open methods?

In the context of the concerns surrounding reproducibility, open methods help to strengthen the transparency of research and ensure it can be repeated by others.

If studies are not reproducible, their findings are not reliable or trustworthy, therefore causing the research to lose its value.

Previously, it has been estimated that in the US alone ‘approximately 28 billion USD a year is spent on preclinical research that is not reproducible’. By strengthening transparency and reproducibility, open methods help ensure a return on investment for scientific funding by making it usable as a basis for future work.

Moreover, it feeds the wider ethos of Open Access publishing and open science practices by making research methods easily and freely accessible to the global scholarly community.

Publishing methods as open protocols

Instead of sharing incomplete methods to appease journal restrictions, authors can enhance the reproducibility of their research by sharing their work as protocols.

Open protocols are detailed, step-by-step accounts of experimental procedures taken throughout the research process and operate as an individual, citable research output.

They build on the contents of traditional methods sections by providing a comprehensive account of procedural steps, along with a detailed list of materials and reagents used, amongst other details.

Protocols.io is a notable Open Access platform for sharing of protocols and currently holds more than 26,000 reproducible methods. Similarly, the Open Science Framework (OSF) provides another free-to-use repository for sharing protocols, pre-registering studies, and storing files.

Additionally, their technical features make them effective for elucidating the research process:

  • They can be updated and tracked over time
  • They are given DOIs, making them citable
  • They are easily accessible to other researchers

By providing a dedicated and unrestricted platform for methods, this process ensures that experimental workflows can be accessed and replicated. Even further, making your processes citable works to benefit authors as it acknowledges how their own methods have influenced the work of another.

Standardizing materials with RRIDs

The Resource Identification Initiative was designed to improve the traceability, reproducibility and reuse of specific materials reported in the Life Sciences literature.

It operates by allocating unique identifiers called Research Resource Identifiers (RRIDs) to resources such as antibodies, cell lines, and organisms and other tools such as software.

These unique identifiers can then be directly inserted into a manuscript, taking the format of ‘RRID’, followed by the unique identifier. For example, RRID:AB_90755.

RRIDs are becoming increasingly favoured by publishers and funders as it eliminates ambiguity around the specific materials used. Also, it champions reproducibility by driving readers to curated registries or research articles where it is also featured.

RRIDs have become an important component of efforts to improve reproducibility, aligning with FAIR data principles, which require that data is findable, accessible, interoperable and reusable.

They provides clarity into exactly what was used within a study, which is useful for scholars looking to reproduce studies as small differences in resources can lead to producing largely different results.

Open notebooks

A significant volume of scientific knowledge and data is often never shared due to negative or inconclusive results being excluded from publications.

Paper-based notebooks contribute to this loss by being difficult to preserve and share and are easily misplaced. Additionally, when failed experiments are recorded on paper, and are subsequently discarded, copious amounts of data are lost.

Open notebook science addresses this issue by documenting and preserving the entire research process as it unfolds, allowing scholars to access data, research, and protocols in real time. Due to its utility, open notebooks are becoming increasingly popular amongst academics.

Leading project-based initiatives include OpenLabNotebooks, focused on aiding researchers in Life Sciences to publish their experimental records and data online, often utilizing WordPress. Open Source Malaria is another notable provider who leverages its infrastructure towards collaborative drug discovery.

Open notebooks provide a digital, accessible space to document your research process. Not only does this work to reduce the duplication of failed approaches but also extends the principle of transparency across the entire research process.

Open science requires open methods

The success of open science rests on more than making knowledge freely available. It requires a concerted effort to share how knowledge is developed, constructed, and supported through rigorous processes.

Open methods work towards this goal by making research reusable. The examples discussed in this article only represent a portion of open methods infrastructures but together they show how processes, materials, and workflows can be effectively and thoroughly documented.

MDPI’s commitment to open science and transparency

MDPI continues to lead the global transition towards Open Access and understands the value of reproducibility and transparency.

MDPI encourages all of its authors to publish their findings in as much detail as possible. On each journal webpage, MDPI provide instructions for authors to help authors prepare their manuscripts. See the full list of journals here.

We’re dedicated to giving you all the information you need to understand Open Access. Our article All You Need to Know About Open Access covers a range of topics.