Posted by d.barclay on 10 August 2023 - 11:00am 

By Stephen Klusza, Andrea Sánchez-Tapia, and Michael Tso.

This blog post is part of our Collaborations Workshop 2023 speed blog series. 

Despite the vast importance of software, data, and notebooks (and other forms of digital output) in research and academia, the digital output itself is greatly undervalued in comparison to published papers. As a result, it is not uncommon to encounter various scenarios such as selective availability of code/data, incomplete documentation in the materials/methods section, and lack of standardisation. This has serious consequences in terms of sustainability, entry/access barriers, and robust verification of software tools for reproducible workflows.  

In this blog post, we focus on three major areas that we feel are essential to address the significant challenges to normalising digital output itself as scholarly artefacts.

1. Publishing Digital Output Openly With Preprint Initiatives 

In the last decade, there have been great strides in uploading papers as preprints prior to publication in journals, which allowed more timely indicators of productivity for manuscript authors. However, these sites are still relegated only to complete manuscripts and typically do not encourage smaller works to be submitted as-is. At the same time, there are young organisations and initiatives such as ResearchEquals and Authorea that are trying to change this through hard work and transparency to provide space to document your research objects openly and quickly with Digital Identification Numbers (DOIs) and other IDs. Publishing your digital output in these avenues can go a long way toward community buy-in and the viability of these alternatives to the traditional measures of productivity. 

2. Supporting Computational Infrastructures That Lower the Barrier of Entry 

Github and Gitlab are very well-known these days as universal repositories for the easy deposition of software/code, which somewhat alleviates the issue of "code available upon reasonable request" that plagued previous eras of research. However, infrastructures that are devoted to open science, reproducibility, and sharing of research lag far behind in visibility and support in academia and research. The Center For Open Science manages OSF, which offers significant storage for open research projects with the ability to submit preprints and foster collaborations transparently. Others like Datalabs and Cyverse extend access to reproducible workflows and computational resources for many researchers who would be unable to do so otherwise. Incorporating your digital output with these supportive and established infrastructures may also be a great way in providing substantial evidence for recognising your work as scholarly.  

3. Advocating for Scholarly Merit of Digital Output  

As hard as it can be to make a convincing argument that digital output deserves to be publishable as-is, it can be even harder to advocate for changing evaluation procedures in academia and research companies to provide proper recognition for the work that was done. To further complicate matters, you will likely encounter various levels of resistance and opinions depending on where you are working. It is important to emphasise that there are many ways to do advocacy. Joining committees and having meaningful discussions about criteria for recognising the merits of digital output is just one way to do this. If your boss or PI is receptive, you may be able to upload your code as a preprint in one of the services mentioned above and show tangible evidence of reach and impact. If applicable, consider suggesting using available free and open-source software/data as a way to do more research for less money. Advocating in the way that works best for you will help the entire community build a stronger case for recognising digital output as scholarly work. 

Conclusion

Creating positive change in academia and research spaces is never an easy task but we hope that these points help start conversations about how to assign equitable value to created digital outputs. In particular, we encourage established academics and professionals to reach out and learn more about options for publishing digital outputs. In our view, legitimising digital outputs as scholarly artefacts provides much-needed incentives for long-term sustainability and reproducibility of computational work and is an essential component to building a more robust and reproducible foundation for academia and research.