Software and research: the Institute's Blog

Latest version published on 21 November, 2017.

Software CarpentryBy Eilis Hannon, University of Exeter.

On Tuesday, 26th September we held a two day Software Carpentry workshop at the University of Exeter. 37 staff and students attended including a number of PhD students in their first week. The course covered the Unix Shell, Python and Git, and gave Jeremy Metz and me the opportunity to teach our first Software Carpentry workshop, alongside Andrew Walker from University of Leeds who was there to steady the ship and provide experience. This was particularly helpful when it came to debugging and resolving common issues.

I was impressed by the sustained enthusiasm over the two days and how much of the content we got through. Many attendees commented that they appreciated the slightly shorter days (from 10am to 4pm) and regular breaks, as these enabled them to remain focused and digest the content delivered. They were also grateful for the additional support provided by the helpers David Richards, Paul O’Neill, Ben Evans…

Continue Reading

Latest version published on 15 November, 2017.

By Kenji Takeda, Microsoft Research.

Research Software Engineers drive advances in how research can be done more effectively using all manner of software, computing systems and infrastructure. As a community, RSEs drive positive change to progress the state-of-the-art to do better, faster, and more reproducible research. It’s clear that cloud computing is playing an increasingly important role in research, so Microsoft is privileged to be able to support the RSE community, and researchers across the world, to exploit cloud computing across all domains through our Azure for Research program.

We were delighted to see so many high-quality applications to the RSE Cloud Computing Awards call, and so have decided to give all applicants access to Microsoft Azure to pursue the wide-range of exciting activities proposed. We particularly congratulate the successful awardees from across the UK, who can now pursue their plans for training, workshops, community software development, and cutting-edge research using Microsoft Azure.

  • Martin Callaghan, The University of Leeds
  • Christian Cole, University of Dundee
  • Joseph Doyle, University of East London
  • Eilis Hannon, University of Exeter…
Continue Reading

Latest version published on 9 November, 2017.

Software Citation HackathonBy Stephan Druskat (Humboldt-Universität zu Berlin, Germany)

On 26 October 2017, the Force11 Software Citation Implementation Working Group and Force11 Hackathon was hosted at the Force2017 Conference in Berlin, Germany, and led by Neil Chue Hong (Software Sustainability Institute), Lars Holm Nielsen (Zenodo) and Martin Fenner (DataCite). Participants took a full day to exchange, discuss, plan, and hack towards implementations of the software citation workflow.

Software citation is at the very heart of the process to create recognition for software as a scholarly product, and finding and implementing a working solution for this issue is the key to attribution and credit for the creators of scientific software. At the same time, the possibility of citing a software also helps to unlock its potential for sustainability by boosting its accessibility, and by fostering a software’s persistence through the encouragement of tagging its published versions with unique identifiers, such as DOIs. Thus, software citation also becomes a natural path to the reproducibility of research results in the context of open science, where not only the data, but also the complete toolchain employed in a research endeavour, is made openly available.

Software citation is still hard

However, software citation is hard, and for diverse reasons. It is…

Continue Reading

Latest version published on 6 November, 2017.

GRADnetBy Mike Jackson, Software Architect

On 18th October I attended GRADnet's "Moving Forward for 2nd Year PGRs" day in London for physics post-graduates, and ran two sessions on "Writing better software to research".

SEPnet, the South East Physics Network, is a consortium of universities in the south east of England, promoting excellence in physics in both academia and industry, via research, collaboration, training, and outreach. GRADnet is SEPnet's collaborative graduate school which provides professional skills training to PhD students.

GRADnet's "Moving Forward for 2nd Year PGRs" day offered attendees a choice of 5 sessions both morning and afternoon, on Creating impact, How to write a successful Fellowship Application, Research data management, Unconscious Bias and Writing better software for research. 66 students attended the event, held at the Park Crescent Conference Centre, London.

My 2.5 hour session on Writing better software for research provided students with a hands-on code review to get them thinking about the qualities of good, and bad, code. I gave an introduction to a selection of best practices from Wilson et al.'s highly recommended 2014 paper Best Practices for…

Continue Reading

Latest version published on 2 November, 2017.

LUX-Zepelin water tankBy Mike Jackson, Software Architect

In Using Excel for data storage and analysis in LUX-ZEPLIN, I summarised how Excel is both used and managed within the LUX-ZEPLIN (LZ) project and recommendations for improvements. In this second of two blog posts, I describe how LZ could migrate their data within Excel to MongoDB with supporting software, in Python, for computation and presentation. I also describe a proof-of-concept which extracts data from Excel, populates MongoDB with this data, and computes the radiogenic backgrounds expected from a subset of the possible sources of contamination.

As a reminder, the BG table is an Excel spreadsheet, with 43 sheets, used by LZ to calculate radiogenic backgrounds, and the WS Backgrounds Table is a sheet within the BG table which summarises the radiogenic backgrounds expected during the lifetime of the experiment from each source of contamination.

Migrating from Excel to MongoDB and Python

Excel combines data, computation and presentation. For example, a cell with a formula in Excel is a combination of data and computation, in effect a tiny program. The migration plan was based around migrating from the BG table into a solution…

Continue Reading