By Edward Smith, Imperial College London.

The Research Software Engineer (RSE) 2017 conference started with an excellent keynote talk by Chris Woods, emphasising a number of issues. This set the tone for the conference, outlining the aspiration of developing the RSE community and changing perceptions. There seems to be clear progress in this area and a number of challenges still to face. One issue that came up was the misalignment of aspiration in the university selection processes for RSE fellowships candidates compared to those in the national RSE community. For me, the most interesting point was Chris’ emphasis on letting people use the technologies they feel most comfortable with, when working as an RSE. This was a theme from the conference that continued with no less than four talks by software engineers working at or with the MET office. Weather prediction is unique in that it is scientific software with a direct and constant validation, often by angry people without umbrellas. It is also an area where code and hardware reliability is crucial. They have an entire standby supercomputer in case the main one goes down and an extensive testing framework. In this context, it is therefore interesting that the MET RSEs are working on rewriting the entire HPC code used for weather prediction in Fortran. The main reason is because the developers use Fortran, they know Fortran and they think Fortran is best for the job. Certainly on High Performance Computing (HPC) platforms like ARCHER, Fortran is clearly the dominant language.

Snapshot of the ARCHER website, programming languages used for jobs showing Fortran is far from dead (taken on the 16th November 2017)

Fortran also makes people very angry. It is not a generally popular language, failing to make stackoverflow’s list of common languages, even in maths and science. It’s certainly not an area Software Carpentry is focusing on teaching or one that is commonly talked about at RSE meetings in the context of best practice. In fact, overwhelmingly, the focus is on teaching Python within the RSE community; I’ve actually taught three Python courses myself. However, the advice from a talk by Andrew Rowley and Alan Stokes, on “Using Python for a Large Software Project” was quite simply DON’T DO IT! The lack of type checking (duck-typing) as well as the lack of encapsulation and variable hiding quickly causes problems down the line. Interestingly, these are exactly the things people criticise about old school FORTRAN code, with implicit variable and common blocks of global variables.

The message I took from the RSE conference is not forcing people to change the code they use, but changing the code they write, making code more Pythonic and less Fortranic. This bring me to the other big theme (for me at least) discussed at the RSE meeting: testing. In particular, the use of testing on high performance computers. Despite using Fortran, the MET office have extensive testing, unit tests, integration tests and regression tests on the full code. From a discussion on how to test code with the MET office, it seems that getting reproducible results in the form of byte comparisons on HPC actually requires rewriting MPI routines to ensure consistent ordering of operations. This is certainly not a trivial undertaking. The MET office is clearly far in advance of best practice than most academic institutions, where most codes don’t even have testing frameworks for HPC, let alone continuous integration (CI).

As always with these type of conferences, some of the most interesting development happened in the gaps between sessions, giving people a chance to discuss and mix. I talked to a number of people interested in the concept of continuous integration for HPC, a topic which I think the RSE community can aim to make better in the UK. I met Catherine Jones from STFC, who put together a survey on the subject, and Steven Lamerton, who has actually already developed a build service on CCP forge. I also talked with Jeffrey Salmond and Chris Richardson from Cambridge who are making efforts to do this as well as RSEs at UCL.Shoaib Sufi and I discussed the best format for an event to bring people together. As a result, I organised a workshop at Imperial on the 10th November. This was funded by the Software Sustainability Institute. The plan for this day is here, and the outcomes of the various discussion sessions are collated here. I plan to write a blog post and summarise in a document soon.

For me, as a researcher and software engineer working in fluid dynamic, the number of MET talks meant this was an excellent conference. As always with RSE meetings, it’s great to know other people are struggling with the same problems you are. Having a chance to see best practice in testing on HPC inspired me to make things better both at my home institution as well as throughout the UK. Not bad for a two day conference.