So to recap, in Part I we had our questioning hero deciding which programming language to use after joining a new research team, in an environment where others were using Fortran.  So many choices!  One important aspect of the question was longevity of the code, which leads us to a very important point… that of data, and how well a language supports the data formats you need to use.  I thought I’d focus on data today, because this issue keeps cropping up. I answered a question on the subject last week during a panel session at the Software Preservation Workshop and it came up again – only yesterday – at a Dev8D guru session.

It’s not just about picking a language with a secure future: you also need to think about the data formats that the language supports.  There’s a lot to think about.  Which formats are used by your community?  Are those formats supported by the language vendor, an open source project, or by a particular research community.  What about the support itself?  Does it have a sustainable future?

Even if the formats are supported now, you could find that the support disappears in the future – just when you come to upgrade your language compiler and run-time system. Maybe that all-important library that provided the support is no longer developed (or available!) for the language, or the support has been deprecated in favour of a more successful competing format.  If the software forms part of a software pipeline (e.g. data pre-processing, processing, post-processing, visualisation), the obvious risk is that the pipeline breaks and you can’t generate results!

The reproducibility of results is increasingly important in research. And you can’t have reproducibility if the software fails to generate results, because the language’s support for the data format has been deprecated.  When selecting a language, I’d recommend doing a bit of investigation on data formats used within your community and their support, because fixing the data format support problem can prove expensive!

Of course, you may have the time and expertise to re-implement the support, or take up the reins if it’s an abandoned open-source project.  You could take the initiative and heroically maintain the support for the rest of the community…

Obviously, this issue is related to the sustainability of the data formats themselves (i.e. as supported standards), but that’s a whole post in itself.

Until next time(1)!

Steve

PS: Ok, so the time(1) pun is wearing a bit thin now.  Clearly over the top(1).