By James Baker, Curator, Digital Research, British Library @j_w_baker

There are complex challenges in the humanities around software sustainability. For if it is true that humanists rely on software to do research, and increasingly software developed by their community, many if not most do not value the use of software and their nascent systems of credit for good software development and reuse are fragile. And so if the humanities are to make the best of the vast and growing digitised and born-digital corpora held by research libraries, key stakeholders in the field must ascribe the same value to the development of and experimentation with research software as they do to traditional practices such as literature surveys, source critique, and written publications.

In order to deepen my knowledge of these challenges and opportunities, I recently attended Scholarship in Software, Software as Scholarship: From Genesis to Peer Review - a two day meeting at Universität Bern that brought together an international audience of scholars, developers, funders, and associated individuals to consider the status, role, and assessment of software in humanities research. The discussions that interspersed the scheduled short papers, keynotes, and round table were varied, fluid, and expansive in character - indeed even the utility of wissenschaft as a term capable of overcoming the Anglophonic divide between 'the sciences' and 'the humanities' was addressed. Nevertheless, three themes were prominent: theory, community, and practice.

Throughout the meeting methodologies, debates, and controversies from the history and philosophy of science and technology - including the work of Thomas Kuhn, Bruno Latour, and Bernard Stiegler - were recurring focal points for critical thinking on scholarly practice. We asked, for example, "how is software known?" and answered that for the majority by what software does rather than how it does it. It follows from this logic that understanding our experience of software as black boxes is more important than attempting to crack those black boxes open. And yet, it was noted, historicising software as a system and as it was known by its creators can enable sustained and meaningful reflection on the intersections between the what software does and the how it does it. These creators, like the software they made, were also subjects of sustained reflection. For, it was noted, the appropriation of a particular - socially constructed - notion of authorship is a foundation of humanities research, an authorship limited to work undertaken in particular media - books, articles, papers. Unsatisfactory as this may be, it was noted that caution should be exercised as the boundary of authorship are loosened because expanding our notion of authorship is not the same as expanding our notion of scholarship, and so advocates for software as scholarship in the humanities are not best served by advocating a vision of scholarship they do not actually want, a vision that - crucially - our peers may not accept. Indeed, the meeting reinforced above all else that as the world around us and by extension the academe is softwarized, is eaten by software as software pervades, humanists (and especially digital humanists) must remain attentive critics with regards to what that softwarization is doing to research practice (for more on softwarization see David Berry, Critical Theory and the Digital, 2014). The astute observation was made, for example, that today software is increasingly embedded in scholarly reasoning processes as opposed to used primarily as a means of representing the outcomes of reasoning processes. This sort of drift in practice - enabled by softwarization and experienced both consciously and unconsciously by researchers - demands further reflection.

Building on these theoretical musings were reflections on the role of community in the valuation of scholarly work with and on software. Will it take - it was asked - a fundamental reframing of peer review for digital scholarly work in the humanities to get the credit it wants, needs, and deserves? If this heresy was considered to be some time away, it was noted that journals are already proliferating in the humanities that have taken a positive step forwards and accept modes of output and authorship hitherto novel to humanists. In doing so they are, however, likely preaching to the choir, to others in and around the Digital Humanities (hereafter DH). It was left unresolved how we meaningfully reach beyond the choir, beyond the DH bubble, when both the form and the content of work emanating from the field can trouble many humanist peers. Nevertheless, it is useful to note that the DH community, a loose conglomerate of people from various academic, alt-ac, and non-academic backgrounds, does interact with many peer communities - ranging from humanists and computer scientists - and that these interactions are one obvious means of disseminating such revised conceptions of authorship. Other venues considered at the meeting included scholarly associations, funding agencies, and memory institutions, all of which emerged as fertile meeting places for various peer communities, as places where experiments are promoted, a positive range of roles and skills recognised, and digital scholarly publications valued very differently from university departments pre-occupied with high-impact journals and prestige presses. These departments - it seems - may do well, we pondered, to consider how isolated its systems of credit can be from the real business of scholarly communication, knowledge creation, and processes of understanding. And yet, of course, the research communities isolated by the peculiarities of tenure track, research evaluation exercises, and their international equivalents are hardly small. And so while it may seem unreasonable from a DH vantage point that software is not treated as scholarship, reports back from the research software development trenches suggested that most humanists do not see rejecting the scholarly merit of software development as problematic.

The practical considerations, questions, and reflections that emerged from the meeting drew on both these theoretical and community centred discussions. On the peer review of digital scholarly work in the humanities we noted that the author of a work (or main author in a collaborative project) is likely to be the only specialist on the subject at hand and the technical approach used. How then does one go about conducting peer review of such work? And upon what criteria? Software documentation emerged as one potential route, and yet - as a presentation based on a small case-study observed - when budgets and/or time are tight (and perhaps pressures to deliver a finished product mount) documentation will always be dropped before implementation. Here some recommendations offered of the American Historical Association (hereafter AHA) for evaluating digital scholarship by historians are a welcome pragmatic step that might - with appropriate adaptation - by usefully extrapolated for evaluating digital scholarship by humanists. The AHA recommends that all research outputs should be valued for their contribution to the scholarly conversation not for the medium through which they are presented and that digital work - like physical work - should evaluated in its native medium. The adoption of these practices would lead, the AHA hope, to a more capacious professional conception of publication and of forms that can and should be evaluated. To support this evaluation, the AHA suggests that scholars should be offered support and encouragement to document their digital projects.

Taken together, these discussions indicate that the systems of credit for good software development and reuse in the humanities are on a positive trajectory. It is interesting to note that when the meeting focused on evaluating digital scholarly work discussion tended to slip between and elide peer review for funding, peer review of manuscripts submitted to journals and presses, and peer review in the form of post-publication reviews and review articles. This says something about how the often software based work produced by the DH community is already presented in a multiplicity of forms, many of which come into being outside of established scholarly publication frameworks, workflows, and processes. And yet in order to develop practical recommendations for the evaluation of scholarly software by humanists, future meetings of this kind will need to tease out these three and in doing so more crisply define the frames of reference within which criteria for evaluation are discussed.

As for my own contributions, on the first day of the meeting I presented a paper on how the work of the British Library Digital Research Team aims to confront scholars with both the software choices they make and the software choices that are made for them during the course of their research, and how thinking critically about the role of these software can enrich their own research. On the second day, I contributed a position statement to the round table on peer review of digital scholarly work based our experiences working with and alongside many humanities research communities. Here I argued that a pre-requisite for widespread, enduring, and embedded acceptance by the humanities research community of the necessary evolution to peer review that bodies such as the Software Sustainability Institute advocate for and that the round table sought to address, is that more humanists deem playing with software to be acceptable research practice; deem figuring out the limitations and possibilities of command line scripting to be as comparably useful as reading a monograph relevant, if not essential, to one's field of study. Indeed to return to the question of "how is software known?", humanists need to know software as something that is for them, is as much part as their research tool kit as it is for any other discipline.

Slides and notes for my paper 'Removing black boxes: exposing software to researchers' are available on Slideshare and a recording of the round table will appear in due course. My live, partial, incomplete, CC-licensed notes from the meeting are available on GitHub Gist. My thanks for Tara Andrews and Enrico Natale for proposing this stimulating meeting, convening such a critical and engaged audience, and for hosting us all in the beautiful city of Bern.