Posted by s.sufi on 28 April 2016 - 4:40pm 

By Shoaib Sufi, Community Lead, Software Sustainability Institute.

The Institute’s Collaborations Workshop 2016 (CW16) took place from 21-23 March 2016 at the Royal College of Surgeons of Edinburgh. The opening slide set the scene displaying a weighted representation of which software the people attending used in their daily work. Shoaib Sufi’s welcome to attendees was followed by an introduction from the Institute Director, Neil Chue Hong. Neil spoke about the work of the Institute and how to get the most out of a Collaborations Workshop (CW). The clue was very much in the name, the main idea was to meet people, people you may not have met before, thus widening your network of potential collaborators. With over 80 attending it made for a real opportunity to learn and share. We cover how the workshop unfolded below.

Lightning talks

We moved quickly onto lightning talks; these gave attendees 2 minutes to talk about aspects of their work and how this related to the theme and sub-themes of the workshop: Software & Credit, Reproducible Research, Collaborative working, code/data sharing and data science. We had almost 40 lightning talks this year, inspiring and informing the audience on topics as wide ranging as dealing with messy data, credit for multi-author papers, containers & reproducibility, promotion of allied workshops and increasing computational competencies in the UK and beyond.

Social supporting Collaboration

Some of the most productive time for forming collaborations and networking takes place at CW’s during lunch and coffee breaks rather than the workshop sessions. To facilitate one of the main purposes of the workshop neither lunches or coffee time was curtailed. The breaks took place in the aptly named Fellows Library and there was much mingling and conversation. CW16 also had a social programme, that allowed more time to see the sights and sounds of the lovely city of Edinburgh and give further time for getting to know the other attendees. The social programme included the delightful workshop dinner, a guided street tour of Edinburgh by the Institute director who hails from the city, a brisk morning walk up Arthur’s Seat and a guided tour of the Surgeons’ Hall Museums which was fascinating and not for the faint hearted.

Better Communication

CW16 had a real emphasis on the importance of communicating your research. By communicating your research and computational practice to those in your domains or who work in other areas it becomes much easier to share best practice and accelerate the computational maturity of other fields of work, thus increasing the impact of the work that you do. Better writing is often key to such communication. To support the emphasis on better communication  we were very fortunate to have David Robson, Feature Writer at the BBC attend and run a Better Writing workshop. Although some of the room were seasoned writers, it was an enlightening experience for those less use to focusing on promotional writing to explore the art of clear thinking and classic style leading to clear writing that communicates your message well.

Discussion Sessions & Speed blogging

The theory and thoughts on writing were put to good practice at the CW16 discussion sessions. The discussion session topics are suggested at registration time and during the workshop, these are topics that are related to the themes and sub-themes of the workshop. Attendees get to vote on which discussion topic they wanted to attend, this is an example of the unconference philosophy of the CW’s in action. The discussion sessions ran on an extended schedule and the latter part were dedicated to speed blogging the outcomes. The idea was rather than a bullet point list at the end of the discussion session which was only really understandable to those who attended the discussion, a blog (even though quickly produced) would capture the key ideas and thoughts of those discussing a topic in a way which was more communicable and informative to the wider research software community. The CW16 speed blogs will make their way to the main Institute blog, for a full list of what was discussed and the associated speed blog please see the annotated CW16 agenda. In terms of themes, the speed blogs were broadly focused on the mentoring the journeymen (and women!) gaining mastery in computational skills, usability, improving the quality of research software, the beauty of null results, using collaborative tooling and RSE career paths.

The Keynote

The CW16 keynote took place on day two, titled,‘Can scraping the web help us get credit for software?’ and expertly delivered by Euan Adie, founder of Altmetric. Euan gave a tour de force of what credit could be collected by mining the web, both quantitative (mentions/citations) and qualitative(opinions). In his talk he highlighted the facets of research outputs for which credit is sought: quality, engagement and Impact. He also warned about the difference between notoriety vs impact (e.g. mentions of the controversial Treefinder license update by its author). The fact that many people are talking about a piece of software on the internet needs further investigation before one can deduce whether it’s truly impactful or merely notorious. Ultimately, he notes that although data can be collected and enriched automatically it still took people to interpret that data and put it into context. Slides of the talk are available.

Software & Credit Panel

This keynote was followed by a panel on Software and Credit. The panel chair was Institute Director, Neil Chue Hong and the panelists were Euan Adie (our keynote speaker), Martin Hammitzch, Catherine Jones and Prof. James Davenport. The panelists explored their personal motivations around the topic, many of them were tasked with making sure software was credited to help support their own teams and colleagues that wrote software as the main part of their work. The panel then moved onto immediate actions needed to improve the lot of Software and Credit and their thoughts focused on changes needed to making it easier to get credit and the need for cultural acceptance for software that is supporting research. The panel then focused on how to encourage change and whether dealing with the social problems around software’s validity as a research output were more important that producing better technical platforms to capture and track credit.

Thornier questions such as ‘What does credit mean in research software’, who gives such ‘credit’ and how they would give credit were then discussed. A video of the panel will become available on the CW16 agenda page. One of the key messages from the panel was that being careful about whether the focus should be establishing credit for the software that was written or whether it was establishing credit for those who wrote the software. The two were not the same and whereas keeping the software sustained is important, maintaining those who write research software with real career paths was a necessary condition for better software and therefore better research.

Collaborative Ideas

The Collaborative Ideas session are always an interesting and fun way to identify problems in research software and propose solutions, with attendees voting for the best ideas and prizes available, some peer review and excitement was added at CW16. Unlike the discussions session, the teams in the Collaborative Ideas sessions are preset, however the problem/solutions they identify or come up with was not mandated, although there is a  preference for ideas related to the theme or sub-themes of the CW. We used a constraint based system for choosing the teams, as suggested by our CW16 steering committee; there was at least one RSE,  if there were Fellows they were from different inauguration years and we chose people from different Institutions. A collaborative idea session solution was  framed as hackday ideas where a team of 3-6 people could make reasonable headway in 18 hours. Many of the ideas focused on the workshop theme: involving mining resources more deeply to find citations and thus support credit, making it easier to put together portfolios of your work, finding collaborators for computational projects and making it easier to climb the computation competency ladder both emotionally and in terms of learning paths. ‘git blame self’ won first prize and ‘Research Portfolios driven by ORCID, customized by users’ took the second prize; well done to those teams! You can see the entire list of entries on the annotated agenda.

Demo time

The demo sessions at CW16 gave attendees a chance to see research software in action or to present their research software. There was a wide gamut of research software on display and importantly the people behind the software or practice were at hand to offer advice on technical details and how the software or approach might be adapted to meet the needs of interested parties.  The CW15 Hackday winner ‘ReciPy’ had developed from strength to strength and was being demonstrated as a product at CW16. Jupyter notebooks were demonstrated using the nbval system as a way to help keep documentation up to date. The use of Linked Data in research was demonstrated illustrating how it could enable new types of research question.  A tool to help you write effective Software Management Plans was demonstrated; this is a well field tested tool for helping you manage your software related outputs at the bid stage and beyond. Our friends at qLegal came to talk about the legalities of open source licensing and talked through the thorny issue of whether putting software into the public domain actually decreased liability while maintaining credit in an effective manner.

In the second session, the ChemBio Hub system was explained, although the name suggested chemical biology, the system was in fact a Research Data Management (RDC) platform for any domain where tabular data is generated, stored or analysed.  The value of Property Based testing that, e.g. help create test cases for stochastic processes using the ‘Hypothesis’ Python library was demonstrated. And a home-brew method of designing a crowdsourcing platform to help get human identification of data in their research was also shown. The demo sessions are always popular and allow people to see software in action and talk to the people behind the software to see how they may use it or learn from the approach. Slides are available for most of the demonstrations, and for those that took place in the main hall videos should become available in the coming weeks on the annotated agenda.

Effective Training panel

There was a panel on computational training chaired by Raniere Silva (Software Carpentry steering committee member) with panellists Aleksandra Pawlik (The Institute's Training Lead), Jonah Duckles (Software Carpentry Foundation Executive Director) and James Hetherington (Research Software Engineering group leader at UCL).

Jonah introduced the training model of Software Carpentry and the benefits of attending the training to become an instructor. Aleksandra introduced the practicalities of running a Software Carpentry or Data Carpentry workshop in the UK and the support that was available. James talked about the extended Software Carpentry style approach he has introduced at UCL for MRes students comprising 10 sessions of a few hours over a semester. He also advocated using literate programming approaches to generate correct lecture material, allow import and export of materials to support mutual benefit and supported the use of the CC-BY license to facilitate this.

The panel were then open to questions and these touched upon specificity of examples when creating training materials, the difference between Software and Data Carpentry, the mental model pedagogy of Software Carpentry and the power of bringing people from different domains but with similar modelling needs together to learn from each other. The full video will be made available on the annotated agenda page.

CW16 Close

Institute Director, Neil Chue Hong, closed the main Collaborations Workshop 2016. He highlighted the non-stop chat during the breaks as a sure sign of collaboration taking place, the excellent talks, the bright ideas, how the speed blogging exceeded our expectations and on the quality of the demos.

The main themes he drew from the workshop focused on the balance between credit for software and credit for people developing that software and the rise of automation around capturing information about software use.

He noted that many in the room were there because their roles were different than the standard roles in academia and as a software oriented community we needed to keep up the pressure for change to help accommodate the differing roles that were necessary for better research through better software.

CW16 Hackday

The CW16 Hackday started on the evening of the second day (22 March). It began with an overview of the purpose of the event, which was to learn by collaboratively working on something new and do things a bit differently to one's’ normal day job. It was an opportunity to see new tools in operation, learn from your teammates, come together and think about the future impact of what you were putting together. Steve Crouch, the Institute Research Software Group lead also gave a presentation on technical choices, introducing teams to the collaborative tools that would help them work more effectively together.

After a pause for Pizza and drinks, it was onto the pitches. This was where potential hackday leaders got a few minutes to attract people to work on their idea. The ideas came from the Collaborative Ideas session earlier in CW16 and independent ideas brought to the CW16 Hackday by participants. Then there was time for team formation. There was much horse trading as teams formed, merged and ideas rose to the top and others were abandoned.

9 teams then emerged from the 21 pitches that were presented, the successful teams varied in size from two to seven people. Not all formed teams were working on code based projects, some were focusing on producing a paper (aka a paper hackathon) and some were working on standards; this was all part of widening the appeal of CW Hackdays to show the importance of non-code work to research software. An evening and most of a working day of hacking thus began. Some teams worked late into the night, others had an early night ready for the day ahead.

On the 23 March the venue was available from 8am onwards. The teams all sat cabaret style, one team per table in the Playfair Hall. As they designed, coded, performed analysis, integrated components, prepared documentation and artwork etc. Teams were visited twice (once in the morning and once in the afternoon) by the CW16 judges who were determining how they were coming together as a team. There was also Mike Jackson, a senior Architect at the Institute who was designated the ‘safe friend’ who was able to offer advice and some help but could not feedback any information to the judges. The judges had a room on the ground floor where the sat and discussed the scheduled of visits to the team and what the judging criteria meant for each of them in terms of what they would be looking out for in practice.

Then 3.30pm was upon everyone. Each team had 5 minutes to present not only their work but the process of how they worked, how open their product/service was and what impact it would have in the future. The hacks were of high quality making and the judges had a tough time scoring. But with aggregated and normalised scores in, the Judges had 15 minutes to discuss the outcome; in all honesty it could have taken 2 hours to discuss. But then the Judges agreed on the winners, they were as they stood from the marking, judge's discretion was not used. The winning team was Research Software Sentiment Analyser, the product they built helped track what people were saying about specific pieces of software on Twitter. The winning team leader was Raquel Alegre, this was her second appearance on a winning team which means in CW17 she will be asked to be one of the judges! In second place was MatchMakedemia led by Angus Maidment; this service made it easier for researchers to connect with developers to help form collaborations and accelerate computationally based projects. You can see more details about the winners and all entries on the Hackday page.

CW16 Hackday was closed by the CW16 lead organiser and Institute Community Lead, Shoaib Sufi. CW16 and the Hackday were an intense and productive three days and those who stayed till the sweet conclusion were CW16 survivors, or should that be victors; gaining new skills, insights, collaborative practice and potential collaborators.

A big ‘Thank you!’

To our keynote speaker Euan Adie and to David Robson who presented the better writing workshop. Our steering committee members, especially James Davenport, Terhi Nurmikko-Fuller and Olexandr Konovalov. Our Hackday judges especially those from the community: Robin Wilson, Catherine Jones, Martin Callaghan and Rob Davey.

Those who gave lightning talks, lead demo sessions, panellists, lead or scribed for discussion groups and collaborative idea sessions and those who took part in the Hackday. In fact CW16 is what it is because of those who attend, thank you to all those who attended!

A note of thanks also to the Institute staff who helped so much with the preparations and helped make the event run smoothly: Giacomo Peru, Clementine Hadfield, Graeme Smith, Neil Chue Hong, John Robinson, Steve Crouch, Simon Hettrick, Mario Antonioletti and Mike Jackson. Also thanks you to Sean McGeever for organising the recording equipment.

A note of thanks also to the conference team at the Royal College of Surgeons of Edinburgh for their polite and efficient service, especially Andre Soromenho.

Feedback Results

We had 80 attending, this included Institute staff. The analysis below is based on 33 non-staff responses (49% response rate for non-staff who attended which is deemed an excellent response rate).

The main CW16 workshop scored 4.3 out of 5 in terms of usefulness and 4.6 out of 5 in terms of enjoyability. The CW16 Hackday scored 4.2/5 for usefulness and 4.5/5 in terms of enjoyability. Overall a very enjoyable and useful time for the participants.

76% of respondents said that they would come again next year, 88% said they would recommend the event to a colleague. 97% said they would or may come again next year and 100% said they would or may recommend it to a colleague. Attendees thus saw the value in coming again and promoting the event to colleagues.

On average people met 15 new people at the workshop and started or intended to start two new collaborations. This is a great indication of positive future activities enabled by attending the CW.

And there is more

Read all the best tweets at Storify.

Mario Antonioletti published an article on the EPCC blog about CW16.

Collaborations Workshop 2017 (CW17) announced!  #CollabW17

We are very pleased to announce that Collaborations Workshop 2017 (CW17) will take place at the Leeds University Business School, University of Leeds from Monday 27 to Wednesday 29 March 2017; The themes will be ‘Internet of Things and Open Data; implications for research’. We hope to see you there!