Posted by s.aragon on 5 February 2019 - 9:08am Image courtesy of ADP.

By Steve Crouch, Software Sustainability Institute, and John Shepherdson, CESSDA Platform Delivery Director describe the main outcomes of two technical training events which took place in 2018 and plans for the next steps in 2019.

Any research community needs both resources and skills to thrive, and CESSDA's software development community certainly has the resources in the form of their sophisticated Research Infrastructure. The challenge is upskilling new developers to keep pace with all the relevant technologies, guidance, and procedures, whilst bringing existing developers up to date.

The CESSDA Expert Seminar (CES2018) took place in September last year and combined strong theoretical underpinnings with Software Carpentry-inspired practical sessions. It was a successful and well-received approach to this challenge, as was a follow-up event held in December.

The CESSDA Research Infrastructure enables researchers to access, develop and deploy tools and services to assist their research. One such tool is the CESSDA Data Catalogue which provides access to social science research data archives across Europe. Over the last couple of years, CESSDA has built-up and improved a comprehensive infrastructure that provides a development and test environment available to all CESSDA Service Providers – at least those who develop software for CESSDA. This environment makes use of modern technologies such as Bitbucket, Jenkins, Selenium, and SonarQube. To enhance the reproducibility, reliability, and scalability of the components that form part of the CESSDA tools and services portfolio, it also uses containerisation (Docker) and clustering (Kubernetes).

However, having a powerful development and deployment infrastructure for the CESSDA Service Provider community is not enough. It is also important to ensure that all developers that contribute software are developing it in a consistent and reusable way and are aware of what is expected of them.

The latest version of the CESSDA Technical Architecture, published in 2018, embodies a set of development principles, practices and guidelines for designing, developing, and deploying CESSDA tools and services. Whilst many in the community are highly skilled in the infrastructure and its development practices, the ongoing challenge is to ensure newcomers sufficiently increase their skills and ‘best practice’ knowledge in line with their colleagues.

The CESSDA Expert Seminar (CES2018) aimed to address this. It was organised by the CESSDA Technical Working Group, was chaired by John Shepherdson and Steve Crouch presented the practical sessions. Despite covering a great deal of theoretical and practical material in a single day, the feedback received indicated that the event was very effective.

Balancing theory with practice

Held on 26th September at the Slovenian Social Science Data Archives at the University of Ljubljana, CES2018 brought together developers from fifteen CESSDA service providers. The event offered two different approaches: to begin with a seminar-style presentation of the more theoretical aspects so as to  provide a solid foundation, followed by a set of practical sessions. These built on the theoretical aspects and exemplified them, which provided practical experience in accomplishing live-coding tasks and exercises. It was a careful balance, in recognition that either aspect alone is insufficient to impart the skills necessary.

In this way, the needs of this type of event are very different from a typical Carpentry event. The teaching style and techniques adopted by Software and Data Carpentry – now known together as the Carpentries – are proven approaches for teaching new technical skills to researchers. They work because they address the recognised lack of skills at a particular level across the research community. The prior knowledge required to benefit from a Carpentry event, and the skills needed to be learned to progress an attendee to the next level, is widely understood. However, when it comes to training developers on sophisticated research infrastructures like that used by CESSDA, the prior knowledge gap is that much wider. Providing a solid theoretical foundation is required, and scheduling this beforehand allows the practical sessions to reinforce this underpinning whilst maintaining a high degree of practical momentum and achievement.

The practical sessions began with a brief overview of the latest versions of CESSDA’s Technical Architecture and Software Maturity Level standards. This was followed by hands-on practical sessions developed and delivered by Steve that covered live coding and exercises on how to build, test, and deploy an example CESSDA tool within an automated pipeline, with emphasis on what CESSDA provides and what is expected from developers. Following CESSDA practices, attendees used a Bitbucket repository with exemplar code to develop and resolve a deliberate issue with the codebase and add new unit tests. They were also shown how to configure and make use of the CESSDA Jenkins continuous integration server to run these tests automatically. In the spirit of a Carpentry event, Claus-Peter Klas (GESIS) and Wilko Stein (DANS) acted as co-presenters and helpers and provided great assistance to those experiencing any technical issues on the day.

Feedback from event attendees was very positive, and all delegates either agreed or strongly agreed with the statement that ‘the materials for the practical session were appropriate’. When asked to what extent they agreed the practical sessions were effective, 93% said they were effective or very effective. Overall, the event was a great success.

John Shepherdson, leader of the CESSDA Technical Working Group stated that: “Significant progress has been achieved since the last meeting two years ago. This has enabled social science data archives across Europe to develop and deliver high quality tools and services to run on the CESSDA technical infrastructure.”

Preparing for the future

Feedback also indicated a desire to move on to the next level with regards to Jenkins pipelines, Docker containers and Kubernetes clusters. So a half-day follow-up workshop was arranged as a side event to EDDI18, in December 2018, which built on the solid foundations laid by the theoretical and practical content of the CESSDA Expert Seminar.

Once more, feedback from delegates was very positive: 90% either agreed or strongly agreed with the statement that ‘the materials for the practical session were appropriate’. When asked to what extent they agreed the practical sessions were effective, 80% said they were effective or very effective.

In order to maintain momentum, a hackathon is being considered for late 2019.