Software Development: Scientific Research Jeopardized by Black-Box Software

In 2009, a scandal nicknamed ‘Climategate‘ hit the scientific community in the UK after hackers broke into the computers of the University of East Anglia.  The researchers had refused to release the data analysis software for public review that they used for processing their data.  It was discovered that the prominent British climate researchers were manipulating climate change data.  Whether there is ill intent or not, this case calls into question the reliability of software being used in research today.

Dirk Gorissen, former research software engineer at the University of Southampton, told the UK Times Higher Education that “the problem is, academia all comes down to what publications you make. When you spend a lot of time on software it can be difficult to turn that into publications.  The code you write – the quality of it – doesn’t really matter. You put effort in and make sure it’s well developed and up to scratch, but I’ve been told a number of times that it is not important. All that matters is whether you can get numbers out so that we can publish them in a paper: the code that makes it happen gets forgotten.”

Simon Hettrick, deputy director of the Software Sustainability Institute, said that “Look around any lab and you will see software being used in all disciplines and by all seniorities of researchers.  Software is clearly fundamental to research, and I find it terrifying that we don’t understand quite how much we rely on it… It’s a terrible concern, because you can work your way through software development—researchers are intelligent people, they can work this stuff out—but you can’t build software that is reliable.  If you’re producing your results through software, and your software doesn’t produce reproducible results, then your research results aren’t reproducible.”

Hettrick said that “Software use is not going to decline.  We need to be aware of our reliance on software so that we can ensure that the [academic] community has the tools and the skills they need to stay at the forefront of research…  There’s not much institutional memory around software artefacts created by research – and these could very well fit into the impact agenda of the research excellence framework that universities think a lot about. Software is one of the outputs of a lot of research projects, and it deserves to be treated as something that other research groups around the world can use.”