A shifting landscape of data management plans
Alongside yesterday's new research data policy, UKRI has said it will be updating its approach to data management plans and the information it requests, along with new templates and peer review guidance.
This is backed by an independent review from Research Consulting exploring what a pan-UKRI approach to data management plans would look like. These plans are where researchers are required to formally spell out what data will be created or collected during research, and how it will be handled during and after the project – and, as the report observes, there is currently great variation both across UKRI and elsewhere in research funding.
Following a review of the evidence from around the world, the report concludes that there is a “shifting landscape” in which funders are “experimenting” with different planning instruments – and there is still a lack of evaluation on how this impacts on behaviour, burden, and downstream data use.
The report finds that the approach to data management plans, or DMPs, across the different research councils reflects the “semi-independent policy development” that preceded UKRI’s establishment – which is to say, the different councils are requiring very different things. Most are said to follow a “front-loaded” approach: requiring a plan at application, without a specific requirement to update it as research progresses.
ESRC and NERC, on the other hand, have a more “hands-on approach”, expecting updates during projects and using their own data infrastructure to monitor compliance. However, the report suggests that the resourcing requirements would “not currently be feasible” elsewhere in UKRI. And then cross-research council schemes may create further difficulties to navigate.
Outside UKRI, it’s noted that NIHR has no formal requirements at application stage, but rather requires DTPs to be completed during research startup, and that Wellcome requires a plan on application – unless a justification is provided – and that it considers the plan’s commitments around data sharing as part of the end-of-grant reporting process.
Community feedback
The consultancy engaged with 71 stakeholders from funders, research performing organisations, researchers, publishers and infrastructure providers, via a mix of interviews and focus groups. This engagement highlighted that variation in DMP requirements – in everything from word count to template format – creates practical challenges for researchers and organisations. On the other hand, participants made it clear that consistency “should not come at the cost of appropriateness,” given the inevitable diversity in requirements in different disciplines and in different projects.
The research unearthed various other community concerns about data management plans as well. First among these was the question of whether weak plans would ever affect funding decisions – participants thought not – and how they were actually being assessed:
“Evidence from institutions and UKRI councils indicates that grant peer reviewers often feel unqualified to assess technical aspects of DMPs, such as those relating to intellectual property, sensitive data handling, or storage infrastructure, which is a structural weakness in today’s approach.”
The costing of data management arrangements was also found to present difficulties – this is another area where UKRI says it is developing updated guidance. Researchers are reluctant to include visible costs in applications “due to competitiveness/value-for-money perceptions”, the report finds, while for institutions the longer-term costs of data management are hard to match to individual projects.
Other issues highlighted by participants include whether DMPs are actually kept updated as “living documents” in practice, as funders often frame them, and the disparities in resourcing between institutions which means that some organisations have less than one full-time equivalent member of staff on hand for data management support.
Towards a pan-UKRI approach
The report arrives at “five foundational considerations” for what a pan-UKRI approach would need: alignment with the new data policy, appropriate timing, proportionality, “structure flexibility,” and adequate resourcing.
The model proposed by Research Consulting would see minimum requirements for DMPs in place both on application and at end of grant – the latter would involve reflection on research data management. It’s suggested that a twin-track approach should be used: for most opportunities responsibility during the project would lie with grantees and their institutions, whereas for those with high complexity or using UKRI-run data repositories, a full DMP would need to be in place within six months of the award start (addressing Science Europe core requirements).
UKRI is planning updated guidance on data management plans; for now, its new data policy (in place from April) stipulates that applicants will normally be required to provide an outline plan, while full plan “may” be required, and assessed, for some data-intensive funding opportunities. When funding is awarded, a full DMP will be required within six months of the start of the award or studentship project.
The real challenge, the review concludes, is that the system and incentives around data management plans have “turned what should be an integrated part of good research practice into an administrative exercise.” A pan-UKRI approach can address some of the issues identified, but arguably can only scratch the surface of others.