COVID-19 has put the spotlight on a type of monitoring that only slowly emerges: remote monitoring. Remote monitoring is what we need to do if we do not have direct, physical access to people, things, and documents we want to keep track of. So far, we have developed and tested remote monitoring mostly in a humanitarian context, where access to sites is not possible, limited, or too dangerous.
Enter the wonderfully named ‘Delphi’ approach.
In my view, the Delphi technique is dramatically undervalued in our toolbox for monitoring.
What is a Delphi technique?
The Delphi technique structures repeated group communication of experts to arrive at an agreement on complex issues. It includes multiple and iterative rounds of data collection and providing participants with structured summary feedback between rounds. The results can be qualitative or quantitative. It is based on the principle that estimations from a structured group of individuals are more accurate than those from an unstructured group. Further, it relies on repeated discussions among a panel of ‘experts’.
What about Change-Oriented E-Delphi?
E-Delphi works remotely with experts in different geographical locations. As such, we can use it in situations where remote M&E is the only option, like during pandemics or epidemics. An extension of the traditional Delphi method – change-oriented Delphi – is suitable for complex and multidimensional problems, such as fluid and uncertain circumstances.
That’s where it gets interesting: ‘Experts‘ is anyone that is particularly knowledgeable. We are not limited to academics or scientists – but can include community leaders, girls or boys, women and men, local administrators, etc. Basically anyone with in-depth knowledge of the situation on the ground.
In 2020/21, the Global Network on Mental Health and Child Marriage used a change-oriented e-Delphi study to investigate mental health consequences of child marriage and potential solutions. The aim was to generate consensus and priorities for driving action and response. The study included a variety of stakeholders including global leaders and survivors.
Round 1 was blended. It used in-person and online discussion groups via zoom with a selection of invited experts to explore findings from literature reviews on this topic area. Round 2 was entirely remote, with an electronic survey, sent to a wider range of experts directly, and an open call via online advertising of the study. After that, round 3 was in person with survivor focus group discussions about priorities established by academic, policy, and civil society actors, while round 4 – the second-round survey sent back to participants in round 2 – was entirely remote. More here.
We can carry out small-scale Delphi studies with less than 50 experts without a specific tool. Further, we can identify experts, design questionnaires, data collection, analysis, and interpretation using typical desk top software like Google Forms. For more complex E-Delphi studies, there are useful electronic Delphi platforms out there, like Mesydel, Calibrum and eDelphi.