Improving capacity for impact evaluations in middle Africa: The contribution of South-South collaboration
By
Tangang Andrew
Research Fellow, eBASE Africa
The last decade has experienced remarkable growth in the use of impact evaluations, a trend which has been supported by international donors and governments. Most impact evaluations are conducted by European and American researchers, and it is generally assumed that Sub-Saharan Africa lacks in-country researchers capable of conducting high-quality impact evaluations (Altshuler & Staats, 2019).
A scoping study on impact evaluation capacity in Sub-Saharan Africa by Erasmus, Jordan & Stewart (2020) reveals that there is widespread capacity to conduct impact evaluations in Sub-Saharan Africa contrary to previous assumptions. However, Central Africa still has significant gaps in terms of impact evaluation capacity because there was limited or no capacity to conduct impact evaluations in 4 of the 5 countries in the region. Despite Cameroon being mentioned as having some institutions providing training in impact evaluations, there is a gap in terms of publications on impact evaluations. This scoping study notes an over-emphasis on health sectors, and that capacity deficits still exist in non-health sectors which require further capacity support. Therefore, what practical steps have been taken to enhance capacities for impact evaluation in Cameroon?
As an active organisation in the evidence ecosystem in middle Africa, eBASE Africa has co-organised four training sessions on conducting impact evaluations in the education sector. This is in tandem with its evidence implementation projects aimed at generating evidence on whether a comprehensive Menstrual Hygiene Management intervention can improve learning outcomes for adolescent girls, and what Social and Emotional Learning (SEL) strategies can improve learning outcomes for IDP learners in primary schools. The capacity building training sessions were organised in partnership with the West Africa Capacity Building and Impact Evaluation Programme (WACIE) of the International Initiative for Impact Evaluation (3ie), and Le Barometre/Institut Africain de Suivi et Evaluation d’Impact des Politiques, Programmes et Projets de Developpement, and funded by the Education Endowment Foundation (EEF).
The training sessions
Four training sessions were organised including an introduction to impact evaluations, an introduction to pilot evaluations, data collection methods in development economics, and quantitative data analysis. These trainings employed a hybrid approach that combined both online and in-person sessions, and the targeted audiences were evaluators working on eBASE-funded evaluations and partner organisations. The World Bank and UNICEF were interested in the training and recommended to their partners recognising the need for capacity building for evaluators.
The interactive training sessions covered different themes including theory of change and results framework, counterfactual and causal attribution, Randomised Controlled Trials, Difference in Difference, Regression Continuity, Instrumental Variable, Randomisation with STATA and Excel, Propensity Score Matching and pilot phase of an intervention. These themes addressed key aspects of impact evaluations using different methods and practical examples that are applicable in most education projects implemented in the Middle Africa region.
Furthermore, the training sessions were organised as a ‘training of trainers’ approach where in the trainees were required to pass on the knowledge acquired to others in their respective organisations and networks. As such, eBASE Africa organised follow-up training on impact evaluations in Bamenda and Buea to train other partners working on various research projects. Across all four workshops, over 80 participants took part in the online training workshops of which 35 were female while 45 were male. 60 of these participants took part in the in-person workshops comprising 35 males and 25 females.
What lessons for impact evaluation capacity building in middle Africa?
One of the main lessons learned from these training sessions is that the need for impact evaluation capacity in Africa as a whole and middle Africa, in particular, is evident. Beyond the recognition that in most Middle African countries, research evidence does not inform government policies, it is worth questioning the quality of the research evidence that is currently being generated including issues around ethics. Three key points stand out from these trainings. Firstly, the research participants were able to distinguish between monitoring and evaluation, and impact evaluations in development interventions. Some participants started reflecting on the possibility of including research and randomisation aspects when developing their research proposals. Secondly, although the online sessions and training materials provided introduced the concept of impact evaluations, the in-person workshops clarified many of the concepts and workshops that had been introduced and explained online. Thirdly, training on the use of different software for quantitative data analysis should be more targeted and in-depth to enable participants understand the functioning of these platforms.
While recognising the value of these once-off training sessions on various aspects of impact evaluations, it is worth stating that more is needed to effectively improve capacity building for impact evaluations in middle Africa. At eBASE, we strongly recommend further investments in capacity development for impact evaluators to meet up with contemporary approaches to impact evaluation. We believe there will be good returns on investments for development work in Middle Africa general for this and particularly so for educational development programs like the eBASE Teaching and Learning Toolkit. Hence, there is a need to consider moving towards a fellowship approach wherein trainees spend some time working and learning in other African-based organisations that are geared towards building impact evaluation capacities.