Vezzini issop 2018

VEZZINI, Francesca, MONTI, Francesca, FANTUZ, Fiorenzo and TAMBURLINI, Giorgio, 2018. Challenges in evaluating ECD-focused interventions targeting at risk population groups with a non-selective approach (Un villaggio per crescere). In: ISSOP2018 - Early Childhood Intervention: Science, Systems and Policies - Promoting Healthy Development of Vulnerable Children [online]. Bonn, Germany: DGSPJ. 27 September 2018. p. 1–120. [Accessed 4 November 2018]. Available from: https://www.issop.org/cmdownloads/vezzini-issop-2018/

Background: Testing innovative interventions to promote nurturing care for early childhood development is a recognized global priority (1). Interventions need to be multi-sector, multifaceted, highly contextualized (2). Traditional evaluation designs may not be applicable to interventions where target population is not rigidly selected and stable over time (3). We describe the main methodological challenges emerging from the development of an evaluation design of a multicenter project aimed at promoting ECD-focused parental services in disadvantaged communities in Italy. The work is part of a broader effort to develop a cross-countries evaluation framework for an international network of not-for profit organizations’ projects targeting at risk families and their children aged 0 to 6 years. (1) World Health Organization, United Nations Children’s Fund, World Bank Group. Nurturing care for early childhood development: a framework for helping children survive and thrive to transform health and human potential. Geneva: World Health Organization. 2018. (2) Black, MM, Walker, SP, Fernald, LC..., and for the Lancet Early Childhood Development Series Steering Commitee. Early childhood development coming of age: science through the life course. Lancet. 2016 (3) Habicht, Jean-Pierre & G Victora, C & Patrick Vaughan, J. Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. International journal of epidemiology.1999

Method: We firstly identified the dimensions to be included in the impact evaluation, i.e. early child development, parental competencies, stress and self-efficacy and community networks. Then we searched for existing validated tools and assessed them tools for feasibility in non-research settings through a consultative process involving national and international experts.

Results: The overarching challenge of evaluation in real-life, non-research settings is to find a balance between the need of rigorous approach and the intrinsically open, inclusive nature of interventions for population groups which cannot be constrained into rigid evaluation schemes. More specific challenges include: applicability of controlled and blinded approaches; transformation in target population (both children and parents) independently from the intervention; need for assessment tools which do not require highly specialized skills; combination of developmental assessment with promotion and evaluation embedded in a dialogue with caregivers, minimizing attrition and resistances to collect potentially sensitive data; inclusion of qualitative and participative approaches.

Discussion: Scaling up ECD-focused interventions in at risk population groups implies moving from traditional experimental designs applicable to confined and controlled settings towards complex and dynamic quasi-experimental, mix methods approaches (4). The identification of methodological challenges is a pre-requisite for developing evaluation designs which are realistic and feasible and at the same time able to produce valid results (5). Our work intends to contribute to a much-needed international effort and dialogue on evaluating complex ECD-focused interventions. (4) Bamberger, M., Rao, V., Woolcock, M. 2010. Using Mixed Methods in Monitoring and Evaluation: Experiences from International Development. World Bank Policy Research Working Paper No. 5245. Accessed December 17, 2017 (5) Guijt, I, Brouwers, J, Kusters, C, Prins, E, Zeynalova, B. Evaluation Revisited: Improving the Quality of Evaluative Practice by Embracing Complexity – Conference Report. Wageningen: Centre for Development Innovation. 2011

Authors, Institutions: 

Francesca Vezzini (1), Francesca Monti (1), Fiorenzo Fantuz (2), Giorgio Tamburlini (2)  1. Fondazione Generali - The Human Safety Net ONLUS  2. Centro per la salute del bambino - onlus

https://www.dgspj.de/wp-content/uploads/ISSOP-2018-Abstract-Vezzini.pdf