Ensuring Implementation Fidelity of a 10-Week Home Visiting Program in Two Randomized Clinical Trials

Abstract

Introduction Implementation fidelity is a challenge for the adoption of evidence-based programs within social service broadly and child welfare specifically. However, implementation fidelity is critical for maintaining the integrity of clinical trials and for ensuring successful delivery of services in public health settings. Methods Promoting First Relationships ^® (PFR), a 10-week home visiting parenting intervention, was evaluated in two randomized clinical trials with populations of families in child welfare. Seven providers from community agencies participated in the trials and administered PFR. Fidelity data collected included observational measures of provider behavior, provider records, and input from clients to assess training uptake, adherence to content, quality of delivery, program dosage, and participant satisfaction. Results In mock cases to assess training uptake, providers demonstrated an increase in PFR verbalization strategies and a decrease non-PFR verbalizations from pre to post PFR training, and overall this was maintained a year later (Mann–Whitney U's = 0, p's < .01). Adherence to content in actual cases was high, with M = 97% of the program elements completed. Quality of delivery varied across providers, indicated by PFR consultation strategies (Wilks' Lambda F = 18.24, df = 15, p < .001) and global ratings (F = 13.35, df = 5, p < .001). Program dosage was high in both trials (71 and 86% receiving 10 sessions), and participant satisfaction was high (M = 3.9, SD = 0.2; 4 = greater satisfaction). Discussion This system of training and monitoring provides an example of procedures that can be used effectively to achieve implementation fidelity with evidence-based programs in social service practice.