Assessing technical performance and determining the learning curve in cleft palate surgery using a high fidelity cleft palate simulator

Introduction: Technical skills assessment can provide objective measures of surgical performance. This study assessed technical performance in cleft palate repair using a newly developed assessment tool and high fidelity cleft palate simulator through a longitudinal simulation training exercise. Methods: Three residents performed five and one resident performed nine consecutive endoscopically recorded cleft palate repairs using a cleft palate simulator. Two fellows in pediatric plastic surgery and two expert cleft surgeons also performed recorded simulated repairs. A cleft palate objective structured assessment of technical skill (CLOSATS) scoring scale and end-product score were developed to assess performance. Two blinded cleft surgeons assessed the recordings and the final repairs using the CLOSATS, end-product scores and a previously developed global rating scale. Results: The average procedure specific (CLOSATS), global and end-product scores increased logarithmically after each successive simulation session for the residents. Reliability of the CLOSATS (average item ICC=0.85±0.093) and global ratings (average item ICC=0.91±0.02) amongst the raters was high. Reliability of the end-product assessments was lower (average item ICC=0.66±0.15). Standard setting linear regression using an overall cutoff score of 7 out of 10 corresponded to a pass score for the CLOSATS, and global rating score of 44 (maximum 60) and 23 (maximum 30) respectfully. Using logarithmic best fit curves, 6.3 simulation sessions are required to reach the minimum standard. Conclusions: A high fidelity cleft palate simulator has been developed that improves technical performance in cleft palate repair. The simulator and technical assessment scores can be used to determine performance before operating on patients. Financial disclosure: The cleft palate simulator is available commercially and sold by Simulare Medical Corp. (Toronto, Ontario, Canada). Drs. Podolsky, Fisher, Wong, Drake and Forrest are each shareholders of Simulare Medical Corp. Corresponding author: Dale J Podolsky, Division of Plastic and Reconstructive Surgery, The Hospital for Sick Children, 5430-555 University Avenue, Toronto, Ontario, M5G 1X8. E-mail: dale.podolsky@mail.utoronto.ca. ©2018American Society of Plastic Surgeons