arthroscopic sweep

Training better surgeons

Oral and maxillofacial surgery sets new bar with first-ever technical skills exam

The skill level of the surgeon makes a tremendous difference in the successful outcome of surgeries. Yet until now, an objective, effective way to test a graduating surgeon’s skills has not existed for oral and maxillofacial surgery (OMFS) — or for dentistry at all.

That’s all about to change as a collaboration, led by Marco Caminiti, assistant professor and graduate program director of OMFS at the Faculty of Dentistry, launched the first-ever OSATS (objective structured assessment of technical skills) exam for residents of the OMFS specialty, with the aim, says exam co-designer Victoria Driesman, of ensuring that graduating surgeons can “perform surgery well.”

With the OSATS, educators now have a tool to analyze the competency of key surgical skills and develop standardized measures to track students’ progress through their training.

For its inaugural run, the designers asked surgeons and trainees with varying skill levels and experience to take part in the exam, from a first year resident with no training all the way up to a graduated, expert surgeon. That’s to demonstrate that the exam can differentiate between the different skill levels, and will help the designers fine-tune future iterations. Rotating through eight stations in short, 15-minute increments, participants performed specific surgical tasks — not communicated in advance — on models meticulously chosen for their life-like properties, and were graded on their proficiency through each step.

At one station, a jaw model selected for its likeness to human bone sat in a vise. With just 12 minutes to perform the procedure (three minutes were allotted to reading over instructions), participants were to drill holes in a jaw in preparation for repairing a fracture.

plate bending and wiring
Plate bending and wiring, courtesy Marco Caminiti

 

At another station, the task was to insert a probe into a remarkably lifelike silicon head — once the instrument was inserted, the model would actually bleed — and perform a “sweep” of the temporomandibular joint, which was simultaneously imaged on a computer screen. Other skills tested included: performing a tracheostomy, tooth implant placement, suturing, and skin harvesting, among others.

The new testing platform should eventually effect change on an institutional level, too. “It’s an extremely valuable tool to understand where there might exist weaknesses in the programs,” says Howard Tenenbaum, professor at the Faculty of Dentistry and chief of dentistry at Mount Sinai Hospital, which then can be used to shape and strengthen different skill modules within the curriculum.

And while the concept of the OSATS is not new — it was designed by professors at the University of Toronto’s Department of Surgery in 1998 — it has not been universally adopted. It’s become a staple for some medical specialties, though, says Caminiti, who is also head of OMFS at Mount Sinai and Humber River Hospitals, “and the trend is increasing.” Having never before been applied to dentistry or OMFS, the exam’s design team hope to see the OSATS widely embraced by this branch of health education.

 

tracheostomy staton
Surgeon performs at the trachostomy station, courtesy Marco Caminiti

After her work in setting up similar exams in general and colorectal surgery for Canadian and U.S. schools, Driesman, who is heading into her second year of medical school at the University of Rochester, was recruited to help develop the UofT exam. She says one important reason that the OSATS is gaining traction is due to the development of high quality, biomimetic examination models. “We now have the tools to do it,” she says.  

The team, which included members from UofT Dentistry, Mount Sinai Hospital, Sunnybrook Hospital and Humber River Hospital, will present the results of the inaugural exam before the Canadian Academy of Oral and Maxillofacial Surgeons in the spring of 2019. The goal, says Caminiti, is to have the test adopted as an annual assessment tool for OMFS residents across Canada, and could even be used to screen candidates applying to the specialty field.

 

OSATS designers

 

Top photo: head model for arthroscopic probe;  bottom: exam designers Drs. Justin Garbedian, Karl Cuddy, Nick Blanas, Lesley David, Daryn Bikey, Marco Caminiti, Vanessa Mendes, Eric Ebrahimi, courtesy Marco Caminiti