Unique patient complaints study yields novel taxonomy, surprising outcomes
Joint study with Royal College of Dental Surgeons of Ontario and Faculty of Dentistry thought to be the largest of its kind
A ground-breaking study of complaints made to the Royal College of Dental Surgeons of Ontario over an 11-year period is yielding new insights into perceptions surrounding oral health care professionals and their standard of care.
“This is a unique study of its kind internationally,” says Carlos Quiñonez, associate professor at the Faculty of Dentistry and lead researcher of the study, which looked at complaints made to the RCDSO — the regulatory and licensing body governing dentists and their standards of practice — between 2007 and 2017.
Over the course of the nearly two-year study, the research team examined 2,199 letters, drawn from a total of 4,627 total complaints made to the College — making it what is thought to be the largest longitudinal study of patient complaints to date.
“People have conducted similar studies in hospital and other healthcare settings, using questionnaires and patient satisfaction surveys, or case files of regulatory findings once the regulator has weighed in. But no one has taken this volume of complaints to a regulator and quantified them,” adds Quiñonez, who is also the graduate specialty program director in dental public health.
Turning complaints into data
“One’s experience when receiving care is so subjective, so how that translates into the letters themselves is very unique,” explains research associate Monika Roerig, a lead organizer of the study. To overcome that challenge, the researchers piloted a framework that turned qualitative complaints into quantitative data. With a primary batch of 100 letters, the team was able to code individual words or expressions like “examination,” or “pain.”
"No one has taken this volume of complaints to a regulator and quantified them"
Those complaint codes were then organized into domains: clinical care and treatment; management and access to care; and relationships and conduct, with those broken down further into subdomains. The remaining 2,099 complaints, chosen randomly by RCDSO staff involved in the project, were subjected to the same taxonomic coding.
Overall, the team counted a total of 17,752 complaint codes in the analysis process.
A rigorous methodology was applied: the coding framework was revised, tested, and revised again during the initial pilot study, and continually fine-tuned as the study went on, so the team could ensure the coding was as bias-free as possible and asserted the complainants’ perspective. Meeting weekly for a year, the team would often recode the letters, “so we could ensure the rater’s reliability and make sure we were all on the same page,” says Quiñonez.
According to the findings published on the RCDSO’s website, 59 per cent of all complaints made to the RCDSO related to clinical care and treatment; 56 per cent involved interpersonal relationships and conduct, such as how the patient was treated, and issues around informed consent. The lowest volume of complaints — 42 per cent — involved management and access to care. (As there were often more than one issue in a single complaint, the numbers do not add up to 100 per cent).
The complaint analysis allowed the team to observe trends over time, says Julie Farmer, a PhD student at the Faculty of Dentistry who collaborated on the project. Remarkably, while there was a small increase in patient letters over the past few years, potentially contributed to by the development of an online complaint submission process, “It seemed like the number of complaints per category didn’t change. They stayed consistent,” says Farmer.
And while the majority of complaints originated with patients, samples had also been collected from family members, dentists, hygienists, nurses, pharmacists and more. That very universality made the outcome more compelling, say the researchers.
“I think this study really revealed some valuable information from all members of the public,” adds Roerig, who is now a research coordinator at the Institute of Health Policy, Management and Evaluation at the Dalla Lana School of Public Health. “The study shows us that it’s important to have excellent clinical care and safety, but communication and interpersonal skills were extremely important. The two work hand in hand.”
Farmer agrees. “One of the big things was how a large percentage of the complaints were about interaction and communication.”
According to Laura Dempster, associate professor at the Faculty of Dentistry and a key investigator on the study, the findings support what dental educators already know: namely, that, “communication is a core skill integral to clinical care, and communication competence is necessary for all health care providers.”
Now that data can be put to work improving patients’ experiences in the dental office. “The results have implications for the regulator,” says Quiñonez, and education is a key component of that. “It’s really important from an educational perspective. I don’t think we do enough on communication, on relational competency,” he adds. “We can do better from the point of view of social competence in dentistry.”
The study could influence practice standards or trends in continuing education, where content can be tailored to fill those gaps. Roerig cautions, though, that any changes to regulations or curricula cannot rest on the results of the study alone. “I think it’s a tool they can use in conjunction with other research,” she adds.
Still, it’s a very powerful tool. In fact, after giving a presentation on the study at the International Association for Dental Research annual meeting, the team was approached by a group in the U.K. interested in adapting the taxonomy for a similar health study.
“It has been an honour for us to share our findings not only in Canada and the U.S., but in Europe as well,” said RCDSO representatives Krystal Evans and Irwin Fefergrad, Registrar, in a joint statement. “There is huge excitement amongst other regulators wanting to know how they can mine and learn from their own data as we have from ours.”
As well as sharing the methodology of this study with international partners, the RCDSO has committed to investing in data mining that can complement and further the success of this study.
Photo: Associate professor Carlos Quiñonez, courtesy Jeff Comber