Physician turnover poses significant challenges to both patients and healthcare facilities, resulting in disruptions and financial burdens. However, a recent study conducted by Yale researchers has harnessed the power of machine learning to uncover key factors that contribute to this issue.
By analyzing data from a substantial U.S. healthcare system spanning almost three years, the researchers successfully predicted physician departures with an accuracy rate of 97%. This approach has the potential to reduce the negative impacts of physician turnover on both patients and healthcare facilities.
While healthcare facilities usually rely on surveys to monitor physician burnout and job satisfaction, the new study utilized data from electronic health records (EHRs).
Ted Melnick, associate professor of emergency medicine and co-senior author of the new study, states that the problem with surveys is that physicians often feel burdened to respond, resulting in low response rates. "And surveys can tell you what's happening at that moment," he added, "but not what's happening the next day, the next month, or over the following year."
EHRs, in addition to collecting clinical patient data, also generate valuable work-related data that provides a unique opportunity to observe physician behavioral patterns in real-time and over extended periods. Leveraging this advantage, the recent study aimed to analyze three years of de-identified EHR and physician data from a prominent New England health care system. The objective was to investigate whether a three-month stretch of data could accurately predict the likelihood of a physician's departure within the subsequent six months. By utilizing this extensive dataset, the researchers sought to uncover significant insights into the factors influencing physician turnover and enable proactive measures to mitigate such departures.
"We wanted something that would be useful on a personalized level," said Andrew Loza, a lecturer and clinical informatics fellow at Yale School of Medicine and co-senior author of the study, according to MedicalXpress. "So if someone were to use this approach, they could see the likelihood of departure for a position as well as the variables contributing most to the estimate in that moment, and intervene where possible."
Data was collected monthly from nearly 320 physicians across 26 medical specialties over a 34-month period. The collected data included:
Different data portions were used to train, validate, and test the machine-learning model.
Researchers further analyzed the sensitivity and specificity of the model, which measures the proportion of correctly classified departure and non-departure months, resulting in values of 64% and 79%, respectively. The model's capabilities extended beyond prediction, providing valuable insights into the factors influencing turnover risk identifying:
These findings demonstrate the model's robustness and potential for understanding and addressing physician turnover.
"There have been efforts to make machine learning models not black boxes wherein you get a prediction but it's not clear how the model came to it," said Loza. "Understanding why the model produced the prediction it did is particularly useful in this case as those details are going to identify issues that may be leading to physician departure."
Researchers identified several variables contributing to departure risk, the top four being:
The study showed that risk of departure was highest for physicians who were hired recently and those with longer tenures, but lower for those with middling tenure length. Additionally, those aged up to 44 years old were at higher risk of departure, with the risk lower for physicians aged 45 to 64, and higher again for those over the age of 65.
Researchers experienced a shift throughout the study period, taking place between 2018 to 2021, pointing to COVID-19 linking to increase in departure risk.
"As physician burnout is an increasingly recognized problem, health care systems, hospitals, and large groups need to figure out what they need to do to ensure the emotional and physical health and well-being of the physicians and other clinicians who do the actual caring for patients," said Robert McLean, New Haven Regional Medical Director of Northeast Medical Group. “Many health care systems already have wellness officers and wellness committees who could have the responsibility of collecting and analyzing this data and developing conclusions, which then would lead to implementation plans for changes and hopefully improvements."