Exactech Publishes New Machine Learning Research That Evaluates Fairness and Accuracy of AI Predictions

The study shows that the company’s Predict+™ clinical decision support tool creates unbiased personalized predictions for shoulder patients from various socio-demographic groups.

GAINESVILLE, Fla. (November 27, 2023) – Exactech, a developer and producer of innovative implants, instrumentation, and smart technologies for joint replacement surgery, reports a new study¹ that demonstrates its machine learning clinical decision support tool, Predict+™, creates fair and accurate outcomes predictions for shoulder arthroplasty patients of different ethnicity, sex and age. The study, the first of its kind in orthopaedics, has just been published in the Journal of Shoulder and Elbow Surgery.

“This new machine learning study is important because it proposes a new statistical test methodology to evaluate the fairness of AI predictions,” said Chris Roche, Exactech’s Sr. Vice President, Extremities, “and because it establishes the clinically relevant criteria that determine acceptable differences in prediction accuracy between shoulder arthroplasty patients of different demographic status. Perhaps the biggest barrier to adoption of machine learning-based clinical decision support tools is the perception that they are not fair and accurate for all patients. This new clinical outcome study is the first of its kind in the orthopaedic literature to evaluate if an AI prediction is fair.”

To evaluate fairness, the study quantified the accuracy of Predict+ when predicting clinical outcomes for anatomic total shoulder arthroplasty and reverse total shoulder arthroplasty in more than 8,000 patients, stratified by multiple patient sub-groups of age, gender and ethnicity. Comparing differences in accuracy between patient groups, the authors were able to identify precisely which patient groups received fair predictions and which patient groups received unfair predictions. The study reported that Predict+ was deemed fair for 98.6% of regression predictions, 99.4% of substantial clinical benefit (SCB) classification predictions, and 100% of minimal clinically important difference (MCID) classification predictions.

“Bias in AI clinical predictions can adversely impact decision making,” said Vikas Kumar, Exactech’s Vice President of Machine Learning. “Biased or unfair predictions are more likely to occur in patient groups that are underrepresented in the training data. As a result, it is critical to train and evaluate machine learning prediction tools against diverse datasets that are representative of all potential patients. These results were only possible because of the large volume of high-quality clinical data Exactech has collected with the Equinoxe® shoulder system. Using our proposed machine learning testing methodology, we aim to provide the statistical determination of what makes a fair or unfair prediction for all machine learning clinical decision support tools.”

Predict+, part of Exactech’s Active Intelligence® portfolio of smart solutions, was released in November 2020 and was designed to better inform surgeons regarding the expected outcomes that can be achieved after shoulder arthroplasty, based on the clinical experience documented within the world’s largest single-shoulder prosthesis outcomes database, consisting of more than 15,000 patients.

Predict+ is available to surgeons worldwide on a limited basis. Please contact your local Exactech representative for support. For more information, visit www.exac.com/extremities/predict-plus/.

 

References:

1. Christine Allen, MS, Vikas Kumar, PhD, Josie Elwell, PhD, Steven Overman, MD, MPH, Bradley S. Schoch, MD, William Aibinder, MD, Moby Parsons, MD, Jonathan Watling, MD, Jiawei Kevin Ko, MD, Bruno Gobbato, MD, Thomas Throckmorton, MD, Howard Routman, DO, Christopher P. Roche, MSE, MBA. Evaluating the Fairness and Accuracy of Machine Learning Based Predictions of Clinical Outcomes after Anatomic and Reverse Total Shoulder Arthroplasty. J Shoulder Elbow Surg online. Sept. 2023. https://doi.org/10.1016/j.jse.2023.08.005.

About Exactech

Exactech is a global medical device company that develops and markets orthopaedic implant devices, related surgical instruments and the Active Intelligence^® platform of smart technologies to hospitals and physicians. Headquartered in Gainesville, Fla., Exactech markets its products in the United States, in addition to more than 30 markets in Europe, Latin America, Asia and the Pacific. Visit www.exac.com for more information and connect with us on LinkedIn, VuMedi, YouTube, Twitter and Instagram. With Exactech by your side, you’ve got EXACTLY what you need.