Courtney Balentine, MD, MPH, is an associate professor in the department of surgery at the University of Wisconsin School of Medicine and Public Health and director of the Wisconsin Surgical Outcomes Research Program.

Courtney Balentine, MD, MPH

Bringing AI to the OR: Real-World Challenges And What’s Next, With Courtney Balentine, MD, MPH

As AI continues to gain traction across medicine, the operating room presents a uniquely challenging environment for meaningful implementation of this technology.

 provides novel insight into how emerging AI technology, specifically the OR Black Box, is being adopted in real-world surgical environments and why expectations often outpace reality.

By comparing how different hospitals deployed the tool, how clinicians perceived and interacted with it, and what structural or communication factors influenced success, the study uncovered substantial variability in implementation. It also identified a consistent gap between what surgical teams expected the AI to do and what it was actually able to deliver, driven by both technological limitations and challenges in conveying the system’s purpose and capabilities to frontline surgeons.

“[Large language models] are trained primarily on text data, so you give them reams and reams of books and papers and all sorts of fun stuff from the internet to learn from, and that is definitely not how the OR is structured,” study author Courtney Balentine, MD, MPH, an associate professor in the department of surgery at the University of Wisconsin School of Medicine and Public Health and director of the Wisconsin Surgical Outcomes Research Program, told 

. “It's not served up and ready to go. You have a whole bunch of different streams of information, you have multiple people in different roles who are talking to and across each other, communicating for different purposes at different times, and it's hard to parse all that out.”

According to Balentine, this complexity not only challenges the technology but also affects surgeon engagement. If clinicians cannot clearly see the value, he says adoption suffers.

His qualitative study was conducted at 3 large academic centers via semistructured interviews with surgeons and implementation leaders of the AI intervention to identify areas where expectations of the technology misaligned with their experiences. A total of 30 surgeons and 17 implementation leaders from 3 centers that implemented the AI intervention were interviewed.

Results showed most surgeons (57%) had neutral views of the technology, 37% expressed positive views, and 7% had negative views. Interviewees identified 4 major themes that highlighted misalignment between user expectations and the experience of using the technology:

The artificial intelligence model needed considerable additional training to be usable

Accessing data on surgical cases was difficult and time consuming

The program showed limited ability to predict postoperative complications

The program generated few academic deliverables

Balentine noted that the sites involved in the study varied widely in how they introduced the system, communicated its purpose, selected modules, and aligned the tool with local goals, which in turn impacted how well the technology was received and utilized.

Even at sites where leadership invested heavily in education and transparency, he said misunderstanding frequently persisted, with many frontline surgeons still struggling to grasp what the technology could realistically do versus what they assumed AI should be able to do. This disconnect highlights the need for more effective communication strategies and better expectation-setting around AI capabilities.

“Our take home [message] was that more thought needs to go into how to get the information out and break through that ‘It's AI, so it must do everything’ kind of perspective,” Balentine said.

Looking ahead, he expressed optimism about the potential of OR-based AI to improve outcomes beyond the operating room. Early gains, he said, are most achievable in areas like safety checklists, specimen handling, education, and audit-and-feedback processes.

The larger challenge, and ultimate goal, is linking intraoperative behaviors to downstream patient outcomes. While that requires significant case volume and cross-institutional data, Balentine described it as the next critical step in moving from process improvement to true outcome-driven surgical innovation.

Editors’ Note: Balentine reported grants from the US National Institute on Aging during the conduct of the study.

Thornton M, Cher BAY, Macdonald C, et al. Expectations vs Reality of an Intraoperative Artificial Intelligence Intervention. 

Peregin T. Black Box Technology Shines Light on Improving OR Safety, Efficiency. July 10, 2023. Accessed January 20, 2026. 

https://www.facs.org/for-medical-professionals/news-publications/news-and-articles/bulletin/2023/july-2023-volume-108-issue-7/black-box-technology-shines-light-on-improving-or-safety-efficiency/

Videos

Balentine explains the unique challenges of AI implementation in surgery and reviews his research on current barriers to real-world uptake.

Courtney Balentine, MD, MPH

Articles