Managing uncertainty in medical decision making

Can clinicians' use of AT be increased and improved via effective communication of algorithmic uncertainty?

My proposed research investigates this research question by understanding how the communication of uncertainty in AT-based diagnostic tools affects clinicians' decision-making.

Extending upon previous findings on decision-making, the project aims to understand whether communication of algorithmic uncertainty affects decision confidence, advice-seeking behaviour, and the willingness of clinicians to integrate advice into their decisions.

The research will focus on an AT tool developed by Professor Charalambos Antoniades, which offers clinicians predictive data on cardiac risks from CT scans.

This has recently been piloted at the Oxford University Hospital, and provides clinicians with estimates of risk for cardiac events based on results of routine CT scans (Oxford University, 2023).

This project will employ a mixed-methods approach and consist of two phases: an intervention development phase, and a proof-of-concept phase.

Firstly, a systematic review to map the current landscape of uncertainty measurement in AT diagnostic tools will be conducted.

This review will assess the types of uncertainty measures utilized, their impact on clinical decisions, and identify existing gaps.

Following this, semi-structured interviews with clinicians will be conducted to gain a deeper understanding of their first-hand experiences and preferences regarding AT tools and uncertainty communication. The thematic analysis of these interviews will inform the development of uncertainty measures for this AT-tool.

For the experimental phase, a Judge-Advisor System (JAS) method will be used empirically test how the developed uncertainty measure affects clinical decisions (Sniezek & Buckley, 1995).

Clinicians will be presented with simulated case scenarios developed in collaboration with OxSTaR and AT-generated risk estimates, both with and without uncertainty quantification.

Their initial risk assessments and decision-decision confidence for treatment will be recorded, followed by an opportunity to revise their decisions post-advice, closely imitating real-life clinical settings.

By testing psychological mechanisms underpinning decision-making, the ability of effective communication of algorithmic uncertainty to increase or improve will be evaluated. This research project has both practical and theoretical impact.

While providing evidence that is highly practical and ready to be applied within the context of this specific AT tool, how it impacts decision-making will also form a more theoretical understanding that can be applied to other settings.

As AT-based tools are increasingly applied to assist clinicians in diagnostics in hospital settings around the world, this is an important area of inquiry.