UIS professor examines reliability of AI predictions for drug discovery

SPRINGFIELD, Ill. (Nov. 5, 2026) - A new study by University of Illinois Springfield computer science associate professor Yanhui Guo finds that improving average uncertainty estimates in AI models does not guarantee dependable estimates for unfamiliar molecules or better selection of drug-discovery candidates.

Guo, co-director of the Interdisciplinary Scientific AI Supercomputing Hub (ISAS), is the sole author of the open-access study in Computational Biology and Chemistry. The paper is available online ahead of its February 2027 issue date.

In early-stage drug discovery, machine-learning models help researchers prioritize molecules for laboratory testing. A prediction interval gives a range intended to contain a molecule’s measured activity. Guo examined whether those ranges remain reliable when models evaluate new chemical scaffolds, the core frameworks of molecules, rather than structures resembling their training data.

The benchmark used 20,327 target-specific compound records from ChEMBL 37, representing 19,972 unique molecules across six human protein targets. Guo tested two model types using 20 scaffold-disjoint training, calibration and test splits per target. He compared raw random-forest intervals with three conformal-prediction approaches, which adjust intervals using separate calibration data.

Intervals intended to capture 90% of measured activity values covered only 9.5% to 11.4% of test molecules when drawn directly from a random-forest ensemble. Conformal calibration generally brought average coverage back toward 90%, although results varied across targets, models and data splits.

Important gaps remained. Coverage averaged 82.3% among the quarter of molecules least similar to the training data and 78.7% among the quarter with the highest measured activity. Adapting intervals to chemical similarity did not consistently improve the balance between coverage and interval width.

The results show why overall coverage, reliability for challenging molecules and the usefulness of candidate-selection strategies need separate evaluation. Better uncertainty estimates on average did not automatically produce better molecular rankings.

The paper, “Scaffold-shift uncertainty calibration in molecular activity prediction: A multi-target benchmark of coverage, efficiency, and risk-aware selection,” is assigned to Volume 126, Part 1, Article 109371. Read the study online.