Research Area
Clinical decisions and clinician learning
How do clinicians use information, learn from experience, and respond to AI advice? We study clinical reasoning, confidence, feedback, and the design of decision support and training.
Overview
As AI systems are increasingly deployed in clinical settings, rigorous evaluation methods are essential to ensure these tools genuinely improve patient outcomes. Our research focuses on developing and testing AI-assisted clinical decision support systems that complement—rather than replace—human expertise.
Key areas of investigation include:
- Clinician-in-the-loop AI: Understanding how clinicians interact with AI recommendations and designing interfaces that support appropriate trust calibration
- Soft ground truth methods: Developing statistical approaches for evaluating AI when gold-standard labels are uncertain
- Implementation science: Studying how workflow integration, training, and population factors affect AI effectiveness
- Equity evaluation: Ensuring clinical AI performs equitably across demographic groups
Active Projects
Local clinical knowledge for AI
With Gillings Innovation Lab support, we study how structured feedback from clinicians can improve AI-generated cases. Daktari-DX is the interface clinicians use to review those cases and contribute local clinical knowledge.
Active research. Gillings Innovation Lab (GIL) is the funder; Daktari-DX is the clinician-facing interface. No model has demonstrated clinical effectiveness. GIL, RLEF Data Generation, and Daktari-DX are related but not interchangeable names.
AIM AHEAD and AugMed
AugMed is a platform for running clinical case-review experiments. Our AIM-AHEAD colorectal cancer risk-assessment study examines how AI recommendations affect clinicians' judgments, screening recommendations, confidence, and attention to patient information.
Analysis. No proven improvement, sample sizes, effect estimates, publication status, or named health-system partner until a paper is out.
Clinical AI Evaluation Methods
Developing rigorous methodologies to assess AI-assisted clinical decision support systems, from algorithm performance to real-world clinical impact.
Clinician training in Ethiopia
Our developing research agenda examines how case-based practice, feedback, and information presentation affect clinicians' reasoning, confidence calibration, and learning. It is intended to inform CollectiveGood's Ethiopia training effort and future evaluations with local partners.
Developing. No executed training study, learning gains, accredited CME, public platform, or verified deployment. No platform link. Partner names come from the approved lab flyer; the MOU's terms are not described here.
Collective intelligence for curbside consultation
Our NIH NIMHD-funded Phase I STTR project develops and evaluates asynchronous peer consultation for rural clinicians. The research examines rapid skill assessment, selection of relevant reviewers, and aggregation of independent judgments, alongside feasibility and workflow fit. Clinical effectiveness remains to be established.
Phase I, newly awarded. Never state diagnostic superiority, completed clinical validation, or a broadly available clinical service.
Measuring HIV care quality in Vietnam with standardized patients
Unannounced standardized patients measure the quality of HIV care across providers in Vietnam, with a focus on how stigma shapes what patients receive, plus prescribing-pattern analysis from medication photos. Builds on R34MH121251 (MPI Sylvia and Smith, 2019 to 2022), which developed the standardized-patient stigma measures in China.
Active data collection. Not among the flyer's seven selected projects; retained here as substantiated funded work.
Related Publications
View all publicationsEpidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review
Adhikari, S. P., Meng, S., Wu, Yuju, Mao, Yu-Ping, Ye, R., Wang, Qingzhi, Sun, Chang, Sylvia, Sean Yuji, Rozelle, S., Raat, H., Zhou, Huan (2020)
Infectious Diseases of Poverty
DOI: 10.1186/s40249-020-00646-xViolence against health care workers in China, 2013–2016: evidence from the national judgment documents
Cai, Ruilie, Tang, Ji, Deng, Chenhui, Lv, Guofan, Xu, Xiaohe, Sylvia, Sean Yuji, Pan, Jay (2019)
Human Resources for Health
DOI: 10.1186/s12960-019-0440-yExplaining the declining utilization of village clinics in rural China over time: A decomposition approach.
Chen, Yunwei, Sylvia, Sean Yuji, Wu, P., Yi, Hongmei (2022)
Social Science & Medicine (1967)
DOI: 10.1016/j.socscimed.2022.114978Structural Determinants of Child Health in Rural China: The Challenge of Creating Health Equity
Chen, Yunwei, Sylvia, Sean Yuji, Dill, Sarah-Eve, Rozelle, S. (2022)
International Journal of Environmental Research and Public Health
DOI: 10.3390/ijerph192113845Primary Care Quality and Inappropriate Community Antibiotic Use: A Double Machine Learning Instrumental Variable Approach
Chen, Y., Yi, H., Rao, S., Weber, A., Hassmiller-Lich, K., Sylvia, Sean Yuji (2026)
medRxiv
DOI: 10.64898/2026.08.26.26361459Evaluation of a village-based digital health kiosks program: A protocol for a cluster randomized clinical trial
Cheng, Weibin, Zhang, Z., Hoelzer, Samantha, Tang, Weiming, Liang, Yizhi, Du, Yumeng, Xue, Hao, Zhou, Qiru, Yip, W., Ma, Xiaochen, Tian, Junzhang, Sylvia, Sean Yuji (2022)
Digital Health
DOI: 10.1177/20552076221129100Tracking US Health Care Spending by Health Condition and County.
Dieleman, J., Beauchamp, Meera, Crosby, Sawyer W, DeJarnatt, Drew, Johnson, Emily K, Lescinsky, H., McHugh, Theresa A., Pollock, Ian, Sahu, Maitreyi, Swart, Vivianne, Taylor, Kayla V, Thomson, A., Tsakalos, G., Weil, Maxwell, Wilner, Lauren B., Bui, Anthony L, Duber, Herbert C., Haakenstad, Annie, Idrisov, B., Mokdad, A., Naghavi, M., Roth, Gregory, Scott, John W., Templin, Tara, Murray, Christopher J L (2025)
Journal of the American Medical Association (JAMA)
DOI: 10.1001/jama.2024.26790Factors Influencing Adoption and Use of Telemedicine Services in Rural Areas of China: Mixed Methods Study
Du, Yumeng, Zhou, Qiru, Cheng, Weibin, Zhang, Z., Hoelzer, Samantha, Liang, Yizhi, Xue, Hao, Ma, Xiaochen, Sylvia, Sean Yuji, Tian, Junzhang, Tang, Weiming (2022)
JMIR Public Health and Surveillance
DOI: 10.2196/40771Tools & Outputs
Collaborate With Us
We're always looking for research partners, health system collaborators, and clinical sites to advance our work in clinical AI evaluation.
Get in Touch