All Research

Research Area

Clinical decisions and clinician learning

How do clinicians use information, learn from experience, and respond to AI advice? We study clinical reasoning, confidence, feedback, and the design of decision support and training.

Overview

As AI systems are increasingly deployed in clinical settings, rigorous evaluation methods are essential to ensure these tools genuinely improve patient outcomes. Our research focuses on developing and testing AI-assisted clinical decision support systems that complement—rather than replace—human expertise.

Key areas of investigation include:

  • Clinician-in-the-loop AI: Understanding how clinicians interact with AI recommendations and designing interfaces that support appropriate trust calibration
  • Soft ground truth methods: Developing statistical approaches for evaluating AI when gold-standard labels are uncertain
  • Implementation science: Studying how workflow integration, training, and population factors affect AI effectiveness
  • Equity evaluation: Ensuring clinical AI performs equitably across demographic groups

Active Projects

Active

Local clinical knowledge for AI

With Gillings Innovation Lab support, we study how structured feedback from clinicians can improve AI-generated cases. Daktari-DX is the interface clinicians use to review those cases and contribute local clinical knowledge.

Active research. Gillings Innovation Lab (GIL) is the funder; Daktari-DX is the clinician-facing interface. No model has demonstrated clinical effectiveness. GIL, RLEF Data Generation, and Daktari-DX are related but not interchangeable names.

Funding: Gillings Innovation Lab (award ended 30 June 2026; research continues)
View project
Analysis

AIM AHEAD and AugMed

AugMed is a platform for running clinical case-review experiments. Our AIM-AHEAD colorectal cancer risk-assessment study examines how AI recommendations affect clinicians' judgments, screening recommendations, confidence, and attention to patient information.

Analysis. No proven improvement, sample sizes, effect estimates, publication status, or named health-system partner until a paper is out.

Funding: AIM-AHEAD (support ended September 2025; analysis and manuscripts continue)
View project
Active Featured

Clinical AI Evaluation Methods

Developing rigorous methodologies to assess AI-assisted clinical decision support systems, from algorithm performance to real-world clinical impact.

Funding: NIH R01, AHRQ
View project
Developing

Clinician training in Ethiopia

Our developing research agenda examines how case-based practice, feedback, and information presentation affect clinicians' reasoning, confidence calibration, and learning. It is intended to inform CollectiveGood's Ethiopia training effort and future evaluations with local partners.

Developing. No executed training study, learning gains, accredited CME, public platform, or verified deployment. No platform link. Partner names come from the approved lab flyer; the MOU's terms are not described here.

View project
Active Featured

Collective intelligence for curbside consultation

Our NIH NIMHD-funded Phase I STTR project develops and evaluates asynchronous peer consultation for rural clinicians. The research examines rapid skill assessment, selection of relevant reviewers, and aggregation of independent judgments, alongside feasibility and workflow fit. Clinical effectiveness remains to be established.

Phase I, newly awarded. Never state diagnostic superiority, completed clinical validation, or a broadly available clinical service.

Funding: NIH NIMHD STTR Phase I (1R41MD021420-01; CollectiveGood prime, UNC subaward, Sean Sylvia subaward PI)
View project
Active

Measuring HIV care quality in Vietnam with standardized patients

Unannounced standardized patients measure the quality of HIV care across providers in Vietnam, with a focus on how stigma shapes what patients receive, plus prescribing-pattern analysis from medication photos. Builds on R34MH121251 (MPI Sylvia and Smith, 2019 to 2022), which developed the standardized-patient stigma measures in China.

Active data collection. Not among the flyer's seven selected projects; retained here as substantiated funded work.

Funding: NIH/NIMH R01 (PI Kumi Smith, University of Minnesota; Co-I Sean Sylvia)
View project

Related Publications

View all publications

Epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review

Adhikari, S. P., Meng, S., Wu, Yuju, Mao, Yu-Ping, Ye, R., Wang, Qingzhi, Sun, Chang, Sylvia, Sean Yuji, Rozelle, S., Raat, H., Zhou, Huan (2020)

Infectious Diseases of Poverty

DOI: 10.1186/s40249-020-00646-x

Violence against health care workers in China, 2013–2016: evidence from the national judgment documents

Cai, Ruilie, Tang, Ji, Deng, Chenhui, Lv, Guofan, Xu, Xiaohe, Sylvia, Sean Yuji, Pan, Jay (2019)

Human Resources for Health

DOI: 10.1186/s12960-019-0440-y

Explaining the declining utilization of village clinics in rural China over time: A decomposition approach.

Chen, Yunwei, Sylvia, Sean Yuji, Wu, P., Yi, Hongmei (2022)

Social Science & Medicine (1967)

DOI: 10.1016/j.socscimed.2022.114978

Structural Determinants of Child Health in Rural China: The Challenge of Creating Health Equity

Chen, Yunwei, Sylvia, Sean Yuji, Dill, Sarah-Eve, Rozelle, S. (2022)

International Journal of Environmental Research and Public Health

DOI: 10.3390/ijerph192113845

Primary Care Quality and Inappropriate Community Antibiotic Use: A Double Machine Learning Instrumental Variable Approach

Chen, Y., Yi, H., Rao, S., Weber, A., Hassmiller-Lich, K., Sylvia, Sean Yuji (2026)

medRxiv

DOI: 10.64898/2026.08.26.26361459

Evaluation of a village-based digital health kiosks program: A protocol for a cluster randomized clinical trial

Cheng, Weibin, Zhang, Z., Hoelzer, Samantha, Tang, Weiming, Liang, Yizhi, Du, Yumeng, Xue, Hao, Zhou, Qiru, Yip, W., Ma, Xiaochen, Tian, Junzhang, Sylvia, Sean Yuji (2022)

Digital Health

DOI: 10.1177/20552076221129100

Tracking US Health Care Spending by Health Condition and County.

Dieleman, J., Beauchamp, Meera, Crosby, Sawyer W, DeJarnatt, Drew, Johnson, Emily K, Lescinsky, H., McHugh, Theresa A., Pollock, Ian, Sahu, Maitreyi, Swart, Vivianne, Taylor, Kayla V, Thomson, A., Tsakalos, G., Weil, Maxwell, Wilner, Lauren B., Bui, Anthony L, Duber, Herbert C., Haakenstad, Annie, Idrisov, B., Mokdad, A., Naghavi, M., Roth, Gregory, Scott, John W., Templin, Tara, Murray, Christopher J L (2025)

Journal of the American Medical Association (JAMA)

DOI: 10.1001/jama.2024.26790

Factors Influencing Adoption and Use of Telemedicine Services in Rural Areas of China: Mixed Methods Study

Du, Yumeng, Zhou, Qiru, Cheng, Weibin, Zhang, Z., Hoelzer, Samantha, Liang, Yizhi, Xue, Hao, Ma, Xiaochen, Sylvia, Sean Yuji, Tian, Junzhang, Tang, Weiming (2022)

JMIR Public Health and Surveillance

DOI: 10.2196/40771

Collaborate With Us

We're always looking for research partners, health system collaborators, and clinical sites to advance our work in clinical AI evaluation.

Get in Touch