Introduction Valid and low-cost quality assessment tools examining care quality are not readily available. The unannounced standardised patient (USP), the gold standard for assessing quality, is costly to implement while the validity of clinical vignettes, as a low-cost alternative, has been challenged. Computerised virtual patients (VPs) create high-fidelity and interactive simulations of doctor-patient encounters which can be easily implemented via smartphone at low marginal cost. Our study aims to develop and validate smartphone-based VP as a quality assessment tool for primary care, compared with USP. Methods and analysis The study will be implemented in primary health centres (PHCs) in rural areas of seven Chinese provinces, and physicians practicing at township health centres and village clinics will be our study population. The development of VPs involves three steps: (1) identifying 10 VP cases that can best represent rural PHCs’ work, (2) designing each case by a case-specific development team and (3) developing corresponding quality scoring criteria. After being externally reviewed for content validity, these VP cases will be implemented on a smartphone-based platform and will be tested for feasibility and face validity. This smartphone-based VP tool will then be validated for its criterion validity against USP and its reliability (ie, internal consistency and stability), with 1260 VP/USP-clinician encounters across the seven study provinces for all 10 VP cases. Ethics and dissemination Sun Yat-sen University: No. 2017-007. Study findings will be published and tools developed will be freely available to low-income and middle-income countries for research purposes.