Development and validation of Mpox healthcare seeking barriers scale for MSM based on a multicenter study in China (2025)

Monkeypox disease (Mpox), caused by the DNA mpox virus of the genus Orthopoxvirus, is transmitted through zoonotic and human-to-human pathways¹. Since May 2022, there has been a significant global increase in cases of pox diseases, prompting the World Health Organization (WHO) to declare the outbreak a global health emergency². In early 2024, a severe outbreak of Mpox occurred again in the African region, with the WHO reporting over 14,000 new cases and 511 deaths³. On August 14, 2024, the WHO once again declared the Mpox outbreak a public health emergency of international concern⁴. By August 2, 2024, Mpox cases have been reported in 116 countries, with 99,176 laboratory-confirmed cases⁵. China has reported 2,357 confirmed cases, placing it among the ten most affected countries worldwide⁵. According to data from the Chinese center for disease control and prevention, 32 out of 35 provincial-level administrative regions in China have reported confirmed Mpox cases. Figure 1 illustrates the geographical locations of these provinces and the number of confirmed cases.

The most prominent feature of the current Mpox outbreak is the high prevalence of infections among the men who have sex with men (MSM)⁶. According to the latest data from the WHO, 96.0% of reported cases globally in the past six months are among MSM⁵. Data released by the Chinese Center for Disease Control and Prevention indicates that 93.1% of all Mpox infections are among MSM⁷. The MSM population has been identified as a critical group in the prevention and control efforts of this Mpox outbreak.

In China, 94.1% of Mpox cases were identified through proactive medical consultations⁵. Seeking medical help promptly after being infected with the Mpox virus plays a positive role in controlling the epidemic. However, worries about Mpox medical visits may hinder patients from proactively seeking medical help. In addition to worry about medical safety, high healthcare costs, long waiting times, and the quality of care, MSM also worry about social stigmatization and discrimination resulting from the exposure of their sexual behavior^8,9,10. These worries may prompt MSM infected with Mpox to delay or cancel plans to seek medical help, thereby increasing the risk of transmitting the virus to others. Daisuke Mizushima and colleagues prospectively evaluated Mpox virus infections among MSM at the onset of the Mpox outbreak in Tokyo, Japan. They found that the prevalence of undetected or asymptomatic Mpox infections might be underestimated¹¹. Therefore, investigating the specific worries of MSM’s Mpox medical visit worries and taking measures to eliminate or mitigate these barriers to healthcare can help identify more patients and control the Mpox epidemic.

However, there are currently no studies reporting on Mpox medical visit worries among MSM, and the influencing factors remain unclear. This may result in public health campaigns and health education activities failing to effectively meet the needs of the MSM population, leading to suboptimal effectiveness of prevention and control measures.The lack of appropriate measurement tools may be a significant reason for this situation. Through the literature review we found that Lyu X et al. had reported the reliability and validity of the adult health care seeking behavior scale¹². Although the concepts measured by this scale are similar to our focus on medical visit worries, it lacks critical elements such as stigma, discrimination, and personal image. Moreover, the MSM population differs significantly from the general population in terms of sexual behavior patterns, health risks, psychosocial stress, healthcare utilization, and social support^13,14,15. These differences make it unsuitable for MSM to share measurement tools with the general population. Therefore, this study aims to develop and validate the MSM Mpox healthcare seeking barriers scale (MMHSBS), providing a foundational measurement tool for future research.

Sampling

This online anonymous cross-sectional survey was conducted from November 2023 to March 2024 via the QuestionStar platform in six regions: Shanghai, Guangzhou, Xinjiang, Shaanxi, Yunnan, and Shenyang.Due to the typically discreet nature of the MSM, we had specifically commissioned social non-governmental organizations (NGO) that had access to and served the MSM population to assist us in conducting this research.

The staff of NGO leveraged their established trust and resource networks within the MSM community to actively promote and mobilize participation in the study. Through various channels such as social media platforms, offline events, and face-to-face interactions, they disseminated detailed information about the research, thoroughly explaining its background, objectives, and potential societal significance, including raising awareness of MSM health issues and contributing to improved disease prevention and management strategies. They also addressed any questions from MSM, including those related to the participation process, privacy protection measures, and participants’ rights and responsibilities, ensuring a clear understanding of the study’s purpose. Based on this transparent communication, MSM individuals expressing interest were further screened to confirm they met the inclusion criteria.To fully protect the privacy of the subjects, the subjects in this study completed the survey using mobile devices such as phones on the QuestionStar platform.Informed consent was obtained, ensuring participants understood the study’s purpose, procedures, rights, and potential risks.

Data collection adhered to privacy regulations. The inclusion criteria are: (1) being male and at least 18 years old; (2) having engaged in male-to-male sexual behavior in the past six months; (3) primarily residing in one of the selected six provinces, municipalities, or autonomous regions over the past six months. The exclusion criteria are: (1) completing the questionnaire in less than 300s; (2) providing incorrect responses to quality control questions; (3) having an IP address outside the provinces or cities under investigation.All participants were required to complete the MMHSBS and respond to questions related to their knowledge of Mpox, demographic information, sexual behavior and sexual health, history of sexually transmitted disease diagnoses, and health behaviors.The survey is estimated to take approximately 20min to complete, and upon completion, each MSM participant was provided with a $7 remuneration as compensation for their time and effort.

Sample size

A total of 2481 questionnaires were collected in this survey, with 2403 being valid, resulting in a validity rate of 96.86%. The distribution of valid questionnaires was: 569 from Shanghai, 500 from Guangzhou, 320 from Xinjiang, 199 from Shaanxi, 313 from Yunnan, and 502 from Shenyang. The focus of this study was on the healthcare-seeking intentions of MSM who were unaware of their Mpox infection status. Therefore, it was necessary to exclude 56 participants who self-reported that they had already been diagnosed with the virus by a doctor. After exclusions, the sample sizes were: 546 from Shanghai, 490 from Guangzhou, 317 from Xinjiang, 193 from Shaanxi, 307 from Yunnan, and 494 from Shenyang, totaling 2347 questionnaires. Figure 2 illustrates the geographical locations of these provinces and municipalities in China.

Quality control

The design of the survey questionnaire incorporated advice and suggestions from multiple experts. After completing the initial draft, a pre-survey was conducted with 12 MSM participants. Based on the pre-survey results, modifications were made to the questionnaire, including adjustments in vocabulary and expressions to better suit the MSM population, enhancing the survey’s acceptability. For survey implementation, the study primarily relied on local NGO platforms, with NGO staff serving as researchers. Uniform training was provided to ensure these researchers understood the survey’s objectives, content, procedures, methods, and precautions. A data quality monitoring team was established to regularly check and provide feedback on the data within 48h during data collection. After data collection, steps for data cleaning and processing were undertaken, including checking for outliers, missing data, and logical errors, and making necessary corrections and adjustments. Participants were given a malpractice allowance for completing the questionnaire to increase compliance with the survey.

Development of the MMHSBS

Firstly, a comprehensive literature review and expert interviews were conducted to gather existing research findings and professional opinions, which helped identify the key factors the scale should cover. Subsequently, interviews with MSM individuals were performed to understand their perspectives on seeking healthcare after Mpox infection, as well as perceived barriers and worries, to refine the initial items. Based on the literature review, expert opinions, and MSM interviews, initial questionnaire items were generated and formatted using a 5-point Likert scale to facilitate quantitative analysis. Next, a pilot survey was conducted with a small sample of 30 MSM individuals to collect preliminary data and obtain feedback, assessing the clarity and applicability of the items, and identifying and correcting potential issues. Finally, a 14-item MMHSBS was developed. The specific items of the MMHSBS are presented in (Supplementary Table 1).

Statistical methods

Content validity

Content validity assesses how well the items of a scale align with and represent the intended construct for a specific assessment objective. The predominant method for evaluating content validity is the content validity index (CVI)¹⁶. The CVI is calculated both at the item level (I-CVI) and for the entire instrument (Ave-CVI).

The CVI is based on expert evaluations of each item’s relevance or representativeness, typically rated on a 4-point Likert scale from 1 (not relevant or representative) to 4 (highly relevant or representative). The I-CVI is determined by the proportion of experts who rate an item as 3 or 4, indicating a consensus on the item’s content validity¹⁷. An I-CVI of 0.8 or higher is considered acceptable.

The Ave-CVI represents the average proportion of items rated 3 or 4 by different experts, providing an overall measure of the instrument’s content validity^18,19. An Ave-CVI of at least 0.8 is required for acceptability. In this study, a total of 9 experts from the fields of public health, epidemiology, medical statistics, mental health, and health policy evaluated the content validity of the MMHSBS.

Structural validity test by factor analysis

We assessed item and scale reliability, performed exploratory factor analysis (EFA) to identify underlying relationships, and confirmatory factor analysis (CFA) to validate constructs. To ensure stability and avoid overfitting, we split the dataset (n = 2347) into two subsets: dataset A (n = 1173) for EFA and dataset B (n = 1174) for CFA.

Exploratory factor analysis

Bartlett’s test of sphericity and the Kaiser-Meyer-Olkin (KMO) measure assessed sample adequacy and data suitability for factor analysis. Bartlett’s test required p ≤ 0.05, and KMO needed to be > 0.7²⁰. The Kaiser criterion identified significant factors with eigenvalues > 1.0, but recent research advises comparing solutions with ± 1 factor^21,22. The Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) helped select the best model, with lower values indicating better fit. We used principal axis factoring with oblique rotation, retaining factors with loadings > 0.4²³. Dual-loading items are assigned to the most appropriate factor based on theoretical relevance and conceptual alignment, with retention in the higher-loading factor if the difference is significant.It is recommended to include a minimum of 3 items per factor²⁴.

Confirmatory factor analysis

CFA assessed the structural validity of constructs identified in the EFA. We reviewed literature and summarized 8 metrics for validation models. The Standardized Root Mean Square Residual (SRMR) and Root Mean Square Error of Approximation (RMSEA) are key measures, with SRMR ≤ 0.08 and RMSEA ≤ 0.08 indicating good fit^25,26. The Goodness-of-Fit Index (GFI) has a threshold of ≥ 0.90, measuring variance explained by the model. The Normed Fit Index (NFI) and Relative Fit Index (RFI) also require values ≥ 0.90^27,28. Incremental fit indices, including the Incremental Fit Index (IFI) and Tucker-Lewis Index (TLI), need thresholds of ≥ 0.90, and the Comparative Fit Index (CFI) requires ≥ 0.95 for acceptable fit^25,29.

Reliability

Split-half reliability, Cronbach’s α, and McDonald’s ω assess internal consistency of scales. Split-half reliability divides the scale into two halves, correlating scores to ensure each half measures the same construct, with ideal correlations > 0.70. Cronbach’s α evaluates all items collectively; α ≥ 0.9 indicates excellent consistency, 0.7 ≤ α < 0.9 is good, 0.6 ≤ α < 0.7 is acceptable, and α < 0.6 is poor. McDonald’s ω accounts for varying item loadings, with ω > 0.80 indicating good reliability.

Convergent validity

The average variance extracted (AVE) measures the average proportion of variance explained by the indicator variables for a construct. It is the mean of the squared factor loadings, indicating the explanatory power of each variable. An AVE of 0.50 or higher signifies that the construct’s variance is well captured by its indicators³⁰. Composite reliability (CR) assesses the internal consistency of the indicators, factoring in loadings and error variance. A CR of 0.70 or higher indicates high internal consistency and good convergent validity³¹.

The formulas for AVE and CR are as follows:

,

.

(n = number of items for each factor; i = constant = 1,2,3…; λ = factor loadings;δ = item error.)

In this study, a more conservative judgment condition was adopted, requiring both AVE ≥ 0.5 and CR ≥ 0.7 to be considered a factor with Convergent Validity.

All statistical analyses in this study were performed using IBM SPSS Statistics (Version 27), IBM SPSS Amos (Version 27), and R (version 4.3.3).

Demographic characteristics

In this study, a total of 2347 MSM were included. The average age was 30.66 years. 61.1% of the MSM were immigrants from other regions, and 78.5% had an education level of college or above. 85.1% were unmarried. The top three occupations were Freelance (28.6%), Business Services (19.5%), and Students (13.4%). Their monthly income was mainly concentrated in the ranges of 415–830 dollars (35.5%) and 830–1660 dollars (33.1%).

Nearly half of the MSM reported having sexual relations with at least two males within the past six months, and 16.0% reported having sexual relations with at least one female in the same period. Additionally, 36.0% had a habit of consuming alcohol before engaging in sexual activities, while 2.1% used substances prior to sexual intercourse. The self-reported prevalence of HIV, syphilis, gonorrhea, genital warts, genital herpes, and chlamydia trachomatis was 7.8, 3.5, 1.1, 2.6, 2.3, and 1.1%, respectively. There was no significant statistical difference in demographic characteristics between Dataset A and Dataset B, which were randomly assigned. Table1 shows the demographic characteristics of respondents.

Knowledge of Mpox among MSM

The MSM population demonstrates a good overall understanding of Mpox, particularly regarding the nature of the disease, primary transmission routes, and the preventive role of condoms. However, awareness needs improvement in areas such as the potential lethality of Mpox, its asymptomatic incubation period, diverse clinical manifestations, vaccine efficacy, and the ineffectiveness of antibiotics. Supplementary Table 2 shows the responses of MSM to questions related to Mpox.

Content validity

Among the experts involved in this MMHSBS content validity assessment, over 80% hold a PhD, and more than 70% have over 10 years of work experience. Detailed expert information is presented in (Supplementary Table 3), and the experts’ ratings for each MMHSBS item are displayed in (Supplementary Table 4).The I-CVI of each item is above 0.8, and the Ave-CVI for all items is 0.92.

Factor analysis

Exploring scale factor models using EFA

Bartlett’s test of sphericity yielded a p-value less than 0.001, and the KMO measure was 0.948, indicating that the data were suitable for EFA. Principal component analysis coupled with oblique rotation was employed for factor rotation. Factors with eigenvalues greater than 1 were retained. The analysis successfully extracted a two-factor model, accounting for 73.0% of the variance. Subsequently, by setting a fixed number of factors to be extracted, a one-factor model and a three-factor model were derived, with a percentage of variance of 61.1% and 77.7%, respectively. Table2 displays the KMO values for each item and the factor loadings across different factor models.

Comparison of different scale factor model fits using CFA

The model fitting parameters and judgment criteria are presented in (Table3). The one-factor model did not achieve acceptable levels on any fitting indices. The two-factor model met acceptable levels on 4 indices (NFI, RFI, IFI, and TLI), while the three-factor model satisfied 7 indices (GFI, NFI, RFI, IFI, TLI, CFI, and SRMR), indicating that the three-factor model fitted better. Additionally, the AIC and BIC for the three-factor model are the lowest among the three models. Therefore, the three-factor model is clearly the best factor structure for MMHSBS, with the three factors named “medical distrust”, “perceived costs”, and “interpersonal damage”.

Based on the modification indices generated in the factor analysis, the three-factor model was revised and optimized. The RMSEA of the revised three-factor model was reduced to an acceptable level, resulting in a total of 8 qualified indicators.The factor structures of all models are shown in (Supplementary Figs. 1–4).

Reliability of the modified three-factor model

Reliability analysis was conducted on the entire dataset (n = 2347). The Cronbach’s α, McDonald’s ω, and Guttman Split-Half Coefficient for all items were 0.949, 0.950, and 0.844, respectively. Table4 lists the scores for each item, the grouping of items used to calculate split-half reliability, and the Cronbach’s α and McDonald’s ω after removing each item.

Convergent validity of the modified three-factor model

The CR for Factor 1 is 0.906, with an AVE of 0.659. Factor 2 has a CR of 0.890 and an AVE of 0.668. Lastly, Factor 3 demonstrates a CR of 0.935 and an AVE of 0.743.All factors have AVE values greater than 0.5 and CR values greater than 0.7.

Scores of MMHSBS in Chinese MSM

In this study, the mean MMHSBS score among 2,347 MSM from six cities in China was 49.04 ± 12.38. Among the 14 items, item 12 had the highest average score, while item 2 had the lowest.

This study developed a scale to measure Mpox medical visit worries among MSM and validated it using a sample of 2347 MSM from six cities in China. The MMHSBS consists of 14 items and uses a 5-point Likert scale, with scores ranging from 14 to 70. Higher scores indicate higher levels of barriers to seeking health care.

According to the content validity scores given by 9 experts, 14 items in the MMHSBS are highly relevant to MSM’s worries about seeking medical care for Mpox. In our exploration of the MMHSBS factor structure, we validated and compared one-factor, two-factor, and three-factor models. The results of the CFA indicated that the three-factor model performed the best. Factor 1, Factor 2, and Factor 3 comprise 5, 4, and 5 items, respectively. Factor 1 describes worries about uncertainties in the diagnosis and treatment of Mpox, the technical level, drug safety, rationality of treatment, and side effects, and is named “Medical distrust”. Factor 2 describes worries about discrimination, costs, impacts on work, and loss of income during the healthcare-seeking process, and is named “Perceived costs”. Factor 3 describes worries about family and friend relationships, personal image, social discrimination, and privacy issues, encompassing various potential impacts on social interactions and personal privacy, and is named “Interpersonal damage”.

The MMHSBS exhibited high consistency, with overall Cronbach’s α and McDonald’s ω values of 0.949 and 0.950, respectively, indicating very high internal consistency. Additionally, the changes in these reliability coefficients were minimal when any single item was removed, suggesting that no individual item significantly impacted the overall reliability. This implies that while each item in the scale is relatively independent, they collectively maintain consistency. Although the Guttman Split-Half Coefficient was 0.844, which is lower than Cronbach’s α and McDonald’s ω, it still indicates good internal consistency. The relatively lower value of this coefficient may be due to the introduction of more random error when the items are randomly split into two halves³².

All three factors meet the criteria for convergent validity with AVE values greater than 0.5 and CR values greater than 0.7. This indicates that each factor reliably measures the construct it is intended to measure and that the observed variables are highly representative of their respective underlying factors³³. The strong CR and AVE values provide robust evidence that the factors have good internal consistency and that the 3 factors are well-defined and distinct.

In structural equation modeling or CFA, residual correlations between items indicate that, beyond the common variance explained by the latent construct, certain specific information is shared among the items³⁴. Brown provides a detailed discussion on the meaning and potential sources of residual terms in CFA in his book. He notes that, in addition to the shared variance explained by the latent construct, residual correlations between items may arise due to the presence of specific shared information³⁵. This explanation aligns with the observed residual correlations among Items 9, 10, and 11.Specifically, the modification indices suggest a strong residual correlation between item 9 and item 10, primarily because both items pertain to the financial stress imposed on the family due to health-related issues. Item 9 emphasizes financial income loss, whereas item 10 reflects the additional burden placed on the family due to one’s health condition. Thus, aside from capturing general health-related concerns, these two items share specific information related to familial responsibility and economic pressure.Similarly, the residual correlation between item 10 and item 11 can be attributed to their shared reference to negative interpersonal experiences. While item 10 focuses on the burden within the family, item 11 highlights the fear of social distancing within one’s peer network. Both items reflect concerns about potential social rejection or estrangement resulting from health issues, thereby manifesting a shared emotional burden beyond the underlying latent construct. This content-specific overlap between items accounts for the observed residual correlations.It should be noted that this study did not examine the associations between the instrument and other related constructs (e.g., medical mistrust, social support, and financial vulnerability). Future research should focus on investigating the relationships between the instrument and these external variables, as well as assessing its predictive validity for public health-related behaviors.

We found that item 2 (“I am worried about the technical competence of healthcare workers”) had a low score, indicating that Chinese MSM have relatively high confidence in the technical competence of healthcare workers. In contrast, concern about the impact of Mpox on personal image was evident, with item 12 (“I worry that Mpox will affect my personal image”) scoring the highest. These findings suggest that the Chinese MSM population believes that the existing healthcare system can provide effective treatment after they are infected with Mpox. However, worries about medical visits, represented by personal image worry, are troubling them and may even be a huge obstacle to their request for medical help. Public health interventions should focus on mitigating stigma, ensuring confidential and respectful care, and enhancing public awareness to address these worries.

Author notes

1. Shangbin Liu and Ying Gao contributed equally to this work.

Authors and Affiliations

1. Public Health Department, Tongren Hospital, Hongqiao International Institute of Medicine, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China

   Shangbin Liu,Ying Gao,Ying Wang&Yong Cai

2. School of Public Health, Shanghai Jiao Tong University, Shanghai, 200025, China

   Huifang Xu&Gang Xu

3. Dermatology Department, Tongren Hospital, Shanghai Jiao Tong University School of Medicine Shanghai, Shanghai, 200025, China

   Jiechen Zhang

Contributions

Y.C. and J.Z. designed the study; S.L., Y.G. and G.X. designed and completed the questionnaire production; H.X., S.L. and Y.W. created the research database and entered research data; S.L. and Y.G. wrote the first draft of the article; Y.C. and J.Z. have revised the first draft into the final draft.

Corresponding authors

Correspondence to Yong Cai or Jiechen Zhang.

Competing interests

The authors declare no competing interests.

Ethical approval

This study was conducted in accordance with the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of Shanghai University of Medicine & Health Sciences (Approval Number: 2023-MSMMPOX-22-310222197604080237).

Informed consent

Informed consent was obtained from all subjects participating in the study.

Consent for publication

All authors have given their consent for the publication of this research.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions