Original Research

A Comparison of 4 Single-Question Measures of Patient Satisfaction

Journal of Clinical Outcomes Management. 2020 January;27(1)

References

1. Waters S, Edmondston SJ, Yates PJ, Gucciardi DF. Identification of factors influencing patient satisfaction with orthopaedic outpatient clinic consultation: A qualitative study. Man Ther. 2016;25:48-55.

2. Kortlever JTP, Ottenhoff JSE, Vagner GA, et al. Visit duration does not correlate with perceived physician empathy. J Bone Joint Surg Am. 2019;101:296-301.

3. Edwards P, Roberts I, Clarke M, et al. Methods to influence response to postal questionnaires. Cochrane Database Syst Rev. 2001(3):CD003227.

4. Salisbury C, Burgess A, Lattimer V, et al. Developing a standard short questionnaire for the assessment of patient satisfaction with out-of-hours primary care. Fam Pract. 2005;22:560-569.

5. Ross CK, Steward CA, Sinacore JM. A comparative study of seven measures of patient satisfaction. Med Care. 1995;33:392-406.

6. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377-381.

7. Medicine USNLo. ClinicalTrials.gov. Accessed March 18, 2019.

8. Nicholas MK, McGuire BE, Asghari A. A 2-item short form of the Pain Self-efficacy Questionnaire: development and psychometric evaluation of PSEQ-2. J Pain. 2015;16:153-163.

9. Salkovskis PM, Rimes KA, Warwick H, Clark D. The Health Anxiety Inventory: development and validation of scales for the measurement of health anxiety and hypochondriasis. Psychol Med. 2002;32:843-853.

10. Schalet BD, Pilkonis PA, Yu L, et al. Clinical validity of PROMIS depression, anxiety, and anger across diverse clinical samples. J Clin Epidemiol. 2016;73:119-127.

11. Ho AD, Yu CC. Descriptive statistics for modern test score distributions: skewness, kurtosis, discreteness, and ceiling effects. Educ Psychol Meas. 2015;75:365-388.

12. Kim HY. Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor Dent Endod. 2013;38:52-54.

13. NICE Satmetrix. What is net promoter? https://www.netpromoter.com/know/. Accessed March 18, 2019.

14. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34-42.

15. Hasson D, Arnetz BB. Validation and findings comparing VAS vs. Likert scales for psychosocial measurements. Int Electronic J Health Educ. 2005;8:178-192.

16. Vickers AJ. Comparison of an ordinal and a continuous outcome measure of muscle soreness. Int J Technol Assess Health Care. 1999;15:709-716.

17. Jaeschke R, Singer J, Guyatt GH. A comparison of seven-point and visual analogue scales: data from a randomized trial. Control Clin Trials. 1990;11:43-51.

18. Voutilainen A, Pitkaaho T, Kvist T, Vehvilainen-Julkunen K. How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric Likert scale. J Adv Nurs. 2016;72:946-957.

19. Brunelli C, Zecca E, Martini C, et al. Comparison of numerical and verbal rating scales to measure pain exacerbations in patients with chronic cancer pain. Health Qual Life Outcomes. 2010;8:42.

20. Hageman MG, Briet JP, Bossen JK, et al. Do previsit expectations correlate with satisfaction of new patients presenting for evaluation with an orthopaedic surgical practice? Clin Orthop Relat Res. 2015;473:716-721.

21. Keulen MHF, Teunis T, Vagner GA, et al. The effect of the content of patient-reported outcome measures on patient perceived empathy and satisfaction: a randomized controlled trial. J Hand Surg Am. 2018;43:1141.e1-e9.

22. Mellema JJ, O’Connor CM, Overbeek CL, et al. The effect of feedback regarding coping strategies and illness behavior on hand surgery patient satisfaction and communication: a randomized controlled trial. Hand. 2015;10:503-511.

23. Tyser AR, Gaffney CJ, Zhang C, Presson AP. The association of patient satisfaction with pain, anxiety, and self-reported physical function. J Bone Joint Surg Am. 2018;100:1811-1818.

24. Vranceanu AM, Ring D. Factors associated with patient satisfaction. J Hand Surg Am. 2011;36:1504-1508.

25. Stirling P, Jenkins PJ, Clement ND, et al. The Net Promoter Scores with Friends and Family Test after four hand surgery procedures. J Hand Surg Eur. 2019;44:290-295.

26. Wilberforce M, Poll S, Langham H, et al. Measuring the patient experience in community mental health services for older people: A study of the Net Promoter Score using the Friends and Family Test in England. Int J Geriatr Psychiatry. 2019;34:31-37.

Methods

Study Design

All English-speaking new and return patients ages 18 to 89 years visiting an orthopedic surgeon in 1 of 7 clinics located in a large urban area were considered eligible for this study. Enrollment took place intermittently over a 5-month period. We were granted a waiver of written informed consent. Patients indicated their consent by completing the surveys. Patients were randomly assigned to 1 of the 4 questionnaires containing different scale types using an Excel random-number generator. After the visit, patients were asked to complete the survey. All questionnaires were administered on an encrypted tablet via a HIPAA-compliant, secure web-based application for building and managing online surveys and databases (REDCap; Research Electronic Data Capture).⁶This study was approved by our Institutional Review Board and is registered on ClinicalTrials.gov (NCT03686735).⁷

Outcome Measures

Study participants were asked to complete questionnaires regarding demographics (sex, age, race/ethnicity, marital status, level of education, work status, insurance status, comorbidities) and to rate satisfaction with their visit on the scale that was randomly assigned to them: (1) an 11-point Likert scale with 5 anchor points and visible numbers; (2) a 5-point Likert scale with 5 anchor points and no visible numbers; (3) a 0-100 VAS with 3 anchor points and visible numbers; (4) a 0-100 VAS with 3 anchor points and no visible numbers (Figure 1). The 4 scales should not differ in time needed to complete them; however, we did not explicitly measure time to completion. Participants also completed measures of psychological aspects of illness. The 2-item Pain Self-Efficacy Questionnaire (PSEQ-2) was used to measure pain self-efficacy, an effective coping strategy for pain.⁸ Higher PSEQ-2 scores indicate a higher level of pain self-efficacy. The 5-item Short Health Anxiety Inventory scale (SHAI-5) was also administered; higher scores on this scale indicate a greater degree of health anxiety.⁹ The Patient-Reported Outcomes Measurement Information System (PROMIS) Depression was used to measure symptoms of depression.¹⁰ Finally, the diagnosis was recorded by the surgeon (not in table).

Statistical Analysis

We reported continuous variables using mean, standard deviation (SD), median, and interquartile range (IQR). Categorical data are presented as frequencies and percentages. We calculated floor and ceiling effect and the skewness and kurtosis of every scale. We scaled every scale to 10 and also standardized every scale. We used the Kruskal–Wallis test to compare differences in satisfaction between the scales; Fisher’s exact test to compare differences in floor and ceiling effect; and Spearman correlation tests to test the correlation between scaled satisfaction scores and psychological status.

Ceiling effects are present when patients select the highest value on a scale rather than a value that reflects their actual feelings about a certain topic. Floor effects are present when patients select the lowest value in a similar fashion. These 2 effects indicate that an independent variable no longer influences the dependent variable being tested. Skewness and kurtosis are rough indicators of a normal distribution of values. Skewness (γ1) is an index of the symmetry of a distribution, with symmetric distributions having a skewness of 0. If skewness has a positive value, it suggests relatively many low values, having a long right tail. Negative skewness suggests relatively many high values, having a long left tail. Kurtosis (γ2) is a measure to describe tailedness of a distribution. Kurtosis of a normal distribution is 3. Negative kurtosis represents little peaked distribution, and positive kurtosis represents more peaked distribution.^11,12If skewness is 0 and kurtosis is 3, there is a normal, or Gaussian, distribution.

Finally, we manually calculated the NPS for all scales by subtracting the percentage of detractors (people who scored between 0 and 6) from the percentage of promoters (people who scored 9 or 10).¹³ NPS are widely used in the service industry to assess customer satisfaction, and scores range between –100 and 100.

An a priori power analysis indicated that in order to find a difference in satisfaction of 0.5 on a 0-10 scale, with an effect size of 80% and alpha set at 0.05, we needed 128 patients (64 per group). Since we wanted to compare 4 satisfaction scales, we doubled this.