Original Research

A Comparison of 4 Single-Question Measures of Patient Satisfaction

Journal of Clinical Outcomes Management. 2020 January;27(1)

Author(s):

Iris I.M. Kleiss, MD

Joost T.P. Kortlever, MD

Prithvi Karyampudi

David Ring, MD, PhD

Laura E. Brown, PhD

Lee M. Reichel, MD

Matt D. Driscoll, MD

Gregg A. Vagner, MD

PDF Download

References

1. Waters S, Edmondston SJ, Yates PJ, Gucciardi DF. Identification of factors influencing patient satisfaction with orthopaedic outpatient clinic consultation: A qualitative study. Man Ther. 2016;25:48-55.

2. Kortlever JTP, Ottenhoff JSE, Vagner GA, et al. Visit duration does not correlate with perceived physician empathy. J Bone Joint Surg Am. 2019;101:296-301.

3. Edwards P, Roberts I, Clarke M, et al. Methods to influence response to postal questionnaires. Cochrane Database Syst Rev. 2001(3):CD003227.

4. Salisbury C, Burgess A, Lattimer V, et al. Developing a standard short questionnaire for the assessment of patient satisfaction with out-of-hours primary care. Fam Pract. 2005;22:560-569.

5. Ross CK, Steward CA, Sinacore JM. A comparative study of seven measures of patient satisfaction. Med Care. 1995;33:392-406.

6. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377-381.

7. Medicine USNLo. ClinicalTrials.gov. Accessed March 18, 2019.

8. Nicholas MK, McGuire BE, Asghari A. A 2-item short form of the Pain Self-efficacy Questionnaire: development and psychometric evaluation of PSEQ-2. J Pain. 2015;16:153-163.

9. Salkovskis PM, Rimes KA, Warwick H, Clark D. The Health Anxiety Inventory: development and validation of scales for the measurement of health anxiety and hypochondriasis. Psychol Med. 2002;32:843-853.

10. Schalet BD, Pilkonis PA, Yu L, et al. Clinical validity of PROMIS depression, anxiety, and anger across diverse clinical samples. J Clin Epidemiol. 2016;73:119-127.

11. Ho AD, Yu CC. Descriptive statistics for modern test score distributions: skewness, kurtosis, discreteness, and ceiling effects. Educ Psychol Meas. 2015;75:365-388.

12. Kim HY. Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor Dent Endod. 2013;38:52-54.

13. NICE Satmetrix. What is net promoter? https://www.netpromoter.com/know/. Accessed March 18, 2019.

14. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34-42.

15. Hasson D, Arnetz BB. Validation and findings comparing VAS vs. Likert scales for psychosocial measurements. Int Electronic J Health Educ. 2005;8:178-192.

16. Vickers AJ. Comparison of an ordinal and a continuous outcome measure of muscle soreness. Int J Technol Assess Health Care. 1999;15:709-716.

17. Jaeschke R, Singer J, Guyatt GH. A comparison of seven-point and visual analogue scales: data from a randomized trial. Control Clin Trials. 1990;11:43-51.

18. Voutilainen A, Pitkaaho T, Kvist T, Vehvilainen-Julkunen K. How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric Likert scale. J Adv Nurs. 2016;72:946-957.

19. Brunelli C, Zecca E, Martini C, et al. Comparison of numerical and verbal rating scales to measure pain exacerbations in patients with chronic cancer pain. Health Qual Life Outcomes. 2010;8:42.

20. Hageman MG, Briet JP, Bossen JK, et al. Do previsit expectations correlate with satisfaction of new patients presenting for evaluation with an orthopaedic surgical practice? Clin Orthop Relat Res. 2015;473:716-721.

21. Keulen MHF, Teunis T, Vagner GA, et al. The effect of the content of patient-reported outcome measures on patient perceived empathy and satisfaction: a randomized controlled trial. J Hand Surg Am. 2018;43:1141.e1-e9.

22. Mellema JJ, O’Connor CM, Overbeek CL, et al. The effect of feedback regarding coping strategies and illness behavior on hand surgery patient satisfaction and communication: a randomized controlled trial. Hand. 2015;10:503-511.

23. Tyser AR, Gaffney CJ, Zhang C, Presson AP. The association of patient satisfaction with pain, anxiety, and self-reported physical function. J Bone Joint Surg Am. 2018;100:1811-1818.

24. Vranceanu AM, Ring D. Factors associated with patient satisfaction. J Hand Surg Am. 2011;36:1504-1508.

25. Stirling P, Jenkins PJ, Clement ND, et al. The Net Promoter Scores with Friends and Family Test after four hand surgery procedures. J Hand Surg Eur. 2019;44:290-295.

26. Wilberforce M, Poll S, Langham H, et al. Measuring the patient experience in community mental health services for older people: A study of the Net Promoter Score using the Friends and Family Test in England. Int J Geriatr Psychiatry. 2019;34:31-37.

From Dell Medical School, The University of Texas at Austin, Austin, TX.

Abstract

Objective: Satisfaction measures often show substantial ceiling effects. This randomized controlled trial tested the null hypothesis that there is no difference in mean overall satisfaction, ceiling and floor effect, and data distribution between 4 different kinds of single-question scales assessing the helpfulness of a visit. We also hypothesized that there is no correlation between scaled satisfaction and psychological status. Finally, we assessed how the satisfaction scores compared with the Net Promoter Scores (NPS).
Design: Randomized controlled trial.
Methods: We enrolled 258 adult, English-speaking new and returning patients. Patients were randomly assigned to 1 of 4 different scale types: (1) an 11-point ordinal scale with 5 anchor points; (2) a 5-point Likert scale; (3) a 0-100 visual analogue scale (VAS) electronic slider with 3 anchor points and visible numbers; and (4) a 0-100 VAS with 3 anchor points and no visible numbers. Additionally, patients completed the 2-item Pain Self-Efficacy Questionnaire (PSEQ-2), 5-item Short Health Anxiety Inventory scale (SHAI-5), and Patient-Reported Outcomes Measurement Information System (PROMIS) Depression. We assessed mean and median score, floor and ceiling effect, and skewness and kurtosis for each scale. Spearman correlation tests were used to test correlations between satisfaction and psychological status.
Results: The nonnumerical 0-100 VAS with 3 anchor points and the 5-point Likert scale had the least ceiling effect (12% and 20%, respectively). The 11-point ordinal scale had skewness and kurtosis closest to a normal distribution (skew = –0.58 and kurtosis = 4.0). Scaled satisfaction scores had a small but significant correlation with PSEQ-2 (r = 0.17; P = 0.006), but not with SHAI-5 (r = –0.12; P = 0.052) or PROMIS Depression (r = –0.12; P = 0.064). NPS were 35, 16, 67, and 20 for the scales, respectively.
Conclusion: Single-question measures of satisfaction can be adjusted to limit the ceiling effect. Additional research in this area is warranted.

Keywords: patient satisfaction; floor and ceiling effect; skewness and kurtosis; quality improvement.

Patient satisfaction is an important quality metric that is increasingly being measured, reported, and incentivized. A qualitative study identified 7 themes influencing satisfaction among people visiting an orthopedic surgeon’s office: trust, relatedness, expectations, wait time, visit duration, communication, and empathy.¹ However, another study found that satisfaction and perceived empathy are not associated with wait time or visit duration, but rather with the quality of the visit.² Satisfaction measures that incorporate many of these features in relatively long questionnaires are associated with lower response rates³ and overlap with the factors whose influence on satisfaction one would like to study (eg, perceived empathy or communication effectiveness).⁴ Single- and multiple-question satisfaction scores are prone to a strong right skew, with a substantial ceiling effect.⁵ Ceiling effect occurs when a considerable proportion (about half) of participants select 1 of the top 2 scores (or the maximum score). An ideal scale would measure satisfaction independent from other factors, would use 1 or just a few questions, and would have little or no ceiling effect.

In this randomized controlled trial, we examined whether there were significant differences in mean and median satisfaction, floor and ceiling effect, and data distribution (by looking at skewness and kurtosis) between 4 different kinds of satisfaction scales asking about the helpfulness of a visit. Additionally, we hypothesized that there is no correlation between scaled satisfaction and psychological status. Finally, we assessed how the satisfaction scores compared to the Net Promoter Scores (NPS). NPS are commonly used in the service industry to measure customer satisfaction; we are using these scores as a measure of patient satisfaction.