Original Research

A Comparison of 4 Single-Question Measures of Patient Satisfaction

Journal of Clinical Outcomes Management. 2020 January;27(1)

References

1. Waters S, Edmondston SJ, Yates PJ, Gucciardi DF. Identification of factors influencing patient satisfaction with orthopaedic outpatient clinic consultation: A qualitative study. Man Ther. 2016;25:48-55.

2. Kortlever JTP, Ottenhoff JSE, Vagner GA, et al. Visit duration does not correlate with perceived physician empathy. J Bone Joint Surg Am. 2019;101:296-301.

3. Edwards P, Roberts I, Clarke M, et al. Methods to influence response to postal questionnaires. Cochrane Database Syst Rev. 2001(3):CD003227.

4. Salisbury C, Burgess A, Lattimer V, et al. Developing a standard short questionnaire for the assessment of patient satisfaction with out-of-hours primary care. Fam Pract. 2005;22:560-569.

5. Ross CK, Steward CA, Sinacore JM. A comparative study of seven measures of patient satisfaction. Med Care. 1995;33:392-406.

6. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377-381.

7. Medicine USNLo. ClinicalTrials.gov. Accessed March 18, 2019.

8. Nicholas MK, McGuire BE, Asghari A. A 2-item short form of the Pain Self-efficacy Questionnaire: development and psychometric evaluation of PSEQ-2. J Pain. 2015;16:153-163.

9. Salkovskis PM, Rimes KA, Warwick H, Clark D. The Health Anxiety Inventory: development and validation of scales for the measurement of health anxiety and hypochondriasis. Psychol Med. 2002;32:843-853.

10. Schalet BD, Pilkonis PA, Yu L, et al. Clinical validity of PROMIS depression, anxiety, and anger across diverse clinical samples. J Clin Epidemiol. 2016;73:119-127.

11. Ho AD, Yu CC. Descriptive statistics for modern test score distributions: skewness, kurtosis, discreteness, and ceiling effects. Educ Psychol Meas. 2015;75:365-388.

12. Kim HY. Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor Dent Endod. 2013;38:52-54.

13. NICE Satmetrix. What is net promoter? https://www.netpromoter.com/know/. Accessed March 18, 2019.

14. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34-42.

15. Hasson D, Arnetz BB. Validation and findings comparing VAS vs. Likert scales for psychosocial measurements. Int Electronic J Health Educ. 2005;8:178-192.

16. Vickers AJ. Comparison of an ordinal and a continuous outcome measure of muscle soreness. Int J Technol Assess Health Care. 1999;15:709-716.

17. Jaeschke R, Singer J, Guyatt GH. A comparison of seven-point and visual analogue scales: data from a randomized trial. Control Clin Trials. 1990;11:43-51.

18. Voutilainen A, Pitkaaho T, Kvist T, Vehvilainen-Julkunen K. How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric Likert scale. J Adv Nurs. 2016;72:946-957.

19. Brunelli C, Zecca E, Martini C, et al. Comparison of numerical and verbal rating scales to measure pain exacerbations in patients with chronic cancer pain. Health Qual Life Outcomes. 2010;8:42.

20. Hageman MG, Briet JP, Bossen JK, et al. Do previsit expectations correlate with satisfaction of new patients presenting for evaluation with an orthopaedic surgical practice? Clin Orthop Relat Res. 2015;473:716-721.

21. Keulen MHF, Teunis T, Vagner GA, et al. The effect of the content of patient-reported outcome measures on patient perceived empathy and satisfaction: a randomized controlled trial. J Hand Surg Am. 2018;43:1141.e1-e9.

22. Mellema JJ, O’Connor CM, Overbeek CL, et al. The effect of feedback regarding coping strategies and illness behavior on hand surgery patient satisfaction and communication: a randomized controlled trial. Hand. 2015;10:503-511.

23. Tyser AR, Gaffney CJ, Zhang C, Presson AP. The association of patient satisfaction with pain, anxiety, and self-reported physical function. J Bone Joint Surg Am. 2018;100:1811-1818.

24. Vranceanu AM, Ring D. Factors associated with patient satisfaction. J Hand Surg Am. 2011;36:1504-1508.

25. Stirling P, Jenkins PJ, Clement ND, et al. The Net Promoter Scores with Friends and Family Test after four hand surgery procedures. J Hand Surg Eur. 2019;44:290-295.

26. Wilberforce M, Poll S, Langham H, et al. Measuring the patient experience in community mental health services for older people: A study of the Net Promoter Score using the Friends and Family Test in England. Int J Geriatr Psychiatry. 2019;34:31-37.

Discussion

Single-question measures of satisfaction can decrease patient burden and limit overlap with measures of communication effectiveness and perceived empathy. Both long and short questionnaires addressing satisfaction and perceived empathy show substantial ceiling effect. We compared 4 different measures for overall scores, floor and ceiling effect, and skewness and kurtosis, and assessed the correlation between scaled satisfaction and psychological status. We found that scale type influenced the median helpfulness score. As one would expect, scales with less ceiling effect have lower median scores. In other words, if the goal is to collect meaningful information and identify areas for improvement, there must be a willingness to accept lower scores.

Only the nonnumerical VAS was below the threshold of 15% ceiling effect proposed by Terwee et al.¹⁴ This scale with 3 anchor points and no visible numbers showed the least ceiling effect (12%) and minimal skew (–1.0), and was closer to kurtosis consistent with a normal distribution (5.0). However, the 11-point ordinal Likert scale with 5 anchor points and visible numbers had the lowest skewness and kurtosis (–0.58 and 4.0). The low ceiling effect observed with the nonnumerical VAS (12%) might be explained by the fact that the scale does not lead patients to a specific description of the helpfulness of their visit, but rather asks patients to use their own judgement in making the rating. The ordinal scale approached the most normal data distribution, and this might be explained by the presence of numbers on the scale. Ratings based on a 0-10 scale are commonly used, and familiarity with the system might have allowed people to pick a number that represents their actual view of the visit helpfulness, rather than picking the highest possible choice (which would have led to a ceiling effect). Study results comparing Likert scales and VAS are conflicting,¹⁵ with some preferring Likert scales for their responsiveness¹⁶ and ease of use in practice,¹⁷ and others preferring VAS for their sensitivity to describe continuous, subjective phenomenon and their high validity and reliability.¹⁸ Looking at our nonnumerical VAS, adding numbers to a scale might not help avoid, and may actually increase, the presence of ceiling effect. However, with the ordinal scale with visible numbers, we saw a 21% ceiling effect coupled with low skew and kurtosis (–0.58 and 4.0), which indicate that the distribution of scores is relatively normal. This finding is in line with other study results.¹⁹

Our findings demonstrated that feedback concerning self-efficacy, health anxiety, or depression had no or only a small effect on patient satisfaction. Consistent with prior evidence, psychological factors had limited or no correlation with satisfaction.^20-24 Given the effect that priming has on patient-reported outcome measures, the effect of psychological factors on satisfaction could be an area of future study.

The NPS varied substantially based on scale structure. Increasing the spread of the scores to limit the ceiling effect will likely reduce promoters and detractors and increase neutrals. NPS systems have been used in the past to measure patient satisfaction with common hand surgery techniques and with community mental health services.^25,26These studies suggest that NPS could be a helpful addition to commonly used clinical measures of satisfaction, after more research has been done to validate it. The evidence showing that NPS are strongly influenced by scale structure suggests that NPS should be used and interpreted with caution.

Several caveats regarding this study should be kept in mind. This study specifically addressed ratings of visit helpfulness. Differently phrased questions might lead to different results. More work is needed to determine the essence of satisfaction with a medical visit.¹ In addition, the majority of our patient population was white, employed, and privately insured, limiting generalizability to other populations with different demographics. Finally, all patients were seen by an orthopedic surgeon, and our results might not apply to other populations or clinical settings. However, given the scope of this study, we suspect that the findings can be generalized to specialty care in general and likely all medical contexts.