For the full range of values, the model derived on the development sample showed a c-statistic of 0.672, which reduced to 0.632 in the validation sample. A value of 1.0 would indicate perfect discrimination between those who did and did not have radiographic pneumonia, while a value of 0.5 would indicate no better than chance discrimination. Model calibration was not acceptable in the validation sample (Hosmer-Lemeshow goodness-of-fit statistic, P=.008). Inspection suggested the disagreement between predicted and observed probability of pneumonia was primarily with lower-risk estimates.
Because the model performed relatively well at distinguishing subjects very likely to have pneumonia, we created a simple point system aimed at identifying such high-risk individuals. Table 4 shows the scoring system. For 33% of subjects (score Ž3), there was a 56% or higher probability of radiographic pneumonia. An additional 24% of subjects (score of 2) had 44% probability of radiographic pneumonia. However, even those with the lowest scores (-1 to 0, 15% of subjects) still had a 24% probability of pneumonia. The relationship between the score and the probability of radiographic evidence of pneumonia is shown in Figure W1.*
Discussion
In a large community-based sample, we considered presenting symptoms, signs, and laboratory findings associated with radiographic pneumonia. Individual findings discriminated poorly, and we could not separate out a very-low-risk group. However, our simple scoring system identified approximately one third to slightly more than one half with high probability of pneumonia—individuals who might be treated without a confirmatory chest x-ray. If our data are confirmed, they suggest a simple clinical strategy in patients with respiratory or general symptoms Table 1 that might suggest pneumonia: (1) if there are no respiratory symptoms, consider other conditions, such as a urinary tract infection, that might fully explain the symptoms; (2) obtain information to apply our symptom score Table 4; (3) for those with scores of 2 or higher (some might choose 3 instead), treat for pneumonia; (4) for those with scores of -1, 0, or 1, obtain a chest radiograph as a guide to treatment.
Considering individual findings, fever was significantly more common in pneumonia, but only 43% of those with possible or probable pneumonia had a temperature of at least 38°C. This reaffirms common wisdom and previous findings that fever is frequently absent in elderly people with pneumonia.9,19 We also confirmed that few signs or symptoms are the norm for nursing home-acquired pneumonia.
Chest examination findings also do not adequately distinguish patients with and without pneumonia Table 2. Also, even expert physicians frequently differ on lung examination findings.20 Nonetheless, presence of crackles and absence of wheezing contribute to our scoring system. Both findings are seen with multiple conditions, but in our data crackles are slightly more associated with pneumonia, while wheezing is more strongly associated with other diseases.
The other components of our scoring system are clinical factors commonly associated with pneumonia. Though none individually discriminates well between those with and without pneumonia Table 2, several combined serve to identify a high-risk group.
Four previous studies from emergency department or outpatient settings developed clinical prediction rules to identify pneumonia.21-24 Criteria for identifying subjects varied substantially, and each rule has limited accuracy in predicting radiographic pneumonia.20 We had adequate data to evaluate 3 of the rules.21-23 As is usually the case when transporting a prediction rule to a new sample, none performed any better than our rule (data not shown). Our sample created the very difficult challenge for any prediction rule of a very high overall prevalence of pneumonia (45%). That made it unlikely that we could identify a low-risk group in whom x-ray studies could be readily forgone, but we were able to identify a highrisk group.
Limitations
Our findings are subject to several limitations. All facilities in our study were located in central or eastern Missouri, and not all physicians or eligible residents in those facilities participated. Compared with national data, we studied an unusually representative sample of nursing home residents from 36 facilities, including rural and urban locations. Also, in episodes excluded because of physician nonparticipation, residents were very similar to included residents in age, vital signs, and presenting symptoms (data available on request). More important, we lack an independent validation sample from a different cohort. Clinical prediction rules usually do not perform as well in independent samples. This is exemplified by the poor performance of the 3 rules we considered from other settings. Overall, our logistic model was only modest in discriminating and was not well calibrated for low-risk episodes in our reserved validation sample. Although we have developed a promising scoring system to identify residents with high probability of radiographic pneumonia, it needs to be validated in other samples of nursing home residents to determine its ultimate usefulness. For all these reasons, our results may not generalize.