The Boston residue and clearance scale: preliminary reliability and validity testing

Abstract

PURPOSE: To date, there is no validated scale for grading the severity of pharyngeal residue. To address this problem, The Boston Residue and Clearance Scale (BRAGS) was constructed and assessed for reliability and validity. METHODS: A team including 5 Speech Language Pathologists (SLPs) at Boston Medical Center conceptualized BRAGS. BRAGS is an 11-point scale comprised of three main components: 1) amount and location of residue, 2) whether a patient executes clearing swallows spontaneously, and 3) the effectiveness of any clearing swallows. Face validity was confirmed by the team. A movie containing 63 unique swallows was created from previously recorded Fiberoptic Endoscopic Evaluation of Swallowing (FEES) procedures. First, four SLPs independently scored the severity of residue for each swallow using clinical judgment (none, mild, mild-moderate, moderate, moderate-severe, severe). The 63 swallows were re-randomized in a second movie and scored in the same manner one week later. Next, the same SLPs were trained to use BRAGS. A 5th SLP who is an expert in FEES served as a gold standard. When concordance of the four raters and the gold standard was acceptable, the four SLPs were given the initial movie to score using BRAGS. After one week, they scored the second movie with BRAGS. In BRAGS, inter-rater reliability and test-retest (intra-rater) reliability were estimated with intraclass correlation coefficients (ICCs). The correlation between each rater and the 5th SLP in the first clinical judgment and in the BRAGS trial was also examined with ICCs in order to confirm the raters' accuracy in rating. Concurrent validity of BRAGS to clinical judgment was tested with Pearson's correlation coefficient. Internal consistency of BRAGS was tested with Cronbach's alpha. RESULTS: Inter-rater ICC for the first clinical judgment was 0.60, and for the first BRAGS trial was 0.81. Test-retest ICCs for the clinical judgment of four SLPs ranged between 0.72 and 0.86, and for BRAGS it ranged between 0.82 and 0.92. Correlation between each rater and the 5th SLP for the first clinical judgment was 0.69 and for the first BRAGS trial was 0.80. Pearson's correlation coefficient demonstrated high concurrent validity of BRAGS to the clinical judgment (r=0.76). Cronbach's alpha showed good internal consistency of all items included in BRAGS (ICC=0.86). CONCLUSION: Excellent inter-rater and test-retest reliability demonstrated that BRAGS is more reliable than clinical judgment when rating severity of pharyngeal residue. Internal consistency and concurrent validity were confirmed.