Reliability of Judgments of Articulation of Cleft Palate Speakers

Abstract

Articulation test data are usually considered to have a satisfactory degree of reliability. This has been demonstrated by investigators employing correlation techniques (4, 7', 8) and by those reporting percentage of agreement among examiners (1—3). Siegel (5), however, questioned the interpretation of reliability as indicated by a high correlation because he also found significant differences in the absolute test scores determined by the examiners for each subject. Some investigators, such as Sommers and others (6), have found that the level of reliability can be improved by having judges train together in evaluating articulation reSponses. However, under conditions such as those imposed by collaborative research at widely scattered clinical centers or in longitudinal projects, such training may not be possible. Therefore, it was the purpose of this study to determine the reliability of judgments of tape—recorded speech samples of cleft palate subjects when evaluated by speech pathologists using written instructions and definitions of criteria for evaluation. The following specific questions were investigated: a) What is the relia— bility of intra— and interjudge evaluations of the intelligibility of connected speech, the identification of articulation errors, and the classification of types of articulation errors? b) Does the reliability of articulation judgments vary with the phonetic classification of the sounds, the position of the sounds, or the error descriptions? 0) Are there wide variations in the specific articulation scores despite a satisfactory level of reliability as determined by percentage of interjudge agreement? d) If marked variation exists in the specific articulation scores, does it impair the clinical research value of articulation evaluations? Procedure SUBJECTS. Tape recordings were made of the speech of 50 cleft palate subjects.