Advanced students (PR of 90 or above) can present challenges in regard to assessment and interpretation of the results. In the days before Computer Aided Testing (CAT), there were individual test booklets for each grade that were designed to be able to properly assess approximately 90 – 95% of the student population at that grade level. In order to properly assess an advanced student, a test booklet one or even two grade levels higher would have to be given to the student (this was called off level testing). This presented operational as well as interpretational challenges.
Now that were are using the STAR assessment system, off level testing is no longer necessary. All students, whether on grade level or not, can be assigned the same assessment and the computer will quickly make the adjustments to present an appropriate assessment. The scale score (the direct result of an assessment) will mean the same regardless of grade level. The resultant instructional recommendations derived from the scale score are not affected by grade placement.
The percentile rank (PR) and normal curve equivalent (NCE) are derived from the scale score and knowledge of the score distribution at each grade level. (Please see my “All These Numbers” blog post for a deeper discussion of these scales, their derivation, and meaning). While assessing a student in the upper 5% is no longer a problem, deriving meaningful standardized scores and measuring growth remains problematic. Since the PR scale tops out at 99 (and thus the NCE scale derived from it tops out at 99), students in the high 90’s will face this “ceiling” issue. It doesn’t matter how high their scale score is, the maximum NCE score will remain at 99.
The solution to this problem is “off level scoring”, where the standardized scores (PR and NCE) are determined as if the student were in a higher grade. This solution is pictured in Figure 1. Johnny scored a scale score of 620 and is in the third grade by age grouping. If standardized scores were to be derived for him as a third grader,(red rectangle), his PR and NCE scores would be close to 99 (the maximum) and he could show growth in scale score terms, but not in NCE’s (the ceiling effect). If Johnny were scored as a fourth grader (blue rectangle), his NCE score would be approximately 64 (corresponding to a PR of 75), leaving plenty of room to demonstrate growth in NCE terms and generate a learning gain greater than 1.0.
Care must be taken, however, in interpreting and explaining Johnny’s new scores. Johnny’s scale score and instructional recommendations will be accurate, as will his learning gains (due to the removal of the ceiling). Johnny’s PR and NCE scores will be lowered and should not be used for any decision regarding Johnny. In discussing Johnny’s scores with his parents, the reason for (too advanced to be scored on grade level) and results of (artificially lowering the standardized scores) off level scoring should be carefully explained.
Figure 1 – Off Level Scoring Example
