Why are my proficiency scores high yet my learning gains seem to be low?
An Individual assessment score (e.g. Reading -Spring) indicates the academic status of a student at a particular point in time while learning gains measure the growth of a student from one point in time to a later point in time. An individual assessment score is converted to a proficiency level by applying a relative arbitrary cut score (or scores) to divide students into proficient or not proficient or to establish categories of proficiency (e.g. levels 1 – 5). This measure indicates where a student is at a particular point in time but contains very little information regarding the path to that point.
A learning gain indicates the amount of growth on the part of a student between two points in time (e.g. Fall to Spring). This measure indicates how far along the path the student has gone, but contains very little information regarding current location.
A student’s Spring position (score) is a function of both Fall position (score) and how far they have travelled (learning gain):
Spring score = Fall score + learning gain.
Therefore, a student’s proficiency level depends both on how far below (or above) proficiency he/she was in Fall and the learning gain. For students well above proficiency in Fall, even negative learning gains would result in proficiency in Spring and for students well below proficiency in Fall, a large, positive learning gain would not result in proficiency in Spring. This means that, on a school level, we would not expect learning gains to have much of a relationship with proportion proficient.
If we establish an arbitrary cut score on the Spring assessment to represent proficiency (NCE = 50 in this example), we can examine the relationship between learning gains and proficiency using SY 15-16 data. The degree of this relationship would not change for any arbitrary point except for highly extreme ones (e.g. NCE = 10 or NCE = 90). No state is currently near these extreme points.
As you can see in Figure 1, there is a very low (r=0.11) correlation between mean learning gains and proportion proficiency. Schools with high learning gains have both high and low proportion proficiency while schools with low learning gains also have both high and low proportion proficient. From this analysis we can conclude that over 90% of the variation in proportion proficient between schools is unrelated to learning gains.
Figure 1: Relationship between Mean Learning Gain and Proportion Proficient
If learning gains have a weak relationship to proportion proficient, what variable does have a stronger relationship? Much research has shown the single variable normally available that has the strongest relationship to proportion proficient is proportion of students that qualify for free and reduced lunch. This relationship is portrayed in figure 2. The correlation between these two variables is r=0.59. This means that the FRL proportion accounts for approximately 34% of the variation in proportion proficient. It is important to note, however, that a high FRL proportion is NOT a determinate of low proportion proficient (just a tendency). As shown by the circled section, schools that are at or near 100% FRL range in proportion proficient from a low of 12% to a high of 60%.
This entire discussion could be repeated for the Math assessment with the same conclusion. Proportion proficient is weakly related to learning gains and highly related to the students entering the school.
For another excellent discussion of proficiency scores, I recommend this article:
http://www.gse.harvard.edu/news/uk/15/12/when-proficient-isnt-good
Figure 2:Relationship between Proportion qualifying for Free or Reduced Lunch and Proportion Proficient
