Sometimes, a school may have a low score on a measure, but responses to many of the items seem generally positive. But on the survey, students and teachers use the strongly agree and agree categories in ways that are analytically and conceptually distinct.
For example, School A has a score of 30 on Teacher-Teacher Trust, but the percent of teachers agreeing or strongly agreeing with each item ranges from 70 to 88%. If we only look at the frequencies, we might think that this school is doing well on this measure. But when we look at the proportion of respondents who agree and compare it to those who strongly agree, we can see that most respondents are agreeing instead of strongly agreeing:
Compare these frequencies to the frequencies of School B that has a measure score of 70 on Teacher-Teacher Trust:
The percent agreeing or strongly agreeing to each item is somewhat higher than School A—87% to 96%--but note that there is a much greater proportion of respondents in the very top category.
In addition, our method of analysis (Rasch analysis) takes into account that some items are more “difficult” to agree with than others. In Teacher-Teacher Trust, we know from Rasch analysis that “Teachers in this school trust each other” is the most difficult to endorse, while “Teachers feel respected by other teachers” is the “easiest” to endorse.
If a teacher responds in a statistically improbable way, that response is down-weighted in calculating the school’s measure score. For example, if the teacher reports that she strongly disagrees with “Teachers in this school trust each other” but responds “to a great extent” to “Teachers feel respected by other teachers”, our scoring process down-weights that individual because the responses are statistically unreliable.