A few notes on EQAO testing
May. 27th, 2007 01:46 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
velvetpageAKA: Why the whole thing is a crock.
Our grade threes wrote their EQAO tests earlier this week. That is, all thirty-six of them. They started as soon as they arrived, and did two, one-hour blocks each day for three days.
Let's examine that, shall we?
First, there are thirty-six of them. This is about standard for our school; generally, there is one straight grade three class and one split class with between ten and fifteen more grade threes in it. Rarely are there more than forty kids. Let's say, just for the sake of some nice round numbers, that there are usually about 40 grade threes at our school. The provincial goal is to have 75% of those kids meeting or exceeding the provincial standard, which is a level three, or a B. So, thirty kids have to make the grade for the school to succeed. But it's done as an average, which means that if one kid totally bombs and there is no 98% kid to compensate, everyone's scores get pulled down. Similarly, the testing must all be completed within the week. If a child misses a day early in the week, they make it up on Thursday or Friday. A kid who is under the weather on Monday stands a decent chance of still being under the weather on Thursday, but they have to write the test anyway. All it takes is for one or two strong students to stay home, and all of a sudden our scores are dramatically skewed. Conversely, the test is to be written by every grade three in the school - including the kid who just returned from a lengthy (read: before Christmas) vacation in Pakistan. He came back on Tuesday. Yes, he had to write the test.
Simply put, forty kids is too small a sample to get a good picture of what is going on in a school. Even sixty kids is too small a sample. You'd need a lot more than that, and you'd need to correlate it with other information. For example, the child's state of health on the days of the test should be noted and compared against their score when it comes back. The child's diagnosed learning disabilities should be taken into account, as well as the suspected ones that are not yet diagnosed - to be correlated with the test when the diagnosis, if any, is complete. You have to be pretty seriously LD to have a diagnosis by the end of grade three in our board of ed. In order to be eligible for psych testing, you have to be at least two grade levels behind, meaning a grade three student in September has to be operating at a kindergarten level before anyone will look at him. That means that milder LD kids are struggling through the test unaided. Ditto the kids whose parents haven't taken them to the doctor in the three years we've been asking them to do so. Many LDs require a pediatric assessment for a firm diagnosis. No assessment, no LD, and no help. It's just one more way that children of low-income/poorly-educated/slightly neglectful parents end up at a disadvantage.
The usual estimate of LD in the population is 10-15%, but that is skewed towards lower income levels, meaning that a school like ours might have an LD level closer to 25%, while schools in richer neighbourhoods may have less than 10%. So if 25% of our grade threes potentially have some LD, that's ten kids. Assuming that those kids are highly unlikely to be operating at grade level (because if they were, they wouldn't qualify for an LD diagnosis) that means all of the others have to make the grade in order for our school to pass muster. As it is, four of our grade threes have diagnosed LDs. We've already beat the 10% number, and we haven't reached the age yet at which the more subtle, less-than-two-years-behind LDs are commonly diagnosed.
Then there's the ESL problem. 50% of our kids come from homes where English is the second language; about 80% of those speak English only at school. That's sixteen grade threes who spoke no English at all before arriving in junior kindergarten or kindergarten, or perhaps upon arriving in Canada more recently than that, and four more whose parents' English is not entirely fluent. The research suggests that second-language learners of average intelligence or better will catch up by grade six or seven, assuming they start learning English in junior kindergarten - that is, seven to eight years. After that, many of them will actually surpass their unilingual peers, which means their tests in grade nine should show about 70% of the second-language, Canadian-born kids to be ahead of their grade. (Seventy percent, because we're only talking about those of average intelligence or better. Throw in a communication LD or a slow-learning kid, and it'll take longer, if indeed they ever catch up.) That means we're testing twenty kids whom we know will not be at full proficiency in English, because they haven't spoken it long enough - and we're testing them with standards designed for native speakers of English. Even those ESL students on the upper end of the intelligence scale are unlikely to have caught up enough to meet the provincial standard with any consistency on EQAO in grade three, since all the tests - reading, writing, and math - require language proficiency. In our current school, there is no crossover between the diagnosed LD kids and the second-language ones. That's 24 kids out of 40 who we know before we start are at a disadvantage, plus at least four more who are undiagnosed but on the list for psychoeducational testing - because we know something's up with them but we don't know what. We're at 28 kids. What was that magic number again? 30 to make the grade? It's looking unlikely at this point, isn't it?
There's one huge, massive, colossal hole in the way EQAO scores are used, and as a grade four teacher, I've asked about it. You see, in many boards across Ontario, including ours, kids receive a truly standardized intelligence test in grade four. The test varies, but our board is currently using the CCAT. It tests two things: vocabulary, and spatial awareness. The tests are usually of the correlation kind: "Small is to large as _________ is to wide." Then they choose from a list. The tests together give a decent (though not perfect) picture of a kid's IQ, and are scored on a normative curve. The scores are used for only one thing: they serve as initial screening for potential entrance into gifted classes. A child must get eight on one test and nine on the other, for a combined score of at least seventeen (a high sixteen might be taken for more testing) in order to qualify. Approximately one percent of kids qualify according to that standard, and in my opinion, it's too strict. But that's an issue for another day.
What no one has done, because the political climate is dead set against doing it, it to correlate the EQAO scores from the spring, with the CCAT scores from the following fall. I'm seriously considering doing it on the sly this fall with my grade fours. It shouldn't be hard to show a correlation. I'd bet good money that the kids who do poorly on the verbal test will be the same kids who did poorly on EQAO. The problem is that the politicians don't want to use EQAO this way.
So how do they use it?
They use it to rank schools, and to accuse teachers in poor-ranking schools of being bad teachers. They claim it's all our fault. Studies show that good teaching trumps all other factors, they say.
Actually, that's not what the studies show. Studies actually show that the most important factor is the education level of the parents, particularly of the mother, and the second-most important factor (which is, unsurprisingly, closely linked to the first) is socio-economic level. The strength of teaching is a distant third in that equation.
In any case, we should be starting to see a difference over the next few years. Boards across our province have been granted small fortunes for training and resources, all aimed at getting those EQAO scores up. And in some areas, it's working. Many schools have seen dramatic increases in their EQAO scores. Most have not seen increases quite as dramatic as the province wants, though. The number of schools going from a 30% pass rate to a 75% pass rate is vanishingly small, and the demographics suggest that schools like ours usually aren't among them. More often, the teachers work themselves into a nervous breakdown and achieve an increase of about 20-30% - that is, about 50% of the students making the grade. Those are huge gains. You'd think the province would be happy for every last kid who succeeds in that way. But all the teachers hear is that the scores still aren't high enough, we still aren't meeting expectations, we still haven't done enough.
Our grade threes wrote their EQAO tests earlier this week. That is, all thirty-six of them. They started as soon as they arrived, and did two, one-hour blocks each day for three days.
Let's examine that, shall we?
First, there are thirty-six of them. This is about standard for our school; generally, there is one straight grade three class and one split class with between ten and fifteen more grade threes in it. Rarely are there more than forty kids. Let's say, just for the sake of some nice round numbers, that there are usually about 40 grade threes at our school. The provincial goal is to have 75% of those kids meeting or exceeding the provincial standard, which is a level three, or a B. So, thirty kids have to make the grade for the school to succeed. But it's done as an average, which means that if one kid totally bombs and there is no 98% kid to compensate, everyone's scores get pulled down. Similarly, the testing must all be completed within the week. If a child misses a day early in the week, they make it up on Thursday or Friday. A kid who is under the weather on Monday stands a decent chance of still being under the weather on Thursday, but they have to write the test anyway. All it takes is for one or two strong students to stay home, and all of a sudden our scores are dramatically skewed. Conversely, the test is to be written by every grade three in the school - including the kid who just returned from a lengthy (read: before Christmas) vacation in Pakistan. He came back on Tuesday. Yes, he had to write the test.
Simply put, forty kids is too small a sample to get a good picture of what is going on in a school. Even sixty kids is too small a sample. You'd need a lot more than that, and you'd need to correlate it with other information. For example, the child's state of health on the days of the test should be noted and compared against their score when it comes back. The child's diagnosed learning disabilities should be taken into account, as well as the suspected ones that are not yet diagnosed - to be correlated with the test when the diagnosis, if any, is complete. You have to be pretty seriously LD to have a diagnosis by the end of grade three in our board of ed. In order to be eligible for psych testing, you have to be at least two grade levels behind, meaning a grade three student in September has to be operating at a kindergarten level before anyone will look at him. That means that milder LD kids are struggling through the test unaided. Ditto the kids whose parents haven't taken them to the doctor in the three years we've been asking them to do so. Many LDs require a pediatric assessment for a firm diagnosis. No assessment, no LD, and no help. It's just one more way that children of low-income/poorly-educated/slightly neglectful parents end up at a disadvantage.
The usual estimate of LD in the population is 10-15%, but that is skewed towards lower income levels, meaning that a school like ours might have an LD level closer to 25%, while schools in richer neighbourhoods may have less than 10%. So if 25% of our grade threes potentially have some LD, that's ten kids. Assuming that those kids are highly unlikely to be operating at grade level (because if they were, they wouldn't qualify for an LD diagnosis) that means all of the others have to make the grade in order for our school to pass muster. As it is, four of our grade threes have diagnosed LDs. We've already beat the 10% number, and we haven't reached the age yet at which the more subtle, less-than-two-years-behind LDs are commonly diagnosed.
Then there's the ESL problem. 50% of our kids come from homes where English is the second language; about 80% of those speak English only at school. That's sixteen grade threes who spoke no English at all before arriving in junior kindergarten or kindergarten, or perhaps upon arriving in Canada more recently than that, and four more whose parents' English is not entirely fluent. The research suggests that second-language learners of average intelligence or better will catch up by grade six or seven, assuming they start learning English in junior kindergarten - that is, seven to eight years. After that, many of them will actually surpass their unilingual peers, which means their tests in grade nine should show about 70% of the second-language, Canadian-born kids to be ahead of their grade. (Seventy percent, because we're only talking about those of average intelligence or better. Throw in a communication LD or a slow-learning kid, and it'll take longer, if indeed they ever catch up.) That means we're testing twenty kids whom we know will not be at full proficiency in English, because they haven't spoken it long enough - and we're testing them with standards designed for native speakers of English. Even those ESL students on the upper end of the intelligence scale are unlikely to have caught up enough to meet the provincial standard with any consistency on EQAO in grade three, since all the tests - reading, writing, and math - require language proficiency. In our current school, there is no crossover between the diagnosed LD kids and the second-language ones. That's 24 kids out of 40 who we know before we start are at a disadvantage, plus at least four more who are undiagnosed but on the list for psychoeducational testing - because we know something's up with them but we don't know what. We're at 28 kids. What was that magic number again? 30 to make the grade? It's looking unlikely at this point, isn't it?
There's one huge, massive, colossal hole in the way EQAO scores are used, and as a grade four teacher, I've asked about it. You see, in many boards across Ontario, including ours, kids receive a truly standardized intelligence test in grade four. The test varies, but our board is currently using the CCAT. It tests two things: vocabulary, and spatial awareness. The tests are usually of the correlation kind: "Small is to large as _________ is to wide." Then they choose from a list. The tests together give a decent (though not perfect) picture of a kid's IQ, and are scored on a normative curve. The scores are used for only one thing: they serve as initial screening for potential entrance into gifted classes. A child must get eight on one test and nine on the other, for a combined score of at least seventeen (a high sixteen might be taken for more testing) in order to qualify. Approximately one percent of kids qualify according to that standard, and in my opinion, it's too strict. But that's an issue for another day.
What no one has done, because the political climate is dead set against doing it, it to correlate the EQAO scores from the spring, with the CCAT scores from the following fall. I'm seriously considering doing it on the sly this fall with my grade fours. It shouldn't be hard to show a correlation. I'd bet good money that the kids who do poorly on the verbal test will be the same kids who did poorly on EQAO. The problem is that the politicians don't want to use EQAO this way.
So how do they use it?
They use it to rank schools, and to accuse teachers in poor-ranking schools of being bad teachers. They claim it's all our fault. Studies show that good teaching trumps all other factors, they say.
Actually, that's not what the studies show. Studies actually show that the most important factor is the education level of the parents, particularly of the mother, and the second-most important factor (which is, unsurprisingly, closely linked to the first) is socio-economic level. The strength of teaching is a distant third in that equation.
In any case, we should be starting to see a difference over the next few years. Boards across our province have been granted small fortunes for training and resources, all aimed at getting those EQAO scores up. And in some areas, it's working. Many schools have seen dramatic increases in their EQAO scores. Most have not seen increases quite as dramatic as the province wants, though. The number of schools going from a 30% pass rate to a 75% pass rate is vanishingly small, and the demographics suggest that schools like ours usually aren't among them. More often, the teachers work themselves into a nervous breakdown and achieve an increase of about 20-30% - that is, about 50% of the students making the grade. Those are huge gains. You'd think the province would be happy for every last kid who succeeds in that way. But all the teachers hear is that the scores still aren't high enough, we still aren't meeting expectations, we still haven't done enough.
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
(no subject)
Date: 2007-05-27 08:13 pm (UTC)how are they even still in the class when they've missed almost a whole term? that's insane!
(no subject)
Date: 2007-05-27 08:24 pm (UTC)They were legally withdrawn from school after they'd been gone four weeks. When they return, we put them back into age-appropriate classes, which is to say, the same class they were in last fall. But they're in grade three on the day of the test, so they take the test.
I agree. It's every bit as insane as trying to get that poor pre-grade-one student to write the test. We know ahead of time which ones are going to fail - the only real question is about the kids in the middle and the top, whether they'll perform up to the extent of their ability. Surprises are rare if the teacher is any good, because it's our job to know our students, to know their needs and weaknesses, and to be addressing those in class.
We set them up for failure, in the name of being "fair." That's not fair. The whole thing is unfair. Fair would be testing them at a time and in a way where they have a fighting chance of succeeding, and we're not doing that.
(no subject)
Date: 2007-05-28 01:52 am (UTC)All those circumstances that skew your results are present in other schools as well. If one school (presumably with similar roadblocks) can post an 80% then it stands to reason that yours could as well.
Obviously some stats like the percentage of ESL kids do affect you unfairly, but in other schools the smart kid can have a sick day that week too.
(no subject)
Date: 2007-05-28 02:25 am (UTC)(no subject)
Date: 2007-05-28 02:55 am (UTC)(no subject)
Date: 2007-05-28 09:55 am (UTC)