Letter to the Editor: AI should not be used to grade MCAS essays

The following was submitted by Medford resident Jim Kang.

I learned via this report https://www.nbcboston.com/investigations/ai-grading-massachusetts-mcas/3807392/ that the essays in the MCAS are being graded with AI, and that it has graded at least 1,400 essays incorrectly. And that is only the grading that was obviously wrong and discoverable. There are likely more undiscovered critical mistakes.

It is unsettling that my kid's essays and other kids' essays are being evaluated by models that have no real understanding of writing. They use statistical methods to find correlations between sequences of symbols and past scores. In addition to just plain not working to the tune of 1,400+ faulty scorings, this punishes individuality in writing.

As part of my work, I audit AI evaluations of code written by humans to make sure that no erroneous evaluations get out. I know well the limitations of AI evaluations. I would never, ever allow AI to judge children's work. It is the wrong tool for the job.

Consider this: If an AI model said that your child's essay deserved a score of zero, would you trust it? If you then were able to check it and found that it was wrong, would you just say, "Oh, well, AI models operate stochastically and don't have a real understanding of writing, so this is bound to happen sometimes. The efficiency gains and cost savings for the state's favored contractor make it all worth it!" I would guess you would not react like that. And so, you shouldn't react with indifference to the state turning kids' essays over to AI.

Even worse than the gross misapplication of technology is the disrespect to the students. We ask our students to do their best and use their brains when they write and read. We would not find it acceptable for kids to write an essay with ChatGPT and turn it in or to feed assigned reading to ChatGPT and turn in a summary provided by it.

Yet, DESE (the Department of Elementary and Secondary Education) somehow finds it OK to feed student essays to a model and only actually look at 10% of the students' work.

And it isn't DESE's fate to which MCAS outcomes are tied. DESE has little skin in this game. Rather, it's the students and schools that are harmed. The state can take punitive action against districts and schools that do not score well.

DESE has paid Cognia $36.5 million (https://www.mass.gov/info-details/department-of-elementary-and-secondary-education-appendix-a) last year to run MCAS and has signed a five-year, $150-million extension. For $150 million, they're feeding students' essays to a model and spitting back whatever comes back, checking only 10% of the scored essays.

If MCAS essays can no longer be graded by people, then we shouldn't have them, and we should reconsider MCAS as a whole. However, if we are going to continue MCAS testing, and we want to make it valid, then legislators and administrators must ensure they are graded by people rather than a probabilistic process. Otherwise, we are teaching kids that they must submit to nonsense for which we have no well-founded explanation.

If you live in the state of Massachusetts, please call or write to your state congressperson and senator as well as your school district. The state has done something you might think of as ridiculous, but apparently, they don't know that, so you have to let them know.

Jim Kang

Medford