Your Child’s Writing Test Graders Were Recruited on Craigslist

Hey, all you newly graduated English majors, how would you like a job grading kids’ writing tests? Perks are that you get free cocoa, a large break room, and you can wear comfy clothes. Pull on those pajama pants and a hoodie. It’ll be just like pulling an all-nighter! No experience necessary.

You’re probably aware by now that Oklahoma students’ writing tests are not being used to come up with schools’ A-F rating because there have been too many suspicions raised by educators about the scoring. For example, many students who are poor readers made high test scores on the writing test and vice versa. Also, education officials noticed that  too many of the writing test scores were the same.

I can understand. I can’t blame those low-paid test graders for fudging things a bit. I remember when I was teaching, and I had that huge stack of essays sitting in front of me. Ugh. Hours and hours of grading, evaluating, writing extensive notes to students on their papers… Sometimes if I were wearing my comfy sweatpants and having a nice, hot cup of cocoa, I might slip into a little nap now and then, resulting in some pretty sweet essay scores for my students.

While the test-grading job might be relaxing, it isn’t exactly high paying. (Where, exactly, are the millions of dollars our state is paying testing vendors going?) According to Oklahoma Watch (article below), Kelly Services (a temp agency) is responsible for recruiting graders, and the temp job pays $11.50 an hour. Well, that’s more than a public school teacher makes, but it is temporary, and teachers have the opportunity for that cushy tenure. My daughter is a recent college grad. and is looking for a temp job while she’s in graduate school. I’ll have to tell her about this one. It beats waitressing.

Of course, trained and experienced educators (otherwise known as “teachers”) can’t be trusted to evaluate the writing ability of their students. Legislators and standardized testing vendors are the real experts. Teachers must spend their classroom time preparing their students for these high-stakes tests, which will tell the true story of their students’ abilities. Those pesky teachers aren’t professional enough to accurately grade kids that they see all day every day for nine months.

These high-stakes, standardized tests are great measures of our children, aren’t they? Parents, why are we accepting this?

Here’s the whole story by Nate Robson from Oklahoma Watch:

Earlier this week, state Superintendent Joy Hofmeister said she was dropping the state’s writing test from the A-F grading equation for schools this year because the test and results were unreliable.

The test vendor wasn’t an issue, she said.

Oklahoma Watch, however, learned that school officials are questioning the qualifications of people grading the tests. The graders were hired by Kelly Services, a temp agency under contract with Oklahoma’s new test vendor, Measured Progress. That agency also was used by the state’s previous testing vendor, CTB/McGraw-Hill, which was criticized last year for improper grading of the writing exams.

This marks the second year in a row that the writing scores for fifth and eighth graders have been thrown out of the letter-grade calculation by the state Department of Education.

Education officials have said too many of the scores were similar or had glaring discrepancies, such as students who do poorly in reading scoring higher on the writing test than those who read well.

According to a job posting on the Kelly Services website, the pay for test graders needed in a number of states, including Oklahoma, is $11.50 an hour. Applicants must have a bachelor’s degree in English.

The company benefits include free coffee and cocoa, a large break room and casual dress.

The headline for the company’s ad reads: “Welcome! You have taken the next step towards putting your college degree to work!”

Officials with Kelly Services and Measured Progress said any training or education requirements are set by the state Department of Education, and that extensive training is provided.

The scorers are also evaluated daily, and each test is scored by two people.

The graders work out of scoring centers in Dover, New Hampshire, Longmont, Colorado, and Menands, New York.

Kelly Services has also been criticized for recruiting on Craigslist.

Clinton Public Schools Assistant Superintendent Tyler Bridges, a former eighth grade English teacher, said he was shocked by the minimum requirements and the fact that Kelly Services would recruit on Craigslist.

Bridges said he had concerns about the qualifications of test graders, especially if they are fresh out of college and have no experience.

“That just kind of blows me away,” he said.

CTB/McGraw-Hill and Pearson have faced similar criticism over the hiring of test graders.

Measured Progress was awarded two contracts in December worth a combined $48 million through fiscal year 2019 to handle Oklahoma’s testing.

Judith Rubenstein, a spokeswoman for Measured Progress, said in an email that Kelly Services recruits, screens and trains the scoring staff. Measured Progress then directly supervises and operates the scoring process.

The scores are based on the state’s grading rubric.

Writing tests are more difficult and expensive to grade than standard multiple-choice tests because a person must review and score each student’s passages.

Department spokesman Phil Bacharach said this year’s concern is that the scores don’t accurately reflect students’ abilities.

Bacharach said he did not know if department officials knew Measured Progress used the same company to hire testers, but added the state does not have any concerns with the testing company or the test graders.

“Bottom line is, the first thing the superintendent wanted to address is specifically with the issues with the results of the test itself,” Bacharach said. “This is not an issue with the people.”

However, several school officials said the similarity of so many writing scores indicates a problem with the grading, not just the test.

Rick Cobb, assistant superintendent of curriculum and instruction at Moore Public Schools, said he believes poorly trained or qualified test scorers are part of the problem. Too many of his students had similar grades on the exam, he added.

“The whole process is flawed from start to finish,” Cobb said. “We need to do a better job at training teachers – and that’s on the schools. We (also) need to do a better job picking the tasks for the test. And the test vendors – they need to not do this on the cheap.”

Cobb said he understands the grading job will be temporary because of the nature of testing, but he said the sheer volume of exams companies have to grade each year means they have to invest in the right people.

“If they (Measured Progress) are going to keep paying people on the cheap, we’re going to keep getting the same results,” he said. “We need to see a scoring company not try to gouge the state for profits or skimp on their employees.”

Oklahoma Parent Teacher Association President Jeffery Corbett also criticized the writing test and the amount of time students spent in class preparing for the exam, which takes away from instruction.

“It is time to take our classrooms out of the hands of the billion-dollar testing industry. Our children deserve the opportunity to become innovate, critical thinkers, not just great test takers,” Corbett said.

By Nate Robson

Oklahoma Watch is a nonprofit, nonpartisan media organization that produces in-depth and investigative content on a range of public-policy issues facing the state. For more Oklahoma Watch content, go to www.oklahomawatch.org.