Reading & Reacting: Study Examines Cost Savings Through ‘Machine Scoring’ of Tests

cc licensed ( BY NC ) flickr photo shared by cobalt123

By Sean Cavanaugh @ EdWeek’s Marketplace K-12 blog

This recent blogpost about the potential savings of machine scoring writing tests in EdWeek’s Marketplace K-12 blog was another in a absurd line of thinking. While Cavanaugh is really only reporting here, I just keep wondering how this rates worthy enough to even be addressed.

There is no question that cost is always an issue in education. Yet savings is not a bottom line issue as it often is in business, nor is it a always a value proposition.

The real issue is far more problematic. What exactly is the message to students when educators say something akin to “Your writing is so unimportant that it is cheaper and easier to have a machine score it”?

To the best of my knowledge humans have never endeavored to write prose with the intended audience being a machine. What would be the purpose of doing so even? To pass a test of dubious validity anyway?

Somewhere along the line, we have lost the plot even thinking machine scoring of student writing is a valid or even good idea from the beginning.

Of course there is no irony that the organizations requesting this kind of information are all involved in student assessment in some way.

An even more blackly comic notion is that machine scoring of student writing can be done a 20-50% of the cost of humans. For how long exactly? The first time maybe, but exactly how long will it take before ever “better” technology will be used at an even greater cost and, incidentally, steeper profit?

All the while the students are the ones losing as the demands for data and meaningless scores on even more meaningless tests of writing ability drive teachers to coach students to write for a machine, rather than endeavor to communicate with greater sophistication and clarity in the hopes of being understood by another human being, which is kind of the point of writing anything anyway.

What is really saved and what are the true costs with machine scored tests for writing? It is all a bit absurd, really.

Returning, Briefly, with a Response to Ripley

cc licensed ( BY SA ) flickr photo shared by Dave C

Note: It has been a few weeks since I fell off the blog, which was as much unanticipated as it is missed. I made a fairly major change in my teaching career, that was both unexpected and quite sudden. It is nothing tragic or anything, an opportunity arose and I changed my entire slate of courses for the second semester with a week’s preparation. I am sure I will write more on this later, because at the moment the new courses have completely taken over both my time and head-space.

Nevertheless, I still read about our profession along similar lines as I have been, a bit less than I was used to doing, however. So, when inspired I suspect I will still post on occasion. I just can’t keep up with the pace I had been setting for myself with the new challenge.

In Annie Murphy Paul’s recent blogpost, “Author Amanda Ripley On ‘Rigor’ In American Schools,” the writer presented The Smartest Kids in the World author with the question from a previous post on her blog, “Are American kids working hard enough?” It is an interesting read and prompted me to write the response below. I posted it, but am unsure whether it will get approved or not. So here it is with some minor adjustments.

While I am admittedly not a Ripley fan, believing it perpetuates an overly simplified myth that American schools are failing, as well as implying that we are doing something wrong because American students don’t score as highly on a standardized test that is even more dubious than those with a vested interest would ever admit. Still, there is something quite compelling in her response. Her suggestion, “if [parents and students] understand why it matters, and they find enough meaning, small victories and intrigue to make along the way to make the hard days worthwhile,” is perhaps the greatest insight I have read in her writing.

I would submit that this is where we continually get everything wrong in education. Culturally, we have perverted the reason why education matters to equate with material success and status, indoctrinating students into tragically believing all of their efforts should be about a grade, which leads to admission in a better brand name university, which leads to better employment, which leads to greater wealth and success in life. Yet why would they think that is not true when nearly every message they hear advances a similar or same agenda.

Additionally, we are now embarking on a ridiculous double-down accountability path that celebrates student failures as some kind of getting tough on academics strategy that is farcical and anything but helping students identify “small victories.” One need only read Emmaline Alvardo’s insightful testimony in the comments section for evidence. Worse still, think about it from a student’s perspective. When students score too well on tests, most often the two most obvious adult reactions are that they cheated or the test was too easy. Forget small victories, sometimes they can’t get any victories, and the students that intuitively recognize this tend to quit trying too hard.

Finally, think about it from a student’s perspective for a little while. Where is the “intrigue” in being essentially powerless and stripped of your developing agency when you walk through the doors of a school, where you will be told exactly what you are to “learn,” generally punished when you don’t, definitely punished when you exercise too much independence and resist adults prescriptions for your behavior and body of knowledge?

Standards dictate what you will learn. Teachers dictate how you will learn it (if you are even lucky). Tests will enforce to what degree you learned it. None of these factors, in most cases, present you the option to make decisions that are not about doing what someone has told you to do. Really, how often do students truly get to be in the driver seat of their own learning? Nearly all instances that can be suggested are more the exception than the rule. In fact, we typically even label those experiences as “alternative” in all flavored of the word. Wait, we also call them private schools, where the mandates of politicians and lobbyists have far less sway, because the ability to pay affords some autonomy.

Perhaps what is most profoundly ironic is that many of those that are so quick to quote someone like Ripley, advancing even more deeply flawed edreforms like, greater standardization, accountability, and testing, fail to see how they are antithetical to recognizing value, understanding meaning, capitalizing on small victories, and creating intrigue. I’m not even sure Ripley truly recognizes that. It kind of undermines one of the premises of her book.

Reading & Reacting: Did Comcast partner with Khan Academy just to get new, low-income customers — and then drop the ball? (Perhaps.)

cc licensed ( BY NC ND ) flickr photo shared by jon mannion

By Ted Bauer @ The Context of Things blog

This recent blogpost highlights some of the many potentially problematic aspects of for-profit corporate interests pairing with educational efforts. It is definitely worth reading with quality source links and embedded videos. The players are Khan Academy and Comcast. While Khan Academy is also a company and not a school, albeit a non-profit, their primary aim is an educational one. Comcast is a communication delivery and content company, among other core businesses.

Yet, here is an example of a corporation that exists to generate profit, partnering with an educational effort, ultimately to expand its pool of customers. If people, again more appropriately labeled customers rather than learners, benefit great, but the basic game is that Comcast increases their customer base, as well as extending their dominance in the marketplace, while possibly gaining some positive PR along the way. In some ways, it is a marketing play for both enterprises.

Here’s the basics: Comcast is using the program to promote a program of its own called Internet Essentials, which basically offers high-speed broadband to low-income families for $9.95/month, along with vouchers for discounted computers. OK. The important thing to know there, though, is that this program isn’t something Comcast did out of the goodness of their heart — rather, the FCC forces cable giants to provide low-cost options to underserved communities. It’s a regulatory requirement. The Internet Essentials program, then, has critics – who speak frequently to the idea that the supposed “high-speed” broadband isn’t high-speed at all. It’s actually about 5 Mbps, which is slower than the most basic package Comcast offers in a lot of U.S. cities. And the other program? You can only get Internet Essentials at the $9.95 rate if your child is in the National School Lunch program (often just referred to as “free or reduced lunch”). If your child drops out of that program for any reason, your Internet bill would shoot up to what anyone pays — probably $50-$70 a month if you didn’t add cable.

In sum, this may be a customer acquisition program in disguise — Comcast gets a new base of customers, then bait-and-switches them into higher prices down the road. Problem for Comcast is, millions of people are theoretically eligible for this program and yet … only about 250K have signed up.

Whether or not Comcast is looking to run a bait-and-switch may or may not come to pass. However, there is no question that higher prices will be in the future of all those customers caught up in their nets. Higher prices over time is a given. Anyone that pays Comcast for services can undoubtedly express with great exasperation how their bill has slowly escalated with increased services and features that are added without request or recourse, other than canceling the services.

Maybe most interesting of all is Bauer’s assertion about potential solutions, although I am not convinced that what he is proposing will happen.

We can fix these problems if the people benefiting the most from the problems directed some — not all by any means — of their resources back to the problems, and did so without an ulterior motive. That’s what it seems like Comcast did here. They have money and could have put some of it towards promoting and marketing the package and locking in the package at $9.95 regardless of the lunch criteria. Instead, no.

First, Comcast benefits from the problem. Of course, they want to expand their customer base. However, they are less interested in making infrastructure investments in areas where the customers are less able to pay their desired prices. It is not good for their bottom line to spend more money than they have to especially where there is less likely to be significant returns, as in poor families.

Second, as bad as it sounds, bait-and-switch strategies are the kinds of near-term tactics that can boost quarterly accounting results, further contributing to ever more economic nearsightedness. Students, in this case Kahn’s learners, are not the primary concern and may very well be afterthoughts, in fact. Moreover, for-profit imperatives seek scale and efficiency, which do not necessarily correlate with how anyone learns. Yet these are the the forces that many would like to invite into public education.

It should be no surprise that any company would be interested in solving any problems from which they benefit. In fact, there is every reason to believe that they will perpetuate them as long as possible.

Image: iPad

posted via
and flipped to Teaching Today