Stephen Schwartz has an op-ed in the Daily Telegraph today talking about machine marking in education. All up I agree with his arguments – but this paragraph struck me as being wrong.
Critics also claim that computers will never be able to measure creativity. This is not true. Computers learn to assign marks by mimicking those given by humans. If the marking rules require human markers to assign higher marks to creative essays, the computer will mimic the markers and reward creativity as well.
Okay – now maybe we’re just quibbling over the definition of a “creative essay”. If it is possible to a priori identify what a creative essay is (so for example an original short story crafted by the students) then mechanistically the machine could add a random mark to all the grades and “reward” the creativity. That is trivial – but doesn’t to my mind “measure” creativity and then “reward creativity”.
The issue to my mind is actually identifying creativity when you see it. Now I’m happy to accept that computers and machines generally can work longer and harder than humans, are more consistent than humans, and so on. They are also profoundly and fundamentally stupid. Any deviation from predetermined and pre-programmed norms is a innovation to a machine – but we know not all innovations are valuable or correct. Sometimes things students say are incredibly profound, other times just wrong. I’m not convinced that a machine can tell the difference unless a human foresaw every instance of student creativity in advance and programmed an appropriate response into the machine. Yet if it can be foreseen, how is it creative?