The Machine Stops

I was reminded of this Christmas cracker joke tonight when watching the first post-lockdown Premier League football.

What’s the quietest game in the world? 10 Pin Bowling. You can hear a pin drop.

These days all sports are the quietest sports and it’s awful. Forgive the sports chat but watch me pivot like a god-damn pro.

Men made it, do not forget that

There was an incident in the first game. The ball went in the goal, only just, but go in it did. The referees failed to see this, not that they were they looking with any particular keenness.

These days there is the hawk-eye goal-line system. Ball goes in the goal, referee’s watch vibrates. It just works. It’s hard to imagine a less controversial and less problematic addition to the world of football. A resounding and flawless success story. Until today.

The thing that interests me about this incident isn’t that a piece of technology failed to function. I’m more interested in the human behaviours of when an infallible system, silently fails.

The match officials, both those on the ground and those in charge of calling their mistakes in the video room, didn’t consider for a moment that the system could be wrong. So they did nothing.

They didn’t have to look out for the ball going into the goal because the system will tell them if it does. They don’t have to check replays to see if the ball went in the goal, because they know for a fact it didn’t. The output of the system is treated as fait accompli.

You talk as if a god had made the Machine. … I believe that you pray to it when you are unhappy. Men made it, do not forget that.

Even the players wronged by the decision, who saw the ball go into the goal with their own eyes, seemed to accept their fate with remarkably little fanfare (given the nature of footballers). As if they had been gaslit by the technology and the system’s unwavering faith in it.

Increased efficiency and decreased intelligence

This, of course, happens in everyday life. With increasing regularity and far more significant consequences. It’s the sort of failure I always think of (the failure that never happens) whenever I hear about automated systems making decisions that shape lives. Job applications, benefit claims, healthcare options, judicial proceedings.

The popular on-trend example is an “AI” system with a hard-coded or learned bias, but there are a million and one ways this raises it’s head.

It’s both fascinating and terrifying to me, and reminds me I want to read Hello World and Weapons of Math Destruction and then perhaps some others.

“The better a man knew his own duties upon it, the less he understood the duties of his neighbour, and in all the world there was not one who understood the monster as a whole.”