Augmenting White-Collar Work & the Dangers of Blind Trust in Automation

I recently sat down with Peggy Smedley to discuss the work I do with Shaping Tomorrow to augment my work as a strategic futurist with various forms of automation including LLMs which are often called Gen-AI. Peggy Smedley Show: Disrupting White-Collar Work: https://peggysmedleyshow.com/disrupting-white-collar-work

Now, let me give you some context:

While my wife and I were living in Korea, we rented a DVD one weekend. The following weekend, we came back to rent another movie, but we were told that we had turned our previous movie late and that we owed them 3,500 Won.

Well, that didn’t make any sense to us. We turned the movie in on time because, otherwise, we would have still had the movie on a workday. The employee just kept saying that the computer doesn’t lie. In fairness, his English wasn’t perfect, and our Korean was much less competent.

So, we paid the 3,500Won. Simple mistake fixed with the all-mighty won. Who’s mistake? Did we misremember when we dropped off the DVD? Did an employee fail to process the movie on time? Was there a fault in the computer system? Who cares? It cost us no more than 3.50USD.

However, that employee’s face every time he said, “the computer doesn’t lie”. Still replays in my head sometimes. The unflinching, unashamed trust in a machine with no thought for error, whether human or technical, is ridiculous.

I saw the same unflinching, unashamed trust in 2015 when the then CEO of Shaping Tomorrow told me that they were moving away from paying for human labor. They were ready to automate everything. And, it was the same trust that I saw the same man express when he left Shaping Tomorrow to start a competitor company after Shaping Tomorrow kept employing me and others. To be fair, he initiated the contracts with me and the others himself, but he also expected to start a new company from scratch that would meet business needs without human oversight.

I have no desire to shame him because he’s a good man, but he was duped into this persistent belief that machines don’t lie. The funny thing about the company’s advances in automation excelling in 2015 is that the Fujitsu/Post Office scandal in the UK was also ramping up that same year after the final legal conviction for theft, fraud, and false accounting.

UK subpostmasters were told that they were lying and stealing from the Post Office. The UK Post Office provided their subpostmasters with a computing system called Horizon which was developed and maintained by Fujitsu. The first issue with Fujitsu’s computing system was reported in 2000. It took nearly 1000 subpostmasters reporting imbalances over 15 years before the UK government began to wonder if maybe, just maybe a computer was lying. Ten years later, and this saga still continues.

The computer doesn’t lie. Always trust the computer. Let’s be clear, I’m not saying that computers are anything but useful tools, very useful indeed. However, we should never assume that the machine’s output reflects reality. Indeed, this is especially true when a machine is specifically designed and programmed to fool us such as large language models (LLMs), social media algorithms, and virtual reality. The same is even more true when those who profit from machines exaggerate their capabilities and deny their faults.

Are you riveted by this article? Listen to the rest as I speak with Peggy Smedley about my own experience using various forms of automation in my foresight work.

Peggy Smedley Show: Disrupting White-Collar Work: https://peggysmedleyshow.com/disrupting-white-collar-work