The Expertise Squeeze?

2016 was the year of surprises. One of those surprises was Brexit. It was a surprise for David Cameron. It turns out that if you spin the referendum roulette wheel often enough, your luck will run out. Based on their preparation for the consequences, it was also a surprise for Leave campaigners. While many shocking things were said and done during that campaign, one sticks out for me. On the 3rd of June 2016, a few weeks before the referendum day itself, Michael Gove was interviewed by Faisal Islam, a Sky journalist. Islam had just listed a series of authorities – including the leaders of the US, India, Australia, CBI, IMF, NHS and unions – that had counselled against Brexit. Gove responded:

“I think the people of this country have had enough of experts with organisations.. from acronyms.. saying that they know what is best and getting it consistently wrong.”

Gove went on to say that those experts “have a vested financial interest in the European Union” and that this interlocutor was “on the side of the elites”. In turn, his interviewer labelled him “Oxbridge Trump”.

And speaking of the US president, two months earlier, the Real Donald Trump had said at a rally in Wisconsin:

“You look at what China’s doing in the South China Sea, and they say, ‘Oh, Trump doesn’t have experts,’ Let me tell you, I do have experts but I know what’s happening. And look at the experts we’ve had, OK? Look at the experts. All of these people have had experts. You know, I’ve always wanted to say this—I’ve never said this before with all the talking we all do—all of these experts, ‘Oh we need an expert—’ The experts are terrible.”

Trump has gone on to govern with the same erratic attitude to competence that characterised this utterance – every few months, he will come out with a similar statement about a policy area: “Who knew that healthcare / North Korea / (insert policy issue here) was so complicated! Who knew!” Who knew? Who indeed. Experts are only one group among many singled out for attack by Trump so it seems almost unfair to mention this event.

The Australian government does not quite have an equivalent of a Gove or a Trump “expert” statement. While the likes of One Nation promote all kinds of strange ideas and say all kinds of outrageous things, they remain a fringe phenomenon. Meanwhile the current Coalition government may often be at odds with experts on topics as diverse as asylum seekers, energy policy and taxation but this is not unusual in a democracy. However there was an utterance by Prime Minister Malcolm Turnbull that came close to Trumpian heights of absurdity. It relates to the regulation of cryptography:

“The laws of Australia prevail in Australia, I can assure you of that. The laws of mathematics are very commendable, but the only law that applies in Australia is the law of Australia.”

Excluding Australia from the realm of mathematics is certainly an… innovative move (and Turnbull has assured us that he is all about innovation). This is unlikely to end well. If mathematics is pitched against the fear of terrorism then mathematics is sure to lose. Maths may have Number Theory but Fear has the actual voter numbers. When even mathematics is politicised (a field of endeavour at once immensely powerful and often wilfully disengaged from the quotidian), you have to wonder what remains out of bounds.

The public expert-bashing has yielded a number of responses. At least three books have been published with the title “Post Truth” this year (spoilers: the authors are not in favour of it) and US academic Tom Nichols has published a book called The Death Of Expertise. Nichols posits that the general public have turned against experts and now arrogantly glory in their own ignorance. The claimed causes of this are higher education, the internet and the media. The expansion of higher education and the proliferation of low quality learning institutions has led to a graduate glut of ill-educated individuals who believe that their near worthless degrees give them the same standing as ivy league professors. The internet makes available a wide variety of content, only a minority of which is actually true. The credulous can easily find “evidence” to support their viewpoints no matter how bizarre. Finally, the media used to be a mainstream environment focused on truth and has now become a fragmented, partisan free-for-all where experts are chosen for their ideological purity rather than their subject matter competence. Experts become mere technicians called on to fix problems rather than sources of insight and authority for the populace.

If this weren’t depressing enough for experts, a group of cognitive scientists and technologists including Gary Klein, Ben Shneiderman, and Robert Hoffman identify a completely different set of threats. For them the threats against the validity of expertise come from researchers and practitioners in the fields of the sociology of knowledge, decision research, heuristics and biases, evidence-based performance and information technology. While some of these viewpoints have little sway outside academia, some of the others are having a major impact in government and business. Evidence-based performance attempts to replace the judgment of professionals with “best practice” – perhaps documented in a checklist. Decision research replaces expert judgment with simple (“linear”) statistical models. Information technology goes further and claims that human beings can be completely replaced by artificial intelligence, big data, and automation. The promises made on behalf of cognitive computing, machine learning and big data are extensive. An often-quoted University of Oxford study puts 47% of US employment at risk of automation – that includes professionals such as insurance underwriters as well as filing clerks. These threats do not come from the plebian masses that concern the post-truth crowd. Instead they come from above, from the organisations that hire and develop experts.

Why are attempts to displace experts attractive to organisations? Well, the place of experts in organisations is an ambivalent one. On the one hand, experts are often the public face of these organisations. Companies and public bodies may compete to hire the most prestigious in their field. They won’t necessarily use them wisely once hired but bragging rights count for a lot – at least until the initial ardor fades and someone new comes along. On the other hand, experts are expensive and fragile. If they genuinely add a great deal and become indispensable then managers are presented with a significant risk. People get sick or depressed or distracted by family problems or tempted by the offers of competitors. So the thought of getting the benefits of your experts without vulnerability to their downsides is deeply attractive. Hence the investment in the original expert systems that promised to replace experts but largely failed. Some of the new Machine Learning (ML) approaches will likewise end badly but others are already displacing professionals. The large accounting firms take on far fewer graduates than they used to. They no longer need armies of 20-somethings with clipboards to audit companies – much of the work can be done by sucking data out of finance systems and running statistical tests over it to detect malfeasance. The junior auditors have been weighed in the balance (sheet) and found wanting.

So it appears that experts face pressures from two directions. On the one hand, an ignorant public (and the populists who court them) wish to eliminate experts and even the notion of expertise while technologists and managers wish to replace them with machines. Squeezed into irrelevance – who would want to be an expert?

In considering these claims about the death of expertise, we should take a step back and look at some data (we should not use populist methods to investigate populism). Does the public see all experts as the same? Has trust in experts collapsed in recent years? Roy Morgan’s annual survey of the image of the professions actually indicates rising trust in doctors, university lectures and engineers over the last 40 years. It’s not gold stars all round however. Bank managers and ministers of religion have seen significant falls over the same period. Politicians do better than used car salesmen (the lower limit of trust it would seem) but only just. A recent poll by Australian National Centre for Public Awareness of Science shows that the overwhelming majority of the public believe that doctors, scientists and engineers both contribute to the wellbeing of society and should be considered prestigious professions. CSIRO studies on public attitudes towards climate change indicate that the most trusted sources of information on the topic are university scientists and the least are oil companies (although for those that don’t believe climate change is happening, their most trusted source is their family and friends). While the Edelman Trust Bar discusses low and decreasing trust among Australians for government, media and business; academic and technical experts are seen as the most credible spokespeople that the survey considers – higher even than “a person like yourself” and way higher than government officials or businesspeople.

The overwhelming message from this range of sources is that the public does not view all authority figures as equal and that trust in experts has not collapsed – in fact it some areas it has risen.The fall of the clergy is likely caused by the decline in religious observance and exacerbated by the ongoing child abuse scandals. If your business is morality and care then carelessly and immorally protecting predators is fatal. Bankers seem to have traded public trust for profits and wealth. It will be fascinating to see what happens if the profits run out and they are thrown back on the goodwill of the public. Overall, however, none of this implies a widespread rejection of expertise although it does indicate that a number of key social and economic institutions have become disconnected from the population at large.

Now there is a difference between simply saying you respect someone and actually following their advice. Looking at the example of medicine, almost four out of five Australians say that they use the internet to research health information (compared to a third in 2012). More than half of Australians say that they look up information about health conditions on the internet to avoid seeing a medical professional. On the other end of the stethoscope, a fifth of GPs report that “patients dictating their treatment” is an issue (about the same number as answered “maintaining electronic system” – technology can be as much a challenge as an enabler). This behaviour begs the question of what patients want from doctors – medical advice, access to a prescription pad, someone to talk to? We are not about to give up our relationship with our doctors but that relationship may change – as we shall see later.

If one issue has crystallised debates about medical expertise, health information and public safety, it is vaccination. In the 1990s, British doctor Andrew Wakefield claimed to have discovered a link between the widely-used Measles, Mumps and Rubella (MMR) vaccine, bowel disease, and autism. Over the following 20 years, medical authorities found his research fraudulent and he was struck off the UK medical register. Wakefield moved to the US where his ideas have been championed by celebrities such as Jenny McCarthy. N.B. There is no reliable evidence that vaccines cause autism or that not getting vaccinated is better for you than getting your shots. In Australia, until relatively recently anti-vaccination groups have been given media time in the interests of balance – but they are now effectively pariah organisations.

How have these debates resonated with the public? A survey of parents carried out in 2012 indicated that 94% of parents support vaccination of children, 90% believed vaccination was safe and 83% of parents obtained information from their GP. However, there were were other results that complicated matters. The internet was the third most widely used information source for information. Avoiding or delaying vaccination was related to seeking advice from an alternative health practitioner. Significant numbers of parents had concerns about vaccines weakening their child’s immune system or potentially causing autism. Doctors have to operate in world full of competing information but they start from strong position of trust. There is a risk that in focusing on the small number of parents who refuse vaccination, the broader concerns and disconnections with parents are lost. These concerns can only be partially assuaged with facts. Parenting is an intensely emotional (and only occasionally rational) activity. The broader question is whether health professionals have both the skills and the time to work through these parental concerns that may undermine this important program.

Of course Michael Gove was not berating doctors in his broadside against experts (although the relationship between his government and the UK medical profession has not always been cordial). He was specifically picking on the economic experts of the “acronym organisations” (e.g. IMF, ECB) who are associated in the minds of many of the public with the Global Financial Crisis of 2008 and the Eurozone Crises of 2010. These were supposedly the smartest guys in the room and yet the world almost collapsed. Millions of people have lost their jobs or their pensions as a result of fraud and incompetence on the part of these experts.

Gove has been roundly condemned by many commentators for his “experts” comment and its implied anti-intellectualism. However Gove’s jibes have weight because the authorities that he challenges have made serious mistakes. This is not to say that Gove is right (I happen to believe the UK has made a mistake around Brexit) but that he is not wholly wrong. Gove’s attacks deserve to be taken seriously especially where he identifies the limits of expertise and the dangers of technocracy.

Firstly, the limits of expertise. If not all experts are trusted equally by the public then not experts perform equally well. Many people call themselves them an “expert” (just as many of us call ourselves “attractive” or claim a “GSOH”). Some fields seem more amenable to developing expertise than others. Whereas weather forecasters and physicians can develop reliably high performance, clinical psychologists and intelligence analysts are restricted in their abilities and stockbrokers do no better than chance. Psychologist James Shanteau calls these “domain differences”. Some fields are more mature than others in terms of their understanding but also some phenomena are more tractable than others. There are some domains where our expertise is likely always to be limited. Managing complex modern economies will be one such domain. Therefore those claiming expertise in this field require humility and the ability to simultaneously realise that they are probably wrong on any number of issues while still acting where they need to. Like consumers of alcohol, experts should know their limits.

The other risk that Gove alludes to is technocracy – removing political decisions from the hands of the public and elected officials and placing them in the hands of unelected experts. While proponents of rule by expert goes back at least as far as Plato, Technocracy as a movement with that name arose in the early years of the 20th century. American engineers such as Howard Scott proposed a society based on efficiency and engineering insight – where money was replaced by energy certificates with energy as the key metric of value (echoing some of our current energy debates). Technocracy as a public movement hit a peak of popularity at the start of the Great Depression – at about the same time that anti-democratic populist movements reached the peak of their popularity in Europe. Technocratic and anti-intellectual populist movements can be seen as differing but related responses to the failure of democracy.

While they have been portrayed as opposites, both offer something similar – the promise of subduing the mess and tumult of democratic life with order. Order through personal authority and charisma for the populist. Order through the authority of knowledge for the technocrat. Populist and technocrat are two sides of the same coin and we should not be surprised to see the emergence of politicians that attempt to wield technocracy and populism at the same time. Indeed, this is what Trump claimed he was going to do – fill government positions not with time-serving Washington insiders but with brilliant business people whose expertise in deal-making and running organisations would transform government and bring it closer to the people. This does not seem to be what has actually happened in practice (as might be expected from Trump’s earlier comments on expertise). Instead he has selected right-wing ideologues and several very wealthy executives whose experience has not transferred well to their new environments. It turns out that the State Department is not that similar to an oil company (although international trust may also be a non-renewable resource).

It should be noted that technocratic governments do not necessarily need to be led by engineers. In Europe, the technocrats are generally economists and financiers (often with a Goldman Sachs pedigree). The balancing of interest rates, employment, exchange rates, debt, taxes, investment, etc are seen as the primary work of government. However in China, the political elite are “traditional” technocrats in that they have degrees in engineering. The last three presidents of China were trained as a chemical engineer, a water conservancy engineer and an electrical engineer. At the same time, the Chinese state combines its technocracy with populism – promoting a heavily nationalist worldview that is happy to demonise those who are not Han Chinese inside and outside its borders.

There is a great danger of technocracy by itself or the muddier variants of technocratic populism and that is due to the previous problem – the limits of expertise. As we saw earlier, some problems are just not that tractable. The big challenges that societies face are “wicked problems”. Problems like climate change and inequality do not have neat technical solutions, they are fundamentally messy, they are ambiguous, they change dynamically and they require a collective will to action. Experts cannot provide this collective action by themselves. They cannot prevent climate change or reduce inequality. They can engage and contribute but they cannot solve these solutions in an individualistic, heroic fashion.

While wicked problems highlight the limits of experts, they also provide them a way to escape the squeeze. The current generation of artificial intelligence systems are not designed to handle complex problems that require unpredictable engagement with human beings. Humans will need to engage with each other to find solutions to our challenges (although they may use information technology to investigate these solutions). However this means that experts need to be willing to work at communication and engagement to ensure that their expertise has an impact. I am not suggesting that experts should be become politicians (that would be self-defeating – and the world has enough politicians already). Rather that their future will involve a broader skill set than they have been previously used to. Not all of these wicked problems will be dealt with at the most rarefied levels of state (in fact, one of the characteristics of a wicked problem is that it needs to be dealt with at multiple levels) . Working with someone to help them lose weight or manage a chronic condition like diabetes or persuading them that their kids won’t be hurt by their vaccination shots can be a complex task where dispassionately relaying the facts is not enough. Much of the technical detail may be outsourced to machines with more a comprehensive and up-to-date knowledge base however the professional will still need have a broad understanding of the field, will still need to spot where the detail of the context needs to be taken into account and will still need to guide another human being through difficult and traumatic choices.

As Atul Gawande (surgeon, public health policy wonk and possibly the greatest writer on medicine and humanity living today) said about the difficult conversations that doctors need to have with their patients (esp. about end of life issues): “One reason there’s more surgery and less discussion is that the health system will pay a doctor a lot for doing a surgery and basically nothing for having a frank, sensitive, hard conversation about end-of-life choices… we really reward me for being a surgeon and this debate about whether we are going to make it possible for people to be rewarded for being really good at these human sides of the skills”. As someone who has had to either work with or manage experts throughout my career, I would observe that the incentives for experts are not just financial. For many experts, their sense of identity and self-worth is intertwined with what they know and what they can do. They have invested time and energy into developing these parts of themselves (and have been rewarded with money and respect for doing so). Some have also developed wonderful communications skills as well – but many have not. And those skills of communication, engagement and mobilisation will only become more important for them. The force of automation will change the shape of expert knowledge.

The impact of machines on experts will not uniform and neither is there only one possible response to them. Richard and Daniel Susskind identify seven different models for the future of professions – ranging from the traditional “trusted adviser” model to through to increased work undertaken by para-professionals, and the embedding of expert knowledge into machines. Tom Davenport and Julia Kirby present different options for those facing technological displacement including becoming a manager of the technologies, focusing on activities that technologies do not cope well with (see the previous comments about conversations) or designing the technological systems themselves. The options open to a professional will vary not only by professional but by the various specialities within that profession. It will change the tasks that experts undertake.

It will change the attitude of the public as well. The Susskinds talk about the rise “quasi-trust” and “trusted solutions” displacing the present notion of the professional as “trusted adviser”. When people are replaced by products, we no longer rely on the moral characteristics of professionals but on the reliability of the solution. I think they have a valid point but this won’t be uniform. While it is true that you only worry about your car being evil if you’re in a Stephen King novel, there are some actions that we will feel comfortable leaving to a machine and some that we will not. While I might be happy to get a robo-generated will, I might not be happy to discuss end of life care with a machine.

The expertise squeeze is real although its extent is exaggerated by pessimists. The squeeze will change the shape of experts – in terms of how they learn, what they need to know and how they must engage with others. The squeeze will change the shape of society – the shifting fortunes of different professionals is tied to the perceived value they add to our lives and the ability of the institutions they represent to handle scandal and bad decisions. The challenge for the rest of us (non-experts) will be our willingness to engage with, and sift through, different sources of advice. In a world brimming with information, knowledge is no longer power. Discernment and judgement come to the fore. We must choose wisely.

Sources & Further Reading

Tom Nichols – Death of Expertise –

Klein et al – The War On Experts –

Future of Employment:

CPAS poll:

Roy Morgan Image of the Professions:

Climate Change  Survey;

Edelman Trust Barometer – Australia:

Medical & patient behaviour:,-beliefs,-behaviours-and-concerns-towards-childhood-vaccinations-in-australia-a-national-online-survey/

Why task domains (still) matter for understanding expertise –

Only Humans Need Apply:

The Future of the Professions:

This entry was posted in Articles & Papers. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s