< Regulars | 03.01.2018

The bad habits of AI

The bad habits of AI

Duncan Minty examines the increasingly urgent need to address gender bias in artificial intelligence.

Insurance is starting to be transformed. Many aspects of underwriting, claims and distribution are being remodelled. This transformation is being powered by the massive amounts of data we are producing as part of our everyday lives.

Success is measured not by the size of that data, but by the insight you can draw from it. For this, insurers are using a variety of tools under the heading of artificial intelligence (AI). Algorithms are one such tool, being a structure of mathematical formulae for identifying relationships within data.

As data used by insurers grows vast and varied, the capacity of humans to create handwritten algorithms is slowly but surely being reached. The solution has been machine learning, in which algorithms learn for themselves where those relationships within data lie. For machines to learn, they must first be trained on historical data.


Underwriters are using algorithms to expand the risk factors that influence quotes and to reduce the questions needed for a quote. Claims people are using them to reduce the time to settle claims and to spot fraudulent claimants. And marketing people are using algorithms to personalise marketing campaigns and predict consumers’ behaviours.

Three trends are worth noting. AI introduces a more automated level of decision making. It also pays attention to the feelings and contexts that influence our decisions. And it is using machine learning to convert social media chatter into virtual identities.

The common denominator here is the vast amount of historical data needed to create this automated reality. This comes from the decisions and opinions we make in our online lives. While many of these will be conscious decisions, the cleverness of AI lies in identifying the unconscious element that often underlies those conscious decisions.


If you train an algorithm on historical data, it will learn not only the good decisions we have made, but the bad decisions as well. It will learn the biases in society. Research published in the April 2017 edition of the journal Science illustrates this.

Researchers looked at an AI tool called ‘word embedding’, which is transforming the way computers interpret speech and text. What they found was that word embedding helped an algorithm learn how words for flowers were clustered around words linked to pleasantness, while words for insects were closer to words linked to unpleasantness. They then found that words like ‘female’ and ‘woman’ were more closely associated with arts and humanities occupations, while ‘male’ and ‘man’ were more closely associated with maths and engineering occupations. What this tells us is that AI will learn the gender biases in historic data and then propagate it into the automated decisions that insurers make in underwriting, claims and marketing.


Propagating gender bias across insurance decisions could result in women paying more in premiums, receiving less in claims settlements and facing a greater exposure to mis-selling. Such detriment will vary from insignificant to significant, from occasional to widespread. That is missing the point though. Gender bias in analogue decisions is illegal. It is just as illegal in digital decisions.

But surely insurers already have controls in place to guard against such bias? Only to a degree. Two trends make this a real risk for insurers. Firstly, some of the insight that AI generates will come from real historical data, while some will be ‘manufactured’ from the clusters of correlations that algorithms identify. And secondly, both those real and manufactured insights will then be injected into automated decision making, the workings of which become more complex and opaque.

The challenge this presents is of ‘black box’ processes giving insufficient consideration to ethical concerns. Detriment blends into the normality of busy firms using complex systems. Complexity may be the new norm, but that does not mean gender bias should be so too.


Insurance firms should work together on a structured response to the challenge of gender bias. Key principles should be adopted and business leaders should publicly emphasise the need to follow them. Tools for testing historic data and algorithms for discriminatory features should be adopted. Responsibilities should be allocated and training provided. Suppliers and business partners should be told to follow suit.

Individual professionals should keep in mind their obligation to work in the public interest and to uphold the Code of Ethics. Understanding the challenge and how it might affect their particular responsibilities is the starting point. Pressurising for positive change is the next step. And providing leadership across the sector to address the issue is a natural continuation.

Many insurance leaders are extremely concerned about how trust could influence their firm’s growth. Tackling gender bias in automated decisions is an issue that needs to be part of every insurer’s trust agenda.

Duncan Minty is business ethics consultant at Duncan Minty Consulting


Related articles

It’s time we measured ‘trust’

It’s time we measured ‘trust’

The new Public Trust Index from the CII examines how trust in insurance can be measured for the benefit of members and the wider public.

Time for change?

Time for change?

For the past 100 years or so, the shareholder has been king but now there are signs of change, with ever-growing emphasis on the wider stakeholder. So what are the implications for the business of insurance?

Interrupt response time

Interrupt response time

In contingent business interruption, the same sort of claims appear again and again and yet it seems the sector continues to use CBI as a sweetener. Is this the year the profession learns the lessons of the past?