< Regulars | 03.01.2018

The bad habits of AI

The bad habits of AI

Duncan Minty examines the increasingly urgent need to address gender bias in artificial intelligence.

Insurance is starting to be transformed. Many aspects of underwriting, claims and distribution are being remodelled. This transformation is being powered by the massive amounts of data we are producing as part of our everyday lives.

Success is measured not by the size of that data, but by the insight you can draw from it. For this, insurers are using a variety of tools under the heading of artificial intelligence (AI). Algorithms are one such tool, being a structure of mathematical formulae for identifying relationships within data.

As data used by insurers grows vast and varied, the capacity of humans to create handwritten algorithms is slowly but surely being reached. The solution has been machine learning, in which algorithms learn for themselves where those relationships within data lie. For machines to learn, they must first be trained on historical data.


Underwriters are using algorithms to expand the risk factors that influence quotes and to reduce the questions needed for a quote. Claims people are using them to reduce the time to settle claims and to spot fraudulent claimants. And marketing people are using algorithms to personalise marketing campaigns and predict consumers’ behaviours.

Three trends are worth noting. AI introduces a more automated level of decision making. It also pays attention to the feelings and contexts that influence our decisions. And it is using machine learning to convert social media chatter into virtual identities.

The common denominator here is the vast amount of historical data needed to create this automated reality. This comes from the decisions and opinions we make in our online lives. While many of these will be conscious decisions, the cleverness of AI lies in identifying the unconscious element that often underlies those conscious decisions.


If you train an algorithm on historical data, it will learn not only the good decisions we have made, but the bad decisions as well. It will learn the biases in society. Research published in the April 2017 edition of the journal Science illustrates this.

Researchers looked at an AI tool called ‘word embedding’, which is transforming the way computers interpret speech and text. What they found was that word embedding helped an algorithm learn how words for flowers were clustered around words linked to pleasantness, while words for insects were closer to words linked to unpleasantness. They then found that words like ‘female’ and ‘woman’ were more closely associated with arts and humanities occupations, while ‘male’ and ‘man’ were more closely associated with maths and engineering occupations. What this tells us is that AI will learn the gender biases in historic data and then propagate it into the automated decisions that insurers make in underwriting, claims and marketing.


Propagating gender bias across insurance decisions could result in women paying more in premiums, receiving less in claims settlements and facing a greater exposure to mis-selling. Such detriment will vary from insignificant to significant, from occasional to widespread. That is missing the point though. Gender bias in analogue decisions is illegal. It is just as illegal in digital decisions.

But surely insurers already have controls in place to guard against such bias? Only to a degree. Two trends make this a real risk for insurers. Firstly, some of the insight that AI generates will come from real historical data, while some will be ‘manufactured’ from the clusters of correlations that algorithms identify. And secondly, both those real and manufactured insights will then be injected into automated decision making, the workings of which become more complex and opaque.

The challenge this presents is of ‘black box’ processes giving insufficient consideration to ethical concerns. Detriment blends into the normality of busy firms using complex systems. Complexity may be the new norm, but that does not mean gender bias should be so too.


Insurance firms should work together on a structured response to the challenge of gender bias. Key principles should be adopted and business leaders should publicly emphasise the need to follow them. Tools for testing historic data and algorithms for discriminatory features should be adopted. Responsibilities should be allocated and training provided. Suppliers and business partners should be told to follow suit.

Individual professionals should keep in mind their obligation to work in the public interest and to uphold the Code of Ethics. Understanding the challenge and how it might affect their particular responsibilities is the starting point. Pressurising for positive change is the next step. And providing leadership across the sector to address the issue is a natural continuation.

Many insurance leaders are extremely concerned about how trust could influence their firm’s growth. Tackling gender bias in automated decisions is an issue that needs to be part of every insurer’s trust agenda.

Duncan Minty is business ethics consultant at Duncan Minty Consulting


Related articles

Plenty of initiatives already underway

Plenty of initiatives already underway

The fourth Dive In Festival for diversity and inclusion in insurance will reach more cities than ever in 2018.

How to have your quake and beat it

How to have your quake and beat it

As insurance losses from the recent New Zealand earthquake are estimated to reach up to $3.5bn

Telematics for the people

Telematics for the people

Telematics insurance has helped cut car casualties among 17 to 19-year-olds by 35%, clearing the way for mass-market adoption of this groundbreaking technology