AI bias poses danger to the financial sector

AI bias poses danger to the financial sector

A new report from a financial technology expert at the Bank of England says that inherent biases in AI systems makes it dangerous to entrust them with too much decision-making at present.

Credit: Shutterstock

The implicit bias of present-day generative AI models make their rapid adoption in the financial sector hazardous, according to a report by a financial technology expert working for the Bank of England.

Kathleen Blake’s report, published Wednesday, splits AI model bias into two categories — bias based on underlying training data, and bias based on the results of model output. Although both reflect the human biases that developers and creators bring to AI models, the former category is impossible to counteract just by getting rid of data points that indicate, for example, femininity or non-whiteness.

Blake likened data bias to the practice of redlining in mortgage lending. In a redlining system, home insurers and mortgage lenders assess non-white customers as “risky” based on their neighborhood, making credit and insurance more difficult to come by for people of color, while not directly attributing denials or higher prices to race. Similar logic is already visible in AI systems, Blake noted.

“[T]he model may make underlying correlations that lead to biased decision-making based on non-protected features,” Blake wrote. “In other words, the remaining, non-protected features could act as proxies for protected characteristics.”

Societal bias, by contrast, is “where norms and negative legacy from a society cause blind spots.” Blake cited an Amazon recruitment algorithm that tended to recommend more male candidates than female, because the data, historically, showed that males tended to be hired more often.

AI bias is particularly dangerous in the financial sector, Blake warned. Trust in the banking system is at serious risk from the aforementioned biases displayed by AI, given that (while noticeably less discriminatory than human decision-makers), AI still charged higher rates to Black and Latinx mortgage customers when compared white ones.

Moreover, the opaque “black box” nature of proprietary models means that, if a large group of important firms uses them for similar purposes, their actions could have a huge effect on the economy as a whole, and be fairly unpredictable into the bargain.

Blake acknowledged that this type of destabilising AI event hasn’t happened yet, but warned that the risks are very real, citing a 2021 case in which Apple and Goldman Sachs were investigated by the New York State Department of Financial Services for algorithmically offering smaller lines of credit to women.

“Beyond the inherent issues with bias, fairness and ethics, this could potentially lead to stability issues for financial institutions or the financial system as a whole,” Blake wrote.

Follow Us

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags biasartificial intelligence (AI)


EDGE 2024

Register your interest now for EDGE 2024!



How MSPs can capitalise on integrating AI into existing services

How MSPs can capitalise on integrating AI into existing services

​Given the pace of change, scale of digitalisation and evolution of generative AI, partners must get ahead of the trends to capture the best use of innovative AI solutions to develop new service opportunities. For MSPs, integrating AI capabilities into existing service portfolios can unlock enhancements in key areas including managed hosting, cloud computing and data centre management. This exclusive Reseller News roundtable in association with rhipe, a Crayon company and VMware, focused on how partners can integrate generative AI solutions into existing service offerings and unlocking new revenue streams.

How MSPs can capitalise on integrating AI into existing services
Access4 holds inaugural A/NZ Annual Conference

Access4 holds inaugural A/NZ Annual Conference

​Access4 held its inaugural Annual Conference in Port Douglass, Queensland, for Australia and New Zealand from 9-11 October, hosting partners from across the region with presentations on Access4 product updates, its 2023 Partner of the Year awards and more.

Access4 holds inaugural A/NZ Annual Conference
Show Comments