In the first of a series of research notes on bias in artificial intelligence (AI), this literature review examines available literature on bias in the context of supervised machine learning.
Read the research note (PDF)[1]
Supervised machine learning generates predictions of an outcome. In a financial services scenario, an outcome could be defaulting on a credit product or making an insurance claim.
From the literature, we identified the following:
- Data issues arising from past decision-making, historical practices of exclusion, and sampling issues are the main potential source of bias.
- Biases can also arise due to choices made during the AI modelling process itself, such as what variables are included, what specific statistical model is used, and how humans choose to use and interpret predictive models.
- In reviewing technical methods for identifying and mitigating such biases, literature suggests these methods should be supplemented by careful consideration of context and human review processes. As we discuss in the paper, however, technical mitigation strategies may affect model accuracy and could have unintended consequences for model bias on other groups.
Our AI research series
We want to enable the safe and responsible use of AI in UK financial markets, driving growth, competitiveness and innovation in the sector. As part of our effort to deepen our understanding of AI and its potential impact on financial services, our team is undertaking research in the area of AI bias.
We will be publishing a series of research notes on how AI intersects with financial services to spark discussion on these issues, drawing on a variety of academic and regulatory perspectives. We hope that these notes are of interest to those who build models, financial firms, and consumer groups in understanding complex debates on building and implementing AI systems.
Disclaimer
Research notes contribute to the work of the FCA by providing rigorous research results and stimulating debate. While they may not necessarily represent the position of the FCA, they are one source of evidence that the FCA may use while discharging its functions and to inform its views. The FCA endeavours to ensure that research outputs are correct, through checks including independent referee reports, but the nature of such research and choice of research methods is a matter for the authors using their expert judgement. To the extent that research notes contain any errors or omissions, they should be attributed to the individual authors, rather than to the FCA.
Authors
Daniel Bogiatzis-Gibbons, Lawrence Charles, Harry Dewing, Camilla Gretschel, Maria Jomy, Annette Reid, Ryan Slack