Bias management for AI consistent with Human Values

We should avoid bias in decisions. You recognise that? Yes, it’s part of human decision-making recommended practices. But in the last several years, such questions are increasingly posed in Artificial Intelligence – driven decision making. From the “Labeled faces in the wild” [1] dataset, where it took a painstaking 12 yeas to recognise how unrepresentative the human faces image data sample had been [2], to the “word embeddings” debate that considers gender [3] or racial bias already present in textual data, issues pertaining to bias in AI-driven decisions are now at the heart of relevant research. Much as “fake news’ may be spread by social-network repetition and therefore amplification, bias amplifications in Machine Learning outcomes can be very profound too. One of the most recognisable examples is the COMPAS system case [4]. Its formal adoption led to introducing (automated) bias in the criminal justice system, only to be taken off the system when its dramatic effects became all too clear.

Relevant lessons learned have moved from research and scholar debates to practitioners on how to handle bias in AI systems [5] and specifically in:

• Data creation / preparation
• How a problem is posed
• How data is analysed
• How AI-outcomes are validated and tested

But why bias in data-driven decisions only became a major issue in recent years? After all, bias handling in early statistical and control system research and application practice was recognised as an issue to deal with many decades ago. Take any textbook in statistical learning and estimation theory and you immediately recognise techniques for “unbiased estimators” of first and second order statistics. The same applies to control theory textbooks, where unbiased estimators have been put forward when making inferencing based on available observations, aiming to drive control actions.

What is the difference now? Earlier literature focused on systems approaches but had a distinctively non-human direction: a system’s view would mostly consider performance and operational targets. In contrast, AI is now increasingly making impact on human lives. It’s a measure of the great success of the real-world applicability of AI outcomes that this huge leap forward is now an evolving reality. But are we taking a leap-forward to a not properly controlled future? Should there be such a “properly-controlled” future anyway? So much has evolved since the “Three Laws of Robotics” have gained eminence, and yet fundamental problems have not been solved – in fact they are merely starting to being posed. For example, the European Commission, following the recommendations of High-Level Expert Group proposed a framework for Trustworthy AI based on three pillars: ethical, robust, and lawful AI and proposed an assessment list of criteria for Trustworthy AI [6]:

• human agency and oversight
• technical robustness and safety
• privacy and data governance
• transparency
• diversity, non-discrimination and fairness
• environmental and societal well-being and
• accountability

Bias is a key consideration in many of the above criteria. Specifically, from technical robustness and safety, to diversity, non-discrimination and fairness, bias reduction, avoidance or elimination are key aspects to consider.

For us in the STAR project, these are highly important criteria to consider. They are part of the criteria identified in our Human-Centric Design Methodology [7], are included in our evaluation methodology, and are being considered in the way the human involvement in the AI outcomes is being designed, for example through the use of Active Learning.

But let us take a step back for a moment. Pretty much as one cannot eliminate bias in human thinking, wouldn’t be appropriate to refocus part of the debate in AI not in the direction of eliminating bias but properly managing bias? In some cases it may even make sense to seek to strengthen the bias in order to effectively deliver outcomes more consistent with our aims in domain-specific contexts. To do so, we need to recognise bias and work on bias management. The contribution of some of the existing and established sciences towards AI can be important on how bias is managed. Knowledge from medical, production systems, or social sciences-based domains can all be incorporated to bias AI decisions towards domain-specific knowledge and constraints.

Ultimately, a key question is how to “bias” AI to produce outcomes more consistent not just with operational and performance targets, but consistent also with human values. Societies have long accepted humans making hard decisions: the humans making such decisions are ultimately responsible for them. We are now moving towards requirements for human agency and oversight on AI-driven decisions. These are still early days in the ethical debates regarding AI-based outcomes. When the time comes for societies to accept AI decisions on a par level with human decisions, then that should imply that we have properly identified ways of managing the bias of AI solutions in the direction of outcomes consistent with human values. We are still far from meeting such an aim. Still the STAR project is aware of such challenges and contributes towards delivering Human-centric AI. That’s why our technical approaches but also our evaluation methodology, both take on board such aspects, aiming to deliver Human -Centric AI in production environments and we plan more in that direction.

By: Christos Emmanouilidis, University of Groningen

[1] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, ‘Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments’, p. 93940B, 2007.

[2] ‘Labeled Faces in the Wild Home Page’. http://vis-www.cs.umass.edu/lfw/.

[3] T. Bolukbasi, K. W. Chang, J. Zou, V. Saligrama, and A. Kalai, ‘Man is to computer programmer as woman is to homemaker? Debiasing word embeddings’, Adv. Neural Inf. Process. Syst., pp. 4356–4364, 2016.

[4] T. Brennan and W. Dieterich, ‘Correctional Offender Management Profiles for Alternative Sanctions (COMPAS)’, in Handbook of Recidivism Risk/Needs Assessment Tools, First Edition, Wiley, 2017, pp. 49–75.

[5] R. Srinivasan and A. Chander, ‘Biases in AI systems’, Commun. ACM, vol. 64, no. 8, pp. 44–49, 2021, doi: 10.1145/3464903.

[6] HLEG, ‘Assessment List for Trustworthy Artificial Intelligence (ALTAI) for self-assessment’, 2020. [Online]. Available: https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai.

[7] S. Waschull and C. Emmanouilidis, ‘Development and application of a human-centric co-creation design method for AI-enabled systems in manufacturing’, 14th IFAC Workshop on Intelligent Manufacturing Systems (on-line), 29-30 March 2022.