AI is becoming increasingly woven into the fabric of our everyday lives. It is, therefore, imperative that we address the potential harms associated with bias in AI systems – that means eliminating algorithms that perpetuate discrimination, reinforce inequality and yield unfair outcomes.
This article will guide you through how to build a bias measuring and mitigation dashboard app in just five easy steps, using Python alongside the Holistic AI, sklearn and Streamlit libraries.
The Holistic AI library is an open-source tool used to assess and improve the trustworthiness of AI systems. The current version of the library offers a set of techniques to easily measure and mitigate bias across a variety of tasks, facilitating the development of fair, transparent, and ethical AI systems.
We will explore the issue of bias mitigation using the example of admission rates for two distinct applicant groups – group a (white) and group b (non-white).
The initial step in a data science or machine learning project typically involves importing the necessary Python libraries and setting up the front page or user interface of the project.
That is the case in our bias mitigation dashboard too, as expressed in the code snippet below.
For the next step, create a simple data visualization with a pie plot to represent the percentage of white and non-white people in the dataset — and distplot to represent the distribution of students’ undergraduate GPA.
Next, we select a model and create an ROC curve. The ROC curve is created by plotting the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. We also compute the area under the ROC curve (AUC), which we use as a metric to quantify the overall performance of the model.
In this step, we compute bias metrics to evaluate whether there are any disparities in the model’s performance across different subgroups of the population – white and non-white applicants in this example. Bias metrics are used to measure the fairness of the model and to identify any potential sources of bias that may be present in the data or the model.
Finally, we use the reweighing strategy to compute the bias metrics. Reweighing is a commonly used technique in machine learning to mitigate bias in datasets. The strategy involves assigning different weights to different samples in the dataset, based on their group membership, to balance the representation of different groups.
You can access the full implementation in this GitHub Repo.
And that's all there is to it. Using the Holistic AI library and Streamlit framework in Python, you can create a user-friendly interface, allowing you to showcase the results of your bias mitigation efforts in machine learning systems.
With the rapid proliferation of AI throughout society, practical solutions for mitigating algorithmic bias have never been more important.
A customisable dashboard that intuitively displays visualisations, tables and other data is the perfect way to visualise data and present it to your stakeholders.
Written by Kleyton da Costa, Researcher at Holistic AI.
DISCLAIMER: This blog article is for informational purposes only. This blog article is not intended to, and does not, provide legal advice or a legal opinion. It is not a do-it-yourself guide to resolving legal issues or handling litigation. This blog article is not a substitute for experienced legal counsel and does not provide legal advice regarding any situation or employer.
Subscribe to our newsletter!
Join our mailing list to receive the latest news and updates.