Measuring algorithm efficacy gives important insights into how well the model is working and does what it is designed to do. In this blog post, we give an overview of some different metrics that can be used to measure the performance of classification and regression systems.
Today, artificial intelligence (AI) is increasingly present in our lives and becoming a fundamental part of many systems and applications. However, like any technology, it is important to ensure that AI-based solutions are trustworthy and fair. That's where the Holistic AI library comes in. In this blog post, we provide an overview of Holistic AI’s Bias analysis and mitigation framework, defining bias and how it can be mitigated before giving an overview of the bias metrics and mitigations available in the Holistic AI library.
Game Theory was an intellectual advance built at final of World War II and initially had mathematicians as its main contributors. Over time, researchers from other areas began to adopt this theoretical framework, highlighting economists and political scientists. The Shapley value describes a method to distribute the total gain to players if they all collaborate in a specific coalition strategy. We can describe that SHAP (SHapley Additive exPlanation) values attribute to each feature the change in the expected model prediction when conditioning on that feature.
In recent years, the fairness of automated employment decision tools (AEDTs) has received increasing attention. In November 2021, the New York City Council passed Local Law 144, which mandates bias audits of these systems.
Under Local Law 144, employers and employment agencies are required to commission independent, impartial bias audits of their tools, where, under the latest version of the Department of Consumer and Worker Protection’s (DCWP) proposed rules, bias should be determined using impact ratios based on outcomes for different subgroups. In this blog post, we outline the metrics required to conduct the bias audit, how small sample sizes can pose issues, and how they can be dealt with when carrying out audits.
Our automated AI Risk Management platform empowers your enterprise to confidently embrace AIGet Started