Framework for Building Trustworthy AI

Finance and accounting, as highly regulated industries, have very thin thresholds for being wrong about your numbers (and severe penalties for mistakes). Put another way, there’s a high need for trust in finance. With this said within finance, many of the logical steps AI applies are tried and true in non-AI settings.Historically, SOC compliance has played a role in enabling trust and internal controls. As firms increased their processing ability and ability to handle large amounts of information with automation, they haven’t updated this control side. When we can explain in normal human speech how each one of these automated components work and walk through tried-and-true logical steps being applied by AI, this is where explainability is built. On the flip side, the execution of this automation behind the scenes can be very complex. It’s best to try to focus on simplifying down to the basic logical steps in our documentation and external comms. This translation from data science terms to logic and industry-specific terms plays a large role in explainability. Additionally, focus on the use case and business value aids in explainability as well.

In the world of financial and accounting controls, receiving 3rd party reports is normal to understand tests, processes, results, and so forth on systems. This extends across many finance and accounting use cases where teams are used to receiving and then assessing next steps based off reports. Providing similar reports on AI systems generally works for communicating with the larger team, particularly if there’s an emphasis on translating complex data science terms into more fundamental concepts.

Fines up to €35M or 7% of global annual turnover for breaches, subject to changes depending on the nature of the non-compliance and the size of the respective entity, have been outlined in the latest political agreement although these amounts are not yet finalized.

Conservative and risk-averse industries are still looking to automate things that are hard to do at scale. This is an opportunity to start small yet still provide efficiency gains to build trust. You can start with comparing options across a lifecycle of AI applications that are more or less transformative. Starting a point solution or gains in efficiency through automation in minute details of the process can build trust before more dramatic overhauls using AI are attempted.

Transparency around testing, audits, results, and methodology is a core component here. Often “good enough” surfaces where there’s a “human in the loop”. There are many ways and metrics for testing confidence in AI, it’s important to provide some overlap and differing views through testing the performance and safety of an AI system.  Layering many tests over systems removes a single point of failure and allows humans to determine which tests are most predictive of confidence. Human judgment still plays a very large role in parsing through testing and metrics to find the best proxies for explainability and trust.