Chapter 7. The Simplest Way to Keep Artificial Intelligence Ethical
I wrote an AI application designed to discover cohorts for patients. If you are diagnosed with an illness (especially a chronic illness), one of the best things that you can do to improve your long-term health is to actively manage your illness with other patients in a similar situation. Find people like you with a similar diagnosis and support each other. For patients who opt-in to such a program, AI can automatically find patient cohorts and help those with similar situations find one another.
I ran my model to find suitable partners for patients and checked the results of an early version, and I saw a huge red flag: the algorithm segregated people mostly by race and gender. Using AI Forensics, we caught the problem early and adjusted the algorithm’s structure and training to prevent racial and gender bias.
Know Thine AI: Create a Profile of the AI Based on Its Recent Behavior
Bias, ethics, and fairness is a big risk factor in AI. The simplest way of protecting against this risk is to check the results. Part of a standard checklist for protecting against ethical violations with AI is to build AI forensics tools, use those tools to profile the algorithm, use the profile to anticipate behavior, and discuss the anticipated behavior with a diverse risk mitigation team (see Figure 7-1).
Figure 7-1. Performing AI forensics
AI forensics analyzes an AI model’s tendencies based on its output. You need a good log of input and corresponding output. You also need tools that can discover the most influential factors in how the model makes decisions. Using the tools, you can build a profile: the decisions it makes, the factors that it considers, and the weight of each factor.
By building a profile, you gain insight into the model’s behavior and tendencies. Profiling an algorithm doesn’t sacrifice performance. It’s not necessary to alter the function of the underlying algorithm. Nor is it necessary for the inner workings of the underlying algorithm to be fully transparent and explainable.
Anticipate the AI’s Behavior: Know What the AI Will Likely Do Next
Using the profile to anticipate the model’s behavior is a statistically rooted exercise in which you calculate the most likely behavior, given both the profile and the production environment in which the final algorithm will operate. You know, for example, that customer service calls and total charges during the day is, by far, the strongest influencer for the customer churn model. But if there is a small but influential portion of customers who don’t use the customer service phone line, the model will most likely ignore and alienate an important group of customers.
The final, and most important step in the AI-ethics checklist is to discuss the results of forensics with a diverse group of people. Bring together a group of people from all different backgrounds and job roles, present the results, and discuss the risks. Decide whether the risk is acceptable or whether changes need to be made before allowing the algorithm to continue in production.
Even when the algorithm is based on a single machine learning model, the complexity of analyzing its behavior, generating a profile, and anticipating behavior will likely be too high for human being to manage. Some of the most useful profiling and analysis tools will, themselves, be based on machine learning. It turns out that machine learning is one of the best ways to protect against the unintended consequences of machine learning.
Profiling is not just an upfront activity. You must update the profile on a regular basis. An AI based on a machine learning algorithm will acquire new behavior as it learns from new data. To remain ethical, your profile of the model must change as the model evolves.
We have a lot of experience dealing with intelligent beings—one another. Much of the basic common sense that we have developed for such dealings is applicable to machine learning. Observe the AI and understand it. Predict the most likely behavior and decide whether any risk is worth the benefit.