How AI generates more accurate and explainable Adverse Action Notices

Adverse Action Notices (AANs) are provided to potential borrowers after an adverse action (e.g. denial of an application for a loan) and must include explanations as to why they were denied credit. AANs are required by both the Fair Credit Reporting Act and the Equal Credit Opportunity Act to ensure transparency in credit decisioning, prevent discrimination, and allow consumers to review and correct any inaccuracies in their credit information. The Consumer Financial Protection Bureau has also emphasized the importance of providing clear, understandable AANs for similar reasons by publishing circulars on this topic in 2022 and 2023.

This requirement to issue AANs, and the explainability of decisions, is particularly significant in the context of advanced technologies, such as AI. The use of alternative data, coupled with AI, offers a level of accuracy in credit decisioning far beyond that of traditional credit models. Since these alternative variables are more specific than traditional variables, AANs based on AI models have the potential to be more helpful to consumers than AANs based on traditional lending models. The more specific the feedback that applicants get in an AAN, the more likely they can improve their chances of getting approved for credit. 

However, providing more detailed feedback presents certain challenges due to the vast amounts of data that AI models process. AI models sometimes utilize data from varying sources, leverage exponentially more inputs compared to a simple linear regression model, and can also rely on interactions among variables, which can be opaque to both consumers and regulators. At Upstart, we have developed a handful of advanced approaches to address these challenges and provide specific, accurate, and comprehensible AANs to applicants. A few of these techniques are explained below:

Use of SHAP Values for Explainability: Upstart employs SHapley Additive exPlanations (SHAP) to generate AAN reasons for AI model-based declines. SHAP provides a numerical summary of how each feature contributed to the model decision for each individual applicant. These “SHAP values” can also be computed regardless of the type of AI model employed. These properties are advantageous when utilizing complex AI models in lending because they allow us to provide specific and accurate AAN reasons for each applicant individually. Since many features in our AI models represent similar concepts, we aggregate these feature-level SHAP values into predefined components, which are groups of conceptually similar features, and rank them by their adverse impact to generate the most relevant and comprehensible reasons for denial.

AAN Directionality Analysis: To ensure that the reasons provided to applicants accurately reflect the reason behind an adverse action, we developed a method to provide “directional information” on the impact of the features. For example, one risk factor might be that an applicant is carrying high balances on their accounts, indicating a directional component to the risk factor. As such, we want the AAN reason to indicate high balances contributed to the decision as it is more specific and informative. While high balances are often a risk factor, AI models are capable of learning complex relationships, and it’s possible for a specific applicant that low balances were the risk factor. In such cases, we must still ensure the AAN message is consistent for the applicant. By developing a directionality score (D-score) that measures the consistency of the applicant’s feature values with the directional statement (i.e., comparing the applicant’s balances to the statement “balances too high”), we can dynamically choose statements that are more consistent for the particular applicant if the data points elsewhere, such as to low balances. This process helps maintain the accuracy and integrity of the AANs, allowing for more specific and informative reasons.

Accounting for Missing Inputs in AANs:  Sometimes we receive applications from thin-file applicants where common credit characteristics, such as the presence of a wide variety of credit accounts, are missing. Other times, an applicant may actively choose to not provide certain information, such as savings and investment balances. Even in these instances, our AI-powered decision engine still assesses risk and rank orders missing features by their contribution to the decision. Instead of indicating we cannot approve the loan request due to “current savings and investments” (which can imply the data was both provided and insufficient), our solution identifies missing inputs and adjusts the AAN reasons accordingly. In the above example, the reason would instead indicate that we cannot approve the loan request due to “lack of information regarding current savings and investments.”

Semantic Clustering for Enhanced AAN Generation: Semantic clustering is another novel approach we’re developing to generate explainable AANs. This technique involves using natural language processing to group features with similar meanings into clusters, which simplifies the interpretation of model outputs and enhances the clarity of the reasons provided to applicants. For example, features related to credit inquiries, such as the number of recent inquiries and the frequency of inquiries in the past six months, can be grouped together. The technique allows for coherent groupings and provides more precision in explaining adverse actions. In addition, instead of having a human generate the corresponding user-facing messages from scratch, we leverage a Large Language Model (LLM) as an assistant. The LLM assistant drafts an initial message explaining how the features in a group contribute to the decision. This is reviewed and refined by a human before finalizing the new message to ensure it is clear and includes sufficient detail to be meaningful to the applicant. These recent developments of Upstart’s AAN program will be rolled out by the end of 2024.