Many of us have come to believe that all aspects of human life in the coming years will be transformed as we turn to AI to understand the world around us better than we do ourselves. This optimism is abundant as it is applied theoretically to credit underwriting—and not completely without warrant.

In theory, there are infinite data sets that can explain why some people default on credit while others pay as agreed. Applying AI to these massive data sets will eventually dramatically increase the accuracy of credit underwriting models and democratize credit and pricing.

In order for AI to transform credit underwriting, lenders will need access to alternative data that goes beyond what is collected generally in calculating—say—a FICO score. Until there is regular access to other information not available to the credit bureaus, those applying AI to the largest (in depth, breadth, and time span) proprietary dataset will achieve the greatest AI-related improvement in underwriting performance.

Who will have access to the data that will train AI credit underwriting models in ways that are transformative? Well, it won’t be those who’s only underwriting inputs come from a loan application. It will be lenders who have a more extensive relationship with their borrowers and history of performance through credit cycles.

Established consumer lenders like JPMorgan Chase, Capital One, and others, have the competitive advantage for the shere length of time they’ve had customer relationships and the number of customers with whom they’ve had multi-touchpoint relationships.

For the disrupter fintechs, consider SoFi. A vast majority of SoFi’s deposit growth has been from direct deposit relationships. It’s well understood that the direct deposit relationship is the stickiest because this account is the one from which depositors set up their bill-pay relationships and conduct most of their financial transactions. It’s also the relationship that will supply the most informative data to an AI credit underwriting model.  Add to that a product like SoFi’s Relay that rewards members for checking their credit scores and tracking transactions and other financial relationships both internal and external to the existing SoFi relationship, and you’ve got a robust data lake to feed your AI model.

In contrast, UPST’s customer relationship begins and ends with the credit relationship where UPST has only proprietary performance data over a relatively short period of time, perhaps a credit score, and the information collected from the abbreviated application process. While they supplement it with third-party purchased data, that’s data that everyone has access to.

So, even though, arguably, UPST objectively appears less likely to be the entity that will transform credit underwriting as we know it, they are the darling of those who believe that AI will change underwriting as we know it.

Take for example, this opening comment from Motley Fool’s Tom Gardner on UPST:

First, its [UPST’s] AI-driven innovation continues to be a competitive edge. The company deployed 23 new model versions in the first quarter, each trained on more than 100 billion cells of data. Today, Upstart’s AI — as it should be — is more sophisticated and accurate than ever before.

Upstart’s AI-Powered Lending: A Beacon Amid Economic Headwinds

Really? How is it more accurate? Has anybody compared it to credit decisions generated at other companies? Has it been tested through a credit cycle? 23 new models? Why not 50? Why not 10? A 100 billion cells of data? Have any of those data demonstrated the predictive power to result in better risk-adjusted returns over the credit cycle? Has any of that data come from a different phase of a credit cycle?

AI hype has caused investors to ignore these questions, which are the only ones that ultimately matter when it comes to a credit underwriting model. I suggest that investors take a look at the “.com” bubble of the ’90s. For every Amazon, there were 1,000 failed companies.

That covers the question of who is more likely to transform lending through AI. Let’s turn to the next question:

How will we know that the consumer credit landscape has been transformed by AI?

UPST hints at the answer in its 1Q23 10-Q:

“We consider credit performance to be one of the most important measures of the effectiveness of our AI models. We focus on credit performance compared to the expectations set by us at the time of origination. Since 2018 through to the end of 2021, all quarterly vintages of loans retained by our lending partners to date are currently forecasted to meet or exceed the target returns set at the time of loan origination. For loans purchased by institutional investors, all vintages from 2018 through 2020 are forecasted to deliver returns at or in excess of the targets such institutional investors were expecting to receive, while our 2021 through mid-2022 vintages have underperformed relative to target returns. Our most recent vintages show performance reconverging to target yields. This was driven by a combination of underlying macro effects stabilizing (as seen in the Upstart Macroeconomic Index (UMI)) and increased conservatism in underwriting.”

UPST’s 1Q23 10-Q

UPST explains indirectly why its AI model underperformed projections in 2021 and first half of 2022:

“the bulk of the data gathered and the development of our AI models have largely occurred during a period of sustained economic growth, and our AI models have not been extensively tested during a down-cycle economy or recession and have not been tested at all during a down-cycle economy or recession without significant levels of government assistance. There is no assurance that our AI models can accurately predict loan performance under adverse economic conditions.”

UPST’s 1Q23 10-Q

So, for this reason alone, we ought to proceed with caution regarding our expectations for UPST’s AI…except…it was tested…

UPST’s AI Failed. As soon as the consumer-credit COVID tide started to recede, UPST’s AI model underperformed. Should we take it on faith that, in the event of an actual recession (or other economic shock), UPST’s AI underwritten loans will perform better than, say, Capital One’s or Citi’s or JPMorgan’s?

In the history of nonbank lenders that have claimed to have built a better underwriting system, we can’t think of any that have actually survived as an independent company through a business cycle.  Not one.

The corporate graveyard is full of lenders that believed they could generate “higher approval rates and lower interest rates, leading to increased volume.”[again from UPST’s 1Q23 10-Q]

This is the theory that other lenders are leaving money on the table by failing to make loans to creditworthy borrowers at the right price. It assumes that they have overestimated the riskiness of these credits and have mistakenly avoided them. However, lending experience tells us that the farther you travel down the credit spectrum, the more exposed the borrowers are to exogenous shocks to their credit worthiness. Lenders that have watched this play out over multiple business cycles have learned this lesson the hard way.

If this experiment performs well through the business cycle, then we will know that AI has indeed transformed credit underwriting. Until then, this kind of growth strategy that is premised on superior–but untested–ability to differentiate profitable borrowers from unprofitable ones has always proven to reveal itself as the credit related cancer known as adverse selection.