Parallel Timing Curve Calibration
Explaining Upstart’s novel and patent-pending solution for accelerating the calibration of AI lending models.
In any AI-enabled business, the company that builds the most accurate models the quickest wins. But in the world of credit, speed can be a challenge. You don’t get immediate feedback the way you would with, say, autonomous driving. If you want comprehensive feedback on a model powering three-year term loans, you have to wait 36 months.
Until now. Here at Upstart, we’ve found a way to speed up the pace of model calibration and therefore model development using something we call parallel timing curve calibration (PTCC). This patent-pending technique offers a faster read into model performance—across the entire term of a loan—than was possible before.
Understanding 'loan month'
To understand PTCC, you first need to understand our loan-month model. Most modern risk models attempt to quantify the unique risks of default and prepayment for each loan in a portfolio of loans. With our patented loan-month model, we go further—predicting the risk of default and prepayment for each month of each loan. This results in a significantly more precise estimate of expected losses (a default in month six is far worse than a default in month 30!).
Imagine that we’re originating three-year loans and generating a new model version every six months, as shown in Figure 1. In this specific example, each model is used to originate loans for six months, with a 36-month runoff period for those loans once that model version is deprecated.
The traditional calibration process
Now imagine that we launched Model v6 six months ago and want to know how well it’s performing (Figure 2), i.e., whether the model’s predictions of delinquencies for each of the first six months are accurate. This is, of course, valuable information. But, in a traditional calibration process, we would only have visibility into model performance for the first six months of the timing curve. We’d have, at best, guesstimates for what’s likely to happen next.
For example, a model might be extremely pessimistic about the first six months and extremely optimistic about the subsequent months. But we may see observed losses much lower than expected at the six-month mark instead, while observed defaults are much higher than expected as the rest of the loan term plays out.
How PTCC works
Now let’s introduce parallel timing curve calibration. Here’s how it works:
First, we’ll go back in time to the launch of Model v6. While preparing to launch our new model, we ask the model to evaluate all of the outstanding loans on the platform and predict the likelihood of default or prepayment for the remaining months on each of those loans. We call this “re-underwriting.” It is not a backtest. The predictions made across the platform are for payment months that have yet to occur.
As a result, at month six we can observe how accurately the model predicted repayments across the entire timing curve rather than just the first six months (Figure 3). Our feedback loop has shrunk from three years to, in theory, just one month. But, in practice, it may require a few months to ensure that the single month isn’t an outlier.
Net net: PTCC dramatically compresses the time needed for model calibration, which in turn provides a giant head start when developing subsequent versions of the model.
Upstart lenders and investors will come to love PTCC because they welcome any innovation that speeds up the insights they get into the performance of their most recent Upstart-powered loans. We’re very excited about this innovation’s potential to extend our leadership in AI lending and to bring us closer to the day when the credit system works for all of us.
In addition to PTCC, Upstart created the Upstart Macro Index, which isolates and estimates the impact the changing economy has on defaults within Upstart-powered loan portfolios. Read about that here.
