Four Predictive Model Pitfalls: Bankruptcy Scars Are Painful for Risk Professionals

Recent conversations with prospects centered around the failure of our competitors’ financial risk models to identify company distress before bankruptcy filings. In short, those prospects expressed disappointment and were focused on upgrading their risk management processes to avoid future counterparty blowups.

The fundamental question is: “What are the root causes of the failure of risk models to provide adequate warning?” After nearly 25 years of company operation and observation, CreditRiskMonitor® has identified four common problems among competing risk models:

1. Input signals/information lack predictive or have statistical confounding between them
2. Data components have prolonged time lags or latency
3. Model training uses a suboptimal target event
4. Scores/ratings are produced with limited or negligible data

The Cloaking Effect: Bad Decisions

It’s often surprising when a renowned corporation is in the news after filing bankruptcy, but it’s particularly painful when that company is a critical customer, supplier, or counterparty and you are blindsided. That situation, when dealing with public companies, is typically a due to the “Cloaking Effect.” The Cloaking Effect is when a company is financially distressed but still makes prompt invoice payments to avoid signaling financial trouble. Many competitor risk models are based on payment performance, akin to the dollar-weighted Days Beyond Terms (DBT) Index, and CFOs know that maintaining prompt payments is crucial to receiving the inventory and services necessary to create revenue. In general, public companies have access to capital markets and typically issue equity or debt for cash to make prompt payments, despite mounting financial weakness. The byproduct is that using payment performance signals as an indicator of financial distress is typically misleading for risk professionals when dealing with public companies. Most public companies maintain timely payments up until the day they file bankruptcy.

Takeaway No. 1: Many competing risk models that use payment performance are deceived by companies that pay their bills promptly. Credit report providers even suggest businesses pay on time to improve their own scores. CreditRiskMonitor does not include payment performance in the 96%-accurate FRISK® score.

Time Lags: Delayed Decisions

Risk professionals have traditionally leaned on financial statements for risk analysis. However, financial statements are only released periodically and can be slower to signal high-risk situations relative to other data sources. Additionally, financial-based risk models can entirely fail to warn as certain ratios and trends can create the appearance of good financial health. For example, the Altman Z’’-Score is an open-source model that is like many competing financial-based risk models. Recently, there have been five large corporate bankruptcies where the Z’’-score indicated fiscal stability for prolonged periods before the filing, providing limited notice or no warning at all.

CreditRiskMonitor has also found competing public company models that combine payment behavior with financial statement data often resulting in worse predictive performance. For example, if we assume a risk model’s performance or AUC (Area Under the ROC [Receiver Operating Characteristic] Curve) is 67% with payment data and 75% with financial statement data, the combined performance will be lower than the 75% AUC of financial statements alone. So even if competitors state that multiple data sources are used, that does not necessarily translate to improved performance.

Takeaway No. 2: The Altman Z''-Score is like competing financial-based risk models that, even when incorporating additional payment signals, can provide late warning of financial distress and sometimes miss alerting of financial risk until it's too late to mitigate. In contrast, the FRISK® score includes stock market performance, bond agency ratings, and risk professional crowdsourced sentiments, in addition to financial statement ratios in a non-linear, additive model that provides superior accuracy and timeliness.

Inferior Model Training: Suboptimal Decisions

Risk models are trained to predict a specific target event, such as bankruptcy, default, or delinquency. Although understanding the probability of default or payment delinquency can be useful, more often than not, public businesses that default on an obligation or have a period of delinquency continue to operate for long periods after such events.

For delinquency, financially stable businesses can fail to pay their bills promptly but that does not necessarily indicate financial stress. Most large companies are known for stretching their supplier payments to maximize working capital efficiency (see Walmart). All types of businesses can also have one-off delinquent payments but that should not automatically send up a red flag.

Regarding credit defaults, companies have been known to default on their obligations and never actually file bankruptcy. For instance, companies can enter technical or selective defaults, such as entering distressed exchanges or grace periods, retiring debt below par value, or amending their credit agreements.

Specifically, long-term averages suggest that only about 1 in 4 of defaulting public companies file for bankruptcy and those bankruptcies can occur months or even years after the default. Therefore, taking steps to mitigate or cut off counterparties based on such non-bankruptcy events can materially interrupt revenue opportunities and damage the top line of the income statement.

Takeaway No. 3: Risk professionals should not use models trained on delinquency or default to inform their decisions about bankruptcy. Many competing scores use delinquency an default as the target events, which can falsely signal healthy businesses as high-risk or indicate situations are worse than reality. CreditRiskMonitor's FRISK® and PAYCE® scores only use the bankruptcy filing as the target event.

Negligible Data Depth: No Decisions

Credit report providers, including CreditRiskMonitor®, are always seeking to maximize scoring coverage so clients can make informed business decisions on all of their counterparties. However, risk professionals should be equally concerned about data depth. A business report can contain basic firmographic information but that will not provide you with an informed opinion. Your top priority should be understanding what specific data enters the risk model, whether that data is really about the subject business or inferred from basic firmographics, and how the model performs on out-of-sample businesses.

In general, risk models are typically based on financial statements, payment information, firmographics, industry data, and alternative datasets.

1. As previously discussed, payment and financial information can be especially useful but have unique pitfalls if not used properly.
2. Firmographic information can provide insight as larger and older firms tend to be more financially stable. However, the average private company has a marginally higher long-term probability of bankruptcy at about 2% versus 1% with public companies. It is dangerous to assume that large companies, private or public, are generally financially safe.
3. Industry data is directionally useful, i.e., knowing whether a particular industry is low or high risk. However, industry measures should be paired with other data, such as an entity-specific risk model, to make an informed decision.
4. Alternative datasets have rapidly expanded over the last decade, but the data must be proven to have accuracy and consistency to be valid for risk assessment. Otherwise, professionals will be using inferior models inadvertently.

Takeaway No. 4: Adequate data depth in risk scores is necessary. Professionals should know exactly what information folds into their third-party risk models, confirm the sources are high-quality, and that sufficient data is available. CreditRiskMonitor distinguishes its risk models by public and private companies and will only generate FRISK® and PAYCE® scores if a significant amount of data is available to preserve model accuracy. The models have maintained their benchmark accuracy over the full business cycle of 96% and 80+%, respectively.

Bottom Line

If competitor models are failing to provide adequate warning about impending bankruptcies, your business outcomes will be painful. Even one missed bankruptcy can expose your company to receivable write-offs, supply disruptions, lost revenue, profit, and cash flow, as well as reputational damage. These events occur even more frequently during economic downturns, tight credit conditions, and industry-specific bust cycles.

Before bankruptcy rates accelerate further, professionals should reassess their risk management processes and examine their third-party risk models. Contact CreditRiskMonitor to see how our industry-leading bankruptcy models provide risk professionals with timeliness and precision.