Counterparty Insight - Probability of Default (PD) Model Methodology

Document version: 1.2 - v1.2 model in production; SHAP attribution + lead-lag refresh shipped as v1.2 patches; PSI stability monitoring shipped as v1.2.1 Last revised: 2026-06-19 Owner: Counterparty Insight Risk Methodology Audience: customers, model risk reviewers, SOC 2 auditors

Implementation status (2026-06-19). Model v1.2 is live on the Q1 2026 vintage (data as of 2026-03-31), replacing v1.1. v1.2 adds four loss-progression features (reserve_to_npl, allowance_growth_qoq, provision_coverage, texas_ratio_chronic_quarters) that close the Community-Bank-style "high Texas Ratio + positive earnings + deferred recognition" blind spot v1.1 missed, fits Platt + isotonic calibrators in parallel and picks whichever has lower test Brier, and adds a 5-tier empirically-anchored overlay (Severe / High / Elevated / Moderate / Low) calibrated to realized FDIC failure rates from the walk-forward backtest. v1.2.1 kept the original 17-letter agency-style overlay (pd_letter) as the primary UI surface alongside the new risk_tier field; production responses include both. Walk-forward mean ROC AUC 0.9253 across 13 years (2010 - 2025). Both 2026 target failures (Metropolitan Capital, Community Bank & Trust West Georgia) now correctly flagged at High tier. See §13.3 for the v1.1 → v1.2 changelog.

Patches since v1.2 promotion (2026-06-19). Three non-model improvements shipped after v1.2 went live, all attached to the same production model: (A) SHAP top-driver attribution - every per-bank score now carries the three features with the largest SHAP contribution so consumers get a grounded "why is this bank flagged" instead of an empty driver list. (B) Lead-lag refresh - the head-to-head against traditional ratio screens was re-run against v1.2, showing 99.6% capture rate at the Elevated tier and a median 3-year lead time before failure. (C) v1.2.1, PSI stability monitoring, frozen reference distribution from the full pre-2014 training population plus a quarterly PSI computation and an MRM-facing dashboard so model risk teams can satisfy SR 11-7 ongoing-monitoring expectations.

Abandoned v1.3 retrain. A retrain attempt targeting the 2023 weak walk-forward year (rate-shock features: AOCI, HTM share, uninsured deposit ratio) was attempted in the same June 2026 timeframe and abandoned after walk-forward 2023 AUC regressed from 0.702 to 0.507. Documented in §13.5 so future retunes do not repeat the experiment without understanding the failure mode. The next model-version bump after v1.2 will be v1.4.

1. Purpose

The Counterparty Insight Probability of Default (PD) model estimates the probability that a US-domiciled, FDIC-insured depository institution will fail within a 12-month horizon. "Failure" here means the institution is closed by its primary federal regulator and resolved by the FDIC.

The PD score is a complement to, not a replacement for, the existing CI Risk Rating. The Risk Rating is a 16-grade peer-percentile score based on the CAMELS framework (Capital, Assets, Management, Earnings, Liquidity, Sensitivity); it is designed to be intuitive for human review. The PD score is a continuous probability estimate produced by a supervised machine-learning model trained on 17+ years of historical Call Report data and FDIC failure outcomes.

Both metrics are presented side-by-side on the bank profile to give users a familiar rating and a finer-grained, statistically calibrated probability.

2. Data Sources

Production data is held per-institution with all Call Report schedules and all available quarters. The failure label set is sourced from the public FDIC Failed Bank List. The training table is produced by the feature-engineering pipeline described in Section 4.

All raw data is sourced from public regulatory filings. No customer or proprietary data is used in training.

3. Label Definition

For each (institution, quarter-end) observation in the training set, the binary label fails_within_4q is set to 1 if the institution appears in the FDIC Failed Bank List with a closure date that falls within four quarters after the observation quarter, and 0 otherwise.

Closure date is mapped to the calendar quarter-end containing the date (a closure on 2009-04-15 maps to 20090630). Banks closed on the observation date itself are excluded from training to avoid trivial label leakage.

Voluntary mergers, acquisitions, and conversions to a different charter type are not treated as failures. Only FDIC-resolved closures count.

4. Feature Set

The current production model (v1.2) uses 19 engineered features drawn from Call Report schedules RC, RCCI, RCM, RCN, RCRI, RI, RIBI, and RCE. The 15 original features (v0) cover size, capital, asset quality, earnings, liquidity, concentration, and trend. v1.2 added four loss-progression signals (marked v1.2 below) that close the community-bank failure-mode gap v1.1 missed (high Texas Ratio + positive earnings + deferred recognition). The exact field list is locked at training time and versioned (see Section 13).

Notes.

• v1.1 and later extract features from each quarter (Q1, Q2, Q3, Q4). Income-statement items are computed on a TTM (trailing-twelve-months) basis: for Q4 observations TTM equals YTD by construction; for Q1, Q2, and Q3 observations TTM is computed as YTD(current quarter) + YTD(prior year Q4) − YTD(same quarter prior year). Banks lacking the prior-year reach-back data fall back to annualization of the current-period YTD.
• For dual-prefix MDRMs (RCFD/RCON for big-bank vs. small-bank reporting), we try both at extraction time and use whichever is populated.
• Cross-period features (asset_growth_yoy, asset_growth_volatility, allowance_growth_qoq, texas_ratio_chronic_quarters) require at least one same-quarter-prior-year observation per bank; new charters lacking that history get NaN and are handled natively by XGBoost.
• Texas Ratio NPL is computed as the sum of column-B (90+ past due, accruing) + column-C (nonaccrual) across all loan types in RC-N. There is no single "total noncurrent loans" MDRM in the schedule, it must be summed. We deliberately exclude column-A (30-89 day past due) because most cure and the noise dilutes the signal.

Features evaluated and dropped from v0: Tier 1 risk-based ratio, total risk-based ratio (regulatory regime change in 2015 broke historical continuity); ROE (correlated with ROAA, no marginal lift); per-loan-type granular concentration features (construction-only, C&I-only, etc., overlapping with the rolled-up cre_concentration and loans_to_assets).

Top features by gain importance (full-population training): texas_ratio 55-65%, roa 8%, equity_to_assets 5-7%, efficiency 4%, brokered_to_deposits 4%, liquid_to_assets 3-5%. The other features each contribute 1-3%. cre_concentration consistently scores 0% - XGBoost finds it redundant with texas_ratio, which captures the post-stress version of the same signal. We retain it for interpretability and in case future retraining surfaces it.

Older periods may not have all modern fields (some 2006-era Call Reports lack post-Dodd-Frank disclosures like brokered deposits via RCONK220). The XGBoost model handles missing values natively; we do not impute - a missing value carries information about the period in which it was reported.

5. Model Algorithm

We use XGBoost (eXtreme Gradient Boosting) - an ensemble of boosted decision trees. We selected XGBoost over the alternatives for these reasons:

Hyperparameters are tuned via 5-fold time-series cross-validation. Final hyperparameters are recorded in the model artifact metadata.

6. Training Setup

Three-way split: train → calib → test. The production model uses an out-of-time test set with a stratified random train/calib split inside the pre-test data:

• Test (held-out, out-of-time): observations with period_year > 2014. The test period covers the 2015 normalization, 2017-2019 expansion, 2020 COVID, 2023 regional bank stress (SVB, Signature, First Republic), and 2024-2025.
• Pre-test data: observations with period_year ≤ 2014 (most positive-label concentration in the 2008-2010 crisis years).
• Train (XGBoost fit): 80% of pre-test, stratified random by label.
• Calib (Platt scaling fit): 20% of pre-test, stratified random by label.

Row counts shown below reference the v0 annual-cadence training sample (Q4 observations only). v1.1 moved to quarterly observations (roughly 4× the row counts) with positive-label totals essentially unchanged after the v1.1 label-tightening rule (only the four-quarter-prior observation marks positive). The split design is the same in v1.1 and v1.2.

• v0 (annual): ~57k test observations / 30 positives; ~65k pre-test / 457 positives; ~52k train / 366 positives; ~13k calib / 91 positives.

Why stratified-random instead of fully time-based? An earlier iteration used a strict time-based split for the calib set (years 2013–2014 only). That gave 26 positives in calib, all from a recovery cohort - Platt scaling fit on those was unstable and tended toward a near-constant near-zero output. Switching to stratified-random across the full pre-test window gives ~91 calib positives spanning the cycle (74% from 2008–2010 crisis years), which is a much more realistic through-the-cycle sample to calibrate on.

The test set remains strictly out-of-time so out-of-sample performance metrics are honest.

Class imbalance. Failures are rare (~573 over 25 years, ~0.1% per (cert, quarter) observation under v1.1 and later). We use XGBoost's scale_pos_weight set to the inverse class ratio in the training fold so the model doesn't degenerate to "always predict survival."

6.1 Probability Calibration

XGBoost's raw predict_proba output is well-ordered (good discrimination) but absolute probabilities are not directly trustworthy - class-imbalance reweighting and the boosting algorithm distort the magnitudes. We post-process raw scores through Platt scaling (sigmoid):

calibrated_logit = a · raw_logit + b where (a, b) are fit on the calib set's true labels via logistic regression on the raw model logit.

We initially tried isotonic regression for calibration. Isotonic is non-parametric and theoretically stronger, but with only ~91 positives in our calib set it produced a near-constant step function that wiped out discrimination. Platt's parametric form (two parameters) is much more stable for rare-event calibration; ROC AUC is preserved (Platt is monotonic in raw score), and Brier score on the test set drops from ~0.0099 (raw) to ~0.0005 (calibrated).

Platt parameters are persisted alongside the trained model and applied in production scoring before any tier mapping.

6.2 Hyperparameters

XGBoost: n_estimators=300, max_depth=4, learning_rate=0.05, eval_metric='aucpr', early_stopping_rounds=25. Early stopping uses the calib set as the eval set (the closest available out-of-sample cohort). Hyperparameters were chosen by hand against the calib set's PR AUC; future retrains should automate via 5-fold time-series CV.

7. PD-to-Rating Calibration

Current overlays (v1.2.1): the 17-letter agency-style overlay described here remains the primary UI surface. v1.2 introduced an empirically-anchored 5-tier overlay (Severe / High / Elevated / Moderate / Low) calibrated to realized FDIC failure rates from the walk-forward backtest; v1.2.1 kept the letter overlay as primary alongside it. The numeric 1 to 80 score and the calibrated PD are unchanged across all versions. Production responses include both pd_letter and risk_tier fields on v1.2 and later vintages.

The model outputs a continuous calibrated PD between 0 and 1. We map this to an 80-point ordinal scale for display:

• 80 grades, indexed from PD1 (lowest PD, highest credit quality) to PD80 (highest PD, near-failure). Lower number = better credit, matching industry Master Scale conventions.
• Letter overlay (primary on v1.1 and later; restored as primary in v1.2.1 after v1.2 briefly dropped it): each grade rolls up into one of 17 familiar rating letters. The numeric notch is the position within the letter tier; there are no +/- modifiers within a tier - the numeric score itself disambiguates (e.g., PD13 and PD22 are both AA+, but PD13 is materially better).

7.1 Letter-tier boundaries

The boundaries are based on the founder's banking-industry experience; the wider AAA/AA+/B-/CCC tiers reflect that fine PD-level distinctions are less actionable at the extremes (a "very safe" or "distressed" call is the headline; the precise notch is supplementary).

7.2 Hybrid scoring method

The mapping from calibrated PD → 1-80 score uses a hybrid approach that combines rank-based and absolute-PD methods. Both methods were evaluated alone before settling on the hybrid:

The hybrid rationale. The model's calibrated PD distribution is heavily bimodal:

• ~86% of banks cluster at the model's "healthy" floor (PD ≈ 0.01%, with little real spread within the cluster - variance is mostly noise).
• ~14% form a "risky" cluster spanning PD 0.04% to 56% (the Platt sigmoid ceiling).
• Almost nothing in between.

Within the safe cluster, fine PD differences (0.0099% vs. 0.0102%) are not real signal - the model can't differentiate that finely. Forcing a fixed-PD-band mapping crushes the entire safe population into a single tier. Within the risky cluster, the model DOES produce real PD spread (0.05% vs. 5% vs. 50% are meaningfully different), and a fixed-band mapping makes those scores cycle-meaningful (a bank moving from 1% to 5% PD changes tier under fixed bands; under rank-based it might not move if the population shifts together).

The hybrid uses each method where it works:

• Safe cluster (calibrated PD < 0.04%): rank-based mapping within the safe cluster only. Anchored breakpoints assign scores 25-51 (AA boundary down to BBB-) by percentile, with a deliberately steep ramp at the top (top 0.1% gets AA, next 5% gets A+, etc.) so AAA/AA+ stay effectively unreachable and AA is reserved for a handful of banks.
• Risky tail (calibrated PD ≥ 0.04%): fixed PD bands map scores 52-80 (BB+ through CCC). Bands are designed for a meaningful population in the B tier (PD 1-95% maps to B+ through B-) with CCC reserved for near-default (PD > 95%).

CCC threshold caveat. The model's Platt sigmoid saturates at ~55.69% PD - no calibrated PD exceeds that ceiling. Setting CCC = "PD > 95%" therefore yields zero CCC banks in the current run, which is intentional: CCC is reserved for "essentially defaulted" and our model can't currently identify that with confidence (110 banks pile up at the saturation ceiling and land in B-). A future improvement would supplement with the FDIC enforcement actions / problem-bank list as an external CCC overlay.

7.3 Resulting distribution (current v1.2 production run, Q1 2026)

9,284 FDIC-insured commercial banks scored:

Investment grade (BBB- and above): ~91.7%. Speculative (BB+ and below): ~8.3%. The A peak reflects that the FDIC-insured population's median PD (~18 bps) lands at the master scale's A band (12-27 bps), and the Master Scale convention was originally calibrated to corporate-bond defaults rather than to bank failures. See the PD distribution explainer for the full unpacking.

The earlier v0 scoring run (April 2026, Q4-only training cadence and 15 features) was BBB--modal (~32% at BBB-, ~4% at A) reflecting that vintage's different calibration; the change to A-modal under v1.2 is driven by quarterly observations + the four loss-progression features compressing the safe-cluster spread.

7.4 PD score vs. CI Risk Rating

The PD score is not equivalent to the existing CI Risk Rating. The CI Risk Rating is a peer-percentile composite using CAMELS components (Capital, Assets, Management, Earnings, Liquidity, Sensitivity); the PD score is a supervised-learning probability estimate. They are correlated (both proxy credit quality) but use different signals and may disagree at the margin - we display both side-by-side on the bank profile and surface divergences as a feature (Section 8.1).

8. Validation

The model is evaluated using two complementary approaches: a single-split out-of-sample test (Section 6 train/calib/test), and a year-by-year walk-forward backtest.

8.1.0 Single-split test (year > 2014)

Trained on period_year ≤ 2014 (stratified-random 80/20 train/calib), evaluated on period_year > 2014:

8.2 Walk-forward backtest (2010-2025)

The walk-forward backtest is the regulatory gold standard for PD models. For each year Y in 2010-2025 we train on period_year < Y (strictly prior), score the year-Y observations on the v1.2 feature set, and evaluate against year-Y labels (failures in year Y+1).

This is more honest than the single-split test because it measures performance the way the model would actually be used in production: at any point in time, you only have data up to "now," and you're asking the model to predict who fails next year.

Aggregate results (mean across 13 valid eval years; 2017, 2020, 2021 skipped - zero positives in eval cohort):

The lift numbers are lower than the v0 single-split test because v1.2 scores all four quarters per bank-year (~20,000 observations/year vs. v0's ~5,000 Q4-only), so "top 5%" is a larger absolute net that must concentrate failures across four-times-as-many observations. Year-over-year movement is the relevant comparison; the absolute levels are not comparable to a Q4-only run.

Read this as: "Across 2010-2025, the v1.2 model produced mean ROC AUC 0.9253 against the year-ahead failure label, and at any given year-end placed roughly half of the next-year failures in its riskiest 5% of scored bank-quarters."

Per-year detail (v1.2):

Years with very low failure counts have noisy metrics (PR AUC and percentile lifts are very sensitive to the base rate when there are only one or two positives). 2023 is the one year with AUC < 0.80 - the rate-shock cohort (SVB, Signature, First Republic) whose failure pattern was not present in pre-2023 training data; an attempted v1.3 rate-shock retrain regressed 2023 further and was abandoned (see §13.5).

Confidence intervals and the rate-shock gap. Bank failures are sparse, so the per-year and aggregate AUC figures are point estimates carrying wide confidence intervals; they should be read as directional evidence of strong discrimination, not as precise values. The 2023 rate-shock weakness is a known, currently-unmitigated failure mode for the PD model in isolation. As an interim compensating control until a v1.4 model captures it, rate-shock exposure is cross-checked through the CAMELS Risk Rating's Sensitivity and Capital components (HTM unrealized loss versus Tier 1 capital, securities-to-assets, and uninsured-deposit share), which are built to flag exactly the AOCI and deposit-flight dynamics the PD model under-weighted in 2023.

The backtest validates that the model's discrimination is strong across multiple economic regimes (2008-2010 financial crisis, 2011-2014 recovery, 2015-2019 expansion, 2020 COVID, 2022-2025 high-rate stress), not just in-sample on a particular split.

Stability monitoring. Population Stability Index (PSI) monitoring shipped with v1.2.1: a quarterly per-feature and per-prediction PSI check against a frozen reference distribution built from the full pre-2014 training population. Industry-standard thresholds apply (PSI < 0.10 stable; 0.10 to 0.25 investigate; > 0.25 recalibrate). Surfaced via an in-product PD Stability dashboard.

8.2.1 Lead-lag analysis vs. traditional ratio screens

Beyond walk-forward AUC, we ran a head-to-head comparison of when each signal first fires before failure. For each of 497 historical failures with sufficient pre-failure data, we walked backward through the bank's quarterly observation history (v1.2 uses all four quarters, not just Q4) and recorded the earliest quarter each signal crossed an "elevated risk" threshold.

Capture rate (% of failures each signal caught at least once):

PD model thresholds are persisted as model artifacts and are calibrated to the realized failure rate observed at each band's PD floor in the v1.2 walk-forward backtest, not to agency idealized PDs.

Head-to-head - PD model (Elevated tier) vs. the three core ratio screens (Texas + Capital + ROA):

The PD model never missed a failure that a ratio screen caught, and caught 6 failures the ratio screens missed entirely.

Of the 489 failures caught by both, looking at which fired first:

Median PD lead time: 12 quarters (3 years). Median traditional lead time: 11 quarters.

This inverts the v0 analysis's "PD fires first 3.2%" finding. Three things changed between v0 (April 2026) and v1.2 (June 2026): (a) v1.2's larger feature set including reserve_to_npl + allowance_growth_qoq picks up the recognition-lag pattern earlier; (b) v1.2 scores all four quarters per year vs v0's Q4-only; (c) band edges are anchored to the realized failure-rate cliff instead of to agency idealized PDs, so the Elevated cut sits at a meaningfully earlier signal.

Caveat: PD scores on pre-2015 observations are partly in-sample (v1.2 was trained on observations through 2022 in the production model; in the walk-forward, years before 2015 contributed both training and test data depending on the fold). Lead-time numbers above are an upper bound for the 2007-2014 portion of the failure cohort. The walk-forward backtest in Section 8.2 is the unbiased counterpart and gives mean ROC AUC of 0.9253 across 13 evaluation years.

For a deeper validation pack (the per-quarter lead-time tables, reproducible scoring artifacts), contact info@counterpartyinsight.com.

9. Production Inference

Scoring runs quarterly (with ad-hoc reruns at quarter-end as FFIEC Call Reports land). The trained model is applied to every active FDIC-insured institution using its most recent quarterly Call Report; output is a per-bank payload with the calibrated PD, the discretized score, the assigned letter grade, the risk tier, and the contributing feature attributions. Score payloads are persisted as an immutable vintage history so prior-quarter results remain available for trend and backtest comparison.

The score payload (per bank) includes:

Still not yet planned beyond v1.2 patches:

• Daily EventBridge Lambda automation (currently manual; quarterly cadence after each FFIEC release is the operational rhythm).
• Per-quarter snapshot output (pd-scores/snapshots/{repdte}.json) for trend / migration analytics. Currently every quarterly scoring run writes a dated history file at pd-scores/v1_2/history/, which covers this use case until we need a dedicated snapshot index.

PD scores are gated behind the same paid subscription tier as the existing CI Risk Rating.

10. Limitations & Known Caveats

Not a credit rating. Counterparty Insight is not a credit rating agency and is not registered as a Nationally Recognized Statistical Rating Organization (NRSRO) under Section 15E of the Securities Exchange Act. The probability-of-default estimates and letter grades this model produces are independent, quantitative analytical assessments for informational use only. They are not "credit ratings" as defined under the Credit Rating Agency Reform Act, are not a recommendation to buy, sell, or hold any security or to extend or deny credit, and must not be used as the sole basis for any investment, lending, or counterparty decision. The letter scale is used for interpretability; it is not affiliated with, endorsed by, or derived from any rating agency.

This model is one input to a credit decision and is not a substitute for human judgment, regulatory review, or current public information. Specifically:

• Forward-looking events not in Call Reports (regulatory consent orders, large legal settlements, sudden deposit runs) will not be reflected until the next quarter's filing.
• Idiosyncratic failures (fraud, sudden unrealized loss recognition like SVB's AFS portfolio in 2023) are inherently harder to predict from historical financials alone. The model captured 4 of 5 of the 2023 cluster in the top decile but the dynamics were unusual.
• Coverage gaps: institutions newly chartered (less than 4 quarters of history) are excluded from scoring until a baseline trend is available.
• Non-bank counterparties (broker-dealers, fintechs, insurers) are out of scope. We may build separate models for those segments later.
• Foreign-domiciled banks and US branches of foreign banks are out of scope.

11. Governance, Versioning & Change Management

This model is built and operated under the principles of SR 11-7 Supervisory Letter on Model Risk Management, adapted to a small-team SaaS context.

Model versioning. Each retraining produces a new immutable model artifact identified by a semantic version (major.minor.patch):

• Major version bump: feature set changes, label definition changes, or algorithm change.
• Minor version bump: hyperparameter retuning or training-window extension on the existing feature set.
• Patch version bump: routine retrains (e.g., quarterly retraining as new data lands) with no methodology change.

Retraining cadence. Quarterly, after each Call Report release. The new model is shadow-scored against production for one full week; if its out-of-sample metrics are within ±2% of the previous model's, it is promoted.

Change log. Every model release records:

• Training data window used
• Hyperparameter set
• Validation metrics (ROC AUC, PR AUC, top-decile capture, calibration Brier)
• A diff of the feature set against the prior version (if any)
• The reviewer who approved the release

Independent validation. At least annually, the model is reviewed by an independent reviewer (initially: the Risk Methodology owner reviews a third-party-style replication on a sampled subset of the training data). When customer or regulatory pressure warrants, we engage a third-party model validator.

12. Contact

Questions, methodology comments, or replication requests: info@counterpartyinsight.com

A deeper validation pack, including the trained model artifacts, the walk-forward backtest output files, the full feature dictionary, and the per-period scoring vintages, is available to a customer's model-risk-management team on request.

13. Version History

This section summarizes the methodology evolution of the production model. Internal release notes (per-script details, calibration-run outputs, S3 paths) are maintained separately for the model risk management team and are available on request via info@counterpartyinsight.com.

13.1 v0 - April 2026

• Initial production model: XGBoost binary classifier with Platt-scaling calibration on a held-out calibration set.
• Trained on bank-quarter observations from 2006 - 2014 (Q4 only), labeled by 12-month forward FDIC failure.
• 15 engineered features across capital, asset quality, earnings, liquidity, concentration, and trend categories.
• 17-letter master-scale overlay (AAA to CCC) mapping calibrated to preserve approximately an 85% safe / 15% risky split.

13.2 v1.1 - June 2026

• Switched training observations from Q4-only to all four quarters per bank-year, with TTM income smoothing for non-Q4 periods.
• Failure label tightened: only the four-quarter-prior observation counts as positive (eliminates autocorrelation between near-failure quarters of the same bank).
• Corrected CRE concentration feature to use RCFDF161 (non-owner-occupied NFNR) per SR 07-1 intent.
• Production scoring vintage moved from annual to per-quarter cadence.
• Validation: Brier score improved 66% (0.003164 to 0.001074) on the same out-of-time test split. ROC AUC essentially flat (the additional in-quarter noise admitted by quarterly observations is offset by removing label autocorrelation).

13.3 v1.2 - June 2026 (current production)

• Added four loss-progression features (reserve-to-NPL ratio, allowance growth quarter-over-quarter, provision coverage, count of chronic high-Texas-Ratio quarters) that close the v1.1 blind spot on community-bank failures where a high Texas Ratio coexists with positive earnings and deferred recognition.
• Calibration: fits Platt and isotonic regression in parallel and selects whichever gives the lower test-set Brier score.
• Master-scale overlay: added a 5-tier empirical overlay (Severe / High / Elevated / Moderate / Low) anchored to realized FDIC failure rates from the walk-forward backtest, alongside the existing 17-letter agency-style overlay. v1.2 initially dropped the letter overlay; v1.2.1 restored it as the primary UI surface so both overlays now ship in production responses. The empirical 5-tier anchors are the right calibration target for the bank-failure population; the letter overlay is preferred for at-a-glance scanning and customer familiarity.
• Walk-forward validation: mean ROC AUC 0.9253 across 13 evaluation years (2010 - 2025). Both 2026 target failures (Metropolitan Capital, Community Bank & Trust West Georgia) flagged at the High tier.

13.4 v1.2 patches and v1.2.1

Three non-model improvements shipped post-v1.2 against the same production model:

• SHAP top-driver attribution. Every per-bank score includes the three features with the largest SHAP contribution, so consumers get a grounded "why is this bank flagged" answer without re-running the model.
• Lead-lag refresh. Re-ran the head-to-head against traditional ratio screens (Texas Ratio + Capital + ROA) against v1.2: 99.6% capture rate at the Elevated tier with a median 3-year lead time before failure.
• v1.2.1 - PSI stability monitoring. Frozen reference distribution from the full pre-2014 training population; quarterly PSI computation against the new vintage; PSI dashboard with the standard 0.10 / 0.25 alarm-move-retrain bands so MRM teams can satisfy SR 11-7 ongoing-monitoring expectations.

13.5 Abandoned v1.3 retrain

A retrain attempt targeting the 2023 weak walk-forward year (rate-shock features: AOCI as a percent of Tier 1, HTM share of securities, uninsured deposit ratio) was attempted in June 2026 and abandoned after walk-forward 2023 AUC regressed from 0.702 to 0.507 despite plausible feature design. Root cause: the rate-shock pattern that drove 2023 failures (SVB, Signature, First Republic) was not present in pre-2023 training data, so the model learned the features as noise on the 2008 - 2014 cohort and over-fit them to the wrong direction.

Documented here so future retunes do not repeat the experiment without understanding the failure mode. The next model-version bump after v1.2 will be v1.4, and the strategy will involve either putting 2023 - 2026 failures in-sample with a quarterly forward-monitoring loop replacing walk-forward, or moving to a model family that captures sequential erosion of buffers (out of scope for v1.x).