# SPECTRA Generalizability Finding: ESMFold2

## Decision

Claim-valid threshold SPC found.

For exact `biohub/ESMFold2` (not Fast), evaluated as single-chain protein sequence inference with `num_loops=3`, `num_sampling_steps=14`, and one diffusion sample, performance on post-cutoff CAMEO 2025 targets degrades sharply when prospective evolutionary support is low.

## Claim Boundary

This claim applies to the local Hugging Face `biohub/ESMFold2` model in single-sequence protein mode without external MSA conditioning. The evaluated pool contains 138 CAMEO targets with usable experimental PDB chain references and sequence length <=650 aa. One eligible 921 aa high-MSA, medium-difficulty target was not evaluated under the predefined cap.

## Primary Axis

Prospective similarity: `log10(nonquery MSA rows + 1)` computed from the CAMEO A3M files before target-model scoring. Higher values mean more evolutionary-family support. Split membership did not use ESMFold2 errors, ESMFold2 confidence, or reference/prediction comparisons.

The densified MSA curve shows a threshold around `log10(nonquery MSA rows + 1) = 3.0` (about 1,000 nonquery MSA rows):

- High MSA support (`>=3.0`): n=105, mean C-alpha lDDT=0.919, mean GDT-TS=0.793.
- Low MSA support (`<3.0`): n=33, mean C-alpha lDDT=0.723, mean GDT-TS=0.556.
- Drop: C-alpha lDDT=0.196; GDT-TS=0.237.

The continuous MSA-depth association was Spearman rho=0.379, p=4.49e-06. The shape is threshold/localized rather than strictly monotone inside the high-support bins.

## Supporting Axis

CAMEO difficulty, a prospective benchmark metadata field aligned with homology/evolutionary support, independently gives a monotone curve:

- Easy: n=57, mean C-alpha lDDT=0.918, mean GDT-TS=0.855, mean MSA log-depth=3.526.
- Medium: n=67, mean C-alpha lDDT=0.861, mean GDT-TS=0.676, mean MSA log-depth=3.154.
- Hard: n=14, mean C-alpha lDDT=0.737, mean GDT-TS=0.543, mean MSA log-depth=1.696.

## Controls

Fixed extended-chain baseline did not reproduce the ESMFold2 degradation: baseline C-alpha lDDT was near 0.06-0.09 across bins.

Length did not explain the primary degradation in this pool. The predeclared length quintile curve was non-explanatory for mean C-alpha lDDT, and Spearman length vs target C-alpha lDDT was positive (rho=0.318, p=1.42e-04). Reference resolution was also not explanatory (rho=0.043, p=6.19e-01).

The low-MSA coefficient remained about -0.19 C-alpha lDDT after controlling for length, reference resolution, fixed baseline score, and CAMEO difficulty.

## Beyond-MSA Sequence-Composition Result

A follow-up SPECTRA run found a second claim-valid axis beyond evolutionary support: a sequence-intrinsic flexible/non-helical composition proxy, defined as the mean percentile rank of `proline_fraction`, `chou_beta_minus_helix`, and `chou_turn_minus_helix`. Higher values indicate more proline-rich and beta/turn-favored relative to helix-favored composition.

On the fixed 138-target CAMEO pool, the highest-stress level had mean CA-lDDT 0.719 and was -0.179 CA-lDDT, -0.290 GDT-TS, and 4.537 A CA-RMSD worse than lower levels. On held-out confirmation folds 1-2, the highest-stress level was -0.199 CA-lDDT and -0.345 GDT-TS worse than lower levels. The trend is a localized high-tail threshold rather than a globally monotone curve.

The result persisted under the ESMFold2 inference-time scaling setting of 10 loops, 68 sampling steps, and 5 diffusion samples. On all 20 high-tail targets plus 20 matched lower-stress controls, best-by-pTM selection gave high-tail mean CA-lDDT 0.724 versus 0.890 for controls, a -0.166 CA-lDDT gap. The same matched panel had a -0.166 CA-lDDT gap before scaling, so inference-time scaling did not rescue the axis.

An external RCSB same-bucket expansion gave weaker support. Applying the same frozen bucket definition to recent RCSB protein chains released on or after 2025-01-01 produced 17 high-bucket chains and 17 matched lower-bucket controls. Under the same 10-loop, 68-step, 5-sample setting, best-by-pTM selection gave high-bucket mean CA-lDDT 0.837 versus 0.912 for controls, a -0.075 CA-lDDT gap. The paired median delta was +0.003 and high-bucket chains were worse in 7/15 finite matched pairs. This does not invalidate the CAMEO finding, but it means the flexible/non-helical bucket is not yet externally confirmed as a broad portable failure mode.

## Artifacts

- `spectra_ready_dataset/candidate_features.csv`
- `target_model_results.csv`
- `spectral_performance_curve.csv`
- `axis_ledger.json`
- `control_analysis.json`
- `spectra_loop_state.json`
- `commands_run.json`
- `beyond_evolutionary_support_sequence_composition_spc/final_report.md`
- `beyond_evolutionary_support_sequence_composition_spc/inference_time_scaling_10loops68steps_5samples_full_top_tail_matched/scaling_report.md`
- `beyond_evolutionary_support_sequence_composition_spc/external_rcsb_same_bucket_expansion_20260531/external_expansion_report.md`