Summary
A bioRxiv preprint introduces AuditPPI, a framework for examining the signals behind protein-interaction predictions. Its authors report that benchmark performance can reflect cellular context and other cues, with limited evidence that the predictive signals specifically capture interaction interfaces.
A bioRxiv preprint from researchers at Shanghai Jiao Tong University introduces AuditPPI, a framework for examining what protein language models use when predicting protein-protein interactions. The authors report that the framework achieves competitive predictive performance across diverse benchmarks, while its audits find that benchmark scores can reflect contextual signals rather than features specifically tied to where proteins bind.
Turning model features into auditable pairs
Protein-protein interaction (PPI) prediction aims to identify which proteins associate. Sequence-based methods use patterns in amino-acid sequences to make those predictions. But a strong score on a benchmark does not, by itself, show whether a model has learned something specific about a protein pair or has relied on other patterns in the data.
AuditPPI starts with features from a frozen protein language model: its parameters are held fixed while the framework analyses the model’s representations. It uses a sparse autoencoder (SAE) to produce features, then combines them into order-invariant pair fingerprints. “Order-invariant” means the representation does not depend on which partner is listed first. The fingerprints are used in a tabular-learning setup, which lets the researchers inspect signals at both the protein and pair levels.
This approach is intended to make prediction signals easier to audit, rather than treating a model’s accuracy as an explanation for its decisions.
Benchmark results include signals beyond the pair
At the protein level, the authors find that partner-independent participation signals remain substantial in conventional benchmarks, including a protein-disjoint benchmark. These signals were largely insufficient for distinguishing individual pairs in a benchmark controlled for topology and degree—properties describing the structure and connectivity of the interaction network.
At the pair level, protein-disjoint evaluation did not remove all contextual structure. The authors report strong subcellular co-localization signals: the proteins in a pair tend to be associated with the same cellular location. They also find benchmark-dependent feature matching, captured through SAE feature co-activation and absolute differences. These results indicate that separating proteins between training and test sets does not necessarily remove every cue that can help predict benchmark labels.
The structural analyses offer a further distinction between prediction and mechanism. For the protein-disjoint benchmark, apparent enrichment of features at interaction interfaces was no longer detectable after the analysis controlled for surface exposure. Contact-specific enrichment appeared for only a small fraction of the SAE feature pairs that could be tested.
Why the distinction matters
A predicted interaction and a learned signal that corresponds to the physical interface between two proteins are different kinds of evidence. AuditPPI’s central contribution, as described by its authors, is a way to examine that difference across benchmarks. Its results suggest that benchmark performance can be supported by biological context or dataset-specific patterns even when the evidence for interface-specific signals is limited.
That distinction matters when interpreting sequence-based PPI predictions: predictive success can be useful without establishing that a model has captured the molecular basis of an interaction. The work is presented as a bioRxiv preprint.