AlphaFold's near-experimental accuracy did not solve protein folding, DeepMind and Biohub researchers say
AlphaFold predicted protein structures to a median backbone accuracy of 0.96 angstroms in CASP14, matching experimental structures in most cases. Google DeepMind's Pushmeet Kohli and Biohub's Sal Candido call that only the beginning, arguing the protein folding problem is far from solved.
Written for builders.
What happened
- In the CASP14 assessment, AlphaFold's structures were far more accurate than those from every competing method.
- CASP is scored against structures solved just before the contest and withheld from public databases, a blind test the field treats as its gold standard.
- Predicting a protein's three-dimensional shape from its amino acid sequence had been an open research problem for more than 50 years.
- Experimental work has resolved the structures of about 100,000 unique proteins, a sliver of the billions of known protein sequences.
Why it matters
- What it limits constraint A static structure, however accurate, does not capture how a protein moves or where it is disordered, so function remains partly out of reach.
- Who has to decide decision The panel ranks calibrated uncertainty above full interpretability, so the test for trusting a prediction becomes whether the model knows when it is wrong.
- What it limits constraint More compute and more sequence data will not by themselves solve biology, the panel argues, so dataset quality is the limiting factor.
The 0.96 figure is a median: half of AlphaFold's predictions in CASP14, the assessment it entered in mid-2020, sat closer to the experimentally solved structure than that, and half further [6][4]. It is a backbone measure, the root-mean-square deviation of alpha-carbon positions scored at 95 percent residue coverage [6]. The paper's framing is confident: it calls AlphaFold the first computational method that can regularly predict structures to atomic accuracy, even when no similar structure has been solved before [9]. Solving one structure by experiment, for comparison, still takes months to years [2].
Kohli and Candido do not dispute the accuracy. Their case is that an accurate shape is a starting point, not an understanding of the biology behind it [10]. Some of the useful data is counterintuitive: low-quality metagenomic sequence can still improve protein language models [14], and those models may already hold biological knowledge no one has read out of them [15]. Candido ties the longer goal to Biohub's aim of curing all disease, which he frames as a problem that needs 10x breakthroughs, not 10 percent improvements [16].
What to watch
- A benchmark that scores protein dynamics and disorder, not static structure alone, would test the panel's not-solved claim directly.
- Signs that more compute and sequence data keep improving biology models would weigh against the panel's data-quality argument.
- A protein language model shown to contain validated, previously unknown biology would support the claim of hidden knowledge.
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The structures of around 100,000 unique proteins have been determined experimentally, a small fraction of the billions of known protein sequences.
- [2]
Determining a single protein structure requires months to years of painstaking effort.
- [3]
Predicting a protein's three-dimensional structure from its amino acid sequence has been an open research problem for more than 50 years.
- [4]
AlphaFold was entered into the CASP14 assessment, held May to July 2020, as a completely new model under the team name AlphaFold2.
- [5]
CASP is carried out using recently solved structures that have not been deposited in the PDB or publicly disclosed, a blind test long regarded as the gold-standard assessment for structure-prediction accuracy.
- [6]
In CASP14, AlphaFold structures had a median backbone accuracy of 0.96 angstroms r.m.s.d.95, the alpha-carbon root-mean-square deviation at 95% residue coverage.
- [7]
AlphaFold demonstrated accuracy competitive with experimental structures in the majority of cases.
- [8]
In CASP14, AlphaFold structures were vastly more accurate than competing methods.
- [9]
The AlphaFold paper describes it as the first computational method that can regularly predict protein structures with atomic accuracy even when no similar structure is known.
- [10]
Google DeepMind's Pushmeet Kohli and Biohub's Sal Candido argue that AlphaFold's breakthrough was only the beginning and that protein structure prediction is far from solved.
- [11]
The panel argues that scaling compute and data alone will not solve biology.
- [12]
Static structure prediction does not capture protein dynamics and disorder.
- [13]
The panel holds that trustworthiness and uncertainty calibration matter more than full interpretability.
- [14]
Low-quality metagenomic data can improve protein language models.
- [15]
Protein language models may already contain scientific knowledge that researchers have not yet unlocked.
- [16]
Sal Candido argues that curing all disease requires thinking in terms of 10x breakthroughs rather than 10% improvements.
Sources & coverage · 2 publishers
The reporting this story was synthesized from, earliest first. Every link goes to the original.
- pmc.ncbi.nlm.nih.govJul 14Highly accurate protein structure prediction with AlphaFold - PMC