Language models stay blind to atomic geometry even at 70 billion parameters, a materials study finds
LLMs fail to learn from atomic coordinates at up to 70 billion parameters and 2 million structures, a paper on nature.com reports. Specialized geometric architectures still win when structure dominates the property, the authors report.
The Scientist · Science desk
What happened
- The authors tested both causal and masked language models and report that each systematically fails to learn from coordinate information.
- The models did well on category information, meaning which elements are present, and struggled with how atoms are arranged in space.
- The blindness also held across text representation strategies, the abstract says.
- Materials science has long relied on hand-crafted features or graph neural networks for these predictions, and researchers increasingly try LLMs as a general-purpose alternative.
Why it matters
- decision Choosing between an LLM and a geometric model now turns on how much of the target property structure drives, and the authors offer their analysis as guidelines for that choice.
- constraint Within the tested range, neither a larger model nor more data recovered coordinate information, so scale is not the lever for structure-driven tasks.
- contradiction Several earlier studies reported encouraging LLM results on chemistry and materials property prediction, and this paper reports geometric blindness; whether those earlier tasks leaned on composition is the comparison that would reconcile them.
The design is what gives the claim weight. The authors built synthetic datasets in which they could tune how much of a label depends on coordinates and how much on categories, and they paired those with a benchmarking framework, MatText [7]. Because the experimenters set the mix, a model that tracks the category share of the signal and misses the coordinate share cannot blame the data. A single score on real crystals cannot make that separation.
The sample-efficiency result adds a second angle. The authors write that LLMs are "drastically less sample-efficient than conventional approaches when learning from coordinate data" [8]. A slow learner might catch up given enough examples, and scaling the data to 2 million structures [11] is the test of that. The abstract still says LLMs "consistently fail to capture coordinate information" [1].
The thing this doesn't tell you is how big the gap is. The abstract refers to a "GNN-LM wall" in performance benchmarks [6] without reporting error values for it. A team cannot yet weigh that gap against its own accuracy requirement.
The claim is also conditional. The wall appears on tasks dominated by structural effects [6], and the same models excel at category patterns [1]. A MatText score describes the properties in MatText. Whether your target is structure-dominated is a separate measurement.
In our view that measurement comes first. The authors' synthetic datasets vary exactly that quantity, so they are a template for the check.
What to watch
- The full results tables: the size of the error gap between graph networks and LLMs on real MatText properties.
- Whether models above 70 billion parameters, or training sets above 2 million structures, show the same blindness.
- Whether the earlier studies with encouraging LLM results turn out to involve composition-dominated properties when re-scored on tunable synthetic data.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence66
- Adoption
- Insufficient
- Hype gap−5
- Incentives
- Insufficient
- Confidence60
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The authors report that LLMs consistently fail to capture coordinate information in coordinate-category data while excelling at category patterns.
- [2]
The failure to learn from coordinate information persists regardless of model size (up to 70 billion parameters) and dataset scale (up to 2 x 10^6 structures).
- [3]
The failure to learn from coordinate information also persists regardless of the text representation strategy used.
- [4]
The authors report that transformer-based models, both causal and masked language models, systematically fail to learn from coordinate information on coordinate-category data.
- [5]
LLMs excel at learning from category information (which elements are present) but struggle with coordinate relationships (how atoms are arranged in space).
- [6]
For property prediction tasks dominated by structural effects, specialized geometric architectures consistently outperform LLMs, as evidenced by a clear 'GNN-LM wall' in performance benchmarks.
- [7]
The authors ran systematic experiments on synthetic datasets with tunable coordinate and category contributions, and used a benchmarking framework called MatText.
- [8]
The authors write that LLMs are drastically less sample-efficient than conventional approaches when learning from coordinate data.
- [9]
Materials science has long relied on domain-specific models with hand-crafted features or graph neural networks for property prediction, and researchers increasingly turn to LLMs as a general-purpose alternative.
- [10]
Several earlier studies applying LLMs to property prediction in chemistry and materials science reported encouraging results.
- [11]
The largest dataset scale in the study, 2 x 10^6 structures, is 2 million structures.
- [12]
The authors say their analysis provides concrete guidelines for architecture selection in scientific machine learning.
Sources
1 independent publisher whose own reporting we read for this story.
- nature.comLess can be more for predicting properties with large language models
2 articles · October 12, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
- Materials informaticsFollow
- Large language modelsFollow
- Graph Neural NetworksFollow