Functional-Group Knowledge Is Structured in Language Models Before It Is Spoken

| Source: arXiv AI

Tags: Interpretability, Representation Probing, Chemistry LLMs, Functional Groups, Fine-Tuning

A probing study of five open-source models finds they organize chemical functional groups internally by family even when they name them wrongly, suggesting output-only chemistry evals understate what models know.

Details

Chemistry and materials agents rely on language models to read molecular structure, but outputs alone cannot tell whether a model missed a feature or saw it and failed to say so. The authors read internal representations of five open-source models on prompts covering 92 molecules and 20 functional groups. Without chemistry training, the models group functional groups by chemical family and encode differences as consistent directions; for example, the thioether-to-thiol shift added to ether lands nearest alcohol. This structure appears abruptly in the middle layers. The base model names the right group in only 58.5% of prompts, the lowest of the five, yet still encodes the families, and its errors are mostly non-answers. Chemistry fine-tuning reorganizes representations around functional groups rather than wording.