From Language to Cognition: How LLMs Outgrow the Human Language Network - ACL Anthology

This academic paper discusses how large language models (LLMs) exhibit similarities to neural activity in the human language network, but the underlying properties and changes across training remain unclear. The study benchmarks 34 training checkpoints across 8 model sizes and finds that brain alignment is more closely related to formal linguistic competence than functional competence. While functional competence continues to develop, its relationship with brain alignment is weaker, suggesting the human language network primarily encodes formal linguistic structure. The correlation between next-word prediction, behavioral alignment, and brain alignment fades once models surpass human language proficiency. The study also shows that model size is not a reliable predictor of brain alignment and that language brain alignment benchmarks remain unsaturated, highlighting opportunities for improving future models.