研究：大语言模型难以复现人脑对语序与句子结构的表征
Large language models poorly capture how word order shapes representations of sentence meaning in the human brain
一项基于 7T fMRI 的研究发现，基于 transformer 的大语言模型在表征需要语序和语义角色（谁对谁做了什么）的句子时，与人脑语言区的表征匹配较差。研究分析了 30 名参与者各阅读 108 个句子的脑活动，这些句子经过设计以区分句子结构与词义；作者用表征相似性分析比较了 8 个 transformer 和 11 个其他语义模型与语言区脑活动的对应关系。结果显示，transformer 整体优于完全忽略词序的模型，但显著不如显式编码词间结构关系的模型。作者据此强调句子结构在人脑句子表征中的关键作用，并对 transformer 与人脑以相似方式表征句子含义的说法提出质疑。
来源：eLife · doi.org
Large language models based on the transformer architecture are now capable of producing human-like language. But do they encode and process linguistic meaning in a human-like way? Here, we address this question by analysing 7T fMRI data from 30 participants reading 108 sentences each. These sentences are carefully designed to disentangle sentence structure from word meaning, thereby testing whether transformer representations of sentence meaning resemble those formed by the brain. Using representational similarity analysis, we compared representations from eight transformers and eleven other semantic models to brain activity in language regions of interest. We found that while transformer models match brain representations better than models that completely ignore word order, all transformers poorly match human brain representations of sentences that require sensitivity to structural relations such as semantic roles (who did what to whom). Further, transformers were significantly inferior to models explicitly designed to encode the structural relations between words. Our results provide insight into the nature of sentence representation in the brain, highlighting the critical role of sentence structure. They also cast doubt on the claim that transformers represent sentence meaning similarly to the human brain.