SNPBERT 在 UK Biobank 基因型上自监督预训练，产出可迁移表征
Biobank-scale self-supervised genotype pretraining yields transferable representations for human genetics
研究团队提出 SNPBERT，一个在 485 923 名 UK Biobank 参与者的基因型上用掩码 rsID 预测做自监督预训练的 transformer。其嵌入在 UK Biobank 和三个外部数据集中恢复了已知群体结构，表现与主成分分析相当，并支持识别祖源信息标记 SNP。用基于注意力的多示例头微调后，可生成疾病特异风险评分，为传统多基因风险评分提供补充信息，Nagelkerke R2 相对提升 0.7% 至 3.3%，在所考察的 8 种慢性病中有 7 种观察到独立关联。
来源：Nucleic Acids Research · doi.org
Biobank-scale genotyping has created unprecedented cohorts for human genetics, yet the field still lacks scalable methods for learning transferable representations from largely unlabeled individual-level genotype data. Here, we introduce SNPBERT, a self-supervised transformer pretrained on genotypes from 485 923 UK Biobank participants using masked rsID prediction. SNPBERT embeddings recovered known population structure across UK Biobank and three external datasets, performed comparably to principal component analysis, and supported the identification of ancestry-informative marker SNPs. Fine-tuning with an attention-based multiple-instance head generated disease-specific risk scores that provided complementary information beyond conventional polygenic risk scores. The relative increase in Nagelkerke’s R2 ranged from 0.7% to 3.3%, and independent associations were observed for seven of the eight chronic diseases examined. In celiac disease, attention prioritization recapitulated major histocompatibility complex signals while elevating variants in CCHCR1 and NOTCH4 relative to genome-wide association study (GWAS) ranking. These results support biobank-scale genotype pretraining as an early step toward foundation modeling for individual-level genotype data.