|
| | | | |
| | | |
| Stat |
Members: 3657
Articles: 2'599'751
Articles rated: 2609
14 October 2024 |
|
| | | | |
|
Article overview
| |
|
|
Pre-training via Denoising for Molecular Property Prediction | | Sheheryar Zaidi
; Michael Schaarschmidt
; James Martens
; Hyunjik Kim
; Yee Whye Teh
; Alvaro Sanchez-Gonzalez
; Peter Battaglia
; Razvan Pascanu
; Jonathan Godwin
; | | Date: |
1 Jun 2022 | | Abstract: | Many important problems involving molecular property prediction from 3D
structures have limited data, posing a generalization challenge for neural
networks. In this paper, we describe a pre-training technique that utilizes
large datasets of 3D molecular structures at equilibrium to learn meaningful
representations for downstream tasks. Inspired by recent advances in noise
regularization, our pre-training objective is based on denoising. Relying on
the well-known link between denoising autoencoders and score-matching, we also
show that the objective corresponds to learning a molecular force field --
arising from approximating the physical state distribution with a mixture of
Gaussians -- directly from equilibrium structures. Our experiments demonstrate
that using this pre-training objective significantly improves performance on
multiple benchmarks, achieving a new state-of-the-art on the majority of
targets in the widely used QM9 dataset. Our analysis then provides practical
insights into the effects of different factors -- dataset sizes, model size and
architecture, and the choice of upstream and downstream datasets -- on
pre-training. | | Source: | arXiv, 2206.00133 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
|
| |
|
|
|
REGISTER