{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n\n# Train Your Own Neural Network Potential\n\nThis example shows how to use TorchANI to train a neural network potential\nwith the setup identical to NeuroChem. We will use the same configuration as\nspecified in `inputtrain.ipt`_\n\n https://github.com/aiqm/torchani/blob/master/torchani/resources/ani-1x_8x/inputtrain.ipt\n\n
TorchANI provide tools to run NeuroChem training config file `inputtrain.ipt`.\n See: `neurochem-training`.
The training setup used in this file is configured to reproduce the original research\n at `Less is more: Sampling chemical space with active learning`_ as much as possible.\n That research was done on a different platform called NeuroChem which has many default\n options and technical details different from PyTorch. Some decisions made here\n (such as, using NeuroChem's initialization instead of PyTorch's default initialization)\n is not because it gives better result, but solely based on reproducing the original\n research. This file should not be interpreted as a suggestions to the readers on how\n they should setup their models.
Besides defining these hyperparameters programmatically,\n :mod:`torchani.neurochem` provide tools to read them from file.
Pytorch default initialization for the weights and biases in linear layers\n is Kaiming uniform. See: `TORCH.NN.MODULES.LINEAR`_\n We initialize the weights similarly but from the normal distribution.\n The biases were initialized to zero.
The weight decay in `inputtrain.ipt`_ is named \"l2\", but it is actually not\n L2 regularization. The confusion between L2 and weight decay is a common\n mistake in deep learning. See: `Decoupled Weight Decay Regularization`_\n Also note that the weight decay only applies to weight in the training\n of ANI models, not bias.