Physics informed neural network consisting of two decoupled stages

Abstract

This paper proposes a two-stage physics informed neural network (PINN) along with an effective training approach for it. The first stage network output that roughly approximates the solution of a partial differential equation (PDE) is fed as input to the second stage which yields a significantly improved approximation. Due to the use of different sets of training samples for the two stages, and particularly, the decoupling of the two stages by not backpropagating the gradients through the second stage to the first stage to update its parameters, the proposed system effectively generalizes the prediction of the PDE solution to the test sample points. For four PDEs of interest, the proposed two-stage system is shown to yield significantly lower test losses than the reference system (original PINN) that has the same total number of layers and similar total training complexity, and whose layers are all together trained with a single set of training samples. It is shown that the performance advantage is due to the long tailed nature of the distribution of frequencies of the PDE solution.