Advances on Model Merging for Physics-Informed Neural Networks (PINNs)

Abstract

Physics-Informed Neural Networks (PINNs) have emerged as a powerful paradigm for solving partial differential equations by incorporating physical laws directly into neural network training. However, scaling PINNs to large domains and capturing high-frequency solution components remains challenging due to increased complexity and optimization difficulties. In this work, we introduce a novel nonlinear model merging approach for PINNs that extends beyond traditional linear combinations of model parameters. We demonstrate our approach on the linear damped oscillator problem. The experimental results show consistent improvements over both pretrained baselines and linear merging approaches, suggesting that the novel technique can effectively assist not only PINNs on larger domains but also facilitate integration of specialized submodels for large-scale foundation models.