LaTeX to Word

---
title: Attention-Weighted Physics-Informed Neural Networks for the Unsteady Navier–Stokes Equations
author: Ming Wang^1^, Hua Li^1,2^, Siyuan Chen^2^
date: September 2026
abstract: Physics-informed neural networks (PINNs) often fail to converge on unsteady flows at high Reynolds numbers because the gradients of their loss terms are badly out of balance. We propose Attn-PINN, which uses an attention mechanism to weight the residuals adaptively, and prove an upper bound on its approximation error. For flow past a cylinder ($\mathrm{Re}=3\,900$), Attn-PINN reaches a relative $L^2$ error of $1.7\times10^{-3}$ — **an order of magnitude** below the baseline — while training 41% faster.
bibliography: refs.bib
number-sections: true
header-left: Attn-PINN for the Unsteady Navier–Stokes Equations
---

# Introduction

PINNs embed the residual of a partial differential equation in the loss of a neural network [@raissi2019]. They need no mesh and solve forward and inverse problems alike, and have become a core method of scientific machine learning [@karniadakis2021]. For the strongly nonlinear Navier–Stokes equations, however, the gradients of the individual loss terms can differ by more than $10^{3}$, trapping training in a “gradient pathology” [@wang2021]. Neural operators [@li2021] ease the problem but need large amounts of labelled data. Borrowing the attention mechanism of the Transformer [@vaswani2017], we let the network **learn** the weight of every collocation point during training.

# Problem statement {#sec:problem}

Consider incompressible viscous flow in a domain $\Omega\subset\mathbb{R}^2$. The velocity $\mathbf{u}=(u,v)^{\top}$ and pressure $p$ satisfy

$$
\left\{
\begin{aligned}
\frac{\partial \mathbf{u}}{\partial t} + (\mathbf{u}\cdot\nabla)\mathbf{u}
  &= -\frac{1}{\rho}\nabla p + \nu\,\nabla^{2}\mathbf{u} + \mathbf{f},
  && (\mathbf{x},t)\in\Omega\times(0,T],\\
\nabla\cdot\mathbf{u} &= 0,
  && (\mathbf{x},t)\in\Omega\times(0,T],\\
\mathbf{u}(\mathbf{x},0) &= \mathbf{u}_0(\mathbf{x}),
  && \mathbf{x}\in\Omega,
\end{aligned}
\right.
$$ {#eq:ns}

where $\nu$ is the kinematic viscosity. With networks $\hat{\mathbf{u}}_{\theta}$ and $\hat{p}_{\theta}$ approximating the solution, the momentum residual is
$\mathbf{r}_{\theta} = \partial_t\hat{\mathbf{u}}_{\theta} + (\hat{\mathbf{u}}_{\theta}\cdot\nabla)\hat{\mathbf{u}}_{\theta} + \rho^{-1}\nabla\hat{p}_{\theta} - \nu\nabla^{2}\hat{\mathbf{u}}_{\theta} - \mathbf{f}$.

# Method {#sec:method}

## Attention-weighted residuals

For $N_r$ collocation points $\{\mathbf{x}_i\}$, the weights come from scaled dot-product attention over residual features:
…