Physics-informed neural network based on Lobatto method and Legendre polynomials for solving differential-algebraic equations

doi:10.11772/j.issn.1001-9081.2024030313

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (3): 911-919.DOI: 10.11772/j.issn.1001-9081.2024030313

• Advanced computing • Previous Articles Next Articles

Physics-informed neural network based on Lobatto method and Legendre polynomials for solving differential-algebraic equations

Shuai LAI¹, Juan TANG¹^,²(), Kun LIANG², Jiasheng CHEN²

^1.School of Computer Science and Cyber Engineering，Guangzhou University，Guangzhou Guangdong 510006，China
^2.Institute of Computing Science and Technology，Guangzhou University，Guangzhou Guangdong 510006，China

Received:2024-03-21 Revised:2024-05-12 Accepted:2024-05-15 Online:2024-06-06 Published:2025-03-10
Contact: Juan TANG
About author:LAI Shuai， born in 1997， M. S. candidate. His research interests include scientific machine learning， artificial intelligence.
LIANG Kun， born in 1999， M. S. candidate. His research interests include scientific machine learning， artificial intelligence.
CHEN Jiasheng， born in 1998， M. S. candidate. His research interests include scientific machine learning， artificial intelligence.
Supported by:
National Natural Science Foundation of China(12201144)

基于Lobatto方法和Legendre多项式的PINN求解微分代数方程

赖帅¹, 唐卷¹^,²(), 梁锟², 陈佳盛²

^1.广州大学计算机科学与网络工程学院，广州 510006
^2.广州大学计算科技研究院，广州 510006

通讯作者: 唐卷
作者简介:赖帅（1997—），男，广东茂名人，硕士研究生，主要研究方向：科学机器学习、人工智能
梁锟（1999—），男，广东茂名人，硕士研究生，主要研究方向：科学机器学习、人工智能
陈佳盛（1998—），男，广东揭阳人，硕士研究生，主要研究方向：科学机器学习、人工智能。
基金资助:
国家自然科学基金资助项目(12201144)

Abstract

Abstract:

Current neural network methods solving Differential-Algebraic Equations （DAEs） basically adopt data-driven strategies， and require a large number of datasets. So that， there are problems such as sensitive structure and parameter selection of neural networks， low accuracy of solution， and poor stability. In response to these issues， a Physics-Informed Neural Network based on Lobatto method and Legendre polynomials （LL-PINN） was proposed. Firstly， based on the discrete Physics-Informed Neural Network （PINN） computing framework， combined with the advantages of high accuracy and high stability of Lobatto IIIA method solving DAEs， the physical information of DAEs was embedded in the Lobatto IIIA time iteration format， and PINN was used to solve the approximate numerical value of this time iteration. Secondly， a neural network structure with single hidden layer was utilized， by using the approximation capability of Legendre polynomials， these polynomials were applied as activation functions to simplify the process of adjusting the network model. Finally， a time domain decomposition scheme was employed to construct the network model， which a differential neural network and an algebraic neural network were used for each equally divided sub-time domain one by one， enabling high-precision continuous-time prediction of DAEs. Results of numerical examples demonstrate that the LL-PINN based on Legendre polynomials and the 4th-order Lobatto method achieves high-precision solutions for DAEs. Compared to the Theory of Functional Connections （TFC） trial solution method and PINN model， LL-PINN significantly reduces the absolute error between the predicted and exact solutions of differential variables and algebraic variables， and improves accuracy by one or two orders of magnitude. Therefore， the proposed solution model exhibits good computational accuracy for solving DAE problems， providing a feasible solution for challenging partial DAEs.

Key words: Differential-Algebraic Equation (DAE), Physics-Informed Neural Network (PINN), Lobatto IIIA method, Legendre polynomial, time domain decomposition

摘要：

当前求解微分代数方程（DAE）的神经网络方法基本都采用数据驱动策略，需要大量的数据集，因此存在对神经网络的结构和参数选择敏感、求解结果精度低、稳定性差等问题。针对这些问题，提出一种基于Lobatto方法和Legendre多项式的物理信息神经网络（LL-PINN）。首先，基于离散型物理信息神经网络（PINN）的计算框架，结合Lobatto IIIA方法求解DAE高精度和高稳定性的优点，将DAE的物理信息嵌入Lobatto IIIA时间迭代格式中，并使用PINN对该时间迭代进行近似数值求解；其次，采用单隐藏层的神经网络结构，利用勒让德多项式展开项的逼近能力，应用这些多项式作为激活函数来简化网络模型调整的过程；最后，采用时间区域分解方案构建网络模型，即对每个等分的子时间区域依次使用一个微分神经网络和一个代数神经网络，从而实现DAE的高精度连续时间预测。数值算例结果表明，基于勒让德多项式和4阶的Lobatto方法的LL-PINN实现了对DAE的高精度求解。与函数连接理论（TFC）试验解模型和PINN模型相比，LL-PINN的微分变量和代数变量的预测解与精确解的绝对误差显著降低，精度提高了一个或两个量级。因此，所提求解模型对求解DAE问题具有较好的计算精度，可为解决具有挑战性的偏DAE提供可行的解决方案。

关键词: 微分代数方程, 物理信息神经网络, Lobatto IIIA方法, 勒让德多项式, 时间区域分解

CLC Number:

O241

Shuai LAI, Juan TANG, Kun LIANG, Jiasheng CHEN. Physics-informed neural network based on Lobatto method and Legendre polynomials for solving differential-algebraic equations[J]. Journal of Computer Applications, 2025, 45(3): 911-919.

赖帅, 唐卷, 梁锟, 陈佳盛. 基于Lobatto方法和Legendre多项式的PINN求解微分代数方程[J]. 《计算机应用》唯一官方网站, 2025, 45(3): 911-919.

Figures/Tables 11

Tab. 1 Coefficient table of Lobatto IIIA method

$c j$	$b 1$	$b 2$	$…$	$b s - 1$	$b s$
$c 1 = 0$	$a 11$	$a 12$	$…$	$a 1, s - 1$	$a 1 s$
$c 2$	$a 21$	$a 12$	$…$	$a 2, s - 1$	$a 2 s$
$⋮$	$⋮$	$⋮$		$⋮$	$⋮$
$c s - 1$	$a s - 1,1$	$a s - 1,2$	$…$	$a s - 1, s - 1$	$a s - 1, s$
$c s = 1$	$a s 1$	$a s 2$	$…$	$a s, s - 1$	$a s s$

Tab. 1 Coefficient table of Lobatto IIIA method

$c j$	$b 1$	$b 2$	$…$	$b s - 1$	$b s$
$c 1 = 0$	$a 11$	$a 12$	$…$	$a 1, s - 1$	$a 1 s$
$c 2$	$a 21$	$a 12$	$…$	$a 2, s - 1$	$a 2 s$
$⋮$	$⋮$	$⋮$		$⋮$	$⋮$
$c s - 1$	$a s - 1,1$	$a s - 1,2$	$…$	$a s - 1, s - 1$	$a s - 1, s$
$c s = 1$	$a s 1$	$a s 2$	$…$	$a s, s - 1$	$a s s$

Fig. 1 Structure of Legendre polynomial activation function

Fig. 2 Structures of parallel differential network and algebraic network

Fig. 3 Framework of LL-PINN

Fig. 4 Structure of time domain decomposition scheme

Fig. 5 High-precision continuous-time prediction scheme

Fig. 6 Comparison of absolute errors among predicted and exact solutions of three models （numerical example 1）

Tab. 2 Comparison of L2 errors among predicted and exact solutions of three models （numerical example 1）

模型

x 1

x 2

x 3

x 4

x 5

TFC

试验解

8.94E-07

4.09E-06

1.02E-06

4.03E-06

1.76E-06

Tab. 2 Comparison of L2 errors among predicted and exact solutions of three models （numerical example 1）

模型

x 1

x 2

x 3

x 4

x 5

TFC

试验解

7.32E-04

1.22E-04

5.40E-04

2.07E-04

2.51E-03

PINN

8.04E-04

2.36E-03

7.17E-04

2.31E-03

1.61E-03

LL-PINN

8.94E-07

4.09E-06

1.02E-06

4.03E-06

1.76E-06

Fig. 7 Single pendulum model

Fig. 8 Comparison of absolute errors among predicted and exact solutions of three models （numerical example 2）

Tab. 3 Comparison of L2 errors among predicted and exact solutions of three models （numerical example 2）

模型

x

u

y

v

λ

TFC

试验解

2.51E-05

2.64E-04

6.09E-05

9.49E-05

1.90E-04

Tab. 3 Comparison of L2 errors among predicted and exact solutions of three models （numerical example 2）

模型

x

u

y

v

λ

TFC

试验解

3.46E-03

3.61E-02

1.72E-02

1.43E-02

2.25E-02

PINN

2.15E-03

4.68E-03

5.48E-03

8.24E-03

1.96E-02

LL-PINN

2.51E-05

2.64E-04

6.09E-05

9.49E-05

1.90E-04

References 36

1	HAUG E. An index 0 Differential-Algebraic equation formulation for multibody dynamics： holonomic constraints ［J］. Mechanics Based Design of Structures and Machines， 2017， 45（4）： 479-506.
2	MÄRZ R， TISCHENDORF C. Recent results in solving index-2 differential-algebraic equations in circuit simulation ［J］. SIAM Journal on Scientific Computing， 1997， 18（1）： 139-159.
3	CAMPBELL S， ILCHMANN A， MEHRMANN V， et al. Applications of differential-algebraic equations： examples and benchmarks ［M］. Cham： Springer， 2019.
4	GEAR C. Simultaneous numerical solution of differential-algebraic equations ［J］. IEEE Transactions on Circuit Theory， 1971， 18（1）： 89-95.
5	LAMOUR R， MÄRZ R， TISCHENDORF C. Differential-algebraic equations： a projector based analysis ［M］. Berlin： Springer， 2013.
6	UNGER J， KRÖNER A， MARQUARDT W. Structural analysis of differential-algebraic equation systems-theory and applications ［J］. Computers and Chemical Engineering， 1995， 19（8）： 867-882.
7	LOGSDON J S， BIEGLER L T. Accurate solution of differential-algebraic optimization problems ［J］. Industrial and Engineering Chemistry Research， 1989， 28（11）： 1628-1639.
8	ASCHER U M， PETZOLD L R. Projected implicit Runge-Kutta methods for differential-algebraic equations ［J］. SIAM Journal on Numerical Analysis， 1991， 28（4）： 1097-1120.
9	ASCHER U M， PETZOLD L R. Computer methods for ordinary differential equations and differential-algebraic equations ［M］. Philadelphia， PA： SIAM， 1998.
10	CASH J R. Modified extended backward differentiation formulae for the numerical solution of stiff initial value problems in ODEs and DAEs ［J］. Journal of Computational and Applied Mathematics， 2000， 125（1/2）： 117-130.
11	HARIER E， WANNER G. Solving ordinary differential equations Ⅱ： stiff and differential algebraic problems， SSCM 14 ［M］. 2nd revised ed. Berlin： Springer， 1996.
12	KUNKEL P， MEHRMANN V. Differential-algebraic equations： analysis and numerical solution ［M］. Helsinki： European Mathematical Society， 2006.
13	SOLTANIAN F， KARBASSI S M， HOSSEINI M M. Application of He’s variational iteration method for solution of differential-algebraic equations ［J］. Chaos， Solitons and Fractals， 2009， 41（1）： 436-445.
14	LeVEQUE R J. Finite difference methods for ordinary and partial differential equations： steady-state and time-dependent problems［M］. Philadelphia， PA： SIAM， 2007.
15	SARAVI M， BABOLIAN E， ENGLAND R， et al. System of linear ordinary differential and differential-algebraic equations and pseudo-spectral method ［J］. Computers and Mathematics with Applications， 2010， 59（4）： 1524-1531.
16	ÇELIK E， BAYRAM M， YELOĞLU T. Solution of Differential-Algebraic Equations （DAEs） by Adomian decomposition method［J］. International Journal of Pure and Applied Mathematical Sciences， 2006， 3（1）： 93-100.
17	NEWMAN C K. Exponential integrators for the incompressible Navier-Stokes equations ［D/OL］. ［2024-02-12］. .
18	陆见光，唐卷，秦小林，等. 改进的保群算法及其在混沌系统中的应用［J］. 物理学报， 2016， 65（11）： No.110501.
	LU J G， TANG J， QIN X L， et al. Modified group preserving methods and applications in chaotic systems ［J］. Acta Physica Sinica， 2016， 65（11）： No.110501.
19	LIU C S， CHEN W， LIU L W. Solving mechanical systems with nonholonomic constraints by a Lie-group differential algebraic equations method ［J］. Journal of Engineering Mechanics， 2017， 143（9）： No.0001298.
20	TANG J， LU J. Modified extended Lie-group method for Hessenberg differential algebraic equations with index-3 ［J］. Mathematics， 2023， 11（10）： No.2360.
21	LEAKE C， MORTARI D. Deep theory of functional connections： a new method for estimating the solutions of partial differential equations ［J］. Machine Learning and Knowledge Extraction， 2020， 2（1）： 37-55.
22	LIU HONGLIANG， LIU HUINI， XU J， et al. Jacobi neural network method for solving linear differential-algebraic equations with variable coefficients ［J］. Neural Processing Letters， 2021， 53（5）： 3357-3374.
23	RAISSI M， PERDIKARIS P， KARNIADAKIS G E. Physics-informed neural networks： a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations ［J］. Journal of Computational Physics， 2019， 378： 686-707.
24	MOYA C， LIN G. DAE-PINN： a physics-informed neural network model for simulating differential algebraic equations with application to power networks ［J］. Neural Computing and Applications， 2023， 35（5）： 3789-3804.
25	SCHIASSI E， FURFARO R， LEAKE C， et al. Extreme theory of functional connections： a fast physics-informed neural network method for solving ordinary and partial differential equations ［J］. Neurocomputing， 2021， 457： 334-356.
26	YANG M， FOSTER J T. Multi-output physics-informed neural networks for forward and inverse PDE problems with uncertainties［J］. Computer Methods in Applied Mechanics and Engineering， 2022， 402： No.115041.
27	尹聪，胡汉平. 基于时间注意力机制的时滞混沌系统参数辨识模型［J］. 计算机应用， 2023， 43（3）： 842-847.
	YIN C， HU H P. Parameter identification model for time-delay chaotic systems based on temporal attention mechanism ［J］. Journal of Computer Applications， 2023， 43（3）： 842-847.
28	KOZLOV D S， TIUMENTSEV Y V. Neural network based semi-empirical models for dynamical systems represented by differential-algebraic equations of index 2 ［J］. Procedia Computer Science， 2018， 123： 252-257.
29	杨钊，兰钧，吴勇军. 几类微分-代数方程的神经网络求解法［J］. 应用数学和力学， 2019， 40（2）：115-126.
	YANG Z， LAN J， WU Y J. On solutions to several classes of differential-algebraic equations based on artificial neural networks ［J］. Applied Mathematics and Mechanics， 2019， 40（2）： 115-126.
30	KOEKOEK J， KOEKOEK R. Differential equations for generalized Jacobi polynomials ［J］. Journal of Computational and Applied Mathematics， 2000， 126（1/2）： 1-31.
31	PAO Y H， PHILLIPS S M. The functional link net and learning optimal control ［J］. Neurocomputing， 1995， 9（2）： 149-164.
32	JAY L O. Lobatto methods ［M］// ENGQUIST B. Encyclopedia of applied and computational mathematics. Berlin： Springer， 2015： 817-826.
33	ISERLES A. A first course in the numerical analysis of differential equations ［M］. 2nd ed. Cambridge： Cambridge University Press， 2009.
34	CHIHARA T S. An introduction to orthogonal polynomials ［M］. Mineola， NY： Dover Publications， 2011.
35	AGARWAL R P， O’REGAN D， AGARWAL R P， et al. Legendre polynomials and functions ［M］// Ordinary and partial differential equations： with special functions， Fourier series， and boundary value problems， UTX. New York： Springer， 2009： 47-56.
36	KINGMA D P， BA J L. Adam： a method for stochastic optimization［EB/OL］. ［2023-10-02］. .

[1]	CAO Junli, LI Jufeng. Improved ellipse fitting algorithm based on Letts criterion [J]. Journal of Computer Applications, 2017, 37(1): 273-277.
[2]	ZHENG Supei FENG Jianhu. Fourth-order high-resolution entropy consistent algorithm [J]. Journal of Computer Applications, 2013, 33(09): 2416-2418.

Physics-informed neural network based on Lobatto method and Legendre polynomials for solving differential-algebraic equations

基于Lobatto方法和Legendre多项式的PINN求解微分代数方程

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 36

Related Articles 2

Recommended Articles

Metrics