In this talk, we consider fast and effective numerical methods for optimization problems where partial differential equations (PDEs) act as constraints, so-called PDE-constrained optimization. Such problems have numerous applications across science and engineering, for instance in fluid flow control problems, chemical and biological processes, mathematical finance, and medical imaging, to name a few. To give an example of a problem structure, consider the following formulation:

\begin{equation*} \ \min _{y,u}~~\frac {1}{2}\|y-\widehat {y}\|_{Q_1(\Omega )}^2+\frac {\beta }{2}\|u\|_{Q_2(\Omega )}^2\qquad \text {s.t.}~~~\mathcal {D}y=u~~\text {in }\Omega , \end{equation*}

where \(y\) and \(u\) denote one or more state variables (PDE variables) and optimal control variables respectively, \(\widehat {y}\) is a desired state, \(\beta >0\) is a regularization parameter, and \(\mathcal {D}\) represents a differential operator equipped with boundary conditions. The problem is posed on a (generally space–time) domain \(\Omega \), with \(Q_1\) and \(Q_2\) two (given) norms. It is possible to impose additional algebraic constraints on the states and/or controls.

The vast majority of work on the optimal control of PDEs has involved local PDEs, where the behaviour of the PDE at a point in \(\Omega \) can be described by problem features within a small neighbourhood of that point. In this talk we consider the emerging field of nonlocal PDE-constrained optimization, including problems with fractional derivatives, integro-differential equations, or (integral) kernel functions. On the numerical linear algebra level, this leads to dense linear systems, as opposed to the sparse matrices obtained from many discretizations of local PDEs. However, by exploiting the structures of the relevant matrices, we are nonetheless able to construct viable and robust schemes for problems of dimensions that would otherwise be out of reach.

Specifically, we derive preconditioned iterative methods to tackle huge-scale linear(ized) systems that result from nonlocal PDE-constrained optimization, and carefully utilize structures that arise in such systems to enhance the efficiency of solvers. For example,

• • We build a spectral-in-time Newton–Krylov method for solving multiscale particle dynamics problems, closely related to mean-field games and mean-field control, to high accuracy [1] (see also [2]). At each Newton iteration for the nonlinear problem, we may apply column operations to the Jacobian matrix to ensure invertibility of the leading block, then construct structured, Kronecker product-based preconditioners for the resulting Schur complement. This leads to rapid GMRES convergence for large and dense systems.

• • We devise a new technology for nonlocal image denoising problems [3], applying an unnormalized extended Gaussian ANOVA kernel within a bilevel optimization routine. Matrix–vector multiplications may be applied via a (matrix-free) Nonequispaced Fast Fourier Transform, and the Conjugate Gradient method accelerated using a novel change of basis approach coupled with a diagonal preconditioning strategy. As a result, rapid and effective denoising for very large problems may be achieved with modest storage requirements on a computer.

• • We tackle optimization problems constrained by certain space–time fractional differential equations with additional state and/or control constraints [4]. This is accomplished by exploiting the multilevel Toeplitz structure of many of the matrix sub-blocks, deriving multilevel circulant preconditioners based on this, and designing a recursive linear algebra which leads to very low storage requirements and operation costs.

This talk will outline some of the progress made in the above areas. In each case, as opposed to exploiting the property of sparsity as one would do for local problems, we utilize structure which the problem provides us with (Kronecker-product approximation, Gaussian kernel which allows a fast discrete transform, or multilevel Toeplitz) in a bespoke way. We will also provide an outlook of the subject area, discussing new applications of nonlocal PDEs and optimization problems, and outlining how the above methods could be adapted to resolve these challenges.

References

• [1]  M. Aduamoah, B. D. Goddard, J. W. Pearson, and J. C. Roden, Pseudospectral methods and iterative solvers for optimization problems from multiscale particle dynamics. BIT Numerical Mathematics, 62 (4): pp.  1703–1743, 2022.

• [2]  S. Güttel and J. W. Pearson, A spectral-in-time Newton–Krylov method for nonlinear PDE-constrained optimization. IMA Journal of Numerical Analysis, 42 (2): pp.  1478–1499, 2022.

• [3]  A. Miniguano-Trujillo, J. W. Pearson, and B. D. Goddard, Efficient nonlocal linear image denoising: Bilevel optimization with Nonequispaced Fast Fourier Transform and matrix-free preconditioning. arXiv preprint arXiv:2407.06834, 2024.

• [4]  S. Pougkakiotis, J. W. Pearson, S. Leveque, and J. Gondzio, Fast solution methods for convex quadratic optimization of fractional differential equations. SIAM Journal on Matrix Analysis and Applications, 41 (3): pp. 1443–1476, 2020.