Calculus of Variations

A functional is a map $J: \mathcal{A} \to \mathbb{R}$, where $\mathcal{A}$ is a subset of a function space. We write $J[y]$ rather than $J(y)$ to distinguish functionals (which eat functions) from ordinary functions (which eat numbers).

The arc length of a curve $y: [a,b] \to \mathbb{R}$ is

This maps each $C^1$ function $y$ to a non-negative real number.

The Dirichlet energy

measures the total “kinetic energy” stored in the gradient of $y$. Minimizing $E[y]$ subject to boundary conditions $y(a) = \alpha$, $y(b) = \beta$ yields the straight line $y(x) = \alpha + (\beta - \alpha)\frac{x - a}{b - a}$.

In classical mechanics, the action is

where $L = T - V$ is the Lagrangian (kinetic minus potential energy). Hamilton’s principle: the physical trajectory extremizes the action.

A PINN loss is a functional on the space of neural network functions:

The parameters $\theta$ parameterize a submanifold of function space; optimization over $\theta$ is a finite-dimensional proxy for the infinite-dimensional variational problem.

We will use the inner-product and Hilbert-space machinery from Topic 31 throughout this topic. Recall: a Hilbert space is a complete inner-product space. The projection theorem guarantees a unique closest point in any closed convex subset. The Riesz representation theorem identifies the dual space $H^*$ with $H$ itself. These are not just background — they are the active ingredients in the direct method (Section 8) and the Lax-Milgram theorem (Section 9).

The first variation of $J$ at $y$ in the direction $\eta$ is

provided the limit exists for all admissible perturbations $\eta$ (typically $\eta \in C_0^\infty(a,b)$, meaning smooth functions vanishing at the boundary). If $\delta J[y; \eta] = 0$ for all $\eta$, we call $y$ a critical point (or extremal) of $J$.

If $f \in C([a,b])$ and

then $f(x) = 0$ for all $x \in [a,b]$.

Proof. Suppose for contradiction that $f(x_0) \neq 0$ at some $x_0 \in (a,b)$. Without loss of generality, assume $f(x_0) > 0$. By continuity of $f$, there exists $\delta > 0$ such that $f(x) > f(x_0)/2$ for all $x \in (x_0 - \delta, x_0 + \delta) \subset (a,b)$.

Choose $\eta$ to be a smooth bump function supported on $(x_0 - \delta, x_0 + \delta)$ with $\eta \geq 0$ and $\eta(x_0) > 0$. For instance, take

Then $\eta \in C_0^\infty(a,b)$, and

This contradicts $\int f \eta = 0$ for all $\eta$. Therefore $f \equiv 0$. $\square$

For $E[y] = \int_0^1 \frac{1}{2}y'^2 \, dx$:

Integrating by parts (with $\eta(0) = \eta(1) = 0$): $\delta E[y;\eta] = -\int_0^1 y''\eta \, dx$. Setting this to zero for all $\eta$ and applying the fundamental lemma gives $y'' = 0$ — the extremal is a straight line.

For $L[y] = \int_0^1 \sqrt{1+y'^2} \, dx$, a similar computation gives

After integration by parts, the Euler-Lagrange equation is $\frac{d}{dx}\frac{y'}{\sqrt{1+y'^2}} = 0$, which implies $y'$ is constant — again, a straight line (which minimizes arc length between two points in the plane).

The first variation is a Taylor expansion in function space. Writing $J[y + \varepsilon\eta]$ as a power series in $\varepsilon$ (cf. Topic 6):

The first variation is the linear term; the second variation (Section 6) is the quadratic term. A critical point has $\delta J = 0$; a minimum requires $\delta^2 J \geq 0$.

Let $L = L(x, y, y')$ be $C^2$ in all arguments, and let $J[y] = \int_a^b L(x, y, y') \, dx$. If $y \in C^2([a,b])$ is an extremal of $J$ (i.e., $\delta J[y;\eta] = 0$ for all $\eta \in C_0^\infty(a,b)$), then $y$ satisfies the Euler-Lagrange equation:

Proof. We proceed in six steps.

Step 1: Expand the perturbed functional. Replace $y$ by $y + \varepsilon\eta$ in $J$:

Step 2: Differentiate under the integral. By the chain rule:

Step 3: Integrate the second term by parts. Writing $u = \frac{\partial L}{\partial y'}$ and $dv = \eta' \, dx$, we get $v = \eta$ and $du = \frac{d}{dx}\frac{\partial L}{\partial y'} dx$:

Step 4: Apply the boundary conditions. Since $\eta(a) = \eta(b) = 0$, the boundary term vanishes:

Step 5: Combine and factor. Substituting back:

for all $\eta \in C_0^\infty(a,b)$.

Step 6: Apply the fundamental lemma. By Theorem 1, since the expression in brackets is continuous and its integral against every smooth compactly-supported $\eta$ vanishes, we conclude

on $(a,b)$. $\square$

For $L = \sqrt{1+y'^2}$: $\frac{\partial L}{\partial y} = 0$ and $\frac{\partial L}{\partial y'} = \frac{y'}{\sqrt{1+y'^2}}$. The Euler-Lagrange equation gives $\frac{d}{dx}\frac{y'}{\sqrt{1+y'^2}} = 0$, so $y' = \text{const}$. The extremal is a straight line — as expected.

The action $S[q] = \int_0^T (\frac{1}{2}m\dot{q}^2 - \frac{1}{2}kq^2)\,dt$ has Lagrangian $L = \frac{1}{2}m\dot{q}^2 - \frac{1}{2}kq^2$. The Euler-Lagrange equation yields $m\ddot{q} + kq = 0$ — Newton’s second law for a spring.

The proof of the Euler-Lagrange equation assumed $\eta(a) = \eta(b) = 0$ (Dirichlet/essential boundary conditions). If we allow $\eta(b) \neq 0$, the boundary term $\frac{\partial L}{\partial y'}\big|_{x=b} \cdot \eta(b)$ must vanish independently, giving the natural boundary condition $\frac{\partial L}{\partial y'}\big|_{x=b} = 0$. This is important in finite element methods.

Problem: Find the curve of fastest descent under gravity from point $A$ to point $B$, starting from rest.

The descent time is $T[y] = \int_0^{x_1} \sqrt{\frac{1+y'^2}{2gy}}\,dx$. The Euler-Lagrange equation (with Lagrangian $L = \sqrt{(1+y'^2)/(2gy)}$) yields $2yy'' + y'^2 + 1 = 0$. The solution is a cycloid:

where $R$ is determined by the endpoint condition. The cycloid is about 19% faster than the straight-line path.

A geodesic minimizes the arc-length functional $L[\gamma] = \int_a^b \sqrt{g_{ij}\dot{\gamma}^i\dot{\gamma}^j}\,dt$ (cf. Topic 20). On a sphere of radius $R$, the Euler-Lagrange equation yields great circles — the shortest paths between points on the sphere.

A flexible chain of uniform density hanging under gravity adopts the shape that minimizes potential energy $V[y] = \int_{-a}^a \rho g\, y\sqrt{1+y'^2}\,dx$ subject to the constraint of fixed length. The Euler-Lagrange equation with a Lagrange multiplier gives the catenary:

where $c > 0$ depends on the chain length and endpoint separation.

Problem: Among all closed curves of fixed perimeter $P$, which one encloses the maximum area?

This is a constrained variational problem. We maximize $A[\gamma] = \frac{1}{2}\oint (x\,dy - y\,dx)$ subject to $L[\gamma] = P$. Using a Lagrange multiplier $\lambda$, the Euler-Lagrange equation for $J = A - \lambda L$ yields a circle of radius $R = P/(2\pi)$.

Constrained variational problems (like the isoperimetric problem) add a side condition $K[y] = c$ to the functional $J[y]$. The method of Lagrange multipliers in function space: extremize $J[y] - \lambda K[y]$ over all $y$ and $\lambda$. This is the infinite-dimensional prototype for Lagrangian duality in optimization.

The second variation of $J$ at an extremal $y$ in the direction $\eta$ is

For $J[y] = \int_a^b L(x,y,y')\,dx$, this becomes

If $y$ is a local minimizer of $J$, then

Proof. If $y$ is a minimizer, then $\delta^2 J[y;\eta] \geq 0$ for all admissible $\eta$. Take $\eta$ to be a tall, narrow bump centered at $x_0$. In the limit, the $\eta'^2$ term dominates, and we need $L_{y'y'}(x_0) \geq 0$. Formally: choose $\eta_n(x) = n\phi(n(x-x_0))$ where $\phi$ is a standard mollifier. Then $\eta_n' = n^2\phi'(n(x-x_0))$, and

For $\delta^2 J \geq 0$ as $n \to \infty$, we need $L_{y'y'}(x_0) \geq 0$. $\square$

A point $c \in (a,b)$ is conjugate to $a$ along the extremal $y$ if there exists a non-trivial solution $h$ of the Jacobi equation

satisfying $h(a) = h(c) = 0$. Conjugate points are where nearby extremals refocus.

If $y$ is an extremal of $J$ with $L_{y'y'} > 0$ on $[a,b]$ (strengthened Legendre), and there are no conjugate points in $(a,b]$, then $y$ is a (strict) local minimum.

Proof outline. The absence of conjugate points ensures that the Jacobi equation has no non-trivial solutions vanishing at both endpoints. This means the quadratic form $\delta^2 J[y;\eta]$ is positive definite on the space of admissible variations, which implies $y$ is a local minimum. The full proof uses the theory of fields of extremals. $\square$

For $E[y] = \int_0^1 \frac{1}{2}y'^2 \, dx$: $L_{y'y'} = 1 > 0$, $L_{yy} = 0$, $L_{yy'} = 0$. So $\delta^2 E[y;\eta] = \int_0^1 \eta'^2 \, dx \geq 0$, with equality only if $\eta' \equiv 0$, i.e., $\eta \equiv 0$ (since $\eta(0)=\eta(1)=0$). The Jacobi equation is $h'' = 0$ with $h(0)=0$, giving $h(x) = cx$. This has no zero in $(0,1]$, so there are no conjugate points. The straight line is a strict minimum.

The second variation is the function-space analogue of the Hessian matrix. In $\mathbb{R}^n$, a critical point is a minimum if the Hessian is positive definite. In function space, a critical point is a minimum if $\delta^2 J$ is positive definite on the space of admissible perturbations. This connects the calculus of variations to the spectral theory of differential operators — the Jacobi equation defines a self-adjoint operator whose eigenvalues determine whether $\delta^2 J$ is positive definite. This closes the obligation planted in Topic 29 about the variational characterization of extrema in infinite dimensions.

A function $u \in L^2(\Omega)$ has a weak derivative $u' \in L^2(\Omega)$ if

This is the integration-by-parts formula with no boundary term (since $\varphi$ vanishes at the boundary). The weak derivative agrees with the classical derivative when $u$ is classically differentiable, but extends to a much larger class of functions.

The Sobolev space $H^1(\Omega)$ consists of all $L^2(\Omega)$ functions with weak derivatives in $L^2(\Omega)$:

equipped with the inner product $\langle u, v \rangle_{H^1} = \int_\Omega (uv + u'v')\,dx$.

The subspace $H^1_0(\Omega) \subset H^1(\Omega)$ consists of functions that vanish on the boundary $\partial\Omega$ (in the trace sense). This is the natural space for Dirichlet boundary conditions.

$H^1_0(\Omega)$ with the $H^1$ inner product is a Hilbert space (a complete inner product space).

Proof. We need to show completeness. Let $\{u_n\}$ be a Cauchy sequence in $H^1_0(\Omega)$. Then $\{u_n\}$ and $\{u_n'\}$ are both Cauchy in $L^2(\Omega)$. Since $L^2$ is complete, there exist $u, v \in L^2(\Omega)$ with $u_n \to u$ and $u_n' \to v$ in $L^2$. We verify $v$ is the weak derivative of $u$: for any $\varphi \in C_0^\infty$,

so $u' = v$ in the weak sense. Hence $u \in H^1(\Omega)$. Since $u_n \in H^1_0$, the boundary values (traces) converge to zero, so $u \in H^1_0$. Thus $u_n \to u$ in $H^1_0$. $\square$

For bounded $\Omega \subset \mathbb{R}^n$, there exists $C > 0$ such that

Proof (for $\Omega = (0,L)$). By the fundamental theorem of calculus and Cauchy-Schwarz:

Integrating over $x \in (0,L)$: $\int_0^L |u|^2\,dx \leq L^2\int_0^L |u'|^2\,dx$. So $C = L$. $\square$

The Poincaré inequality says that on $H^1_0$, the $H^1$ seminorm $\|u'\|_{L^2}$ is equivalent to the full $H^1$ norm. This is crucial for the coercivity arguments in the direct method.

For bounded $\Omega \subset \mathbb{R}^n$ with Lipschitz boundary, the embedding $H^1(\Omega) \hookrightarrow L^2(\Omega)$ is compact: every bounded sequence in $H^1(\Omega)$ has a subsequence converging in $L^2(\Omega)$.

Proof deferred. The full proof requires approximation theory and the Arzelà-Ascoli theorem (cf. Topic 29). We state this result because it is essential for the direct method: it converts weak convergence in $H^1$ to strong convergence in $L^2$.

The tent function $u(x) = \min(x, 1-x)$ on $[0,1]$ has a corner at $x = 1/2$ and is not classically differentiable there. But it has a weak derivative: $u'(x) = +1$ on $(0, 1/2)$ and $u'(x) = -1$ on $(1/2, 1)$. Since both $u$ and $u'$ are in $L^2$, we have $u \in H^1(0,1)$. Its squared $H^1$ norm is $\|u\|_{H^1}^2 = \int_0^1 u^2 + (u')^2 \, dx = 1/12 + 1 = 13/12 \approx 1.083$, so $\|u\|_{H^1} \approx 1.04$.

In contrast, $u(x) = x^{-1/3}$ on $(0,1]$ has $u' = -\frac{1}{3}x^{-4/3}$, and $\int_0^1 (u')^2 \, dx = \frac{1}{9}\int_0^1 x^{-8/3}\,dx = \infty$. So $u \notin H^1$.

The key point: $H^1_0(\Omega)$ with the inner product $\langle u, v\rangle_{H^1} = \int (uv + u'v')\,dx$ is a Hilbert space (Theorem 5). This means we can apply the entire Hilbert-space toolkit from Topic 31 — projection theorem, Riesz representation, spectral theory — to variational problems posed on Sobolev spaces. This closes the obligation from Topic 31 and Topic 23 to explain why Sobolev spaces are the right function spaces for PDEs.

A functional $J: H \to \mathbb{R}$ on a Hilbert space $H$ is:

• Coercive if $J[u] \to +\infty$ as $\|u\|_H \to \infty$.
• Weakly lower semicontinuous (w.l.s.c.) if $u_n \rightharpoonup u$ (weak convergence) implies $J[u] \leq \liminf_{n\to\infty} J[u_n]$.

Let $H$ be a reflexive Banach space (e.g., a Hilbert space) and $J: H \to \mathbb{R} \cup \{+\infty\}$ be coercive and weakly lower semicontinuous. Then $J$ attains its infimum: there exists $u^* \in H$ with $J[u^*] = \inf_{u \in H} J[u]$.

Proof. We proceed in four steps.

Step 1: Bounded minimizing sequence. Let $m = \inf J$ and choose a minimizing sequence $\{u_n\}$ with $J[u_n] \to m$. By coercivity, $\|u_n\|$ is bounded: if $\|u_n\| \to \infty$, then $J[u_n] \to \infty$, contradicting $J[u_n] \to m < \infty$.

Step 2: Weak compactness. Since $H$ is reflexive and $\{u_n\}$ is bounded, by the Eberlein-Šmulian theorem (the sequential version of the Banach-Alaoglu theorem for reflexive spaces), there exists a subsequence $\{u_{n_k}\}$ and $u^* \in H$ with $u_{n_k} \rightharpoonup u^*$ (weak convergence).

Step 3: Lower semicontinuity. Since $J$ is weakly lower semicontinuous:

Step 4: Conclusion. By definition, $m = \inf J \leq J[u^*]$. Combined with Step 3, $J[u^*] = m$. The infimum is attained. $\square$

The Dirichlet energy $E[u] = \int_\Omega \frac{1}{2}|\nabla u|^2\,dx$ on $H^1_0(\Omega)$ satisfies:

• Coercivity: By Poincaré, $E[u] \geq c\|u\|_{H^1_0}^2$.
• Weak lower semicontinuity: The norm is w.l.s.c., and $E[u] = \frac{1}{2}\|u\|^2$ (using the equivalent norm from Poincaré).

By the direct method, a minimizer exists. This is the variational proof of existence for the Poisson equation $-\Delta u = 0$ with Dirichlet boundary conditions.

Given $f \in L^2(\Omega)$ and a closed subspace $V \subset L^2(\Omega)$, the best approximation problem is: minimize $J[v] = \|f - v\|_{L^2}^2$ over $v \in V$. This is a quadratic functional on a Hilbert space — coercive and w.l.s.c. The direct method guarantees existence, and the minimizer is the orthogonal projection $P_V f$ (cf. Topic 31 and Topic 15).

In finite dimensions, the extreme value theorem says: a continuous function on a compact set attains its extrema. The direct method is the infinite-dimensional version. Coercivity provides the “compact set” (bounded sublevel sets), weak lower semicontinuity provides “continuity” (in the weak topology), and reflexivity provides “compactness” (Eberlein-Šmulian). This closes the staircase from the Bolzano-Weierstrass theorem in Topic 29 to the direct method here.

A function $u \in H^1_0(\Omega)$ is a weak solution of the boundary value problem $-\Delta u = f$ (with $u|_{\partial\Omega} = 0$) if

This is obtained by multiplying the PDE by a test function $v$, integrating by parts, and dropping the boundary term (since $v \in H^1_0$).

A bilinear form $a: H \times H \to \mathbb{R}$ on a Hilbert space $H$ is:

• Bounded (continuous) if $|a(u,v)| \leq M\|u\|\|v\|$ for some $M > 0$.
• Coercive if $a(u,u) \geq \alpha\|u\|^2$ for some $\alpha > 0$.

Let $H$ be a Hilbert space, $a: H \times H \to \mathbb{R}$ a bounded, coercive bilinear form, and $F \in H^*$ a bounded linear functional. Then there exists a unique $u \in H$ such that

Proof. The key is to use the Riesz representation theorem from Topic 31.

Step 1: Define the operator $A$. For each fixed $u \in H$, the map $v \mapsto a(u,v)$ is a bounded linear functional on $H$. By the Riesz representation theorem, there exists a unique $Au \in H$ such that $a(u,v) = \langle Au, v\rangle$ for all $v$. The map $u \mapsto Au$ is linear and bounded with $\|A\| \leq M$.

Step 2: Show $A$ is bijective. Coercivity gives $\langle Au, u\rangle = a(u,u) \geq \alpha\|u\|^2$, so:

This means $A$ is injective and has closed range.

Step 3: Show the range is dense. If $w \perp \text{Range}(A)$, then $\langle Aw, w\rangle = 0$, but coercivity gives $\alpha\|w\|^2 \leq \langle Aw, w\rangle = 0$, so $w = 0$. Hence $\text{Range}(A)^\perp = \{0\}$, meaning $\text{Range}(A)$ is dense in $H$.

Step 4: Conclude. Since $\text{Range}(A)$ is both closed and dense, $A$ is surjective. Hence $A$ is bijective with bounded inverse ($\|A^{-1}\| \leq 1/\alpha$).

Now apply Riesz again: $F \in H^*$ corresponds to some $f \in H$ with $F(v) = \langle f, v\rangle$. Set $u = A^{-1}f$. Then $a(u,v) = \langle Au, v\rangle = \langle f, v\rangle = F(v)$ for all $v$. $\square$

For $-\Delta u = f$ on $\Omega$ with $u|_{\partial\Omega} = 0$: take $H = H^1_0(\Omega)$, $a(u,v) = \int \nabla u \cdot \nabla v\, dx$, $F(v) = \int fv\,dx$. By Poincaré, $a$ is coercive: $a(u,u) = \int |\nabla u|^2 \geq C_P^{-2}\|u\|_{H^1_0}^2$. By Cauchy-Schwarz, $a$ is bounded. Lax-Milgram gives the unique weak solution $u \in H^1_0$.

For $-(pu')' + qu = f$ on $(a,b)$ with $u(a)=u(b)=0$: take $a(u,v) = \int_a^b (pu'v' + quv)\,dx$. If $p \geq p_0 > 0$ and $q \geq 0$, then $a$ is coercive and bounded. Lax-Milgram applies.

The finite element method discretizes the Lax-Milgram problem: replace $H$ by a finite-dimensional subspace $V_h$ (e.g., piecewise-linear functions on a mesh), and solve $a(u_h, v_h) = F(v_h)$ for all $v_h \in V_h$. This is a finite linear system $K\mathbf{u} = \mathbf{f}$ where $K_{ij} = a(\phi_j, \phi_i)$ is the stiffness matrix. The Lax-Milgram theorem guarantees that the stiffness matrix is invertible (because $a$ is coercive on $V_h$ too).

The Sturm-Liouville eigenvalue problem is: find non-trivial $u \in H^1_0(0,\pi)$ and $\lambda \in \mathbb{R}$ such that

The eigenvalues are $\lambda_n = n^2$ ($n = 1, 2, 3, \ldots$) with eigenfunctions $u_n(x) = \sin(nx)$.

The first eigenvalue of $-u'' = \lambda u$ on $(0,\pi)$ with Dirichlet conditions is

is the Rayleigh quotient. The minimum is achieved by $u_1 = \sin(x)$ with $\lambda_1 = 1$.

Proof. We use the weak formulation. Multiply $-u'' = \lambda u$ by $v \in H^1_0$ and integrate by parts:

Setting $v = u$: $\int u'^2 = \lambda\int u^2$, so $\lambda = R[u]$ for any eigenfunction.

Now we show $\lambda_1$ is the minimum of $R$. Expand $u$ in the eigenfunction basis: $u = \sum c_n\sin(nx)$. By Parseval’s identity:

Therefore $R[u] = \frac{\sum n^2 c_n^2}{\sum c_n^2} \geq \frac{1^2 \sum c_n^2}{\sum c_n^2} = 1 = \lambda_1$, with equality when $c_n = 0$ for $n \geq 2$, i.e., $u = c_1\sin(x)$. $\square$

The $k$-th eigenvalue is

Proof outline. This follows from the spectral theorem for compact self-adjoint operators applied to the inverse of $-d^2/dx^2$ (cf. Topic 31). The min-max characterization is equivalent to the variational principle: $\lambda_k$ is the minimum of $R[u]$ over the orthogonal complement of the first $k-1$ eigenfunctions. $\square$

The eigenfunctions $u_n(x) = \sin(nx)$ have eigenvalues $\lambda_n = n^2$. For the trial function $u(x) = \sin(x)$:

The variational characterization of eigenvalues is the bridge between the spectral theorem (Topic 31) and the calculus of variations. The compact self-adjoint operator $T = (-d^2/dx^2)^{-1}$ on $L^2(0,\pi)$ has eigenvalues $\mu_n = 1/n^2$ and eigenfunctions $\sin(nx)$. The spectral theorem decomposes $T$ as $Tf = \sum \frac{1}{n^2}\langle f, u_n\rangle u_n$. The Rayleigh quotient $R[u] = \langle u, (-d^2/dx^2)u\rangle / \langle u, u\rangle$ is the inverse: $R = 1/\mu$. Minimizing $R$ is equivalent to maximizing $\mu$ — finding the largest eigenvalue of $T$.

A PINN parameterizes the solution of a PDE $\mathcal{L}u = f$ as a neural network $u_\theta$ and minimizes the variational loss

This is literally a calculus-of-variations problem: $J$ is a functional on the space of network functions. The Euler-Lagrange equation for $J$ recovers the PDE — the PINN finds an approximate solution by minimizing the variational residual. The direct method guarantees that a minimizer exists in $H^1_0$ (the Sobolev space), and the neural network approximates it.

The Monge-Kantorovich problem seeks the transport map $T$ minimizing the total cost $\int c(x, T(x))\,d\mu(x)$. In its relaxed (Kantorovich) formulation, we minimize over transport plans $\gamma$: