Fourier Series & Orthogonal Expansions

A trigonometric polynomial of degree $n$ is a function of the form

$T_n(x) = \frac{a_0}{2} + \sum_{k=1}^{n}(a_k \cos kx + b_k \sin kx)$

where $a_0, a_1, \ldots, a_n, b_1, \ldots, b_n$ are real constants. The factor $\frac{1}{2}$ on $a_0$ is a normalization convention that simplifies the coefficient formulas below.

For a function $f$ integrable on $[-\pi, \pi]$, the Fourier coefficients of $f$ are

$a_n = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x)\cos(nx)\,dx \quad \text{for } n \geq 0$

$b_n = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x)\sin(nx)\,dx \quad \text{for } n \geq 1$

The Fourier series of $f$ is the formal trigonometric series

$f(x) \sim \frac{a_0}{2} + \sum_{n=1}^{\infty}(a_n \cos nx + b_n \sin nx)$

The symbol $\sim$ (rather than $=$) indicates that convergence has not yet been established — the series may or may not converge, and if it converges, it may or may not converge to $f$.

Taylor coefficients $f^{(n)}(c)/n!$ are computed from derivatives at a single point — a local operation. Fourier coefficients $a_n, b_n$ are computed by integrating over the entire period $[-\pi, \pi]$ — a global operation. This is why the Fourier series can represent discontinuous functions that the Taylor series cannot: the coefficients “see” the whole function, not just the behavior at one point.

Let $f(x) = \text{sgn}(x)$ for $x \in (-\pi, \pi)$ — the square wave that takes value $+1$ for $x > 0$ and $-1$ for $x < 0$. Since $f$ is odd, $a_n = 0$ for all $n$. Computing $b_n$:

$b_n = \frac{1}{\pi}\int_{-\pi}^{\pi} \text{sgn}(x)\sin(nx)\,dx = \frac{2}{\pi}\int_0^{\pi}\sin(nx)\,dx = \frac{2}{n\pi}(1-\cos n\pi) = \frac{2}{n\pi}(1-(-1)^n)$

So $b_n = \frac{4}{n\pi}$ for $n$ odd and $0$ for $n$ even. The Fourier series is

$\text{sgn}(x) \sim \frac{4}{\pi}\sum_{k=0}^{\infty}\frac{\sin((2k+1)x)}{2k+1} = \frac{4}{\pi}\left(\sin x + \frac{\sin 3x}{3} + \frac{\sin 5x}{5} + \cdots\right)$

Let $f(x) = x$ for $x \in (-\pi, \pi)$. Again $f$ is odd, so $a_n = 0$. Integration by parts gives

$b_n = \frac{2}{\pi}\int_0^{\pi} x\sin(nx)\,dx = \frac{2(-1)^{n+1}}{n}$

The Fourier series is $x \sim 2\sum_{n=1}^{\infty}\frac{(-1)^{n+1}}{n}\sin(nx) = 2\left(\sin x - \frac{\sin 2x}{2} + \frac{\sin 3x}{3} - \cdots\right)$.

Let $f(x) = |x|$ for $x \in (-\pi, \pi)$. This is even, so $b_n = 0$. We get $a_0 = \pi$ and for $n \geq 1$:

$a_n = \frac{2}{\pi}\int_0^{\pi} x\cos(nx)\,dx = \frac{2}{n^2\pi}((-1)^n - 1)$

So $a_n = -\frac{4}{n^2\pi}$ for $n$ odd and $0$ for $n$ even. The series is

$|x| \sim \frac{\pi}{2} - \frac{4}{\pi}\sum_{k=0}^{\infty}\frac{\cos((2k+1)x)}{(2k+1)^2}$

The trigonometric basis $\{\cos nx, \sin nx\}$ is natural for periodic functions because: (i) they are the only bounded solutions of $y'' + n^2 y = 0$ — eigenfunctions of $-d^2/dx^2$ with eigenvalue $n^2$; (ii) they are periodic with period $2\pi/n$, so sums of such functions are $2\pi$-periodic; and (iii) they are orthogonal under the natural inner product — as the next section demonstrates.

For functions $f, g$ integrable on $[-\pi, \pi]$, define the inner product

$\langle f, g \rangle = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x)\, g(x)\,dx$

and the corresponding norm $\|f\| = \sqrt{\langle f, f \rangle} = \sqrt{\frac{1}{\pi}\int_{-\pi}^{\pi} f(x)^2\,dx}$.

The factor $1/\pi$ is a normalization that makes $\|\cos nx\| = 1$ and $\|\sin nx\| = 1$ for $n \geq 1$, while $\|1/\sqrt{2}\| = 1$.

The functions $\{1/\sqrt{2}, \cos x, \cos 2x, \ldots, \sin x, \sin 2x, \ldots\}$ form an orthonormal system under $\langle \cdot, \cdot \rangle$:

$\langle \cos mx, \cos nx \rangle = \delta_{mn}, \quad \langle \sin mx, \sin nx \rangle = \delta_{mn}, \quad \langle \cos mx, \sin nx \rangle = 0$

for all $m, n \geq 1$, and $\langle 1, \cos nx \rangle = 0$, $\langle 1, \sin nx \rangle = 0$ for $n \geq 1$, where $\delta_{mn}$ is the Kronecker delta.

By direct computation using product-to-sum formulas. For $\langle \cos mx, \cos nx \rangle$ with $m \neq n$:

$\frac{1}{\pi}\int_{-\pi}^{\pi}\cos(mx)\cos(nx)\,dx = \frac{1}{2\pi}\int_{-\pi}^{\pi}[\cos((m-n)x) + \cos((m+n)x)]\,dx = 0$

since both $\cos((m-n)x)$ and $\cos((m+n)x)$ integrate to zero over a full period (as $m \pm n \neq 0$). For $m = n$:

$\frac{1}{\pi}\int_{-\pi}^{\pi}\cos^2(nx)\,dx = \frac{1}{2\pi}\int_{-\pi}^{\pi}[1 + \cos(2nx)]\,dx = 1$

The $\langle \sin mx, \sin nx \rangle$ and $\langle \cos mx, \sin nx \rangle$ cases follow similarly using the product-to-sum identities $\sin\alpha\sin\beta = \frac{1}{2}[\cos(\alpha-\beta) - \cos(\alpha+\beta)]$ and $\cos\alpha\sin\beta = \frac{1}{2}[\sin(\alpha+\beta) - \sin(\alpha-\beta)]$.

Verify: $\langle \cos 2x, \sin 3x \rangle = \frac{1}{\pi}\int_{-\pi}^{\pi}\cos(2x)\sin(3x)\,dx = \frac{1}{2\pi}\int_{-\pi}^{\pi}[\sin(5x) + \sin(x)]\,dx = 0$. This is a specific instance of the general orthogonality: cosines and sines of different (or same) frequencies are always orthogonal.

Orthogonality gives a coordinate formula. If $f = \frac{a_0}{2} + \sum(a_n \cos nx + b_n \sin nx)$, then taking the inner product with $\cos mx$ and using orthogonality:

$\langle f, \cos mx \rangle = a_m$

Similarly $\langle f, \sin mx \rangle = b_m$. The Fourier coefficients are the orthogonal projections of $f$ onto each basis function — exactly as vector components are dot products with unit vectors in $\mathbb{R}^n$. The coefficient formulas in Definition 2 are not ad hoc; they are forced by orthogonality.

The $n$th Fourier partial sum of $f$ is

$S_n(x) = \frac{a_0}{2} + \sum_{k=1}^{n}(a_k \cos kx + b_k \sin kx)$

This can be written as a convolution: $S_n(x) = \frac{1}{\pi}\int_{-\pi}^{\pi} f(t)\, D_n(x - t)\,dt$ where

$D_n(t) = \frac{1}{2} + \sum_{k=1}^{n}\cos(kt) = \frac{\sin\left((n+\tfrac{1}{2})t\right)}{2\sin(t/2)}$

is the Dirichlet kernel.

If $f$ is integrable on $[-\pi, \pi]$, then $a_n \to 0$ and $b_n \to 0$ as $n \to \infty$.

For a step function $g = c \cdot \mathbf{1}_{[a,b]}$, direct computation gives $\int_a^b c\cos(nx)\,dx = c[\sin(nb) - \sin(na)]/n \to 0$. Every integrable function can be approximated in the $L^1$ norm by step functions (from the definition of the Riemann integral). For general $f$: given $\varepsilon > 0$, choose a step function $g$ with $\int|f - g| < \varepsilon$. Then

$|a_n(f)| \leq |a_n(f - g)| + |a_n(g)| \leq \frac{1}{\pi}\int|f - g| + |a_n(g)| < \frac{\varepsilon}{\pi} + |a_n(g)|$

Since $a_n(g) \to 0$, we get $\limsup|a_n(f)| \leq \varepsilon/\pi$. Since $\varepsilon$ is arbitrary, $a_n(f) \to 0$. The argument for $b_n$ is identical.

If $f$ is piecewise smooth on $[-\pi, \pi]$ (meaning $f$ and $f'$ are piecewise continuous), then the Fourier series converges pointwise:

$S_n(x) \to \frac{1}{2}[f(x^+) + f(x^-)] \quad \text{for every } x$

At points of continuity, this gives $S_n(x) \to f(x)$. At jump discontinuities, the partial sums converge to the average of the left and right limits.

At $x = 0$ (discontinuity of $\text{sgn}(x)$), Dirichlet’s theorem gives $S_n(0) \to \frac{1}{2}[f(0^+) + f(0^-)] = \frac{1}{2}[1 + (-1)] = 0$. At $x = \pi/2$ (point of continuity), $S_n(\pi/2) \to f(\pi/2) = 1$.

The Dirichlet kernel $D_n(t) = \sin((n+\frac{1}{2})t)/(2\sin(t/2))$ is the analog of the “evaluation functional” for Fourier series. As $n \to \infty$, $D_n$ concentrates near $t = 0$ while oscillating rapidly elsewhere — a nascent delta function. The increasingly wild oscillations of $D_n$ are what cause convergence difficulties (including the Gibbs phenomenon in the next section). Compare with power series, where no such kernel appears because the monomial basis is not orthogonal under an integral inner product.

Let $f$ have a jump discontinuity at $x_0$ with jump height $[f] = f(x_0^+) - f(x_0^-)$. Then the maximum of $S_n(x)$ near $x_0$ overshoots the target value $f(x_0^+)$ by approximately

$\frac{[f]}{2}\left(\frac{2}{\pi}\int_0^{\pi}\frac{\sin t}{t}\,dt - 1\right) \approx 0.0895 \cdot [f]$

The overshoot ratio $\frac{2}{\pi}\int_0^{\pi}\frac{\sin t}{t}\,dt \approx 1.1790$ is independent of $n$ — increasing $n$ compresses the overshoot into a narrower region near $x_0$ but does not reduce its height.

The square wave $f(x) = \text{sgn}(x)$ has jump $[f] = 2$ at $x = 0$. The maximum of $S_n(x)$ near $x = 0^+$ approaches approximately $1 + 2 \times 0.0895 = 1.179$ as $n \to \infty$, not $1$. The location of the maximum moves toward $x = 0$ as $n$ increases (at approximately $x = \pi/n$), but its height remains $\approx 1.179$.

The Gibbs phenomenon means that $S_n \to f$ is not uniform near a jump discontinuity, even though it is pointwise. This is fundamentally different from power series, where convergence inside the radius of convergence is uniform on compact subsets (Power Series & Taylor Series, Theorem 4). For Fourier series, uniform convergence requires global smoothness (see Section 7).

(a) If $f$ is $C^k$ (i.e., $k$ times continuously differentiable) and $2\pi$-periodic, then $a_n, b_n = O(1/n^k)$ as $n \to \infty$. In particular:

• Discontinuous $\Rightarrow$ $O(1/n)$
• $C^1 \Rightarrow O(1/n^2)$
• $C^\infty \Rightarrow$ faster than any polynomial (superpolynomial decay)

(b) The mechanism: integration by parts $k$ times transfers $k$ powers of $n$ from the coefficient to the function. Specifically, if $f$ is $C^k$ and periodic, then the Fourier coefficients of $f^{(k)}$ satisfy $(f^{(k)})^\wedge(n) = (in)^k \hat{f}(n)$, so $|c_n| = |c_n^{(k)}|/n^k$ where $c_n^{(k)}$ are the (bounded) coefficients of $f^{(k)}$.

(a) The square wave has $b_n = O(1/n)$ — discontinuous function, slowest possible decay.

(b) The triangle wave has $a_n = O(1/n^2)$ — continuous but not differentiable (corner at $x = 0$), one degree faster.

(c) The smooth periodic function $f(x) = 1/(2 - \cos x)$ has coefficients that decay exponentially: $a_n = O(r^n)$ with $r = 2 - \sqrt{3} \approx 0.268$. This is the Fourier counterpart of the smooth-vs-analytic distinction from Power Series & Taylor Series.

If $f$ is $C^1$ (continuously differentiable) and $2\pi$-periodic, then the Fourier series of $f$ converges uniformly to $f$.

More generally, if $f$ is continuous, $2\pi$-periodic, and the Fourier coefficients satisfy $\sum_{n=1}^{\infty}(|a_n| + |b_n|) < \infty$, then the Fourier series converges uniformly by the Weierstrass M-test from Uniform Convergence.

The triangle wave $f(x) = |x|$ on $(-\pi, \pi)$ has $a_n = O(1/n^2)$. Since $\sum 1/n^2 < \infty$ ($p$-series with $p = 2$ from Series Convergence & Tests), the Fourier series converges uniformly to $f$ by the Weierstrass M-test. There is no Gibbs phenomenon because $f$ is continuous (though not differentiable). But the convergence rate is $O(1/n)$ in the sup-norm — much slower than exponential convergence for smooth functions.

For any integrable $f$ on $[-\pi, \pi]$,

$\frac{a_0^2}{2} + \sum_{n=1}^{N}(a_n^2 + b_n^2) \leq \frac{1}{\pi}\int_{-\pi}^{\pi} f(x)^2\,dx = \|f\|^2$

for every $N$. Taking $N \to \infty$: $\frac{a_0^2}{2} + \sum_{n=1}^{\infty}(a_n^2 + b_n^2) \leq \|f\|^2$.

Expand $\|f - S_N\|^2 \geq 0$:

$0 \leq \frac{1}{\pi}\int_{-\pi}^{\pi}\bigl(f(x) - S_N(x)\bigr)^2\,dx = \|f\|^2 - 2\langle f, S_N \rangle + \|S_N\|^2$

By orthogonality, $\langle f, S_N \rangle = \frac{a_0^2}{2} + \sum_{n=1}^{N}(a_n^2 + b_n^2)$ and $\|S_N\|^2 = \frac{a_0^2}{2} + \sum_{n=1}^{N}(a_n^2 + b_n^2)$. Substituting:

$0 \leq \|f\|^2 - \frac{a_0^2}{2} - \sum_{n=1}^{N}(a_n^2 + b_n^2) \qquad \square$

If $f$ is square-integrable on $[-\pi, \pi]$, then

$\frac{a_0^2}{2} + \sum_{n=1}^{\infty}(a_n^2 + b_n^2) = \|f\|^2 = \frac{1}{\pi}\int_{-\pi}^{\pi} f(x)^2\,dx$

Equality holds in Bessel’s inequality: no energy is lost in the Fourier decomposition.

The square wave $\text{sgn}(x)$ has $\|f\|^2 = \frac{1}{\pi}\int_{-\pi}^{\pi} 1\,dx = 2$. With $b_n = 4/(n\pi)$ for odd $n$, Parseval gives

$\sum_{k=0}^{\infty} \frac{16}{(2k+1)^2\pi^2} = 2$

which rearranges to $\sum_{k=0}^{\infty}\frac{1}{(2k+1)^2} = \frac{\pi^2}{8}$ — a non-trivial identity derived purely from Parseval.

The sawtooth $f(x) = x$ has $\|f\|^2 = \frac{1}{\pi}\int_{-\pi}^{\pi} x^2\,dx = \frac{2\pi^2}{3}$. With $b_n = 2(-1)^{n+1}/n$, Parseval gives

$\sum_{n=1}^{\infty}\frac{4}{n^2} = \frac{2\pi^2}{3}$

which yields $\sum_{n=1}^{\infty}\frac{1}{n^2} = \frac{\pi^2}{6}$ — the Basel problem, one of the most celebrated results in analysis, solved here by Fourier theory.

If $f$ is integrable on $[-\pi, \pi]$ with Fourier series $f(x) \sim \frac{a_0}{2} + \sum(a_n \cos nx + b_n \sin nx)$, then for any $x \in [-\pi, \pi]$:

$\int_{-\pi}^{x} f(t)\,dt = \frac{a_0}{2}(x + \pi) + \sum_{n=1}^{\infty}\left(\frac{a_n \sin nx}{n} + \frac{b_n(1 - \cos nx)}{n}\right)$

The integrated series converges uniformly, even if the original Fourier series does not.

The integrated series has coefficients $O(a_n/n)$ and $O(b_n/n)$. By Bessel’s inequality from Theorem 2, $\sum(a_n^2 + b_n^2) < \infty$. By Cauchy-Schwarz,

$\sum_{n=1}^{\infty}\left(\frac{|a_n|}{n} + \frac{|b_n|}{n}\right) \leq \sqrt{\sum(a_n^2 + b_n^2)} \cdot \sqrt{\sum \frac{1}{n^2}} < \infty$

The Weierstrass M-test from Uniform Convergence then gives uniform convergence.

Integrating $\text{sgn}(x) \sim \frac{4}{\pi}\sum_{k=0}^{\infty}\frac{\sin((2k+1)x)}{2k+1}$ from $-\pi$ to $x$ gives

$|x| - \pi \sim -\frac{4}{\pi}\sum_{k=0}^{\infty}\frac{1 - \cos((2k+1)x)}{(2k+1)^2}$

which converges uniformly — the integrated series of a non-uniformly convergent Fourier series is uniformly convergent. Integration improves convergence by one order.

Term-by-term differentiation of Fourier series is not generally valid. Differentiating the square wave’s Fourier series gives $\frac{4}{\pi}\sum_{k=0}^{\infty}\cos((2k+1)x)$, which diverges everywhere. Differentiation multiplies the $n$th coefficient by $n$, making decay worse. For term-by-term differentiation to be valid, $f$ must be $C^1$ and periodic (so that $f'$ also has a convergent Fourier series). Compare with power series, where differentiation preserves the radius of convergence — Fourier series are more delicate.

Using $\cos nx = (e^{inx} + e^{-inx})/2$ and $\sin nx = (e^{inx} - e^{-inx})/(2i)$, the Fourier series can be written

$f(x) \sim \sum_{n=-\infty}^{\infty} c_n e^{inx} \qquad \text{where} \quad c_n = \frac{1}{2\pi}\int_{-\pi}^{\pi} f(x)e^{-inx}\,dx$

The complex form is more compact and connects directly to the Fourier transform (the non-periodic analog, extending this to all of $\mathbb{R}$). In the complex form, Parseval’s identity becomes $\sum_{n=-\infty}^{\infty}|c_n|^2 = \frac{1}{2\pi}\int_{-\pi}^{\pi}|f(x)|^2\,dx$.