Brownian Bridge™

Setup

Every matrix — whether square or rectangular, rank-deficient or full-rank — admits a singular value decomposition (SVD). This is a strictly stronger result than eigendecomposition, which applies only to square matrices and fails for non-diagonalisable ones. The SVD is the canonical tool for understanding the geometry of a linear map and the sensitivity of the linear system $Ax = b$ .

Where this lives on a desk. The SVD appears in three distinct quant workflows:

Calibration and least-squares fitting. When you fit a volatility surface or calibrate a yield curve model, you solve an overdetermined system $\min_\theta \|F(\theta) - \text{market}\|^2$ . The normal equations $A^\top A\theta = A^\top b$ are solved via the pseudoinverse $A^+ = V\Sigma^+ U^\top$ — the SVD-based answer that minimises the residual with minimum-norm $\theta$ .
Risk factor models. Returns matrices $X \in \mathbb{R}^{T \times n}$ are decomposed into orthogonal factors. SVD of $X$ gives the principal components directly without forming $X^\top X$ — numerically more stable when $T$ is small relative to $n$ .
Numerical stability diagnostics. The condition number $\kappa_2(A) = \sigma_1 / \sigma_n$ quantifies how much the solution $x$ of $Ax = b$ amplifies errors in $b$ . An ill-conditioned calibration problem signals that the model has redundant parameters or the data is insufficient to identify them separately.

Mathematical setting. Let $A \in \mathbb{R}^{m \times n}$ with $m \geq n$ (the overdetermined case common in calibration). No symmetry assumption. $r = \text{rank}(A) \leq \min(m, n)$ .

Notation. $\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_r > 0 = \sigma_{r+1} = \cdots = \sigma_n$ are the singular values of $A$ . Subscripts follow the usual convention: $\sigma_1$ is the largest (spectral norm of $A$ ).

INSIGHT

Financial Insight. SVD makes the geometry of the calibration problem explicit: $U$ rotates the output space (market observables), $V$ rotates the input space (model parameters), and $\Sigma$ stretches/compresses each independent direction by its singular value. Directions with $\sigma_i \approx 0$ are directions in parameter space that barely affect market observables — the model is near-unidentifiable in those directions.

Theory

1. The Singular Value Decomposition

THEOREM

Theorem 4.1 (SVD Existence). For any $A \in \mathbb{R}^{m \times n}$ there exist orthogonal matrices $U \in \mathbb{R}^{m \times m}$ and $V \in \mathbb{R}^{n \times n}$ , and a matrix $\Sigma \in \mathbb{R}^{m \times n}$ with $\Sigma_{ii} = \sigma_i \geq 0$ and $\Sigma_{ij} = 0$ for $i \neq j$ , such that $A = U \Sigma V^\top.$ The diagonal entries of $\Sigma$ , taken in non-increasing order $\sigma_1 \geq \sigma_2 \geq \cdots \geq 0$ , are unique and called the singular values of $A$ .

Derivation. The key is to relate singular values to eigenvalues of the symmetric matrices $A^\top A$ and $A A^\top$ .

Since $A^\top A \in \mathbb{R}^{n \times n}$ is symmetric positive semi-definite (SPSD), the Spectral Theorem (Module 3) guarantees $A^\top A = V \Lambda V^\top, \quad \Lambda = \text{diag}(\lambda_1, \ldots, \lambda_n), \quad \lambda_i \geq 0, \quad V \text{ orthogonal.}$

Define $\sigma_i = \sqrt{\lambda_i}$ for $i = 1, \ldots, r$ (the non-zero eigenvalues). For $i \leq r$ , set $u_i = \frac{1}{\sigma_i} A v_i \in \mathbb{R}^m.$

These are orthonormal: for $i \neq j$ , $\langle u_i, u_j \rangle = \frac{1}{\sigma_i \sigma_j} v_i^\top A^\top A v_j = \frac{1}{\sigma_i \sigma_j} v_i^\top V \Lambda V^\top v_j = \frac{\lambda_j}{\sigma_i \sigma_j} \delta_{ij} = 0.$

Extend $\{u_1, \ldots, u_r\}$ to an orthonormal basis $\{u_1, \ldots, u_m\}$ for $\mathbb{R}^m$ . Then $A = U \Sigma V^\top$ holds by construction.

Note: $\sigma_i^2 = \lambda_i(A^\top A) = \lambda_i(A A^\top)$ for $i \leq \min(m,n)$ — the non-zero eigenvalues of $A^\top A$ and $A A^\top$ coincide.

DEFINITION

Definition 4.1 (Thin SVD). The thin (economy) SVD retains only the $r$ non-zero singular values: $A = U_r \Sigma_r V_r^\top,$ where $U_r \in \mathbb{R}^{m \times r}$ , $\Sigma_r = \text{diag}(\sigma_1, \ldots, \sigma_r) \in \mathbb{R}^{r \times r}$ , $V_r \in \mathbb{R}^{n \times r}$ . For $m \gg r$ this is far cheaper to compute and store than the full SVD.

2. Geometric Interpretation

The decomposition $A = U \Sigma V^\top$ factors the action of $A$ into three steps:

$V^\top$ : rotate the input vector $x$ into the right singular vector basis.
$\Sigma$ : independently scale each coordinate by $\sigma_i$ (and discard the null space).
$U$ : rotate the stretched vector into the output (column) space.

INSIGHT

Financial Insight. Think of $V^\top$ as transforming raw model parameters into uncorrelated "eigen-parameters". $\Sigma$ tells you how sensitively observable prices respond to each eigen-parameter. Eigen-parameters with near-zero singular values are unobservable from the data — the calibration is degenerate along those directions.

3. The Four Fundamental Subspaces via SVD

The SVD gives explicit orthonormal bases for all four subspaces first seen in Module 1.

THEOREM

Theorem 4.2 (Fundamental Subspaces). Let $A = U \Sigma V^\top$ with rank $r$ . Then:

Column space (image): $\text{col}(A) = \text{span}(u_1, \ldots, u_r)$ — first $r$ left singular vectors.
Left null space: $\text{null}(A^\top) = \text{span}(u_{r+1}, \ldots, u_m)$ — last $m - r$ left singular vectors.
Row space: $\text{row}(A) = \text{span}(v_1, \ldots, v_r)$ — first $r$ right singular vectors.
Null space: $\text{null}(A) = \text{span}(v_{r+1}, \ldots, v_n)$ — last $n - r$ right singular vectors.

Proof sketch. $Av_i = \sigma_i u_i$ for $i \leq r$ (so $v_i \in \text{row}(A)$ , $u_i \in \text{col}(A)$ ), and $Av_i = 0$ for $i > r$ (so $v_i \in \text{null}(A)$ ). The orthogonality of $U$ and $V$ gives the subspace dimensions. $\square$

4. The Moore-Penrose Pseudoinverse

When $Ax = b$ is overdetermined ( $m > n$ , more equations than unknowns) the system generally has no exact solution. The least-squares solution minimising $\|Ax - b\|_2$ is:

DEFINITION

Definition 4.2 (Pseudoinverse). The Moore-Penrose pseudoinverse of $A$ is $A^+ = V \Sigma^+ U^\top,$ where $\Sigma^+$ replaces each non-zero $\sigma_i$ by $1/\sigma_i$ and leaves zero entries as zero.

Why this solves the LS problem. For $A = U\Sigma V^\top$ with full column rank ( $r = n$ ): $A^+ = (A^\top A)^{-1} A^\top = V \Sigma^{-1} U_n^\top.$ This is the same formula as the normal equations, but computed via SVD — numerically stable even when $A^\top A$ is ill-conditioned.

Among all minimisers, $x^* = A^+ b$ is the one with smallest norm $\|x\|_2$ — relevant when the system has a null space and you want the minimum-norm parameter vector.

EXAMPLE

Example 4.1 (Surface calibration as LS). Fitting $n = 15$ SABR parameters to $m = 60$ market option prices gives an overdetermined system. The pseudoinverse $\hat\theta = A^+ b_\text{market}$ finds the minimum-residual parameter set. If the condition number $\kappa_2(A) \approx 10^8$ , then a $10^{-4}$ relative error in market quotes translates to a $10^4$ relative error in the inferred parameters — the calibration is numerically singular and requires regularisation.

5. Condition Number and Perturbation Theory

DEFINITION

Definition 4.3 (Condition number). The 2-norm condition number of $A \in \mathbb{R}^{n \times n}$ invertible is $\kappa_2(A) = \|A\|_2 \|A^{-1}\|_2 = \frac{\sigma_1}{\sigma_n}.$ For a rectangular $A$ (or rank-deficient), use $\kappa_2(A) = \sigma_1 / \sigma_r$ where $\sigma_r$ is the smallest non-zero singular value.

Why it matters. Consider $Ax = b$ . If $b$ is perturbed by $\delta b$ (market quote error, floating-point rounding), the perturbed solution $\hat x$ satisfies:

THEOREM

Theorem 4.3 (Perturbation bound). Let $A\hat x = b + \delta b$ . Then $\frac{\|\delta x\|_2}{\|x\|_2} \leq \kappa_2(A) \cdot \frac{\|\delta b\|_2}{\|b\|_2}.$

Proof. $\delta x = A^{-1} \delta b$ , so $\|\delta x\| \leq \|A^{-1}\| \|\delta b\| = \sigma_n^{-1} \|\delta b\|$ . Combine with $\|b\| \leq \|A\| \|x\| = \sigma_1 \|x\|$ . $\square$

REMARK

Remark. The condition number is a worst-case amplification factor. In practice, errors in $b$ are not aligned with the worst-case direction (the left singular vector $u_n$ ), so the actual amplification is often much smaller. However, $\kappa_2(A)$ is the right diagnostic for "is this problem well-posed?"

Intuition for $\kappa_2$ values:

$\kappa_2(A)$	Interpretation
$\approx 1$	Well-conditioned; errors not amplified
$10^3$	Machine precision $\approx 10^{-16}$ → solution accurate to $10^{-13}$
$10^8$	Solution may have only 8 significant figures
$\geq 10^{16}$	Numerically singular at double precision

6. Low-Rank Approximation (Eckart-Young Theorem)

The spectral truncation from Module 3 extends naturally to non-symmetric matrices via SVD.

THEOREM

Theorem 4.4 (Eckart-Young, 1936). Among all matrices of rank at most $k$ , the best approximation to $A$ in both the spectral norm $\|\cdot\|_2$ and the Frobenius norm $\|\cdot\|_F$ is $A_k = \sum_{i=1}^k \sigma_i u_i v_i^\top.$ The approximation errors are: $\|A - A_k\|_2 = \sigma_{k+1}, \qquad \|A - A_k\|_F = \sqrt{\sum_{i=k+1}^r \sigma_i^2}.$

Comparison with eigendecomposition (Module 3). For symmetric $A$ , singular values equal absolute values of eigenvalues: $\sigma_i = |\lambda_i|$ . The Eckart-Young theorem for the symmetric case used $\sqrt{\sum_{i>k} \lambda_i^2}$ — identical to the SVD formula when eigenvalues are non-negative.

EXAMPLE

Example 4.2 (Returns matrix compression). A returns matrix $X \in \mathbb{R}^{252 \times 100}$ (252 daily returns, 100 assets) has rank 100. The rank- $k$ SVD approximation $X_k$ captures the $k$ dominant risk factors. The ratio $\sum_{i=1}^k \sigma_i^2 / \sum_{i=1}^{100} \sigma_i^2$ measures the fraction of the total Frobenius-norm squared (proportional to total variance) explained by the first $k$ factors — equivalent to the PCA variance explained ratio from Module 3.

7. Regularisation and Truncated SVD

When $\kappa_2(A)$ is large, the naive pseudoinverse $A^+ = V\Sigma^+ U^\top$ amplifies noise in $b$ . Two standard remedies:

Truncated SVD (TSVD). Retain only the $k$ largest singular values; set $\sigma_{k+1}^+ = \cdots = \sigma_n^+ = 0$ : $x_k^{\text{TSVD}} = \sum_{i=1}^k \frac{u_i^\top b}{\sigma_i} v_i.$ Bias increases but variance (noise amplification) decreases as $k$ decreases.

Tikhonov regularisation. Solve $\min_x \|Ax - b\|_2^2 + \lambda \|x\|_2^2$ , which has the closed-form solution: $x_\lambda^{\text{Tikh}} = (A^\top A + \lambda I)^{-1} A^\top b = \sum_{i=1}^n \frac{\sigma_i}{\sigma_i^2 + \lambda} (u_i^\top b) v_i.$ This smoothly down-weights directions with $\sigma_i \ll \sqrt{\lambda}$ rather than hard-truncating them.

WARNING

Warning. Choosing $\lambda$ (or $k$ ) is a model selection problem, not a linear algebra problem. Too small: noise amplified. Too large: the solution is biased toward zero. Cross-validation or L-curve methods are required. On a calibration desk, the regularisation parameter often encodes prior information about parameter magnitude — equivalent to a Bayesian prior.

Validation

The companion notebook verifies:

SVD factorisation: $\|A - U\Sigma V^\top\|_F < \varepsilon_\text{machine}$ for a random $5 \times 3$ matrix.
Orthogonality: $\|U^\top U - I\|$ , $\|V^\top V - I\|$ both near zero.
Four subspaces: $Av_i = \sigma_i u_i$ for $i \leq r$ ; $Av_i \approx 0$ for $i > r$ .
Pseudoinverse: $A^+ b$ minimises $\|Ax - b\|$ over all $x$ ; verified by perturbing from $A^+b$ and checking residual increases.
Condition number: Hilbert matrix $H_n$ with $H_{ij} = 1/(i+j-1)$ is notoriously ill-conditioned; $\kappa_2(H_n)$ grows exponentially with $n$ .
Eckart-Young error: $\|A - A_k\|_F = \sqrt{\sum_{i>k} \sigma_i^2}$ checked against direct computation.

PRACTICE

Hand exercise (before running the notebook). Let $A = \begin{pmatrix} 3 & 0 \\ 0 & 2 \\ 0 & 0 \end{pmatrix}$ . By inspection: (a) What are the singular values of $A$ ? (b) What is $A^\top A$ ? What are its eigenvalues? (c) Write down $U$ , $\Sigma$ , $V^\top$ explicitly. (d) What is $A^+$ ? Verify $A^+ A = I_2$ . (Answer: $\sigma_1 = 3$ , $\sigma_2 = 2$ ; $A^\top A = \text{diag}(9, 4)$ ; $U = [e_1 \, e_2 \, e_3]$ , $\Sigma = \begin{pmatrix}3&0\\0&2\\0&0\end{pmatrix}$ , $V = I_2$ ; $A^+ = \begin{pmatrix}1/3&0&0\\0&1/2&0\end{pmatrix}$ .)

Limitations

WARNING

Warning: SVD cost vs. eigendecomposition. Full SVD of $A \in \mathbb{R}^{m \times n}$ costs $O(\min(m,n)^2 \max(m,n))$ flops. For large, square symmetric matrices (covariance matrices in risk), eigh (which exploits symmetry) is faster. Use SVD when $A$ is rectangular, not symmetric, or when you need the pseudoinverse explicitly.

WARNING

Warning: condition number is a worst-case bound. $\kappa_2(A) = 10^8$ does not mean the solution is wrong by a factor of $10^8$ — only that it could be. If the right-hand side $b$ is nearly orthogonal to the worst-case left singular vector $u_n$ , the actual error is much smaller. But in calibration with many market quotes, the error vector can project onto any direction, so worst-case is relevant.

WARNING

Warning: truncated SVD vs. Tikhonov at the boundary. TSVD is optimal when the signal and noise occupy disjoint singular value subspaces (rare in practice). Tikhonov is optimal when noise is Gaussian and the prior on $x$ is Gaussian. In the common case where neither holds exactly, both are approximations and the choice of threshold / $\lambda$ dominates the solution quality.

Model failure modes:

Near-degenerate singular values: like near-equal eigenvalues (Module 3), the singular vector basis is unstable. A small perturbation in $A$ rotates $v_i$ and $v_j$ arbitrarily if $\sigma_i \approx \sigma_j$ .
Rank determination: floating-point $\sigma_i$ are never exactly zero. A threshold $\tau$ must be chosen to determine effective rank. The common choice $\tau = \max(m,n) \cdot \varepsilon_\text{machine} \cdot \sigma_1$ is the numpy.linalg.matrix_rank default.
Double-precision overflow: for matrices with $\sigma_1 \approx 10^{150}$ , intermediate computations in $A^\top A$ overflow. Work with the matrix $A$ directly via scipy.linalg.lstsq rather than forming $A^\top A$ explicitly.

Interview Angle

PRACTICE

L1 (Junior Quant Developer / Junior Quant).

Expected depth: definitions, the formula, one worked example.

"What is the singular value decomposition? How does it differ from eigendecomposition?" Answer: SVD works for any matrix $A = U\Sigma V^\top$ (rectangular, non-symmetric). Eigendecomposition requires a square matrix and may fail (no full eigenbasis). For symmetric $A$ , singular values = |eigenvalues|, $U = V = Q$ from the spectral theorem.
"What does the condition number tell you?" Answer: $\kappa_2(A) = \sigma_1/\sigma_n$ ; it bounds how much relative errors in $b$ are amplified in the solution to $Ax = b$ . For calibration, a large condition number signals near-unidentifiability of some parameters.
"How do you compute the pseudoinverse? When do you need it?" Answer: $A^+ = V\Sigma^+ U^\top$ . Needed for overdetermined LS (calibration, regression), rank-deficient systems (degenerate covariance), and minimum-norm solutions.

PRACTICE

L2 (Senior Quant / Quant Researcher).

Expected depth: derivation, connection to LS, condition number perturbation theory.

"Derive the pseudoinverse from first principles and prove it solves the LS problem." Answer: Start from $A = U\Sigma V^\top$ . LS minimises $\|Ax - b\|^2 = \|U\Sigma V^\top x - b\|^2 = \|\Sigma y - \tilde b\|^2$ where $y = V^\top x$ , $\tilde b = U^\top b$ . Minimise: set $y_i = \tilde b_i / \sigma_i$ for $i \leq r$ , $y_i = 0$ for $i > r$ (minimum norm). Back-substitute: $x^* = Vy = V\Sigma^+ U^\top b = A^+b$ .
"A calibration of 20 SABR parameters to 80 market prices gives condition number $10^{10}$ . What is the implication and how do you fix it?" Answer: A $10^{-6}$ relative bid-ask spread translates to $10^4$ relative error in some parameter directions — numerically meaningless. Fix: (i) identify near-zero singular values (those below threshold $\tau$ ); (ii) either truncate (TSVD) or regularise (Tikhonov with $\lambda$ chosen by L-curve or cross-validation); (iii) consider reducing model complexity — the ill-conditioning signals over-parameterisation.

PRACTICE

L3 (Quant Researcher / Model Risk).

Expected depth: original analysis, regularisation theory, model risk implications.

"Compare TSVD and Tikhonov regularisation. Under what data-generating processes is each optimal, and how would you choose between them for vol surface calibration?" Answer: TSVD is optimal when the signal lives in a low-dimensional subspace (top $k$ singular directions) and noise is in the complement. Tikhonov is the minimum-variance unbiased estimator under Gaussian prior on $x$ and Gaussian noise on $b$ . For vol surfaces, neither assumption holds strictly: skew structure creates a soft separation. Practical choice: use Tikhonov with $\lambda$ determined by GCV (generalised cross-validation) or the L-curve. TSVD is preferable when interpretability matters (clear separation of priced vs. unpriced risk factors).
"How does the SVD of the forward sensitivity matrix $\partial F / \partial \theta$ inform model risk for a structured product?" Answer: Singular values of the Jacobian measure how sensitively each market observable responds to each parameter direction. Directions with $\sigma_i \approx 0$ are parameter combinations that cannot be identified from the hedging instruments. These directions represent model risk: their contribution to the price is not hedged. Monitoring these via daily re-calibration SVD is a model risk control. If a previously identifiable singular direction becomes near-zero, it signals either a regime change or a model breakdown.