Orthogonal projection
Orthogonal projection finds the closest vector in a subspace under the Euclidean norm. The residual is perpendicular to that subspace.
Definition
Orthogonal projection finds the closest vector in a subspace under the Euclidean norm. The residual is perpendicular to that subspace.
Intuition
It is the shadow of a vector on the directions the subspace can represent.
Worked example
Projecting (3, 4) onto the horizontal axis gives (3, 0), with residual (0, 4).
The math
Onto a nonzero vector u: .
In machine learning
Projection underlies least squares, feature residualization, and decomposition of a signal into components.
Sources
More in Linear algebra
Assembled from the ReLU.chat curated knowledge base. These explanations are concise on purpose; check the sources for anything important.