Inner Product

Definition

The inner product (dot product, scalar product) of two vectors is the matrix product of a row vector with a column vector, yielding a scalar.

Intuition

Geometrically, ${\mathbf{u}}^{\top }\mathbf{v}=\Vert \mathbf{u}\Vert \Vert \mathbf{v}\Vert \cos \theta$ where $\theta$ is the angle between the two vectors. It measures how much one vector “projects onto” the other. When the inner product is zero, the vectors point in completely perpendicular directions; when it equals the product of their norms, they are perfectly aligned.

Formal Description

For $3\times 1$ column vectors $\mathbf{u},\mathbf{v}$:

$${\mathbf{u}}^{\top }\mathbf{v}=\left(\begin{array}{ccc}{u}_1& u_2& u_3\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\\ v_3\end{array}\right)=u_1{v}_1+u_2{v}_2+u_3{v}_3$$

• If ${\mathbf{u}}^{\top }\mathbf{v}=0$ for nonzero $\mathbf{u},\mathbf{v}$, the vectors are orthogonal.
• The norm (Euclidean length) of a vector: $\Vert \mathbf{u}\Vert =({\mathbf{u}}^{\top }\mathbf{u}{)}^{1/2}=(u_1^2+u_2^2+u_3^2{)}^{1/2}$.
• A vector with $\Vert \mathbf{u}\Vert =1$ is normalised.
• A set of vectors that are mutually orthogonal and each normalised is called orthonormal.
• The inner product is commutative: ${\mathbf{u}}^{\top }\mathbf{v}={\mathbf{v}}^{\top }\mathbf{u}$.

Applications

• Computing similarity between vectors (e.g. cosine similarity in NLP and recommendation systems).
• Defining the Euclidean norm and distance for optimisation and nearest-neighbour search.
• Checking orthogonality; forming orthonormal bases via Gram–Schmidt.

Trade-offs

• The standard inner product assumes the Euclidean metric; in weighted or non-Euclidean spaces a different inner product (e.g. ${\mathbf{u}}^{\top }\mathbf{M}\mathbf{v}$ for positive-definite $\mathbf{M}$) may be more appropriate.
• For complex vectors, the inner product is ${\mathbf{u}}^{\ast }\mathbf{v}$ (conjugate transpose), not ${\mathbf{u}}^{\top }\mathbf{v}$; confusing the two breaks conjugate symmetry and positive-definiteness.
• The inner product collapses two vectors to a single number, discarding directional information — use the outer product when the full rank-1 interaction matrix is needed.

Links

• Matrix Operations
• Outer Product
• Special Matrices (orthogonal matrices)