Row Space and Rank

Definition

For an $m\times n$ matrix $\mathrm{A}$, there are two additional fundamental vector spaces beyond the column space and null space:

• Row space $\mathrm{Row}(\mathrm{A})$: the column space of ${\mathrm{A}}^T$ — the span of the rows of $\mathrm{A}$, a subspace of ${\mathbb{R}}^n$.
• Left null space ${\mathrm{Null}(\mathrm{A}}^T)$: the null space of ${\mathrm{A}}^T$ — all $\mathrm{y}$ with ${\mathrm{A}}^T\mathrm{y}=0$, a subspace of ${\mathbb{R}}^m$.

Intuition

A matrix has four fundamental subspaces that come in two orthogonal pairs: the row space and null space partition ${\mathbb{R}}^n$ (the input space), while the column space and left null space partition ${\mathbb{R}}^m$ (the output space). Rank is the “effective dimensionality” of the mapping — how many independent output directions the matrix can reach.

Formal Description

A basis for the row space can be read directly from $\mathrm{rref}(\mathrm{A})$: the non-zero rows (written as column vectors) with pivot entries form a basis.

The null space of $\mathrm{A}$ is orthogonal to the row space: every vector in $\mathrm{Null}(\mathrm{A})$ is orthogonal to every vector in $\mathrm{Row}(\mathrm{A})$. These two subspaces are orthogonal complements within ${\mathbb{R}}^n$, i.e. their dimensions sum to $n$. Similarly, $\mathrm{Col}(\mathrm{A})$ and $\mathrm{Null}({\mathrm{A}}^{\mathrm{T}})$ are orthogonal complements within ${\mathbb{R}}^m$.

The dimensions of the column space and row space are equal:

$$\dim (\mathrm{Col}(\mathrm{A}))=\dim (\mathrm{Row}(\mathrm{A})).$$

This common value is the rank of $\mathrm{A}$:

$$\mathrm{rank}(\mathrm{A})\le \min (m,n).$$

When equality holds, $\mathrm{A}$ is of full rank. For a square matrix of full rank, $\dim (\mathrm{Null}(\mathrm{A}))=0$ and $\mathrm{A}$ is invertible.

Applications

• Rank reveals the number of linearly independent constraints or equations in a system.
• Full column rank is necessary and sufficient for ${\mathrm{A}}^T\mathrm{A}$ to be invertible (uniqueness in least squares).
• Rank deficiency signals multicollinearity in regression or redundant features in a dataset.

Trade-offs

Rank computed via RREF can be numerically unreliable for nearly-rank-deficient matrices. Numerical rank is better estimated as the number of singular values exceeding a threshold $\epsilon$ (SVD-based), at the cost of $O(\min (m,n)\cdot mn)$ computation.

Links

• Column Space
• Null Space
• Span, Basis and Dimension
• Vector Space