Column Space

Definition

The column space of a matrix $\mathrm{A}$, denoted $\mathrm{Col}(\mathrm{A})$, is the span of the columns of $\mathrm{A}$. For an $m\times n$ matrix, $\mathrm{Col}(\mathrm{A})$ is a subspace of ${\mathbb{R}}^m$.

Intuition

$\mathrm{Ax}$ is just a weighted sum of the columns of $\mathrm{A}$, with weights given by $\mathrm{x}$. So the column space is exactly the set of all outputs the matrix can produce — the range of the linear map. The system $\mathrm{Ax}=\mathrm{b}$ is solvable precisely when $\mathrm{b}$ lands inside this output set.

Formal Description

For any vector $\mathrm{x}$, the product $\mathrm{Ax}$ is a linear combination of the columns of $\mathrm{A}$:

$$\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)=x\left(\begin{array}{c}a\\ c\end{array}\right)+y\left(\begin{array}{c}b\\ d\end{array}\right).$$

Thus $\mathrm{Ax}\in \mathrm{Col}(\mathrm{A})$ for all $\mathrm{x}$, and the system $\mathrm{Ax}=\mathrm{b}$ is consistent if and only if $\mathrm{b}\in \mathrm{Col}(\mathrm{A})$.

Finding a basis. Row operations preserve the linear dependence relations among columns. The pivot columns of $\mathrm{rref}(\mathrm{A})$ identify which columns of $\mathrm{A}$ (not $\mathrm{rref}(\mathrm{A})$) form a basis for $\mathrm{Col}(\mathrm{A})$.

Example.

$$\mathrm{A}=\left(\begin{array}{ccccc}-3& 6& -1& 1& -7\\ 1& -2& 2& 3& -1\\ 2& -4& 5& 8& -4\end{array}\right),\mathrm{rref}(\mathrm{A})=\left(\begin{array}{ccccc}1& -2& 0& -1& 3\\ 0& 0& 1& 2& -2\\ 0& 0& 0& 0& 0\end{array}\right).$$

The pivot columns are 1 and 3, so a basis for $\mathrm{Col}(\mathrm{A})$ is

$$\left\{\left(\begin{array}{c}-3\\ 1\\ 2\end{array}\right),\left(\begin{array}{c}-1\\ 2\\ 5\end{array}\right)\right\}.$$

Rank-nullity theorem. For an $m\times n$ matrix $\mathrm{A}$:

$$\dim (\mathrm{Col}(\mathrm{A}))+\dim (\mathrm{Null}(\mathrm{A}))=n.$$

The dimension of $\mathrm{Col}(\mathrm{A})$ equals the number of pivot columns; the dimension of $\mathrm{Null}(\mathrm{A})$ equals the number of non-pivot (free) columns.

Applications

• Determining solvability of $\mathrm{Ax}=\mathrm{b}$: consistent iff $\mathrm{b}\in \mathrm{Col}(\mathrm{A})$.
• Least-squares problems project $\mathrm{b}$ onto $\mathrm{Col}(\mathrm{A})$ when no exact solution exists.
• In neural networks, each layer maps inputs into the column space of its weight matrix.

Trade-offs

The column space depends on which columns are chosen as a basis — basis vectors are not unique, though the subspace they span is. Using the actual pivot columns of $\mathrm{A}$ (not $\mathrm{rref}(\mathrm{A})$) preserves the geometric meaning of the original data.

Links

• Null Space
• Span, Basis and Dimension
• Row Space and Rank
• Vector Space