Computing Inverses

Definition

The inverse ${\mathrm{A}}^{-1}$ of an invertible square matrix $\mathrm{A}$ satisfies $\mathrm{A}{\mathrm{A}}^{-1}=\mathrm{I}$. It can be computed by reducing the augmented matrix $[\mathrm{A}\mid \mathrm{I}]$ to $[\mathrm{I}\mid {\mathrm{A}}^{-1}]$ via Reduced Row Echelon Form.

Intuition

${\mathrm{A}}^{-1}$ is the matrix whose columns solve $\mathrm{Ax}={\mathrm{e}}_{\mathrm{i}}$ for each standard basis vector ${\mathrm{e}}_{\mathrm{i}}$. Augmenting $\mathrm{A}$ with the full identity and row-reducing solves all $n$ systems simultaneously in a single pass.

Formal Description

Let the columns of ${\mathrm{A}}^{-1}$ be ${\mathrm{a}}_1^{-1},{\mathrm{a}}_2^{-1},\dots$ and the columns of $\mathrm{I}$ be ${\mathrm{e}}_1,{\mathrm{e}}_2,\dots$. Then $\mathrm{A}{\mathrm{a}}_{\mathrm{i}}^{-1}={\mathrm{e}}_{\mathrm{i}}$ for each $i$. Solving all $n$ systems simultaneously via row reduction on the $n\times 2n$ augmented matrix $[\mathrm{A}\mid \mathrm{I}]$ and continuing until $\mathrm{rref}(\mathrm{A})=\mathrm{I}$ yields ${\mathrm{A}}^{-1}$ on the right-hand side.

Example. Starting from

$$\left(\begin{array}{cccccc}-3& 2& -1& 1& 0& 0\\ 6& -6& 7& 0& 1& 0\\ 3& -4& 4& 0& 0& 1\end{array}\right),$$

row reduction proceeds:

$$\begin{array}{rl}& \to \left(\begin{array}{cccccc}-3& 2& -1& 1& 0& 0\\ 0& -2& 5& 2& 1& 0\\ 0& -2& 3& 1& 0& 1\end{array}\right)\\ & \to \left(\begin{array}{cccccc}-3& 2& -1& 1& 0& 0\\ 0& -2& 5& 2& 1& 0\\ 0& 0& -2& -1& -1& 1\end{array}\right)\\ & \to \left(\begin{array}{cccccc}-3& 0& 4& 3& 1& 0\\ 0& -2& 5& 2& 1& 0\\ 0& 0& -2& -1& -1& 1\end{array}\right)\\ & \to \left(\begin{array}{cccccc}-3& 0& 0& 1& -1& 2\\ 0& -2& 0& -\frac{1}{2}& -\frac{3}{2}& \frac{5}{2}\\ 0& 0& -2& -1& -1& 1\end{array}\right)\\ & \to \left(\begin{array}{cccccc}1& 0& 0& -\frac{1}{3}& \frac{1}{3}& -\frac{2}{3}\\ 0& 1& 0& \frac{1}{4}& \frac{3}{4}& -\frac{5}{4}\\ 0& 0& 1& \frac{1}{2}& \frac{1}{2}& -\frac{1}{2}\end{array}\right).\end{array}$$

Hence,

$${\mathrm{A}}^{-1}=\left(\begin{array}{ccc}-\frac{1}{3}& \frac{1}{3}& -\frac{2}{3}\\ \frac{1}{4}& \frac{3}{4}& -\frac{5}{4}\\ \frac{1}{2}& \frac{1}{2}& -\frac{1}{2}\end{array}\right).$$

${\mathrm{A}}^{-1}$ exists if and only if $\mathrm{rref}(\mathrm{A})=\mathrm{I}$ (equivalently, $\det (\mathrm{A})\ne 0$). If $\mathrm{A}$ is not invertible, the reduction will produce a zero row on the left side before reaching $\mathrm{I}$.

Applications

• Used when the inverse itself is needed explicitly (e.g., symbolic manipulation, deriving normal equations in statistics).
• Confirms invertibility: if reduction fails to reach $\mathrm{I}$ on the left, $\mathrm{A}$ is singular.

Trade-offs

• Computing ${\mathrm{A}}^{-1}$ explicitly costs $O(n^3)$ and requires $O(n^2)$ storage; for solving $\mathrm{Ax}=\mathrm{b}$ it is numerically less stable and no faster than LU factorisation.
• In floating-point arithmetic, the inverse accumulates rounding errors; solving via LU is preferred whenever the inverse is only needed to apply to vectors.
• Only applicable to square, non-singular matrices.

Links

• Reduced Row Echelon Form
• Systems of Linear Equations