Special Matrices

Definition

Several matrix types with restricted structure arise frequently in linear algebra, each with special algebraic properties.

Intuition

Structured matrices trade generality for efficiency and interpretability. Diagonal matrices scale each axis independently; orthogonal matrices rotate (and reflect) without stretching; triangular matrices encode dependency order that can be exploited by forward/back substitution; permutation matrices simply reorder rows or columns. Recognising the structure of a matrix unlocks faster algorithms and stronger theoretical guarantees.

Formal Description

Zero Matrix

Denoted $0$; all elements are zero. Multiplication by a zero matrix yields a zero matrix.

$$0=\left(\begin{array}{cc}0& 0\\ 0& 0\end{array}\right)$$

Identity Matrix

Denoted $\mathbf{I}$; a square matrix with ones on the diagonal and zeros elsewhere. For any conformable square matrix $\mathbf{A}$:

$$\mathbf{AI}=\mathbf{IA}=\mathbf{A}$$

Diagonal Matrix

A (typically square) matrix with nonzero elements only on the main diagonal:

$$\mathbf{D}=\left(\begin{array}{cc}{d}_1& 0\\ 0& d_2\end{array}\right)$$

Banded Matrix

A banded matrix has nonzero elements only on the main diagonal and a finite number of adjacent diagonals. A tridiagonal matrix (bandwidth 1) has nonzero elements on the main diagonal and the diagonals immediately above and below:

$$\mathbf{B}=\left(\begin{array}{ccc}{d}_1& a_1& 0\\ b_1& d_2& a_2\\ 0& b_2& d_3\end{array}\right)$$

Triangular Matrices

A square matrix with all zeros below (upper triangular) or above (lower triangular) the main diagonal:

$$\mathbf{U}=\left(\begin{array}{ccc}a& b& c\\ 0& d& e\\ 0& 0& f\end{array}\right),\mathbf{L}=\left(\begin{array}{ccc}a& 0& 0\\ b& c& 0\\ d& e& f\end{array}\right)$$

Permutation Matrix

A permutation matrix is a row-permuted identity matrix. Left-multiplying by $\mathbf{P}$ permutes the rows of a matrix; right-multiplying permutes columns. For example, the row permutation $\{1,2,3\}\to \{3,1,2\}$ is:

$$\left(\begin{array}{ccc}0& 0& 1\\ 1& 0& 0\\ 0& 1& 0\end{array}\right)\left(\begin{array}{ccc}a& b& c\\ d& e& f\\ g& h& i\end{array}\right)=\left(\begin{array}{ccc}g& h& i\\ a& b& c\\ d& e& f\end{array}\right)$$

A permutation matrix satisfies ${\mathbf{P}}^{-1}={\mathbf{P}}^{\top }$ (it is orthogonal), since it is an orthogonal identity permutation. A $3\times 3$ matrix has $3!=6$ possible row permutations.

Orthogonal Matrix

A square real matrix $\mathbf{Q}$ satisfying ${\mathbf{Q}}^{-1}={\mathbf{Q}}^{\top }$, equivalently:

$$\mathbf{Q}{\mathbf{Q}}^{\top }=\mathbf{I}\text{and}{\mathbf{Q}}^{\top }\mathbf{Q}=\mathbf{I}$$

The columns (and rows) of $\mathbf{Q}$ form an orthonormal set: ${\mathbf{q}}_i^{\top }{\mathbf{q}}_i=1$ and ${\mathbf{q}}_i^{\top }{\mathbf{q}}_j=0$ for $i\ne j$. An orthogonal matrix preserves vector lengths:

$$\Vert \mathbf{Qx}{\Vert }^2=(\mathbf{Qx}{)}^{\top }(\mathbf{Qx})={\mathbf{x}}^{\top }{\mathbf{Q}}^{\top }\mathbf{Q}\mathbf{x}={\mathbf{x}}^{\top }\mathbf{x}=\Vert \mathbf{x}{\Vert }^2$$

Rotation Matrix

A rotation matrix is an orthogonal matrix that rotates vectors without changing their length. The $2\times 2$ matrix rotating a vector counterclockwise by angle $\theta$ in the $x$-$y$ plane is derived from the cosine and sine addition formulae:

$$\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right)=\left(\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)={\mathbf{R}}_{\theta }\left(\begin{array}{c}x\\ y\end{array}\right)$$

• Permutation and rotation matrices are both orthogonal: ${\mathbf{P}}^{-1}={\mathbf{P}}^{\top }$, ${\mathbf{R}}_{\theta }^{-1}={\mathbf{R}}_{\theta }^{\top }={\mathbf{R}}_{-\theta }$.
• Products of orthogonal matrices are orthogonal.
• Triangular matrices arise in LU decomposition; diagonal matrices arise in diagonalisation and SVD.

Applications

• Diagonal matrices enable $O(n)$ inversion and multiplication, used in preconditioning and eigendecomposition.
• Permutation matrices appear in LU decomposition with pivoting ($\mathbf{PA}=\mathbf{LU}$) to improve numerical stability.
• Orthogonal matrices appear in QR decomposition and are numerically stable as they preserve condition numbers.
• Rotation matrices are the building block of rigid-body transformations in robotics and computer graphics.

Trade-offs

• An orthogonal matrix in the real case can represent a rotation or a reflection ($\det \mathbf{Q}=\pm 1$); pure rotations require $\det \mathbf{Q}=+1$.
• Diagonal and triangular structure is generally lost under addition; sums of triangular matrices are triangular, but sums of diagonal matrices may become triangular.
• Banded structure can be destroyed by matrix inversion — the inverse of a banded matrix is typically dense.

Links

• Matrix Operations
• Inverse Matrix
• LU Decomposition