2 Matrices and Matrix Operations

Reading and representing matrices

A matrix is a rectangular array organized by rows and columns. If it has $m$ rows and $n$ columns, its size is $m\times n$, and an entry is written $a_{ij}$, where $i$ is the row number and $j$ is the column number.

For example,

has size $2\times3$, and its entry $a_{23}$ is $5$. A square matrix has the same number of rows and columns. A row matrix has one row, while a column matrix has one column.

The identity matrix of size $n\times n$ is written $I_n$. Its main diagonal contains ones and all other entries are zero. The zero matrix, written $0$, contains zeros everywhere.

Matrices provide a compact form for linear systems. The equations

can be written as

The associated augmented matrix is

Takeaway: Read matrix indices as row first, column second, and use dimensions to determine which operations are possible.

Addition, equality, and scalar multiplication

Addition and subtraction are defined only for matrices with the same dimensions. The operations are performed entry by entry:

For example,

A scalar multiple multiplies every entry by the same scalar. Thus,

The zero matrix is the additive identity because $A+0=A$. Matrix equality also depends on both dimensions and corresponding entries: two matrices are equal exactly when they have the same dimensions and every corresponding entry is equal.

These operations should not be confused with Matrix multiplication. Entrywise addition is possible only for matching dimensions, whereas multiplication uses a row-by-column rule and has a different dimension condition.

Takeaway: Before adding or subtracting, verify equal dimensions; for scalar multiplication, apply the scalar to every entry.

Computing matrix products

For matrices $A$ of size $m\times n$ and $B$ of size $n\times p$, the product $AB$ is defined and has size $m\times p$. The inner dimensions must match.

Each entry is obtained by taking the dot product of a row of $A$ with a column of $B$:

For

we get

Matrix multiplication is generally not commutative: $AB\ne BA$, and $BA$ may not even be defined. It is associative and distributive when all products involved are defined:

The identity matrix acts as a multiplicative identity whenever the dimensions are compatible:

Multiplying a matrix by a column vector can also be viewed as forming a linear combination of the matrix's columns. If the columns are $\mathbf{a}_1,\ldots,\mathbf{a}_n$, then

Takeaway: Check compatibility first, then compute each result entry using one row of the first matrix and one column of the second.

Transposes and structural identities

The transpose $A^T$ exchanges rows and columns. An $m\times n$ matrix becomes an $n\times m$ matrix, with entries satisfying

For example,

Important identities are

For a product, the order reverses:

The reversal is required to preserve dimension compatibility. A square matrix satisfying $A^T=A$ is symmetric. For complex matrices, the analogous conjugate transpose both transposes the matrix and takes the complex conjugate of each entry.

Takeaway: Transpose by reflecting entries across the main diagonal, and reverse factor order when transposing a product.

Inverses and solving with matrices

An invertible matrix is a square matrix with an inverse. The inverse $A^{-1}$ satisfies

If $A\mathbf{x}=\mathbf{b}$ and $A$ is invertible, multiplying by the inverse gives

For

an inverse exists when $ad-bc\ne0$, and then

The quantity $ad-bc$ is the determinant for this $2\times2$ matrix. If it is zero, the matrix is singular and has no inverse.

To find an inverse by Gauss–Jordan elimination, augment the matrix with an identity matrix and row-reduce:

For example,

Therefore,

Useful identities include

The order reversal in $(AB)^{-1}$ is essential. Although the inverse gives a theoretical formula for solving systems, numerical computation usually favors a direct solve rather than explicitly forming the inverse.

Takeaway: An inverse exists only for an invertible square matrix; use row reduction or an appropriate factorization to compute or apply it.

Row operations and elimination

An elementary matrix results from applying one elementary row operation to an identity matrix. The three operations are:

1. Interchange two rows: $R_i\leftrightarrow R_j$.

2. Multiply a row by a nonzero scalar: $R_i\to cR_i$, where $c\ne0$.

3. Add a multiple of one row to another: $R_i\to R_i+cR_j$, where $i\ne j$.

If $E$ represents one of these operations, then left multiplication applies it to another matrix:

For example, adding three times row one to row two in a $3\times3$ matrix uses

Every elementary matrix is invertible, and its inverse reverses the associated row operation. Thus, a sequence of row operations can be represented as

This explains why row reduction preserves the solution set of a linear system: each elementary row operation produces an equivalent system.

Gaussian elimination transforms an augmented matrix $[A\mid\mathbf{b}]$ into row-echelon form, after which back-substitution finds the solution. Gauss–Jordan elimination continues to reduced row-echelon form. For a unique solution, the final augmented matrix has the form

For example,

The second row gives $y=2$, and substituting into the first gives $x=1$.

Takeaway: Elementary matrices encode row operations, while Gaussian elimination uses those operations to solve systems without changing their solutions.

Reliable numerical computation

Exact arithmetic permits any nonzero pivot, but floating-point arithmetic can magnify rounding errors. Partial pivoting reduces this risk by interchanging the current row with a lower row whose entry in the pivot column has the largest absolute value. This also helps avoid division by a very small pivot.

A matrix can be mathematically invertible but numerically ill-conditioned. In that situation, small errors in the matrix entries or right-hand side can produce relatively large changes in the computed solution. The condition number measures one aspect of this sensitivity.

When several systems share the same coefficient matrix, LU factorization is efficient. Write

where $P$ is a permutation matrix, $L$ is lower triangular, and $U$ is upper triangular. To solve $A\mathbf{x}=\mathbf{b}$, solve two triangular systems:

The factorization requires an initial setup, but each additional right-hand side can then be handled efficiently.

For numerical work, solve a system directly rather than computing $A^{-1}\mathbf{b}$ explicitly whenever possible. Direct solvers generally avoid unnecessary work and reduce numerical error. In NumPy, numpy.linalg.solve(A, b) is the standard choice for a square system, .T represents a transpose, and @ performs matrix multiplication.

Common checks:

• Do not add or subtract matrices with different dimensions.

• Do not multiply entries position by position when matrix multiplication is required.

• Do not assume $AB=BA$.

• Do not replace $(AB)^T$ with $A^TB^T$.

• Do not invert a nonsquare or singular matrix.

• Do not divide by a zero or extremely small pivot without considering a row interchange.

Takeaway: Choose algorithms based on numerical reliability: pivot carefully, reuse LU factors for repeated systems, and prefer direct solves to explicit inverses.

• size of a matrix