Linear Transformations and Matrices

2.1 Linear Transformations

Let \(V\) and \(W\) be vector spaces over a field \(F\). A function \(T : V \to W\) is called a linear transformation from \(V\) to \(W\) if, for all \(x, y \in V\) and \(c \in F\), the following conditions hold: \[ \text{(a)}\quad T(x + y) = T(x) + T(y) \]

\[ \text{(b)}\quad T(cx) = c\,T(x) \]

  • A transformation \(T : V \to W\) is linear if and only if \[ T(cx + y) = c\,T(x) + T(y) \] for all \(x, y \in V\) and all scalars \(c \in F\).

The identity transformation \(I_V : V \to V\) is defined by \[ I_V(x) = x \quad \text{for all } x \in V. \]

We often write \(I\) instead of \(I_V\).

The zero transformation \(T_0 : V \to W\) is defined by \[ T_0(x) = 0 \quad \text{for all } x \in V. \]

  • It is clear that both of these transformations are linear.

Let \(V\) and \(W\) be vector spaces, and let \(T : V \to W\) be linear. We define the null space (or kernel) of \(T\), denoted \(N(T)\), to be the set of all vectors \(x \in V\) such that \(T(x) = 0\). That is,

\[ N(T) = \{\, x \in V : T(x) = 0 \,\}. \]

We define the range (or image) of \(T\), denoted \(R(T)\), to be the subset of \(W\) consisting of all images \(T(x)\) of vectors \(x \in V\). That is,

\[ R(T) = \{\, T(x) : x \in V \,\}. \]

Theorem 2.1 Let \(V\) and \(W\) be vector spaces and \(T : V \to W\) be linear. Then \(N(T)\) and \(R(T)\) are subspaces of \(V\) and \(W\), respectively.

Let \(V\) and \(W\) be vector spaces, and let \(T : V \to W\) be linear. If \(N(T)\) and \(R(T)\) are finite-dimensional, then we define the nullity of \(T\), denoted \(\operatorname{nullity}(T)\), and the rank of \(T\), denoted \(\operatorname{rank}(T)\), to be the dimensions of \(N(T)\) and \(R(T)\), respectively.

Theorem 2.2 Let \(V\) and \(W\) be vector spaces, and let \(T : V \to W\) be linear. If \(\beta = \{v_1, v_2, \ldots, v_n\}\) is a basis for \(V\), then \[ R(T) = \operatorname{span}(T(\beta)) = \operatorname{span}(\{T(v_1), T(v_2), \ldots, T(v_n)\}). \]

Theorem 2.3 (Dimension Theorem) Let \(V\) and \(W\) be vector spaces, and let \(T : V \to W\) be linear. If \(V\) is finite-dimensional, then \[ \operatorname{nullity}(T) + \operatorname{rank}(T) = \dim(V). \]

Theorem 2.4 Let \(V\) and \(W\) be vector spaces, and let \(T: V \to W\) be linear. Then \(T\) is one-to-one if and only if \(N(T) = \{0\}\).

Theorem 2.5 Let \(V\) and \(W\) be vector spaces of equal finite dimension, and let \(T: V \to W\) be linear. Then the following are equivalent:

\(\text{(a)}\quad T\) is one-to-one.

\(\text{(b)}\quad T\) is onto.

\(\text{(c)}\quad \operatorname{rank}(T) = \dim(V)\).

Theorem 2.6 Let \(V\) and \(W\) be vector spaces over \(F\), and suppose that \(\{v_1, v_2, \ldots, v_n\}\) is a basis for \(V\). For \(w_1, w_2, \ldots, w_n\) in \(W\), there exists exactly one linear transformation \(T: V \to W\) such that \(T(v_i) = w_i\) for \(i = 1, 2, \ldots, n\).

Corollary 2.1 Let \(V\) and \(W\) be vector spaces, and suppose that \(V\) has a finite basis \(\{v_1, v_2, \ldots, v_n\}\). If \(U, T : V \to W\) are linear and \(U(v_i) = T(v_i)\) for \(i = 1, 2, \ldots, n\), then \(U = T\).

2.2 The Matrix Representation of a Linear Transformation

Let \(V\) be a finite-dimensional vector space. An ordered basis for \(V\) is a basis for \(V\) endowed with a specific order; that is, an ordered basis for \(V\) is a finite sequence of linearly independent vectors in \(V\) that generates \(V\).

For the vector space \(\mathsf{F}^n\), we call \(\{e_1, e_2, \dots, e_n\}\) the standard ordered basis for \(\mathsf{F}^n\). Similarly, for the vector space \(\mathsf{P}_n(F)\), we call \(\{1, x, \dots, x^n\}\) the standard ordered basis for \(\mathsf{P}_n(F)\).

Let \(\beta = \{u_1, u_2, \dots, u_n\}\) be an ordered basis for a finite-dimensional vector space \(V\). For \(x \in V\), let \(a_1, a_2, \dots, a_n\) be the unique scalars such that \[ x = \sum_{i=1}^{n} a_i u_i. \]

We define the coordinate vector of \(x\) relative to \(\beta\), denoted \([x]_\beta\), by \[ [x]_\beta = \begin{pmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \end{pmatrix}. \]

Suppose that \(V\) and \(W\) are finite-dimensional vector spaces with ordered bases \(\beta = \{v_1, v_2, \dots, v_n\}\) and \(\gamma = \{w_1, w_2, \dots, w_m\}\), respectively. Let \(T : V \to W\) be linear. Then for each \(j\), \(1 \le j \le n\), there exist unique scalars \(a_{ij} \in F\), \(1 \le i \le m\), such that \[ T(v_j) = \sum_{i=1}^{m} a_{ij} w_i \qquad \text{for } 1 \le j \le n. \]

We call the \(m \times n\) matrix \(A\) defined by \(A_{ij} = a_{ij}\) the matrix representation of \(T\) in the ordered bases \(\beta\) and \(\gamma\) and write \[ A = [T]_{\beta}^{\gamma}. \] - If \(V = W\) and \(\beta = \gamma\), then we write simply

\[ A = [T]_{\beta}. \] - Notice that the \(j\) th column of \(A\) is simply \([\,T(v_j)\,]_{\gamma}\). Also observe that if \(U : V \to W\) is a linear transformation such that \([U]_{\beta}^{\gamma} = [T]_{\beta}^{\gamma}\), then \(U = T\).

Let \(T, U : V \to W\) be arbitrary functions, where \(V\) and \(W\) are vector spaces over \(F\), and let \(a \in F\). We define \(T + U : V \to W\) by \[ (T + U)(x) = T(x) + U(x) \qquad \text{for all } x \in V, \]

and \(aT : V \to W\) by

\[ (aT)(x) = a\,T(x) \qquad \text{for all } x \in V. \]

  • Let \(V\) and \(W\) be vector spaces over \(F\). We denote the vector space of all linear transformations from \(V\) into \(W\) by \(\mathcal{L}(V, W)\). In the case that \(V = W\), we write \(\mathcal{L}(V)\) instead of \(\mathcal{L}(V, W)\).

Theorem 2.7 Let \(V\) and \(W\) be finite-dimensional vector spaces with ordered bases \(\beta\) and \(\gamma\), respectively, and let \(T, U : V \to W\) be linear transformations. Then \[ \text{(a)}\quad [T + U]_{\beta}^{\gamma} = [T]_{\beta}^{\gamma} + [U]_{\beta}^{\gamma} \]

\[ \text{(b)}\quad [aT]_{\beta}^{\gamma} = a [T]_{\beta}^{\gamma} \qquad \text{for all scalars } a. \]

2.3 Composition of Linear Transformations and Matrix Multiplication

Theorem 2.8 Let \(V\), \(W\), and \(Z\) be vector spaces over the same field \(F\), and let \(T : V \to W\) and \(U : W \to Z\) be linear. Then \(UT : V \to Z\) is linear.

Let \(T : V \to W\) and \(U : W \to Z\) be linear transformations, and let \(A = [U]_{\beta}^{\gamma}\) and \(B = [T]_{\alpha}^{\beta}\), where \(\alpha = \{v_1, v_2, \dots, v_n\}\), \(\beta = \{w_1, w_2, \dots, w_m\}\), and \(\gamma = \{z_1, z_2, \dots, z_p\}\) are ordered bases for \(V\), \(W\), and \(Z\), respectively.

We would like to define the product \(AB\) of two matrices so that \[ AB = [UT]_{\alpha}^{\gamma}. \]

Consider the matrix \([UT]_{\alpha}^{\gamma}\). For \(1 \le j \le n\), we have

\[ (UT)(v_j) = U(T(v_j)) = U \!\left( \sum_{k=1}^{m} B_{kj} w_k \right) = \sum_{k=1}^{m} B_{kj} \, U(w_k) \]

\[ = \sum_{k=1}^{m} B_{kj} \left( \sum_{i=1}^{p} A_{ik} z_i \right) = \sum_{i=1}^{p} \left( \sum_{k=1}^{m} A_{ik} B_{kj} \right) z_i \]

\[ = \sum_{i=1}^{p} C_{ij} \, z_i, \]

where

\[ C_{ij} = \sum_{k=1}^{m} A_{ik} B_{kj}. \]

This computation motivates the following definition of matrix multiplication.

We define the Kronecker delta \(\delta_{ij}\) by \(\delta_{ij} = 1\) if \(i = j\) and \(\delta_{ij} = 0\) if \(i \ne j\). The \(n \times n\) identity matrix \(I_n\) is defined by \((I_n)_{ij} = \delta_{ij}\).

  • Thus, for example, \[ I_1 = (1), \qquad I_2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \qquad \text{and} \qquad I_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}. \]

Theorem 2.9 Let \(A\) be an \(m \times n\) matrix, \(B\) and \(C\) be \(n \times p\) matrices, and \(D\) and \(E\) be \(q \times m\) matrices. Then

\(\text{(a)}\quad A(B + C) = AB + AC\) and \((D + E)A = DA + EA\).

\(\text{(b)}\quad a(AB) = (aA)B = A(aB)\) for any scalar \(a\).

\(\text{(c)}\quad I_m A = A = A I_n\).

\(\text{(d)}\quad\)If \(V\) is an \(n\)-dimensional vector space with an ordered basis \(\beta\), then \([I_V]_{\beta} = I_n\).

Theorem 2.10 Let \(V\) and \(W\) be finite-dimensional vector spaces having ordered bases \(\beta\) and \(\gamma\), respectively, and let \(T : V \to W\) be linear. Then, for each \(u \in V\), we have \[ [T(u)]_{\gamma} = [T]_{\beta}^{\gamma} [u]_{\beta}. \]

Let \(A\) be an \(m \times n\) matrix with entries from a field \(F\). We denote by \(L_A\) the mapping \(L_A : F^n \to F^m\) defined by \(L_A(x) = Ax\) for each column vector \(x \in F^n\). We call \(L_A\) a left-multiplication transformation.

Theorem 2.11 Let \(A\) be an \(m \times n\) matrix with entries from \(F\). Then the left-multiplication transformation \(L_A : F^n \to F^m\) is linear. Furthermore, if \(B\) is any other \(m \times n\) matrix (with entries from \(F\)) and \(\beta\) and \(\gamma\) are the standard ordered bases for \(F^n\) and \(F^m\), respectively, then we have the following properties.

\(\text{(a)}\quad [L_A]_{\beta}^{\gamma} = A\).

\(\text{(b)}\quad L_A = L_B\) if and only if \(A = B\).

\(\text{(c)}\quad L_{A+B} = L_A + L_B\) and \(L_{aA} = a L_A\) for all \(a \in F\).

\(\text{(d)}\quad\)If \(T : F^n \to F^m\) is linear, then there exists a unique \(m \times n\) matrix \(C\) such that \(T = L_C\). In fact, \(C = [T]_{\beta}^{\gamma}\).

\(\text{(e)}\quad\)If \(E\) is an \(n \times p\) matrix, then \(L_{AE} = L_A L_E\).

\(\text{(f)}\quad\)If \(m = n\), then \(L_{I_n} = I_{F^n}\).

2.4 Invertibility and Isomorphisms

Let \(V\) and \(W\) be vector spaces, and let \(T : V \to W\) be linear. A function \(U : W \to V\) is said to be an inverse of \(T\) if \(TU = I_W\) and \(UT = I_V\). If \(T\) has an inverse, then \(T\) is said to be invertible.

  • We often use the fact that a function is invertible if and only if it is both one-to-one and onto.

Theorem 2.12 Let \(V\) and \(W\) be vector spaces, and let \(T : V \to W\) be linear and invertible. Then \(T^{-1} : W \to V\) is linear.

Theorem 2.13 Let \(T\) be an invertible linear transformation from \(V\) to \(W\). Then \(V\) is finite-dimensional if and only if \(W\) is finite-dimensional. In this case, \(\dim(V) = \dim(W)\).

Let \(A\) be an \(n \times n\) matrix. Then \(A\) is invertible if there exists an \(n \times n\) matrix \(B\) such that \(AB = BA = I\). If \(A\) is invertible, then the matrix \(B\) such that \(AB = BA = I\) is unique.

Theorem 2.14 Let \(V\) and \(W\) be finite-dimensional vector spaces with ordered bases \(\beta\) and \(\gamma\), respectively. Let \(T : V \to W\) be linear. Then \(T\) is invertible if and only if \([T]_{\beta}^{\gamma}\) is invertible. Furthermore,\([T^{-1}]_{\gamma}^{\beta} = ([T]_{\beta}^{\gamma})^{-1}\).

Let \(V\) and \(W\) be vector spaces. We say that \(V\) is isomorphic to \(W\) if there exists a linear transformation \(T : V \to W\) that is invertible. Such a linear transformation is called an isomorphism from \(V\) onto \(W\).

Theorem 2.15 Let \(V\) and \(W\) be finite-dimensional vector spaces (over the same field). Then \(V\) is isomorphic to \(W\) if and only if \(\dim(V) = \dim(W)\).

Theorem 2.16 Let \(V\) and \(W\) be finite-dimensional vector spaces of dimensions \(n\) and \(m\), respectively. Then \(\mathcal{L}(V, W)\) is finite-dimensional of dimension \(mn\).

2.5 The Change of Coordinate Matrix

Let \(\beta\) and \(\beta'\) be two ordered bases for a finite-dimensional vector space \(V\), and let \(Q = [I_V]_{\beta'}^{\beta}\). Then

\(\text{(a)} \quad Q\) is invertible.

\(\text{(b)}\quad\)For any \(v \in V\), \([v]_{\beta} = Q [v]_{\beta'}\).

The matrix \(Q = [I_V]_{\beta'}^{\beta}\) is called a change of coordinate matrix.

Theorem 2.17 Let \(T\) be a linear operator on a finite-dimensional vector space \(V\), and let \(\beta\) and \(\beta'\) be ordered bases for \(V\). Suppose that \(Q\) is the change of coordinate matrix that changes \(\beta'\)-coordinates into \(\beta\)-coordinates. Then \[ [T]_{\beta'} = Q^{-1} [T]_{\beta} Q. \]

Let \(A\) and \(B\) be matrices in \(M_{n \times n}(F)\). We say that \(B\) is similar to \(A\) if there exists an invertible matrix \(Q\) such that \[ B = Q^{-1} A Q. \]

2.6 Dual Spaces

For a vector space \(V\) over \(F\), we define the dual space of \(V\) to be the vector space \(\mathcal{L}(V, F)\), denoted by \(V^{*}\).

  • \(\dim(V^{*}) = \dim(V).\)

Suppose that \(V\) is a finite-dimensional vector space with the ordered basis \(\beta = \{x_1, x_2, \dots, x_n\}\). we call the ordered basis \(\beta^{*} = \{f_1, f_2, \dots, f_n\}\) of \(V^{*}\) that satisfies \(f_i(x_j) = \delta_{ij}\) (\(1 \le i, j \le n\)) the dual basis of \(\beta\).

Theorem 2.18 For any \(f \in V^{*}\), we have \[ f = \sum_{i=1}^{n} f(x_i)\, f_i. \]

Let \(V\) and \(W\) be finite-dimensional vector spaces over \(F\) with ordered bases \(\beta\) and \(\gamma\), respectively. For any linear transformation \(T : V \to W\), the mapping \(T^{t} : W^{*} \to V^{*}\) defined by \(T^{t}(g) = gT\) for all \(g \in W^{*}\) is a linear transformation with the property that

\[ [T^{t}]_{\gamma^{*}}^{\beta^{*}} = ([T]_{\beta}^{\gamma})^{t}. \]

The linear transformation \(T^t\) is called the transpose of \(T\).

We define the double dual \(V^{**}\) of \(V\) to be the dual of \(V^{*}\).

For a vector \(x \in V\), we define \(\widehat{x} : V^{*} \to F\) by \(\widehat{x}(f) = f(x)\) for every \(f \in V^{*}\). It is easy to verify that \(\widehat{x}\) is a linear functional on \(V^{*}\), so \(\widehat{x} \in V^{**}\).

Lemma 2.1 Let \(V\) be a finite-dimensional vector space, and let \(x \in V\). If \(\widehat{x}(f) = 0\) for all \(f \in V^{*}\), then \(x = 0\).

Theorem 2.19 Let \(V\) be a finite-dimensional vector space, and define \(\psi : V \to V^{**}\) by \(\psi(x) = \widehat{x}\). Then \(\psi\) is an isomorphism.

Theorem 2.20 Let \(V\) be a finite-dimensional vector space with dual space \(V^{*}\). Then every ordered basis for \(V^{*}\) is the dual basis for some basis for \(V\).