1 Limits and Continuity of Multivariate Functions

1.1 Examples of Multivariate Functions

Multivariate Function

An $m$-variable function is a mapping $f : D \to \mathbb{R}$, where $D$ is $\mathbb{R}^m$ or a subset of it, and the function value $y$ and independent variables $x_1, x_2, \dots, x_m$ satisfy $y = f(x_1, x_2, \dots, x_m)$.

Multivariate Mapping

The values of multiple variables $y_1, \dots, y_n$ are determined by a set of independent variables $x_1, \dots, x_m$:

$$\begin{equation*} \begin{pmatrix} y_1 \\ \vdots \\ y_n \end{pmatrix} = \begin{pmatrix} f_1(x_1, \dots, x_m) \\ \vdots \\ f_n(x_1, \dots, x_m) \end{pmatrix}, \quad \text{i.e.,} \quad \begin{cases} y_1 = f_1(x_1, \dots, x_m), \\ \vdots \\ y_n = f_n(x_1, \dots, x_m). \end{cases} \end{equation*}$$

1.2 Distance and Limits of Sequences in $\mathbb{R}^m$

Distance on $\mathbb{R}^m$

A bivariate function $d : \mathbb{R}^m \times \mathbb{R}^m \to \mathbb{R}$ is called a distance on $\mathbb{R}^m$ if it satisfies:

1. For any $\mathbf{x}, \mathbf{y} \in \mathbb{R}^m$, $d(\mathbf{x}, \mathbf{y}) = d(\mathbf{y}, \mathbf{x}) \ge 0$;

2. For any $\mathbf{x}, \mathbf{y} \in \mathbb{R}^m$, $d(\mathbf{x}, \mathbf{y}) = 0$ if and only if $\mathbf{x} = \mathbf{y}$;

3. For any $\mathbf{x}, \mathbf{y}, \mathbf{z} \in \mathbb{R}^m$, $d(\mathbf{x}, \mathbf{z}) \le d(\mathbf{x}, \mathbf{y}) + d(\mathbf{y}, \mathbf{z})$.

• For $r > 0$, denote

  $$\begin{equation*} B(\mathbf{x}, r) := \{\mathbf{y} \in \mathbb{R}^m \mid d(\mathbf{x}, \mathbf{y}) < r\}, \end{equation*}$$

  which is called the open ball centered at $\mathbf{x}$ with radius $r$.

• A distance $d$ on $\mathbb{R}^m$ is said to be translation-invariant if for any $\mathbf{x}, \mathbf{y}, \mathbf{z} \in \mathbb{R}^m$,

$$\begin{equation*} d(\mathbf{x} + \mathbf{z}, \mathbf{y} + \mathbf{z}) = d(\mathbf{x}, \mathbf{y}). \end{equation*}$$

Norm on $\mathbb{R}^m$

A function $\|\cdot\| : \mathbb{R}^m \to \mathbb{R}$ is called a norm on $\mathbb{R}^m$ if it satisfies:

1. For any $\mathbf{x}, \mathbf{y} \in \mathbb{R}^m$, $\|\mathbf{x} + \mathbf{y}\| \le \|\mathbf{x}\| + \|\mathbf{y}\|$;
2. For any $\mathbf{x} \in \mathbb{R}^m$, $\|\mathbf{x}\| \ge 0$; $\|\mathbf{x}\| = 0$ if and only if $\mathbf{x} = \mathbf{0}$;
3. For any $\mathbf{x} \in \mathbb{R}^m$ and any $\lambda \in \mathbb{R}$, $\|\lambda \mathbf{x}\| = |\lambda| \|\mathbf{x}\|$.

Matrix Norm

Given a norm $\|\cdot\|$ in $\mathbb{R}^n$ and a square matrix $A$ of order $n$, the matrix norm of $A$ is defined as:

$$\begin{align*} \|A\| = \max_{\|\mathbf{v}\|=1} \|A\mathbf{v}\|. \end{align*}$$

Also

$$\begin{align*} \|A\| = \max_{\mathbf{v}\neq 0} \frac{\|A\mathbf{v}\|}{\|\mathbf{v}\|}. \end{align*}$$

Theorem 1.1.

1. For any matrix $A$, the norm is always non-negative, and it is zero if and only if the matrix itself is the zero matrix.

$$\begin{align*} \|A\| \geq 0, \quad \text{and} \quad \|A\| = 0 \iff A = \mathbf{0} \end{align*}$$

2. Multiplying a matrix by a scalar $\alpha$ scales the norm by the absolute value of that scalar.

$$\begin{align*} \|\alpha A\| = |\alpha| \cdot \|A\| \end{align*}$$

3. The norm of the sum of two matrices is less than or equal to the sum of their individual norms.

$$\begin{align*} \|A + B\| \leq \|A\| + \|B\| \end{align*}$$

4. For induced norms, the norm of a product of matrices is bounded by the product of their norms.

$$\begin{align*} \|AB\| \leq \|A\| \cdot \|B\| \end{align*}$$

5. By the very definition of the induced norm, the transformation of any vector $\mathbf{v}$ by matrix $A$ is bounded by the matrix norm:

$$\begin{align*} \|A\mathbf{v}\| \leq \|A\| \cdot \|\mathbf{v}\| \end{align*}$$

Inner Product on $\mathbb{R}^m$

A bivariate function $\langle \cdot, \cdot \rangle : \mathbb{R}^m \times \mathbb{R}^m \to \mathbb{R}$ is called an inner product on $\mathbb{R}^m$ if it satisfies:

1. For any $\mathbf{x}, \mathbf{y} \in \mathbb{R}^m$, $\langle \mathbf{x}, \mathbf{y} \rangle = \langle \mathbf{y}, \mathbf{x} \rangle$;
2. For any $\mathbf{x}, \mathbf{y}, \mathbf{z} \in \mathbb{R}^m$ and any $\alpha, \beta \in \mathbb{R}$, $$\begin{equation*} \langle \alpha \mathbf{x} + \beta \mathbf{y}, \mathbf{z} \rangle = \alpha \langle \mathbf{x}, \mathbf{z} \rangle + \beta \langle \mathbf{y}, \mathbf{z} \rangle; \end{equation*}$$
3. For any $\mathbf{x} \in \mathbb{R}^m$, $\langle \mathbf{x}, \mathbf{x} \rangle \ge 0$; and $\langle \mathbf{x}, \mathbf{x} \rangle = 0$ if and only if $\mathbf{x} = \mathbf{0}$.

Definition. For any $p \ge 1$,

$$\begin{equation*} \|\mathbf{x}\|_p := (|x^1|^p + \dots + |x^m|^p)^{\frac{1}{p}} \end{equation*}$$

defines a norm on $\mathbb{R}^m$ ,which induces a distance on $\mathbb{R}^m$:

$$\begin{equation*} d_p(\mathbf{x}, \mathbf{y}) := (|x^1 - y^1|^p + \dots + |x^m - y^m|^p)^{\frac{1}{p}}. \end{equation*}$$

Theorem 1.2.

1. $\|\mathbf{x}\|_\infty := \max\{|x^1|, \dots, |x^m|\}$.

2. $\|\mathbf{x}\|_\infty \le \|\mathbf{x}\|_p \le m^{\frac{1}{p}} \|\mathbf{x}\|_\infty$.

3. $\frac{1}{m^{\frac{1}{q}}} \|\mathbf{x}\|_q \le \|\mathbf{x}\|_p \le m^{\frac{1}{p}} \|\mathbf{x}\|_q$.

Corollary 1. Any norm $\|\cdot\|$ on $\mathbb{R}^m$ is equivalent to the norm $\|\cdot\|_\infty$,

Limits of Sequences in $\mathbb{R}^m$

Let $\{\mathbf{x}_n\}_{n \ge 1}$ be a sequence in $\mathbb{R}^m$.

1. The sequence $\{\mathbf{x}_n\}_{n \ge 1}$ is said to be bounded if there exists $M > 0$ such that for any positive integer $n$, $\|\mathbf{x}_n\| \le M$.

2. The sequence $\{\mathbf{x}_n\}_{n \ge 1}$ is said to be convergent if there exists $A \in \mathbb{R}^m$ such that for any $\varepsilon > 0$, there exists a positive integer $N(\varepsilon) > 0$ such that for any positive integer $n$, whenever $n \ge N(\varepsilon)$, we have $\|\mathbf{x}_n - A\| < \varepsilon$. Such an $A$ is called a limit of the sequence $\{\mathbf{x}_n\}_{n \ge 1}$. We denote $A = \lim\limits_{n \to +\infty} \mathbf{x}_n$.

3. The sequence $\{\mathbf{x}_n\}_{n \ge 1}$ is said to be a Cauchy sequence if for any $\varepsilon > 0$, there exists a positive integer $N(\varepsilon) > 0$ such that for any positive integers $p, q$, whenever $p, q \ge N(\varepsilon)$, we have $\|\mathbf{x}_p - \mathbf{x}_q\| < \varepsilon$.

Theorem 1.3. Let $\{\mathbf{x}_n\}$ be a sequence in $\mathbb{R}^m$, where $\mathbf{x}_n = (x_n^1, \dots, x_n^m)^T$. Then:

1. $\{\mathbf{x}_n\}_{n \ge 1}$ is a bounded sequence if and only if for any $1 \le k \le m$, the sequence $\{x_n^k\}_{n \ge 1}$ is a bounded sequence.
2. $\{\mathbf{x}_n\}_{n \ge 1}$ is a Cauchy sequence if and only if for any $1 \le k \le m$, the sequence $\{x_n^k\}_{n \ge 1}$ is a Cauchy sequence.
3. $\lim\limits_{n \to +\infty} \mathbf{x}_n = A$ if and only if for any $1 \le k \le m$, $\lim\limits_{n \to +\infty} x_n^k = A_k$.

Theorem 1.4.

1. A sequence in $\mathbb{R}^m$ is convergent if and only if it is a Cauchy sequence.
2. Any bounded sequence in $\mathbb{R}^m$ has a convergent subsequence.

Closed Set

A set $E \subseteq \mathbb{R}^m$ is said to be a closed set if: for any sequence $\{\mathbf{x}_n\}_{n \ge 1}$ in $E$, if the sequence has a limit $\lim\limits_{n \to +\infty} \mathbf{x}_n$, then this limit is also in $E$.

1.3 Continuity of Multivariable Functions

Continuity of Multivariable Function

Let $E \subseteq \mathbb{R}^m$. A mapping $f: E \to \mathbb{R}^p$ is said to be continuous at $\mathbf{x}_0 \in E$, if for any $\epsilon > 0$, there exists a $\delta > 0$ such that for any $\mathbf{x} \in E$, as long as

$$\begin{align*} \|\mathbf{x} - \mathbf{x}_0\| < \delta \end{align*}$$

the following inequality holds:

$$\begin{align*} \|f(\mathbf{x}) - f(\mathbf{x}_0)\|_* < \epsilon \end{align*}$$

Here, $\|\cdot\|$ and $\|\cdot\|_*$ denote the norms in the domain space and the codomain space of $f$, respectively.

• $f$ is said to be continuous on $E_0 \subseteq E$, if $f$ is continuous at every point $\mathbf{x}_0 \in E_0$.
• $f$ is called a continuous mapping, if it is continuous on its entire domain $E$.

Theorem 1.5. Any linear mapping on $\mathbb{R}^m$ is continuous.

Theorem 1.6. Any bilinear mapping $B: \mathbb{R}^m \times \mathbb{R}^n \to \mathbb{R}^p$, i.e., $\forall \alpha_1, \alpha_2, \beta_1, \beta_2 \in \mathbb{R}, \mathbf{x}_1, \mathbf{x}_2, \mathbf{x} \in \mathbb{R}^m, \mathbf{y}_1, \mathbf{y}_2, \mathbf{y} \in \mathbb{R}^n$

$$\begin{align*} B(\alpha_1 \mathbf{x}_1 + \alpha_2 \mathbf{x}_2, \mathbf{y}) &= \alpha_1 B(\mathbf{x}_1, \mathbf{y}) + \alpha_2 B(\mathbf{x}_2, \mathbf{y}), \\ B(\mathbf{x}, \beta_1 \mathbf{y}_1 + \beta_2 \mathbf{y}_2) &= \beta_1 B(\mathbf{x}, \mathbf{y}_1) + \beta_2 B(\mathbf{x}, \mathbf{y}_2), \end{align*}$$

is continuous.

Corollary 1. Any multilinear mapping is a continuous mapping.

Theorem 1.7. Vector addition, scalar multiplication, and the inner product in $\mathbb{R}^m$, as well as real number multiplication, are all continuous mappings.

Theorem 1.8. Let $E \subseteq \mathbb{R}^m$, $F \subseteq \mathbb{R}^p$, and $f: E \to \mathbb{R}^p$ be continuous at $\mathbf{x}_0 \in E$. If $g: F \to \mathbb{R}^q$ is continuous at $\mathbf{y}_0 = f(\mathbf{x}_0) \in F$, then the composite mapping

$$\begin{align*} g \circ f: E \cap f^{-1}(F) \to \mathbb{R}^q \end{align*}$$

is continuous at $\mathbf{x}_0$.

Theorem 1.9. The determinant is a continuous function of the matrix; therefore, small perturbations of an invertible matrix remain invertible.

Theorem 1.10. Let $E \subseteq \mathbb{R}^m$ and $f: E \to \mathbb{R}^p$ be a continuous mapping. Then:

1. For any bounded closed subset $K$ of $E$, the image $f(K)$ is also a bounded closed set.

2. If $p = 1$, then for any non-empty bounded closed subset $K$ of $E$, $f$ attains both a maximum value and a minimum value on $K$.

Theorem 1.11. Let $\langle \cdot, \cdot \rangle$ be an inner product on $\mathbb{R}^n$, and let $A$ be a symmetric matrix of order $n$, i.e.,

$$\begin{align*} \langle A\mathbf{x}, \mathbf{y} \rangle = \langle \mathbf{x}, A\mathbf{y} \rangle, \quad \forall \mathbf{x}, \mathbf{y} \in \mathbb{R}^n. \end{align*}$$

There exist $n$ orthonormal unit vectors $\mathbf{v}_1, \dots, \mathbf{v}_n$ in $\mathbb{R}^n$ and real numbers $\lambda_1, \dots, \lambda_n$ such that

$$\begin{align*} A\mathbf{v}_k = \lambda_k \mathbf{v}_k, \quad k = 1, \dots, n. \end{align*}$$

Theorem 1.12. Let $E \subseteq \mathbb{R}^m$ and $f: E \to \mathbb{R}^p$ be a continuous mapping. For any bounded closed subset $K$ of $E$, $f$ is uniformly continuous on $K$.

This means that for any $\epsilon > 0$, there exists a $\delta > 0$ such that for any $\mathbf{x} \in K$ and any $\mathbf{y} \in E$, as long as

$$\begin{align*} \|\mathbf{x} - \mathbf{y}\| < \delta \end{align*}$$

the following holds:

$$\begin{align*} \|f(\mathbf{x}) - f(\mathbf{y})\|_* < \epsilon \end{align*}$$

Here, $\|\cdot\|$ and $\|\cdot\|_*$ denote the norms in the domain space and the codomain space of $f$, respectively.

Path

A set $E \subseteq \mathbb{R}^m$ is called a path-connected set if, for any $\mathbf{x}, \mathbf{y} \in E$, there exists a continuous mapping $f : [0, 1] \to \mathbb{R}^m$ such that

$$\begin{align*} f(0) = \mathbf{x}, \quad f(1) = \mathbf{y}, \quad \text{and } f(t) \in E \text{ for all } t \in [0, 1]. \end{align*}$$

Such a continuous mapping $f$ is called a path from $\mathbf{x}$ to $\mathbf{y}$ in $E$.

1.4 Limits of Multivariable Functions

Limits of Multivariable Function

Let $E \subseteq \mathbb{R}^m$. A point $\mathbf{x}_0 \in \mathbb{R}^m$ is called a cluster point (or accumulation point) of $E$ if, for any $\epsilon > 0$, there exists $\mathbf{x} \in E$ such that

$$\begin{align*} 0 < d(\mathbf{x}, \mathbf{x}_0) < \epsilon \end{align*}$$

If $\mathbf{x}_0$ is both a cluster point of $E \subseteq \mathbb{R}^m$ and a cluster point of its complement $E^c = \mathbb{R}^m \setminus E$ (meaning every neighborhood of $\mathbf{x}_0$ contains points from both $E$ and $\mathbb{R}^m \setminus E$), then $\mathbf{x}_0$ is called a boundary point of $E$. The set of all boundary points of $E$ is denoted as $\partial E$ and is called the boundary set of $E$.

Let $E$ be a subset of the domain of mapping $f$, and let $\mathbf{x}_0 \in \mathbb{R}^m$ be a cluster point of $E$. We say that $\mathbf{A} \in \mathbb{R}^p$ is the limit of $f(\mathbf{x})$ as $\mathbf{x} \in E$ approaches $\mathbf{x}_0$, denoted as

$$\begin{align*} \lim_{E \ni \mathbf{x} \to \mathbf{x}_0} f(\mathbf{x}) = \mathbf{A} \end{align*}$$

if for any $\epsilon > 0$, there exists a $\delta > 0$ such that for any $\mathbf{x} \in E$, as long as $0 < d(\mathbf{x}, \mathbf{x}_0) < \delta$, we have

$$\begin{align*} d_2(f(\mathbf{x}), \mathbf{A}) < \epsilon \end{align*}$$

Here, $d_2$ is the distance in the codomain space of $f$.

• When $E$ is the entire domain of mapping $f$, $\lim\limits_{E \ni \mathbf{x} \to \mathbf{x}_0} f(\mathbf{x})$ is simply written as $\lim\limits_{\mathbf{x} \to \mathbf{x}_0} f(\mathbf{x})$.

Theorem 1.13. Let $E$ be the domain of the composite function $g \circ f$, with $\lim\limits_{E \ni \mathbf{x} \to \mathbf{x}_0} f(\mathbf{x}) = \mathbf{y}_0$ and $\lim\limits_{\mathbf{y} \to \mathbf{y}_0} g(\mathbf{y}) = A$. Define

$$\begin{align*} \tilde{g}(\mathbf{y}) = \begin{cases} g(\mathbf{y}), & \mathbf{y} \neq \mathbf{y}_0; \\ A, & \mathbf{y} = \mathbf{y}_0. \end{cases} \end{align*}$$

Then $\lim\limits_{\mathbf{x} \to \mathbf{x}_0} \tilde{g}(f(\mathbf{x})) = A$. In particular, if $g$ is continuous at $\mathbf{y}_0$, then

$$\begin{align*} \lim_{\mathbf{x} \to \mathbf{x}_0} g(f(\mathbf{x})) = A. \end{align*}$$