Measure Theory
1.1 Probability Spaces
A probability space is a triple \((\Omega, \mathcal{F}, P)\) where \(\Omega\) is a set of “outcomes,” \(\mathcal{F}\) is a set of “events,” and \(P : \mathcal{F} \to [0,1]\) is a function that assigns probabilities to events. We assume that \(\mathcal{F}\) is a \(\sigma\)-field (or \(\sigma\)-algebra), i.e., a (nonempty) collection of subsets of \(\Omega\) that satisfy
if \(A \in \mathcal{F}\) then \(A^c \in \mathcal{F}\), and
if \(A_i \in \mathcal{F}\) is a countable sequence of sets then \(\bigcup_i A_i \in \mathcal{F}\).
A measure is a nonnegative countably additive set function; that is, a function \(\mu : \mathcal{F} \to \mathbb{R} \cup \{\infty\}\) with
\(\mu(A) \ge \mu(\emptyset) = 0\) for all \(A \in \mathcal{F}\), and
if \(A_i \in \mathcal{F}\) is a countable sequence of disjoint sets, then
\[ \mu\left(\bigcup_i A_i\right) = \sum_i \mu(A_i). \]
If \(\mu(\Omega) = 1\), we call \(\mu\) a probability measure. Probability measures are usually denoted by \(P\).
In general, a measure may take the value \(\infty\), i.e.,
\[ \mu : \mathcal{F} \to [0,\infty]. \]
Theorem 1.1 Let \(\mu\) be a measure on \((\Omega, \mathcal{F})\) (a \(\sigma\)-field).
monotonicity. If \(A \subset B\) then \(\mu(A) \le \mu(B)\).
subadditivity. If \(A \subset \bigcup_{m=1}^{\infty} A_m\) then \[ \mu(A) \le \sum_{m=1}^{\infty} \mu(A_m). \]
continuity from below. If \(A_i \uparrow A\) (i.e., \(A_1 \subset A_2 \subset \cdots\) and \(\bigcup_i A_i = A\)) then \[ \mu(A_i) \uparrow \mu(A). \]
continuity from above. If \(A_i \downarrow A\) (i.e., \(A_1 \supset A_2 \supset \cdots\) and \(\bigcap_i A_i = A\)), with \(\mu(A_1) < \infty\), then \[ \mu(A_i) \downarrow \mu(A). \]
If we are given a set \(\Omega\) and a collection \(\mathcal{A}\) of subsets of \(\Omega\), then there is a smallest \(\sigma\)-field containing \(\mathcal{A}\). We call this the \(\sigma\)-field generated by \(\mathcal{A}\) and denote it by \(\sigma(\mathcal{A})\).
Let \(\mathbb{R}^d\) be the set of vectors \((x_1,\ldots,x_d)\) of real numbers and \(\mathcal{R}^d\) be the Borel sets, the smallest \(\sigma\)-field containing the open sets. When \(d=1\) we drop the superscript.
Measures on the real line. Measures on \((\mathbb{R},\mathcal{R})\) are defined by giving a Stieltjes measure function with the following properties:
\(F\) is nondecreasing.
\(F\) is right continuous, i.e. \(\lim_{y\downarrow x} F(y)=F(x)\).
Theorem 1.2 Associated with each Stieltjes measure function \(F\) there is a unique measure \(\mu\) on \((\mathbb{R}, \mathcal{R})\) with \[ \mu((a,b]) = F(b) - F(a) \]
When \(F(x) = x\) the resulting measure is called Lebesgue measure.
1.2 Random Variables
A function \(X : \Omega \to S\) is said to be a measurable map from \((\Omega, \mathcal{F})\) to \((S, \mathcal{S})\) if
\[ X^{-1}(B) \equiv \{\omega : X(\omega) \in B\} \in \mathcal{F} \quad \text{for all } B \in \mathcal{S}. \]
If \((S, \mathcal{S}) = (\mathbb{R}^d, \mathcal{R}^d)\) and \(d > 1\), then \(X\) is called a random vector. Of course, if \(d = 1\), \(X\) is called a random variable.
We define \(\sigma(X)\) to be the smallest \(\sigma\)-algebra on \(\Omega\) such that \(X\) is measurable.
Theorem 1.3 For \(\mathcal{A}\) generates \(\mathcal{S}\), and for all \(A \in \mathcal{A}\), \(X^{-1}(A) \in \mathcal{F}\), then \(X\) is a measurable map from \((\Omega, \mathcal{F})\) to \((S, \mathcal{S})\).
Theorem 1.4 If
\[ X : (\Omega, \mathcal{F}) \to (S, \mathcal{S}) \]
and
\[ f : (S, \mathcal{S}) \to (T, \mathcal{T}) \]
are measurable maps, then \(f(X)\) is a measurable map from \((\Omega, \mathcal{F})\) to \((T, \mathcal{T})\).
1.3 Distributions
The indicator function of a set \(A \in \mathcal{F}\):
\[ 1_A(\omega) = \begin{cases} 1, & \omega \in A, \\ 0, & \omega \notin A. \end{cases} \]
If \(X\) is a random variable, then \(X\) induces a probability measure on \(\mathbb{R}\), called its distribution by setting \(\mu(A) = P(X \in A)\) for Borel sets \(A\). The right-hand side can be written as \(P(X^{-1}(A))\).
The distribution of a random variable \(X\) is usually described by giving its distribution function,
\[ F(x) = P(X \le x). \]
Theorem 1.5 Any distribution function \(F\) has the following properties:
\(F\) is nondecreasing.
\[ \lim_{x\to\infty}F(x)=1, \qquad \lim_{x\to-\infty}F(x)=0. \]
\(F\) is right continuous, i.e., \[ \lim_{y\downarrow x}F(y)=F(x). \]
If \[ F(x-)=\lim_{y\uparrow x}F(y), \] then \[ F(x-)=P(X<x). \]
\[ P(X=x)=F(x)-F(x-). \]
Theorem 1.6 If \(F\) satisfies properties (i), (ii), and (iii), then \(F\) is the distribution function of some random variable.
1.4 Integration
1.4.1 Simple Functions
\(\varphi\) is said to be a simple function if
\[ \varphi(\omega) = \sum_{i=1}^n a_i 1_{A_i} \]
and \(A_i\) are disjoint sets with \(\mu(A_i) < \infty\). If \(\varphi\) is a simple function, we let
\[ \int \varphi \, d\mu = \sum_{i=1}^n a_i \mu(A_i). \]
Theorem 1.7 \(\varphi \ge \psi\) \(\mu\)-almost everywhere (or \(\varphi \ge \psi\) \(\mu\)-a.e.) means
\[ \mu(\{\omega : \varphi(\omega) < \psi(\omega)\}) = 0. \]
Let \(\varphi\) and \(\psi\) be simple functions.
If \(\varphi \ge 0\) a.e. then
\[ \int \varphi \, d\mu \ge 0. \]
For any \(a \in \mathbb{R}\),
\[ \int a\varphi \, d\mu = a \int \varphi \, d\mu. \]
\[ \int (\varphi + \psi) \, d\mu = \int \varphi \, d\mu + \int \psi \, d\mu. \]
If (i) and (iii) hold then we have:
If \(\varphi \le \psi\) a.e. then
\[ \int \varphi \, d\mu \le \int \psi \, d\mu. \]
If \(\varphi = \psi\) a.e. then
\[ \int \varphi \, d\mu = \int \psi \, d\mu. \]
If, in addition, (ii) holds when \(a=-1\) we have
- \[ \left| \int \varphi \, d\mu \right| \le \int |\varphi| \, d\mu. \]