Statistical Mechanics

Statistical View of Entropy

Microscopic and Macroscopic States

Microscopic State

A microscopic state describes the exact arrangement of all particles (atoms or molecules) in a system, specifying details like positions, velocities, and spins for each individual particle.

Macroscopic State

A macroscopic state is defined by bulk properties (state functions) of the system, such as temperature (T), pressure (P), volume (V), and entropy (S). These properties characterize the system as a whole, independent of individual particle details.

Equal-Probability Hypothesis

An isolated system is in equilibrium iff all accessible microstates have equivalent probabilities. Which have the same energy (E), volume (V), and number of particles (N).

Boltzmann’s Entropy Formula

The entropy (S) of a system is related to the number of accessible microstates () by the formula: [ S = k_B ] where (k_B) is Boltzmann’s constant.

Stirling’s approximation: (N! N N - N) for large (N).

Statistical Interpretation of the Entropy

In classical statistical mechanics, we calculate the entropy of an isolated system (microcanonical ensemble) by analyzing its Phase Space. A system containing \(N\) identical, distinguishable particles possesses \(3N\) spatial coordinates and \(3N\) velocity (momentum) coordinates.

Here is the step-by-step rigorous derivation of separating these components to calculate the total entropy.

The total number of accessible microstates \(\Omega\) is defined by the volume of accessible phase space divided by the fundamental quantum volume unit \(h^{3N}\): \[ \Omega = \frac{1}{h^{3N}} \int \cdots \int d^{3N}\mathbf{r} \, d^{3N}\mathbf{p} \]

\[ \Omega = \frac{1}{h^{3N}} \left( \int_{V} d^{3N}\mathbf{r} \right) \times \left( \int_{E} d^{3N}\mathbf{p} \right) \]

Each particle \(i\) can occupy any position within the macroscopic volume \(V\). Integrating the three-dimensional coordinates \(\mathbf{r}_i = (x_i, y_i, z_i)\) over the volume yields \(V\). For \(N\) independent particles, the spatial integral evaluates to: \[ \int_{V} d^{3N}\mathbf{r} = V^N \]

For an isolated ideal gas, the total internal energy \(E\) is rigidly fixed. Since there is no potential energy, \(E\) is entirely made of kinetic energy: \[ E = \sum_{i=1}^{3N} \frac{p_i^2}{2m} \implies \sum_{i=1}^{3N} p_i^2 = 2mE \]

This constraint mathematically outlines the surface of a \(3N\)-dimensional hyper-sphere with a radius of \(R = \sqrt{2mE}\). The momentum integral is proportional to the surface area of this hyper-sphere: \[ \int_{E} d^{3N}\mathbf{p} \propto (2mE)^{\frac{3N-1}{2} } \approx E^{\frac{3N}{2}} \]

Combining both factored integrals yields the total microstates equation: \[ \Omega = C \cdot V^N \cdot E^{\frac{3N}{2}} \]

We now map these microstates directly to macroscopic entropy using Boltzmann’s core postulate.

\[ S = k_B \ln \left( C \cdot V^N \cdot E^{\frac{3N}{2}} \right) \]

By exploiting the algebraic rules of logarithms (\(\ln(ABC) = \ln A + \ln B + \ln C\)), we can cleanly decouple the entropy into distinct coordinate and velocity expressions: \[ S = \underbrace{k_B \ln V^N}_{\text{Coordinate Contribution}} + \underbrace{k_B \ln E^{\frac{3N}{2}}}_{\text{Velocity (Energy) Contribution}} + k_B \ln C \]

Bringing the exponents to the outside of the logarithms: \[ S = N k_B \ln V + \frac{3N}{2}k_B \ln E + \text{Constant} = n R \ln V + \frac{3}{2} n R \ln E + \text{Constant} \]

Microscopic Ensembles and Thermodynamic Laws

Microcanonical Ensemble

A microcanonical ensemble represents an isolated system with fixed energy (E), volume (V), and number of particles (N).

Principle of Maximum Entropy Postulate

The system described by a microcanonical ensemble stays on the macroscopic state with maximum Boltzmann entropy.

Temperature and the Zeroth Law

Temperature and the Zeroth Law from Statistical Mechanics

Consider a situation where two subsystems, denoted as System 1 and System 2, are separated by a diathermal (heat-conducting) wall. The combined total system \(1 + 2\) is completely isolated from the rest of the universe.

Let the energy of System 1 be \(E_1\). Since the total energy is conserved, the energy of System 2 is constrained to be: \[ E_2 = E - E_1 \]

While \(E\) is constant, \(E_1\) can fluctuate and vary between different elements of the ensemble. Our goal is to determine the most probable value of \(E_1\) when the systems reach thermal equilibrium.

According to the Fundamental Postulate of Statistical Mechanics (the principle of equal a priori probabilities), the probability \(\Pr(E_1)\) of finding System 1 with an internal energy \(E_1\) is directly proportional to the number of microstates available to the combined system: \[ \Pr(E_1) = \frac{\Omega_1(E_1)\Omega_2(E - E_1)}{\Omega(E)} \]

where \(\Omega(E)\) represents the total number of microstates within the fixed total energy shell \([E, E + \Delta]\).

To find the most probable macroscopic state, we maximize the likelihood by taking the natural logarithm of the probability (\(\ln \Pr(E_1)\)) and setting its partial derivative with respect to \(E_1\) equal to zero: \[ 0 = \partial_{E_1} \ln \Pr(E_1) = \partial_{E_1} \left( \ln \Omega_1(E_1) + \ln \Omega_2(E - E_1) - \ln \Omega(E) \right) \]

Since \(\ln \Omega(E)\) is a constant with respect to \(E_1\), its derivative vanishes. We then apply the chain rule to explicitly track how changing \(E_1\) impacts System 2 via \(E_2\): \[ 0 = \partial_{E_1} \ln \Omega_1(E_1) + \frac{\partial E_2}{\partial E_1} \partial_{E_2} \ln \Omega_2(E_2) \]

Since \(E_2 = E - E_1\) and \(E\) is a constant, the derivative of \(E_2\) with respect to \(E_1\) is strictly: \[ \frac{\partial E_2}{\partial E_1} = -1 \]

Substituting this back into the optimization equation yields the equilibrium condition: \[ \partial_{E_1} \ln \Omega_1(E_1) = \partial_{E_2} \ln \Omega_2(E_2) \]

The condition above proves that when two systems are in thermal equilibrium (\(\Pr(E_1)\) is extremized), a specific macro-property becomes perfectly equal between them. This property depends strictly on the parameters of each individual subsystem, which matches the definition of a temperature parameter.

We can therefore establish the formal definition of temperature in statistical mechanics: \[ \frac{1}{k_B T} = (\partial_E \ln \Omega(E))_{dW=0} \]

where \(k_B\) is Boltzmann’s constant, and \(dW = 0\) denotes that no mechanical work is done on the system. Using this definition, the equilibrium condition simplifies beautifully to: \[ \frac{1}{k_B T_1} = \frac{1}{k_B T_2} \implies T_1 = T_2 \]

Temperature with Statistical Mechanics

From classical thermodynamics, the inverse temperature \(\frac{1}{T}\) is defined by the partial derivative of entropy \(S\) with respect to internal energy \(U\) at constant volume \(V\) and particle number \(N\): \[ \frac{1}{T} = \left( \frac{\partial S}{\partial E} \right)_{V, N} \]

By substituting Boltzmann’s entropy formula into this definition, we can express the temperature in terms of the number of accessible microstates \(\Omega\): \[ \frac{1}{T} = k_B \left( \frac{\partial \ln \Omega(E, V, N)}{\partial E} \right)_{V, N} \]

As derived from the hyper-surface area of a \(3N\)-dimensional sphere in momentum space, the total number of accessible microstates \(\Omega\) is factored as: \[ \Omega(E, V, N) = C \cdot V^N \cdot E^{\frac{3N}{2}} \]

where \(C\) is a constant independent of \(E\) and \(V\). Taking the natural logarithm of \(\Omega\) to match Boltzmann’s formula yields: \[ \ln \Omega = \ln C + N \ln V + \frac{3N}{2} \ln E \]

We apply the statistical mechanics definition of temperature by taking the partial derivative of this expanded expression with respect to \(E\): \[ \frac{1}{T} = k_B \cdot \frac{\partial}{\partial E} \left( \ln C + N \ln V + \frac{3N}{2} \ln E \right) \]

Since \(\ln C\) and \(N \ln V\) are held constant during this partial differentiation, their derivatives vanish completely, leaving: \[ \frac{1}{T} = k_B \cdot \frac{3N}{2} \cdot \frac{1}{E} \]

Rearranging the equation yields the canonical relationship between internal energy and macroscopic temperature: \[ E = \frac{3}{2} N k_B T \]

Solving directly for \(T\) reveals its final physical definition as the average thermal energy scaling factor: \[ T = \frac{2}{3k_B} \left( \frac{E}{N} \right) \]

\[ P = T \left( \frac{\partial S}{\partial V} \right)_{E, N} = \frac{2E}{3V} \]

\[ S = n R \ln V + \frac{3n}{2} R \ln E + C = n R \ln V + \frac{3n}{2} R \ln T + C \]

Statistical Distributions of Microscopic Particles

Kinds of Particles

Types of Particles in Statistical Mechanics

Distinguishable identical particles: \[ \Omega\{n_i\} = N! \prod_{i} \frac{g_i^{n_i}}{n_i!} \]
Bosons: \[ \Omega\{n_i\} = \prod_{i} \frac{(n_i + g_i - 1)!}{n_i!(g_i - 1)!} \]
Fermions: \[ \Omega_F\{n_i\} = \prod_{i} \frac{g_i!}{n_i!(g_i - n_i)!} \]

Particle Distributions

Maxwell-Boltzmann Distribution \[ n_i = g_i e^{-\alpha - \beta \epsilon_i} \]
Bose-Einstein Distribution \[ n_i = \frac{g_i}{e^{\alpha + \beta \epsilon_i } - 1} \]
Fermi-Dirac Distribution \[ n_i = \frac{g_i}{e^{\alpha + \beta \epsilon_i } + 1} \]