Skip to main content

Probability Theory: for stat mech

Probability Theory: The Language of Statistical Mechanics

Statistical Mechanics bridges the microscopic world of particles with the macroscopic world of thermodynamics. To do this, it relies entirely on the mathematical framework of probability. This post defines the foundational concepts—from sets and axioms to covariance—that are necessary to understand ensembles, entropy, and fluctuations.

[INSERT DIAGRAM OF SAMPLE SPACE AND EVENTS HERE]

1. Sets and Events (Foundations)

In set theory, a set is simply a collection of objects.

  • \( x \in A \): Element \( x \) belongs to set \( A \).
  • \( B \subset A \): \( B \) is a subset of \( A \).

The Physics Connection

In probability theory applied to physics:

  • Sample Space (\( \Omega \)): The set of all possible outcomes. In statistical mechanics, this is the set of all possible microstates. \[ \Omega = \{ \text{all possible outcomes} \} \]
  • Event (\( A \)): A subset of \( \Omega \). It is not just a single element but often represents a macroscopic constraint. \[ A \subset \Omega \]

Example (Coin Toss):
Sample Space: \( \Omega = \{H, T\} \)
Event: \( A = \{H\} \)

2. Probability Axioms (Kolmogorov)

A valid probability measure \( P: \mathcal{F} \to [0,1] \) must satisfy three axioms:

  1. Non-negativity: Probabilities cannot be negative. \[ P(A) \ge 0 \quad \forall A \subset \Omega \]
  2. Normalization: The probability of the entire sample space is 1. \[ P(\Omega) = 1 \]
  3. Countable Additivity: For mutually exclusive (disjoint) events \( A_i \) (where \( A_i \cap A_j = \varnothing \) for \( i \neq j \)): \[ P\left(\bigcup_i A_i\right) = \sum_i P(A_i) \]

3. Why This Matters in Statistical Mechanics

These abstract definitions map directly to physical reality:

  • \( \Omega \) represents the ensemble of all microstates.
  • An event \( A \) represents a macroscopic state, such as "The energy of the system lies between \( E \) and \( E + \delta E \)".

The axioms guarantee that our ensemble averages are mathematically well-defined and that thermodynamic quantities are self-consistent.

4. Joint Probability

For two events \( A, B \subset \Omega \), the joint probability is denoted by \( P(A \cap B) \). It represents the probability that both conditions \( A \) and \( B \) are satisfied simultaneously.

Physics Example:
\( A \): The particle has energy \( E \).
\( B \): The particle lies in volume \( V \).
Joint probability encodes the correlations between these physical properties.

5. Conditional Probability

This is defined as the probability of \( B \) occurring given that \( A \) is known to have occurred:

\[ P(B|A) = \frac{P(A \cap B)}{P(A)}, \quad \text{where } P(A) \neq 0 \]

Physical Meaning: It implies restricting the ensemble. We ignore all microstates except those satisfying condition \( A \), and then ask how often \( B \) occurs. This logic underpins the definition of the Canonical and Grand Canonical ensembles.

6. Bayes’ Theorem

Bayes' theorem relates conditional probabilities:

\[ P(A|B) = \frac{P(B|A)\,P(A)}{P(B)} \]

Or equivalently:

\[ P(A \cap B) = P(A|B)P(B) = P(B|A)P(A) \]

This theorem is vital for inverse problems: inferring microscopic parameters (like intermolecular forces) from observed macroscopic data.

7. Random Variables & Expectation Values

In physics, we measure "observables," which are essentially random variables (e.g., Energy, Magnetization). The expectation value \( \langle X \rangle \) corresponds to the thermodynamic observable.

Discrete Case

\[ \langle f(X) \rangle = \sum_{x} f(x)\,p(x) \]

Example (Average Energy): \( \langle E \rangle = \sum_i E_i\,p_i \)

Continuous Case

\[ \langle f(X) \rangle = \int f(x)\,p(x)\,dx \]

8. Covariance and Fluctuations

Covariance measures how two variables fluctuate together:

\[ C_{ij} = \langle (x_i - \langle x_i\rangle)(x_j - \langle x_j\rangle) \rangle \]

Significance:
If \( C_{ij} = 0 \), the variables are uncorrelated. This concept is central to the Fluctuation-Dissipation Theorem and Phase Transitions. For example, the specific heat capacity is directly related to energy fluctuations:

\[ C_{EE} = \langle E^2 \rangle - \langle E \rangle^2 \]

Visualizing These Concepts

1. Venn Diagram: Visualize \( \Omega \) as a large rectangle. Event \( A \) is a circle inside it. \( A \cap B \) is the overlapping region of two circles.
2. Probability Distribution Graph: A bell curve (Gaussian) representing the probability density \( p(x) \) vs variable \( x \). The area under the curve is 1 (Normalization). The peak is the expectation value \( \langle X \rangle \) (for symmetric distributions).