2026-08-05-Preview

2026-08-05-Preview · Probability Theory Ch2 Axioms of Probability（概率公理）

Ref: Lecture slides Ch2 · Ross Ch2 Axioms of Probability

Key Concepts

Term	Definition
Sample space（样本空间）	The set $S$ of all possible outcomes of an experiment
Event（事件）	Any subset $E$ of $S$; $E$ occurs if the outcome lies in $E$
Union / Intersection（并集 / 交集）	$E\cup F$: outcomes in $E$ or $F$; $E\cap F=EF$: outcomes in both
Complement（补集）	$E^c$: outcomes in $S$ not in $E$; $E\cup E^c=S$, $E\cap E^c=\varnothing$
Axioms of probability（概率公理）	$0\le P(E)\le 1$; $P(S)=1$; countable additivity for mutually exclusive events
Equally likely outcomes（等可能结果）	$P(E)=\frac{\#E}{\#S}$ when $P(\{i\})=\frac{1}{N}$ for all $i$

Per-Subsection: Definition & Examples

2.2 Sample Space and Events（样本空间与事件）

Definition

The sample space $S$ of an experiment (whose outcome is uncertain) is the set of all possible outcomes. Any subset $E$ of $S$ is known as an event; if the outcome is in $E$, we say that $E$ has occurred.

Example: horse race（赛马）— 12 horses, the experiment is the order of finish.

• $S$ = all $12!$ permutations of $(1,2,\dots ,12)$;
• $E=\{$outcomes starting with $7\}$ = the race was won by horse 7.
• Key idea: an event is just a subset of $S$; here $|E|=11!$ because horse 7 is fixed in first place.

Set Theory: Operations on Events（集合运算）

Definition

$E\cup F$: all outcomes in $E$ or $F$ (or both); $E\cap F=EF$: outcomes in both; $E^c$: outcomes in $S$ not in $E$.

De Morgan’s laws: $({\cup }_{i=1}^n{E}_i{)}^c={\cap }_{i=1}^n{E}_i^c$, $({\cap }_{i=1}^n{E}_i{)}^c={\cup }_{i=1}^n{E}_i^c$.

Proof (De Morgan)

• $x\in ({\cup }_i{E}_i{)}^c⟺x\notin E_i$ for all $i⟺x\in E_i^c$ for all $i⟺x\in {\cap }_i{E}_i^c$;
• the second law follows by applying the first to the events $E_i^c$ and taking complements.

Example: two coins — $E=\{$at least one H$\}=\{(H,H),(H,T),(T,H)\}$, $F=\{$at least one T$\}=\{(H,T),(T,H),(T,T)\}$.

• $E\cap F=\{(H,T),(T,H)\}$ = exactly one H and one T;
• $E\cup F=S$（两个事件覆盖整个样本空间）.

2.3 Axioms of Probability（概率公理）

Definition

Axiom 1: $0\le P(E)\le 1$.

Axiom 2: $P(S)=1$.

Axiom 3: for any sequence of mutually exclusive events $E_i$ ($E_i\cap E_j=\varnothing$, $i\ne j$),

$P({\cup }_{i=1}^{\infty }{E}_i)={\sum }_{i=1}^{\infty }P(E_i)$

Direct consequences: $P(\varnothing )=0$; finite additivity $P({\cup }_{i=1}^n{E}_i)={\sum }_{i=1}^{n}P(E_i)$.

Example: two coins — first unbiased, second biased with $P(H)=\frac{2}{3}$, $P(T)=\frac{1}{3}$.

• $P(\{H,H\})=P(\{T,H\})=\frac{1}{2}\cdot \frac{2}{3}=\frac{1}{3}$;
• $P(\{H,T\})=P(\{T,T\})=\frac{1}{2}\cdot \frac{1}{3}=\frac{1}{6}$;
• Axiom 3 lets us add singletons: $P(\{H,H\})+P(\{H,T\})+P(\{T,H\})+P(\{T,T\})=1$.

2.4 Some Simple Propositions（基本命题）

Definition

(a) $P(E^c)=1-P(E)$.

(b) If $E\subset F$, then $P(E)\le P(F)$.

(c) $P(E\cup F)=P(E)+P(F)-P(E\cap F)$.

Inclusion-Exclusion: $P({\cup }_{i=1}^n{E}_i)={\sum }_{i}P(E_i)-{\sum }_{i<j}P(E_i{E}_j)+{\sum }_{i<j<k}P(E_i{E}_j{E}_k)-\cdots +(-1{)}^{n+1}P(E_1\cdots E_n)$.

Proof (c)

• $E\cup F=E\cup (E^c\cap F)$ with disjoint pieces ⇒ $P(E\cup F)=P(E)+P(E^c\cap F)$;
• $F=(E^c\cap F)\cup (E\cap F)$ with disjoint pieces ⇒ $P(E^c\cap F)=P(F)-P(E\cap F)$;
• combine: $P(E\cup F)=P(E)+P(F)-P(E\cap F)$（(a)(b) 用同样的分解技巧）.

Example: restaurant（点菜）— like dish 1 with 0.6, dish 2 with 0.4, both with 0.3.

• $P(\text{like at least one})=0.6+0.4-0.3=0.7$;
• $P(\text{like neither})=1-0.7=0.3$.

2.5 Sample Spaces having Equally Likely Outcomes（等可能样本空间）

Definition

If $S=\{1,2,\dots ,N\}$ and $P(\{i\})=\frac{1}{N}$ for each $i$, then for any event $E$,

$P(E)=\frac{\#\text{ outcomes in }E}{\#\text{ outcomes in }S}$

Example: Chevalier de Méré（梅雷问题）— throw a die 4 times; probability of at least one “6”.

• Complement: no “6” in 4 throws, probability ${\left(\frac{5}{6}\right)}^4$;
• $P=1-{\left(\frac{5}{6}\right)}^4\approx 0.5177$；
• Two dice thrown 24 times, at least one double “6”: $1-{\left(\frac{35}{36}\right)}^{24}\approx 0.4914<0.5$ — why de Méré lost money（直觉与数学的分歧）.

2.6 Probability as a Continuous Set Function（概率的连续性）

Definition

If $\{E_n\}$ is increasing ($E_1\subset E_2\subset \cdots$) with ${\lim }_{n\to \infty }{E}_n={\cup }_{i=1}^{\infty }{E}_i$, or decreasing with ${\lim }_{n\to \infty }{E}_n={\cap }_{i=1}^{\infty }{E}_i$, then

${\lim }_{n\to \infty }P(E_n)=P({\lim }_{n\to \infty }{E}_n)$

Proof (increasing case)

• “Donuts” decomposition（甜甜圈分解）: $F_1=E_1$, $F_n=E_n\cap E_{n-1}^c$; then $F_i$ are mutually exclusive and ${\cup }_{i=1}^n{F}_i=E_n$;
• by Axiom 3: $P({\cup }_i{E}_i)=P({\cup }_i{F}_i)={\sum }_{i}P(F_i)={\lim }_n{\sum }_{i=1}^{n}P(F_i)={\lim }_{n}P(E_n)$;
• decreasing case: apply the result to the increasing complements $E_n^c$.

Example: urn paradox（无限坛子悖论）— infinitely large urn, at times $1-\frac{1}{2^n}$ add 10 balls and withdraw one.

• Withdraw the highest-numbered ball: $9(n+1)$ balls remain after $n+1$ withdrawals ⇒ infinitely many at time 1;
• Withdraw the lowest-numbered ball: ball $n$ is gone by time 1 ⇒ urn is empty;
• Withdraw randomly: let $E_n$ = ball 1 is still in the urn after $n$ withdrawals, then $$P(E_n)=\prod _{i=1}^n\frac{9i}{9i+1}\to 0$$ and $\{E_n\}$ is decreasing, so $P({\cap }_i{E}_i)={\lim }_{n}P(E_n)=0$ ⇒ urn is empty with probability 1.