Lecture 2

Sep 16, 2026

Conditional probabilities

Let $A$ and $B$ be events. Recall that $A \cap B =$ {both $A$ and $B$ occur}.

Assume we now find out that $B$ occurs. Given this information, what is the probability that $A$ occurs?

Definition

The conditional probability for the event $A$ given $B$ is calculated by $P (A | B) = \frac{P (A \cap B)}{P (B)}, if P (B) > 0.$

Note that if $A \cap B = \emptyset,$ then $P (A | B) = 0.$ Also if $B \subseteq A$ , then $P (A | B) = 1.$

Example

When throwing dice, we know that the probability of getting 5 dots: $P (5) = \frac{1}{6} .$ But $P (5 | odd number of dots) = \frac{1 / 6}{1 / 2} = \frac{1}{3} .$

Independant events

Definition

We say that $A$ and $B$ are unconditional events if $P (A \cap B) = P (A) P (B) .$

Note that if $A$ and $B$ are independant, and the probability $P (B) > 0$ , then $P (A | B) = \frac{P (A \cap B)}{P (B)} = \frac{P (A) P (B)}{P (B)} = P (A) .$

Independence of Three Events

The events $A$ , $B$ , and $C$ are said to be pairwise independent if

P (A \cap B) = P (A) P (B),

P (A \cap C) = P (A) P (C),

and

P (B \cap C) = P (B) P (C) .

The events $A$ , $B$ , and $C$ are said to be mutually independent if they are pairwise independent and

P (A \cap B \cap C) = P (A) P (B) P (C) .

Pairwise independence does not necessarily imply mutual independence. However, mutual independence always implies pairwise independence.

Example

Example of a situation where we have pairwise independence but not mutual independence:

Let $Ω = 1,2,3,4$ and $P$ be the uniform measurement on $Ω .$

Let $A = 1,2, B = 1,3, and C = 1,4 ⟹ P (A) = P (B) = P (C) = \frac{1}{2} .$

$P (A \cap B) = \frac{1}{4}, P (A \cap C) = \frac{1}{4}, and P (B \cap C) = \frac{1}{4} . But P (A \cap B \cap C) = \frac{1}{4} ≠ {(\frac{1}{2})}^{3} = \frac{1}{8} .$ So these three events are pairwise independent, but not mutually independent.

The law of total probability

Recall: If $A$ and $B$ are events, then $P (A | B)$ is the conditional probability for $A$ given $B$ .

Definition

Let $Ω$ be a sample space. The events $A_{1}, \dots, A_{n}$ form a partition of $Ω$ if

A_{i} \cap A_{j} = \emptyset whenever i \neq j,

and

⋃_{i = 1}^{n} A_{i} = Ω .

Theorem (Law of Total Probability)

Let $A_{1}, \dots, A_{n}$ be a partition of $Ω$ such that $P (A_{i}) > 0$ for every $i$ . If $B$ is an event, then

P (B) = \sum_{i = 1}^{n} P (B | A_{i}) P (A_{i}) .

Proof.

Write

B = ⋃_{i = 1}^{n} \underset{pairwise disjoint}{\underset{⏟}{(B \cap A_{i})}} .

We see that

\begin{matrix} P (B) & = P (⋃_{i = 1}^{n} (B \cap A_{i})) \\ \overset{Kolmogorov’s axioms}{=} \sum_{i = 1}^{n} P (B \cap A_{i}) \\ \overset{definition of conditional probability}{=} \sum_{i = 1}^{n} P (B | A_{i}) P (A_{i}) . □ \end{matrix}

Theorem (Bayes theorem)

Let $A_{1}, \dots, A_{n}$ be a partition of $Ω$ , and let $B$ be an event with $P (B) > 0$ . For $i = 1,2, \dots, n$ , we have:

P (A_{i} | B) = \frac{P (B | A_{i}) P (A_{i})}{\sum_{j = 1}^{n} P (B | A_{j}) P (A_{j})} .

Proof.

For $i = 1, \dots, n$ , we have

\begin{matrix} P (A_{i} | B) & = \frac{P (A_{i} \cap B)}{P (B)} & (definition of conditional probability) \\ = \frac{P (B | A_{i}) P (A_{i})}{P (B)} & (definition of conditional probability) \\ = \frac{P (B | A_{i}) P (A_{i})}{\sum_{j = 1}^{n} P (B | A_{j}) P (A_{j})} & (law of total probability). □ \end{matrix}