Lecture 3

Robin Sandquist Robin Sandquist

Random Variables

Robin Sandquist

September 2026

Random Variables

Definition (Random variable)

A random variable is a function

X:Ω⟶ℝ

that assigns a real number X(u) to each outcome u in the sample space Ω.

For every t∈ℝ, the set

{u∈Ω:X(u)≤t}

is an event to which a probability can be assigned.

Example (The sum of two die)

Let the sample space

Ω={(u1,u2):u1,u2∈{1,2,3,4,5,6}}.

A random variable on this sample space is

X(u1,u2)=u1+u2.

Discrete Random Variables

Definition (Discrete random variable)

A random variable X is called discrete if it takes values in a finite or countably infinite set S with probability one:

ℙ(X∈S)=1.
Definition (Probability mass function)

The probability mass function of a discrete random variable X is

pX(k):=ℙ(X=k),k∈ℝ.

Note: pX(k)=0 for values outside the set of possible values of X.

Example (Students arriving late)

Suppose that three students each have probability 1/5 of arriving late, independently of one another. Let X be the number of students who arrive late.

Then X∈{0,1,2,3}, and

pX(k)=(3k)(15)k(45)3−k,k=0,1,2,3.

Unequivocally,

k0123pX(k)6412548125121251125

the probability that at least two students arrive late is

ℙ(X≥2)=pX(2)+pX(3)=13125.
Theorem (Properties of the probability mass function)

Let X be a discrete random variable. Then:

  1. For every k∈S,

    0≤pX(k)≤1.
  2. The probabilities sum to one:

    ∑kpX(k)=1.
  3. For a,b∈ℝ with a≤b,

    ℙ(a≤X≤b)=∑a≤k≤bpX(k).

    If X is integer-valued and a,b are integers, this becomes

    ℙ(a≤X≤b)=∑k=abpX(k).
  4. For every a∈ℝ,

    ℙ(X≤a)=∑k≤apX(k).
  5. For every a∈ℝ,

    ℙ(X≥a)=∑\k≥apX(k)=1−ℙ(X<a).

Proof of property 3 .

Suppose that a≤b are integers. We can write

{a≤X≤b}=⋃k=ab{X=k}.

The events {X=k} are pairwise disjoint. By additivity of probability,

ℙ(a≤X≤b)=ℙ(⋃k=ab{X=k})=∑k=abℙ(X=k)=∑k=abpX(k).

□

Cumulative Distribution Functions

Definition (Cumulative distribution function)

For any random variable X, the cumulative distribution function is defined by

FX(t):=ℙ(X≤t),t∈ℝ.

This definition applies to both discrete and continuous random variables.

The function determines the probability distribution of X.

Theorem (Properties of the cumulative distribution function)

Let FX be the cumulative distribution function of a random variable X. Then FX is non-decreasing and right continuous. That is,

s≤t⟹FX(s)≤FX(t),

and

limh↓0⁡FX(t+h)=FX(t).
  1. limt→∞⁡FX(t)=1.
  2. limt→−∞⁡FX(t)=0.
  3. For a<b,

    ℙ(a<X≤b)=FX(b)−FX(a).
  4. For every a∈ℝ,

    ℙ(X>a)=1−FX(a).
  5. For every a∈ℝ,

    ℙ(X<a)=limh↓0⁡FX(a−h)=FX(a−),

    where FX(a−) denotes the left limit of FX at a.

Continuous Random Variables

Example (Waiting for a bus)

Suppose that a bus arrives exactly every ten minutes and that a passenger arrives at a unifor random time between two buses.

Let X be the passengers waiting time measured in minutes. Then X is uniformly distributed on [0,10], with density

fX(x)={110,0≤x≤10,0,otherwise.

For example,

ℙ(2≤X≤5)=∫25110dx=310.
Definition (Probability density function)

A random variable X is called absolutely continuous if there exists a non-negative measurable function

fX:ℝ⟶[0,∞)

such that, for every A⊆ℝ,

ℙ(X∈A)=∫AfX(x)dx.

The function fX is called a probability density function of X.

In these notes, ``continuous random variable’’ refers to a random variable with such a density.

Remark (The probability of a single value)

If X has a density, then for every fixed t∈ℝ,

ℙ(X=t)=ℙ(t≤X≤t)=∫ttfX(x)dx=0.

Consequently, including or excluding the endpoints of an interval does not change its probability.

This does not apply to a constant random variable: if X=c with probability one, then

ℙ(X=c)=1.
Theorem (Properties of a probability density)

If fX is a probability density function, then

fX(x)≥0for almost every x∈ℝ,

and

∫−∞∞fX(x)dx=1.

the total probability is

1=ℙ(X∈ℝ)=∫−∞∞fX(x)dx.

The relationship between the PDF and CDF

If X has density fX, then

FX(t)=ℙ(X≤t)=∫−∞tfX(x)dx.

It follows that

FX′(t)=fX(t)for almost every t∈ℝ.

Expected Values

Definition (Expected value)

Let X be a discrete random variable with probability mass function pX. Its expected value is

𝔼[X]=∑kkpX(k),

provided that

∑k|k|pX(k)<∞.

If X has density fX, its expected value is

𝔼[X]=∫−∞∞xfX(x)dx,

provided that

∫−∞∞|x|fX(x)dx<∞.

The expected value is an average of the possible values of X.

Remark

The expected value dont have to be the most likely outcome, and it doesn’t need to be a value that X can take.

For example, a random variable taking the values 0 and 1, each with probability 1/2, has

𝔼[X]=0⋅12+1⋅12=12,

even though X never equals 1/2.

Variance and Standard Deviation

Theorem (Expected value of a function of a random variable)

Let X be a random variable and let g:ℝ→ℝ be a function such that

𝔼[|g(X)|]<∞.

Then:

  1. If X is discrete with probability mass function pX,

    𝔼[g(X)]=∑k∈Sg(k)pX(k).
  2. If X has density fX,

    𝔼[g(X)]=∫−∞∞g(x)fX(x)dx.
Definition (Variance)

The variance of X, denoted by V(X) or Var⁡(X), is

V(X)=Var⁡(X):=𝔼[(X−𝔼[X])2]=𝔼[(X−μ)2].

Variance measures the spread of a random variable around its expected value. A small variance means that the deviation from the mean is close to it.

Definition (Standard deviation)

The standard deviation of X is

D(X):=V(X).

The variance and standard deviation are often denoted by

σ2=V(X),σ=D(X).

Note that the standard deviation has the same units as X.

For a discrete random variable,

V(X)=∑k∈S(k−μ)2pX(k).

For a continous variable with density fX,

V(X)=∫−∞∞(x−μ)2fX(x)dx.
Theorem (Computational formula for variance)

If 𝔼[X2]<∞, then

V(X)=𝔼[X2]−(𝔼[X])2.

Proof.

Let μ=𝔼[X]. Expanding the square and using linearity of expectation, we obtain

V(X)=𝔼[(X−μ)2]=𝔼[X2−2μX+μ2]=𝔼[X2]−2μ𝔼[X]+μ2=𝔼[X2]−2μ2+μ2=𝔼[X2]−μ2=𝔼[X2]−(𝔼[X])2.

□