PHIL 7001: Fundamentals of AI, Data, and Algorithms
Lecture 3
Probability Spaces and Distributions
Boris Babic,
HKU 100 Associate Professor of Data Science, Law and Philosophy
Boris Babic, HKU Week 2 1 / 33
Today
Learning goals
• Introduction to Probability Distributions
• Discrete vs. Continuous Distributions
• Binomial Distribution and Examples
• Normal Distribution and its Role
• Properties of the Normal Distribution
Boris Babic, HKU Week 2 2 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Last week
Review of last week
• Introduction to probability
• Sample space, outcomes, events
• Properties of sets (union, intersection, complement)
• Laws of probability
• Bayes’ Rule
Boris Babic, HKU Week 2 3 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Random Variables
Boris Babic, HKU Week 2 4 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Recap on sample space
• In working with probability questions, it is important to know our sample
space!
• What are all the possible outcomes? Ex: a die can land on 1...6. What
are the events of interest? Ex: the likelihood that a die lands on an even
number. Even number = {2, 4, 6}.
Boris Babic, HKU Week 2 5 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Calculating Probability by Counting Outcomes from the Sample
Space Ω
• We can use this method if:
• All outcomes of the sample space are equally likely.
• The sample space Ω is finite.
• Let A be an event defined on the sample space Ω.
• The probability of event A can be calculated as:
P (A) =
number of outcomes satisfying event A
Total number of outcomes in Ω
Example 1: What is the probability of getting a head if a fair coin is tossed?
• The sample space Ω = {H,T}
• Event: We are looking for a head.
• Out of the two outcomes, one satisfies our condition.
• Hence, P (H) = 1
2
.
Boris Babic, HKU Week 2 6 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Random variables
• We will use capital end of alphabet letters, ...W,X, Y, Z, to indicate
random variables.
• Random variables map outcomes in the sample space to natural or real
numbers.
• In this sense, they are a function: X : S → R.
• For example: If we are modeling a coin toss, we might say we have a
random variable X, which can take the value 1 for heads, or 0 for tails.
• If we are interested in students’ heights, then we can assume X ∈ R+.
• Random variables can be discrete, or continuous.
• We will use capital X to denote the random variable and lowercase x to
indicate values it might take.
Boris Babic, HKU Week 2 7 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Probability Distributions
Boris Babic, HKU Week 2 8 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Probability distributions
• A probability distribution is a function f of the random variable X: f(X)
• It maps values of the random variable to probabilities.
• This function can only take values between 0 and 1.
• Also, this function is additive: for two independent events, the probability of
their sum is the sum of their probabilities.
Ex: Pr (die lands on 4) + Pr (die lands on 6) = Pr (die lands on 4 or die lands
on 6).
• Finally, the total probability of the whole space must sum to 1.
• Example: Imagine you roll a fair six-sided dice. The outcomes are the numbers
1 through 6. Because the dice is fair, each number has an equal probability of
1
6
. The probability distribution can be represented as:
Outcome 1 2 3 4 5 6
Probability 1
6
1
6
1
6
1
6
1
6
1
6
Boris Babic, HKU Week 2 9 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Two Main Types of Distributions
1 Discrete Probability Distribution: For discrete random variables.
Outcomes take on distinct/separate values.
2 Continuous Probability Distribution: For continuous random variables.
Outcomes take on a range of values without gaps.
Boris Babic, HKU Week 2 10 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Discrete Probability Distribution
Definition: In the discrete case, it tells us how probable it is that the random
variable X will take a specific value x. We often denote the function f with
Pr. Possible values are countable and distinct.
Example: Tossing a coin, Rolling a die.
Visualization: Probability mass function (PMF).
Characteristics:
• Probability for any specific value is non-negative: P (X = x) ≥ 0.
• Sum of all probabilities is 1: ∑P (X = x) = 1.
• Example: Rolling a fair six-sided die. Each number 1-6 has a 1
6
chance of
occurring. All possibilities summed together equal 1.
Boris Babic, HKU Week 2 11 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Continuous Probability Distribution
Definition: In the continuous case, it tells us how probable it is that the
random variable X will fall within some specified region between a and b.
Visualization: Probability density function (PDF).
Characteristics:
• Probability that X takes on any specific value is always 0.
• Area under the entire curve (PDF) is 1.
• To find the probability that X lies in a range of values, calculate the area
under the curve for that range.
• Example: Height of adult men being between 5’8” and 6’2”.
Boris Babic, HKU Week 2 12 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Binomial Distribution
Boris Babic, HKU Week 2 13 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Binomial Distribution
Boris Babic, HKU Week 2 14 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Binomial Distribution
Boris Babic, HKU Week 2 15 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Binomial Distribution
• Suppose we are tossing a coin once, and we want to know the probability
that it lands on heads (p), or tails (1− p).
• In general, we can write this as,
px(1− p)1−x
• If we toss the coin n times, then this becomes
px(1− p)n−x
• But we must account for the number of different ways we can observe x
successes in n experiments.
• Example: What is the probability of observing 2 heads in 3 tosses? First,
we have p2(1− p)3−2. If the coin is fair, then this is
0.52(1− 0.5)1 = 0.53 = 0.125.
• But there are three ways to observe two heads in three tosses of a coin:
HHT, HTH, THH. So we need to multiply 0.125 by
(
3
2
)
= 3. This is
0.375.
• So the full binomial distribution is given by,
P(X = x) =
(
n
x
)
px(1− p)n−x,
where
(
n
x
)
= n!
x!(n−x)! .
Boris Babic, HKU Week 2 16 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Exercise
Exercise: In a factory, 95% of products pass quality control. If you randomly
select 20 products, what’s the probability that exactly 18 of them pass the
quality control?
Answer:
P(X = 18) =
(
20
18
)
(0.95)18(0.05)2 ≈ 0.1887
Boris Babic, HKU Week 2 17 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Exercise
Exercise: In a factory, 95% of products pass quality control. If you randomly
select 20 products, what’s the probability that exactly 18 of them pass the
quality control?
Answer:
P(X = 18) =
(
20
18
)
(0.95)18(0.05)2 ≈ 0.1887
Boris Babic, HKU Week 2 17 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Exercise
Exercise: You’re taking a multiple-choice test with 10 questions. Each
question has 4 choices and only one is correct. If you guess on each question,
what’s the probability of getting at least 8 questions correct?
Take 5 minutes to work on this in groups of 2.
Answer:
P(X ≥ 8) = P(X = 8) + P(X = 9) + P(X = 10)
=
(
10
8
)(
1
4
)8(
3
4
)2
+
(
10
9
)(
1
4
)9(
3
4
)1
+
(
10
10
)(
1
4
)10(
3
4
)0
≈ 0.00042
Boris Babic, HKU Week 2 18 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Exercise
Exercise: You’re taking a multiple-choice test with 10 questions. Each
question has 4 choices and only one is correct. If you guess on each question,
what’s the probability of getting at least 8 questions correct?
Take 5 minutes to work on this in groups of 2.
Answer:
P(X ≥ 8) = P(X = 8) + P(X = 9) + P(X = 10)
=
(
10
8
)(
1
4
)8(
3
4
)2
+
(
10
9
)(
1
4
)9(
3
4
)1
+
(
10
10
)(
1
4
)10(
3
4
)0
≈ 0.00042
Boris Babic, HKU Week 2 18 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Exercise Solution using Python
• In Python, the function binom.pmf(k, n, p) returns the probability of k
successes in n trials (which is the same as x in the problem), given
probability p.
• So you can use the code:
binom.pmf(k=8, n=10, p=0.25) + binom.pmf(k=9, n=10, p=0.25) +
binom.pmf(k=10, n=10, p=0.25)
• More efficiently, the function binom.sf(k, n, p) returns the probability to
the right of k, given n and p.
• Hint: If you use binom.cdf(k, n, p) instead, it will return the probability
to the left of k, including k.
• Be careful: binom.sf(k, n, p) returns Pr(X > x). But binom.cdf(k, n, p)
returns Pr(X ≤ x). So just remember this, depending on whether you
want to include k or not.
• So you can simply write: binom.sf(k=7, n=10, p=0.25)
• Notice that here we wrote 7, and not 8. Since the question said ”at least
8” we want to include 8 in our calculation.
Boris Babic, HKU Week 2 19 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Binomial Distribution in Python
Python Code for Binomial Probabilities
1 from scipy.stats import binom
2
3 # Calculate individual probabilities (PMF)
4 # P(X=8) + P(X=9) + P(X=10) for n=10, p=0.25
5 prob_sum = (binom.pmf(k=8, n=10, p=0.25) +
6 binom.pmf(k=9, n=10, p=0.25) +
7 binom.pmf(k=10, n=10, p=0.25))
8
9 # Calculate P(X > 7) using survival function
10 prob_greater_than_7 = binom.sf(k=7, n=10, p=0.25)
11
12 # Alternative: Calculate P(X > 7) using CDF
13 prob_greater_than_7_alt = 1 - binom.cdf(k=7, n=10, p=0.25)
Key Functions in scipy.stats.binom
• binom.pmf(k, n, p): Probability mass function - returns P (X = k)
• binom.cdf(k, n, p): Cumulative distribution function - returns P (X ≤ k)
• binom.sf(k, n, p): Survival function - returns P (X > k)
Boris Babic, HKU Week 2 20 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
CDF vs SF Relationship
• Relationship: binom.sf(k, n, p) = 1 - binom.cdf(k, n, p)
• For ”at least k” successes: use binom.sf(k-1, n, p)
• Example: P (X ≥ 8) = P (X > 7) = binom.sf(7, 10, 0.25)
Boris Babic, HKU Week 2 21 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Normal Distribution
Boris Babic, HKU Week 2 22 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Normal Distribution
Boris Babic, HKU Week 2 23 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Normal Distribution
Boris Babic, HKU Week 2 24 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Normal Distribution
• Variables that have a normal distribution are ubiquitous in real life,
provided we have enough data.
• Age of HKU students, height of HKU students.
Boris Babic, HKU Week 2 25 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Boris Babic, HKU Week 2 26 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Personality of Normal Parameters
Boris Babic, HKU Week 2 27 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
The norm.cdf and norm.sf Functions in Python
• Suppose that the test scores of a course exam at HKU are normally
distributed with a mean of 72 and a standard deviation of 15.2. What is
the probability that a randomly chosen student received above 84?
• 1 - norm.cdf(84, loc=72, scale=15.2) or norm.sf(84, loc=72,
scale=15.2).
• Approximately 21%.
• We use 1 - norm.cdf() or norm.sf() in order to get the area from x
to ∞.
• If you want the area to the left of x, then use norm.cdf() directly.
• Example: Find the percentage of otters that weigh less than 33 kilograms
in a population with mean = 40 and sd = 8.
• norm.cdf(33, loc=40, scale=8) = 0.19.
Boris Babic, HKU Week 2 28 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
The norm.sf function in Python
• The weekly salaries of the employees of a large corporation are assumed
to be normally distributed with mean $450 and standard deviation $40.
• What is the probability that a randomly chosen employee earns more than
$500 per week?
Boris Babic, HKU Week 2 29 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
The norm.sf function in Python
• The weekly salaries of the employees of a large corporation are assumed
to be normally distributed with mean $450 and standard deviation $40.
• What is the probability that a randomly chosen employee earns more than
$500 per week?
Python Code for Normal Distribution Probabilities
1 norm.sf(500, 450, 40)
Approximately 10%
Boris Babic, HKU Week 2 30 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Tips
• Probabilities correspond to areas
• Probabilities sum to 1: Pr(X < k) = 1− Pr(X > k)
• Symmetry: Pr(X < −k) = Pr(X > k)
• For intervals, use subtraction:
Pr(a < X < b) = Pr(X < b)− Pr(X < a)
Boris Babic, HKU Week 2 31 / 33
Week 2
Boris
Babic,
HKU
Review
from last
class
Random
Variables
Probability
Distribu-
tions
Binomial
Distribution
Normal
Distribution
Summary
• Binomial: Discrete, characterized by two parameters n, p.
• Normal: Continuous, characterized by two parameters, µ (loc), σ (scale).
• Binomial in Python:
• scipy.stats.binom.cdf(k, n, p),
• scipy.stats.binom.pmf(k, n, p),
• scipy.stats.binom.sf(k, n, p).
• Normal distribution in Python:
• scipy.stats.norm.cdf(x, loc, scale),
• scipy.stats.norm.pdf(x, loc, scale),
• scipy.stats.norm.sf(x, loc, scale).
• Now you know how to calculate normal and binomial probabilities for any
event without ever having to use one of those complicated look up tables
at the back of statistics textbooks! WOW!!
Boris Babic, HKU Week 2 32 / 33
Resources
• Whenever you need help with Python, I highly recommend to Google
search the package documentation. For example, for the binom function
in Python, everything can easily be found here: https://docs.scipy.
org/doc/scipy/reference/generated/scipy.stats.binom.html
Boris Babic, HKU Week 2 33 / 33

学霸联盟