微信客服：xiaoxionga100
微信客服：ITCS521
程序代写案例-ECSE 509
时间：2021-11-07
ECSE 509 - Probability and Random Signals 2
Vahid Partovi Nia
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Contents
1 Conditional Probability and Expectation
Definition
Conditional Expectation
Conditional Variance
Examples
Extreme Values
Order Statistics
2/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Section 2
Conditional Probability and Expectation
3/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Definition
Subsection 1
Definition
4/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Definition
What is a stochastic process
• A stochastic process Xt is a collection of random variables
on an ordered univariate t > 0.
• Random field is for multi-dimensional t..
• What does mean Xt = x? what sort of random variables
are we dealing with?
• Chain, Process, Diffusion ?
Why do we need conditional probabilities?
• Markov chain of order k is
P (Xt | Xt−1, . . . , X1, X0) = P (Xt | Xt−1, . . . , Xt−k)
• Markov chain is
P (Xt | Xt−1, . . . , X1, X0) = P (Xt | Xt−1)
• What about irregular time t measurements?
• What about r.v. Xt with uncountable support?
5/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Definition
Definitions
• p(x | y)
• f(x | y)
• How to recover marginals p(x), p(y), f(x), f(y) from
conditionals
• IEX,Y {g(X,Y )} =
∫ ∫
g(x, y)f(x, y)dxdy
• IV{g(X,Y )} = IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
6/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Definition
Monty Hall Problem
Find the solution with conditional probabilities.
https://en.wikipedia.org/wiki/Wikipedia_talk:
Requests_for_mediation/Monty_Hall_problem/Conditional_probability_solution
7/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Expectation Subsection 2
Conditional Expectation
8/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Expectation
Conditional Expectation
IEX,Y {g(X,Y )} = IEX{IEY |Xg(X,Y )} = IEY {IEX|Y g(X,Y )}
IEX|Y {g(X,Y )} =
∫
g(x, y)f(x | y)dx
IEY IEX|Y {g(X,Y )} =
∫ {∫
g(x, y)f(x | y)dx
}
f(y)dy
9/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance Subsection 3
Conditional Variance
10/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance
Conditional Variance
IVX,Y {g(X,Y )} = IVY [IEX|Y {g(X,Y )}]+IEY [ IVX|Y {g(X,Y )}]
IV{g(X,Y )}
= IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
±IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − IE2X,Y {g(X,Y )}
+IEX,Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − [IEY IEX|Y {g(X,Y )}]2︸ ︷︷ ︸
first term
+ IEY IEX|Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}︸ ︷︷ ︸
second term
11/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance
Conditional Variance
IVX,Y {g(X,Y )} = IVY [IEX|Y {g(X,Y )}]+IEY [ IVX|Y {g(X,Y )}]
IV{g(X,Y )}
= IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
±IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − IE2X,Y {g(X,Y )}
+IEX,Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − [IEY IEX|Y {g(X,Y )}]2︸ ︷︷ ︸
first term
+ IEY IEX|Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}︸ ︷︷ ︸
second term
11/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance
Conditional Variance
IVX,Y {g(X,Y )} = IVY [IEX|Y {g(X,Y )}]+IEY [ IVX|Y {g(X,Y )}]
IV{g(X,Y )}
= IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
±IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − IE2X,Y {g(X,Y )}
+IEX,Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − [IEY IEX|Y {g(X,Y )}]2︸ ︷︷ ︸
first term
+ IEY IEX|Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}︸ ︷︷ ︸
second term
11/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance
Conditional Variance
IVX,Y {g(X,Y )} = IVY [IEX|Y {g(X,Y )}]+IEY [ IVX|Y {g(X,Y )}]
IV{g(X,Y )}
= IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
±IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − IE2X,Y {g(X,Y )}
+IEX,Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − [IEY IEX|Y {g(X,Y )}]2︸ ︷︷ ︸
first term
+ IEY IEX|Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}︸ ︷︷ ︸
second term
11/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Examples Subsection 4
Examples
12/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Examples
Compound Variable
Suppose
N∑
i=1
Xi | N while N is another independent (positive
and discrete) random variable.
Example
Suppose Xi are iid from exponential distribution with rate λ1.
Define SN =
∑N
i=1Xi | N, while N is Poisson with mean λ2;
find the (unconditional) mean and the variance of SN .
13/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Examples
Compound Variable
Suppose
N∑
i=1
Xi | N while N is another independent (positive
and discrete) random variable.
Example
Suppose Xi are iid from exponential distribution with rate λ1.
Define SN =
∑N
i=1Xi | N, while N is Poisson with mean λ2;
find the (unconditional) mean and the variance of SN .
13/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Examples
Over-dispersion
Suppose X is Poisson with mean θ, but θ follows Gamma
distribution with parameters Γ(α, λ).
Find the (unconditional) mean and variance of X.
14/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Extreme Values
Subsection 5
Extreme Values
15/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Extreme Values
Density of the maximum
Suppose continuous random variables X1, X2, . . . , Xn
iid∼ F
and X(1) ≤ X(2), . . . ,≤ X(n) are the increasing order statistics.
Remind that dF (x)dx = f(x)
FX(n)(x) = P (X(n) ≤ x)
= P
[
{X1 ≤ x} · · · {Xn ≤ x}
]
= Fn(x)
Therefore
fX(n)(x) =
dFX(n)
dx
=
dFn(x)
dx
= nf(x)Fn−1(x)
16/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Extreme Values
Density of the minimum
Suppose continuous random variables X1, X2, . . . , Xn
iid∼ F and
X(1) ≤ X(2), . . . ,≤ X(n) are the increasing order statistics.
1− FX(1)(x) = P (X(1) > x)
= P
[
{X1 > x} · · · {Xn > x}
]
= {1− F (x)}n
Therefore
fX(1)(x) =
dFX(1)
dx
=
d
[
1− {1− F (x)}n
]
dx
= nf(x){1− F (x)}n−1
17/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Subsection 6
Order Statistics
18/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Marginal
Suppose continuous random variables X1, X2, . . . , Xn
iid∼ F
and X(1) ≤ X(2), . . . ,≤ X(n) are the increasing order statistics.
Denote n =
k∑
i=1
ni(
n
n1, . . . , nk
)
=
n!
n1! · · ·nk!
fX(i)(x) =
(
n
i− 1, 1, n− i
)
{F (x)}i−1f(x){1− F (x)}n−i
One may intuitively explain it as i− 1 data less than X(i), one
data exactly on X(i) and n− i data bigger than X(i)
X1, · · · , Xi−1︸ ︷︷ ︸
i−1
X(i)︸︷︷︸
1
Xi+1, . . . , Xn︸ ︷︷ ︸
n−i
F (x) f(x) 1− F (x)
19/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Pairwise
fX(i),X(j)(xi, xj) =
(
n
i− 1, 1, j − i− 1, 1, n− j
)
{F (xi)}i−1f(xi)
{F (xj)− F (xi)}j−i−1f(xj)
{1− F (xj)}n−j ,
xi ≤ xj
A similar intuition is valid for the distribution of a pair of order
statistics.
X1, · · · , Xi−1︸ ︷︷ ︸
i−1
X(i)︸︷︷︸
1
Xi+1, . . . , Xj−1︸ ︷︷ ︸
j−i−1
X(j)︸︷︷︸
1
Xj+1, · · · , Xn︸ ︷︷ ︸
n−j
F (xi) f(xi) {F (xj)− F (xi)} f(xj) 1− F (xj)
20/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Joint
fX(1),...,X(n)(x1, . . . xn) =
n!f(x1) · · · f(xn)
x1 ≤ · · · ≤ xn
21/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Joint
fX(1),...,X(n)(x1, . . . xn) = n!f(x1) · · · f(xn)
x1 ≤ · · · ≤ xn
21/21
学霸联盟
Vahid Partovi Nia
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Contents
1 Conditional Probability and Expectation
Definition
Conditional Expectation
Conditional Variance
Examples
Extreme Values
Order Statistics
2/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Section 2
Conditional Probability and Expectation
3/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Definition
Subsection 1
Definition
4/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Definition
What is a stochastic process
• A stochastic process Xt is a collection of random variables
on an ordered univariate t > 0.
• Random field is for multi-dimensional t..
• What does mean Xt = x? what sort of random variables
are we dealing with?
• Chain, Process, Diffusion ?
Why do we need conditional probabilities?
• Markov chain of order k is
P (Xt | Xt−1, . . . , X1, X0) = P (Xt | Xt−1, . . . , Xt−k)
• Markov chain is
P (Xt | Xt−1, . . . , X1, X0) = P (Xt | Xt−1)
• What about irregular time t measurements?
• What about r.v. Xt with uncountable support?
5/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Definition
Definitions
• p(x | y)
• f(x | y)
• How to recover marginals p(x), p(y), f(x), f(y) from
conditionals
• IEX,Y {g(X,Y )} =
∫ ∫
g(x, y)f(x, y)dxdy
• IV{g(X,Y )} = IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
6/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Definition
Monty Hall Problem
Find the solution with conditional probabilities.
https://en.wikipedia.org/wiki/Wikipedia_talk:
Requests_for_mediation/Monty_Hall_problem/Conditional_probability_solution
7/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Expectation Subsection 2
Conditional Expectation
8/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Expectation
Conditional Expectation
IEX,Y {g(X,Y )} = IEX{IEY |Xg(X,Y )} = IEY {IEX|Y g(X,Y )}
IEX|Y {g(X,Y )} =
∫
g(x, y)f(x | y)dx
IEY IEX|Y {g(X,Y )} =
∫ {∫
g(x, y)f(x | y)dx
}
f(y)dy
9/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance Subsection 3
Conditional Variance
10/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance
Conditional Variance
IVX,Y {g(X,Y )} = IVY [IEX|Y {g(X,Y )}]+IEY [ IVX|Y {g(X,Y )}]
IV{g(X,Y )}
= IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
±IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − IE2X,Y {g(X,Y )}
+IEX,Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − [IEY IEX|Y {g(X,Y )}]2︸ ︷︷ ︸
first term
+ IEY IEX|Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}︸ ︷︷ ︸
second term
11/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance
Conditional Variance
IVX,Y {g(X,Y )} = IVY [IEX|Y {g(X,Y )}]+IEY [ IVX|Y {g(X,Y )}]
IV{g(X,Y )}
= IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
±IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − IE2X,Y {g(X,Y )}
+IEX,Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − [IEY IEX|Y {g(X,Y )}]2︸ ︷︷ ︸
first term
+ IEY IEX|Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}︸ ︷︷ ︸
second term
11/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance
Conditional Variance
IVX,Y {g(X,Y )} = IVY [IEX|Y {g(X,Y )}]+IEY [ IVX|Y {g(X,Y )}]
IV{g(X,Y )}
= IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
±IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − IE2X,Y {g(X,Y )}
+IEX,Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − [IEY IEX|Y {g(X,Y )}]2︸ ︷︷ ︸
first term
+ IEY IEX|Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}︸ ︷︷ ︸
second term
11/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Conditional
Variance
Conditional Variance
IVX,Y {g(X,Y )} = IVY [IEX|Y {g(X,Y )}]+IEY [ IVX|Y {g(X,Y )}]
IV{g(X,Y )}
= IEX,Y {g2(X,Y )} − IE2X,Y {g(X,Y )}
±IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − IE2X,Y {g(X,Y )}
+IEX,Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}
= IEY IE
2
X|Y {g(X,Y )} − [IEY IEX|Y {g(X,Y )}]2︸ ︷︷ ︸
first term
+ IEY IEX|Y {g2(X,Y )} − IEY IE2X|Y {g(X,Y )}︸ ︷︷ ︸
second term
11/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Examples Subsection 4
Examples
12/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Examples
Compound Variable
Suppose
N∑
i=1
Xi | N while N is another independent (positive
and discrete) random variable.
Example
Suppose Xi are iid from exponential distribution with rate λ1.
Define SN =
∑N
i=1Xi | N, while N is Poisson with mean λ2;
find the (unconditional) mean and the variance of SN .
13/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Examples
Compound Variable
Suppose
N∑
i=1
Xi | N while N is another independent (positive
and discrete) random variable.
Example
Suppose Xi are iid from exponential distribution with rate λ1.
Define SN =
∑N
i=1Xi | N, while N is Poisson with mean λ2;
find the (unconditional) mean and the variance of SN .
13/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Examples
Over-dispersion
Suppose X is Poisson with mean θ, but θ follows Gamma
distribution with parameters Γ(α, λ).
Find the (unconditional) mean and variance of X.
14/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Extreme Values
Subsection 5
Extreme Values
15/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Extreme Values
Density of the maximum
Suppose continuous random variables X1, X2, . . . , Xn
iid∼ F
and X(1) ≤ X(2), . . . ,≤ X(n) are the increasing order statistics.
Remind that dF (x)dx = f(x)
FX(n)(x) = P (X(n) ≤ x)
= P
[
{X1 ≤ x} · · · {Xn ≤ x}
]
= Fn(x)
Therefore
fX(n)(x) =
dFX(n)
dx
=
dFn(x)
dx
= nf(x)Fn−1(x)
16/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Extreme Values
Density of the minimum
Suppose continuous random variables X1, X2, . . . , Xn
iid∼ F and
X(1) ≤ X(2), . . . ,≤ X(n) are the increasing order statistics.
1− FX(1)(x) = P (X(1) > x)
= P
[
{X1 > x} · · · {Xn > x}
]
= {1− F (x)}n
Therefore
fX(1)(x) =
dFX(1)
dx
=
d
[
1− {1− F (x)}n
]
dx
= nf(x){1− F (x)}n−1
17/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Subsection 6
Order Statistics
18/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Marginal
Suppose continuous random variables X1, X2, . . . , Xn
iid∼ F
and X(1) ≤ X(2), . . . ,≤ X(n) are the increasing order statistics.
Denote n =
k∑
i=1
ni(
n
n1, . . . , nk
)
=
n!
n1! · · ·nk!
fX(i)(x) =
(
n
i− 1, 1, n− i
)
{F (x)}i−1f(x){1− F (x)}n−i
One may intuitively explain it as i− 1 data less than X(i), one
data exactly on X(i) and n− i data bigger than X(i)
X1, · · · , Xi−1︸ ︷︷ ︸
i−1
X(i)︸︷︷︸
1
Xi+1, . . . , Xn︸ ︷︷ ︸
n−i
F (x) f(x) 1− F (x)
19/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Pairwise
fX(i),X(j)(xi, xj) =
(
n
i− 1, 1, j − i− 1, 1, n− j
)
{F (xi)}i−1f(xi)
{F (xj)− F (xi)}j−i−1f(xj)
{1− F (xj)}n−j ,
xi ≤ xj
A similar intuition is valid for the distribution of a pair of order
statistics.
X1, · · · , Xi−1︸ ︷︷ ︸
i−1
X(i)︸︷︷︸
1
Xi+1, . . . , Xj−1︸ ︷︷ ︸
j−i−1
X(j)︸︷︷︸
1
Xj+1, · · · , Xn︸ ︷︷ ︸
n−j
F (xi) f(xi) {F (xj)− F (xi)} f(xj) 1− F (xj)
20/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Joint
fX(1),...,X(n)(x1, . . . xn) =
n!f(x1) · · · f(xn)
x1 ≤ · · · ≤ xn
21/21
ECSE 509 -
Probability
and Random
Signals 2
Conditional
Probability
and
Expectation
Order Statistics
Joint
fX(1),...,X(n)(x1, . . . xn) = n!f(x1) · · · f(xn)
x1 ≤ · · · ≤ xn
21/21
学霸联盟
