Portfolio Optimization
Dr. ZHANG Xiaowei
MSBA7021 Prescriptive Analytics
Background
In order to mitigate risk while ensuring a reasonable level of return, investors purchase
a variety of securities and combine these into an investment portfolio
2 / 68
Securities often change together with some classes of securities, and in the opposite
direction of other classes of securities
A portfolio that contains many positively correlated securities is more risky (and
potentially more rewarding) than one that contains a mix of positively and negatively
correlated securities
3 / 68
Goals
Most investors have two goals in forming portfolios:
to obtain a large expected return, and
to obtain a small variance (to minimize risk)
At a given expected return, there is one portfolio which has the lowest risk
The risk-return characteristics of a portfolio change in a nonlinear fashion
4 / 68
Index vs. active funds
Index Fund Active Fund
Goal
Match the performance of a specic
market benchmark or "index" (e.g.
S&P500) as closely as possible
Outperform its benchmark
Strategy
Buys all (or a representative sample) of
the stocks or bonds in the index it's
tracking
Uses the portfolio manager's deep
research and expertise to hand-select
stocks or bonds for the fund
Risk
Aligns directly to the risks involved with
the specic stock or bond market the
fund tracks
Adds the risk that the portfolio
manager may underperform its
benchmark
5 / 68
6 / 68
7 / 68
Asset
Asset: an investment instrument that can be bought and sold
Suppose we invest into an asset now and sell it some time later, receiving X
0
X
t
total return = R =
rate of return = r =
X
t
X
0
X
t
−X
0
X
0
8 / 68
Short sales
It is sometimes possible to sell an asset that you do not own through the process of
short selling the asset
Borrow the asset from someone (the lender) who owns it
Sell the borrowed asset to someone else, receiving an amount
At a later date, purchase the asset for and return it to the lender
Short selling is protable if the asset price declines
If , you'll make a prot of
Note: Short selling is quite risky
The potential loss is unlimited: can increase arbitrarily so can the loss
Short selling is forbidden or supplemented by restrictions within certain nancial
institutions
X
0
X
t
X
t
< X
0
X
0
−X
t
X
t
9 / 68
Portfolio return
Suppose we invest to form a portfolio of different assets
The amount invested in the th asset is
Dene as the fraction of asset in the portfolio:
If short selling is allowed, then some of the 's can be negative
X
0
n
i X
0i
n
∑
i=1
X
0i
= X
0
w
i
i w
i
= X
0i
/X
0
n
∑
i=1
w
i
= 1
w
i
10 / 68
Let and denote the total return and the rate of return of asset
The total amount received by the portfolio
The total return of the portfolio
The rate of return of the portfolio
R
i
r
i
i
n
∑
i=1
R
i
X
0i
=
n
∑
i=1
R
i
w
i
X
0
R = =
n
∑
i=1
R
i
w
i
∑
n
i=1
R
i
w
i
X
0
X
0
r = R− 1 =
n
∑
i=1
R
i
w
i
−
n
∑
i=1
w
i
=
n
∑
i=1
r
i
w
i
11 / 68
Random returns
When an asset is originally acquired, its rate of return is random
We summarize its uncertainty by its
expected value
variance
covariance with other assets of interest
r
E[r] ≡ μ
Var[r] = E[(r− μ)
2
] ≡ σ
2
12 / 68
Import historical stock price data
import pandas as pd
df = pd.read_csv('StockPrices.csv')
# select a recent subperiod
nDays = 300
start = df.index[nDays]
end = df.index[-1]
print('Start Date:', df['Date'][start])
print(' End Date:', df['Date'][end])
Start Date: 22/3/2018
End Date: 31/5/2019
13 / 68
S = df.loc[start:end]
S.head()
Date AAPL AXP BA CAT CSCO CVX DIS GE
5087 22/3/2018 165.6652 89.7283 311.6044 142.5569 41.5126 108.4317 99.0218 12.5799 171
5088 23/3/2018 161.8289 88.7860 312.9596 140.0241 40.8861 107.7451 96.9941 12.3160 167
5089 26/3/2018 169.5112 90.9160 320.7300 144.7792 42.4668 110.0053 99.0710 12.1464 172
5090 27/3/2018 165.1648 89.7381 313.0766 142.6442 41.1367 109.3472 97.8012 12.6647 170
5091 28/3/2018 163.3399 90.5136 312.0042 140.8684 40.1536 106.9059 96.9941 12.8908 170
5 rows × 21 columns
14 / 68
import plotly.express as px
import plotly.io as pio
pio.templates.default = 'plotly_white'
S_melt = S.melt(id_vars = 'Date', var_name = 'Stock', value_name = 'Price')
fig_price = px.line(S_melt, x = 'Date', y = 'Price', color = 'Stock', height = 600)
fig_price.update_layout(xaxis_title = '')
15 / 68
Computing stock returns
Let denote the price of stock at time
Let denote the rate of return of asset from time to time
stocks = S.columns[1:]
tmp = S[stocks]
ret = (tmp.diff()[1:] /tmp.shift(1)[1:])
ret = pd.concat([S['Date'][1:], ret], axis=1)
S
it
i t
r
it
i t− 1 t
r
it
=
S
it
− S
i,t−1
S
i,t−1
16 / 68
ret_melt = ret.melt(id_vars = 'Date', var_name = 'Stock', value_name = 'Return')
fig = px.line(ret_melt, x = 'Date', y = 'Return', color = 'Stock', height = 600)
fig.update_layout(xaxis_title = '', yaxis_title = 'Daily Return')
17 / 68
fig = px.box(ret_melt, x = 'Stock', y = 'Return', color = 'Stock', height = 600)
fig.update_layout(xaxis_title = '', yaxis_title = 'Daily Return')
18 / 68
s = 'MSFT'
fig = px.histogram(ret, x = s, y = s, histnorm = 'probability')
fig.update_layout(xaxis_title = 'Price', yaxis_title = 'Probability', title = s)
19 / 68
Computing means and standard deviations
mu = ret.mean(axis = 0)
sigma = ret.std(axis = 0)
stats = pd.concat([mu, sigma], axis = 1)
stats.columns = ['Mean Return', 'Standard Deviation']
stats.reset_index(inplace = True)
stats.rename(columns = {'index':'Stock'}, inplace = True)
20 / 68
fig = px.bar(
stats.melt(id_vars = ['Stock']), x = 'Stock',
y = 'value', facet_col = 'variable')
fig.update_layout(yaxis2 = {'matches':None}, xaxis_title = '', xaxis2_title = '')
21 / 68
Mean-standard deviation diagram
The random rates of return of assets can be represented on a two-dimensional diagram
Vertical axis: mean rate of return
Horizontal axis: standard deviation
μ
σ
22 / 68
fig_MSD = px.scatter(
stats, x = 'Standard Deviation', y = 'Mean Return',
text = 'Stock', color = 'Stock',
range_x = [0, 0.03], range_y = [-0.002, 0.002])
fig_MSD.update_layout(showlegend = False)
23 / 68
Computing covariances
Covariance between asset and asset :
Correlation between asset and asset :
Note:
cor = ret[stocks].corr()
i j
Cov[r
i
, r
j
] = E[(r
i
− μ
i
)(r
j
− μ
j
)] ≡ σ
ij
i j
Corr[r
i
, r
j
] = =
Cov[r
i
, r
j
]
√Var[r
i
]Var[r
j
]
σ
ij
σ
i
σ
j
σ
ii
= σ
2
i
24 / 68
import plotly.graph_objects as go
fig = go.Figure(data=go.Heatmap(
z=cor,
x=ret.columns[1:],
y=ret.columns[1:],
hoverongaps = False))
fig.show()
25 / 68
Portfolio mean and variance
Mean return of a portfolio
Variance of a portfolio
μ = E[r] = E[w
1
r
1
+⋯+ w
n
r
n
] = w
1
μ
1
+⋯+ w
n
μ
n
σ
2
= E[(r− μ)
2
]
= E[(
n
∑
i=1
w
i
r
i
−
n
∑
i=1
w
i
μ
i
)
2
]
= E[(
n
∑
i=1
w
i
(r
i
− μ
i
))
2
]
= E[
n
∑
i,j=1
w
i
w
j
(r
i
− μ
i
)(r
j
− μ
j
)]
=
n
∑
i,j=1
w
i
w
j
σ
ij
26 / 68
Equally weighted portfolio
An equally weighted portfolio allocates an equal amount of capital to each component
of the portfolio
for all
import numpy as np
n = len(stocks)
w = 1/n * np.ones(n)
cov = ret[stocks].cov()
mu_equal = w@mu
sigma_equal = np.sqrt(w@cov@w)
print('Mean rate of return of the portfolio: {00.4f}'.format(mu_equal))
print('Standard deviation: {00.4f}'.format(sigma_equal))
Mean rate of return of the portfolio: 0.0004
Standard deviation: 0.0099
w
i
= 1/n i = 1,… ,n
27 / 68
fig_MSD.add_scatter(
x = [sigma_equal], y = [mu_equal], text = ['EW Portfolio'],
mode = 'markerstext', marker = {'size':20})
fig_MSD.update_layout(
{'title': 'Risk/Return for an Equally Weighted Portfolio'})
28 / 68
fig_price.update_traces(line = {'dash':'dot'})
fig_price.add_scatter(
x = S['Date'], y = S[stocks].mean(axis = 1),
line = {'width':4, 'color':'navy'},
name = "Equally Weighted Portfolio")
fig_price.update_layout(
title = 'Value of an Equally Weighted Portfolio',
showlegend = False)
29 / 68
30 / 68
The Markowitz model
Given a set of assets, how should one determine the portfolio that has the lowest risk
and yields a high expected return?
This question was answered by Harry Markowitz in the 1950s
He received the Nobel Prize in economics in 1990 for this work
Assumptions
The mean rates of return and the covariances are known, for
Short selling is allowed
μ
i
σ
ij
i, j = 1,… ,n
31 / 68
Decision variables
: the fraction of asset in the portfolio
Objective
Minimizing the variance of the rate of return of the portfolio
Constraints
The expected return of the portfolio meets a target level
The fractions sum to 1
w
i
i
n
∑
i,j=1
w
i
w
j
σ
i,j
m
0
n
∑
i=1
w
i
μ
i
= m
0
n
∑
i=1
w
i
= 1
32 / 68
The minimum-variance portfolio is constructed by solving
In practice, and must be estimated from historical data
minimize
n
∑
i,j=1
w
i
w
j
σ
i,j
subject to
n
∑
i=1
w
i
μ
i
= m
0
n
∑
i=1
w
i
= 1
μ
i
σ
i,j
33 / 68
Pyomo model
import pyomo.environ as pe
# create a model
model = pe.ConcreteModel()
# define decision variables
n = len(stocks)
model.w = pe.Var(range(n), domain=pe.Reals)
# declare objective
model.var = pe.Objective(
expr = sum(model.w[i]model.w[j]cov.iloc[i,j]
for i in range(n) for j in range(n)),
sense = pe.minimize)
34 / 68
# target mean daily return is the same as MSFT
s = 'MSFT'
mu_targ = mu[s]
# constraint: meet target return
model.ret = pe.Constraint(
expr=sum(model.w[i]mu[i] for i in range(n)) mu_targ)
# constraint: fractions sum to one
model.frac = pe.Constraint(
expr=sum(model.w[i] for i in range(n)) 1)
# solve the model
solver = pe.SolverFactory('gurobi')
solver.solve(model);
35 / 68
Optimal solution
fig = px.bar(x = stocks, y = [model.w[i]() for i in range(n)])
fig.update_layout(
xaxis_title = '', yaxis_title = 'Weight',
title = 'Target Daily Mean Return: {00.4f}'.format(mu_targ))
36 / 68
s = 'MSFT'
print('{}:'.format(s))
print('Mean Rate of Return: {00.4f}'.format(mu[s]))
print('Standard Deviation: {00.4f}'.format(sigma[s]))
print('\nMinimum-Variance Portfolio:')
print('Mean Rate of Return: {00.4f}'.format(mu_targ))
sigma_port = np.sqrt(model.var())
print('Standard Deviation: {00.4f}'.format(np.sqrt(model.var())))
MSFT
Mean Rate of Return: 0.0013
Standard Deviation: 0.0166
Minimum-Variance Portfolio:
Mean Rate of Return: 0.0013
Standard Deviation: 0.0077
37 / 68
fig_MSD.add_scatter(
x = [sigma_port], y = [mu_targ], text = ['MV Portfolio'],
mode = 'markerstext', marker = {'size':20})
s = 'MSFT'
fig_MSD.add_scatter(
x = [sigma_port, sigma[s]], y = [mu_targ, mu[s]],
mode = 'lines', line = {'dash': 'dot'})
fig_MSD.update_layout(title = 'Risk/Return for Minimum-Variance Portfolio')
38 / 68
39 / 68
Minimum-variance set
Given a target mean rate of return, there exists a portfolio that has the minimum
variance
By varying the target mean rate of return, we collect the set of minimum-variance
portfolios and call it the minimum-variance set
ret_targ = np.linspace(-0.004, 0.008, 100)
min_std = np.zeros(ret_targ.shape)
for (i, mu_targ) in enumerate(ret_targ):
model.ret.set_value(
sum(model.w[i]mu[i] for i in range(n)) mu_targ)
results = solver.solve(model)
min_std[i] = np.sqrt(model.var())
40 / 68
fig = px.scatter(
stats, x = 'Standard Deviation', y = 'Mean Return',
text = 'Stock', color = 'Stock', range_x = [0, 0.03],
range_y = [ret_targ.min(), ret_targ.max()])
# minimumvariance point (MVP)
MVP_idx = np.argmin(min_std)
MVP = (min_std[MVP_idx], ret_targ[MVP_idx])
fig.add_scatter(
x = [MVP[0]], y = [MVP[1]], mode = 'markerstext', text = ['MVP'],
marker = {'size':12, 'symbol':'diamond'})
fig.add_scatter(
x = min_std, y = ret_targ, mode = 'lines', line = {'dash':'dot'}, name = '')
fig.update_layout(title = 'Minimum-Variance Set', showlegend = False)
41 / 68
42 / 68
Efcient Frontier
Given a standard deviation, there exists a portfolio that has the highest mean rate of
return
Only the upper portion of the minimum-variance set will be of interest to most
investors
This upper portion is called the efcient frontier
43 / 68
fig = px.scatter(
stats, x = 'Standard Deviation', y = 'Mean Return',
text = 'Stock', color = 'Stock', range_x = [0, 0.03],
range_y = [ret_targ.min(), ret_targ.max()])
# minimumvariance point (MVP)
MVP_idx = np.argmin(min_std)
MVP = (min_std[MVP_idx], ret_targ[MVP_idx])
fig.add_scatter(
x = [MVP[0]], y = [MVP[1]], mode = 'markerstext', text = ['MVP'],
marker = {'size':12, 'symbol':'diamond', 'color':'gold'})
fig.add_scatter(
x = min_std[MVP_idx:], y = ret_targ[MVP_idx:],
mode = 'lines', line = {'dash':'dot'}, name = '')
fig.update_layout(title = 'Efficient Frontier', showlegend = False)
44 / 68
45 / 68
What if short-selling is prohibited?
Simply add the nonnegativity constraints to the optimization problem:
Two ways to achieve this in Pyomo :
either change the domain of the variables to be NonNegativeReals
for i in range(n):
model.w[i].domain = pe.NonNegativeReals
or set lower bound of the variables to be 0
for i in range(n):
model.w[i].setlb(0)
With each , the mean return of the portfolio will be ranged between the
minimum and the maximum mean returns of the individual assets
Without such a restriction, the mean return of the portfolio is unbounded
w
i
≥ 0
w
i
≥ 0
46 / 68
for i in range(n):
model.w[i].domain = pe.NonNegativeReals
ret_targ_NS = np.linspace(mu.min(), mu.max(), 50)
min_std_NS = np.zeros(ret_targ_NS.shape)
for (i, mu_targ) in enumerate(ret_targ_NS):
model.ret.set_value(
sum(model.w[i]mu[i] for i in range(n)) mu_targ)
results = solver.solve(model)
min_std_NS[i] = np.sqrt(model.var())
47 / 68
fig = px.scatter(
stats, x = 'Standard Deviation', y = 'Mean Return',
text = 'Stock', color = 'Stock', range_x = [0, 0.03],
range_y = [ret_targ.min(), ret_targ.max()])
MVP_idx = np.argmin(min_std_NS)
fig.add_scatter(
x = min_std_NS[MVP_idx:], y = ret_targ_NS[MVP_idx:],
mode = 'lines', line = {'dash':'dot'}, name = '')
MVP_idx = np.argmin(min_std)
MVP = (min_std[MVP_idx], ret_targ[MVP_idx])
fig.add_scatter(
x = min_std[MVP_idx:], y = ret_targ[MVP_idx:],
mode = 'lines', line = {'dash':'dot'}, name = '')
fig.add_scatter(x = [0.009], y = [0.002], text = 'Allowed', mode = 'text')
fig.add_scatter(x = [0.009], y = [0.001], text = 'Prohibited', mode = 'text')
fig.update_layout(title = 'Efficient Frontier', showlegend = False)
48 / 68
49 / 68
The restriction on short selling reduces the feasible set of portfolios
Given the same standard deviation, the highest expected rate of return that a portfolio
can achieve is also lower if short selling is prohibited
50 / 68
Markowitz model is elegant, but...
To estimate the covariance matrix is challenging
The number of unknown parameters is of order of magnitude of
Sample covariance matrix is not a good estimator

n
2
51 / 68
Mean-absoluate deviation (MAD) portfolio
optimization
Proposed in 1991 by Hiroshi Konno and Hiroaki Yamazaki
Use MAD instead of variance to measure the risk of a portfolio
Var[r] = E[(r− μ)
2
]
MAD[r] = E[|r− μ|] = E[
∣
∣
n
∑
i=1
w
i
(r
i
− μ
i
)
∣
∣
]
52 / 68
Let denote the rate of return of asset observed at time , for and r
it
i t i = 1,… ,n
t = 1,… ,T
μ
i
≈
T
∑
t=1
r
it
≡ μ^
i
MAD[r] ≈
T
∑
t=1
∣
∣
n
∑
i=1
w
i
(r
it
− μ^
i
)
∣
∣
1
T
1
T
53 / 68
The minimum-MAD portfolio is constructed by solving
The objective function involves absolute value functions, making it nonlinear and non-
differentiable
Directly feeding it to a solver would not work
minimize
T
∑
t=1
∣
∣
n
∑
i=1
w
i
(r
it
− μ^
i
)
∣
∣
subject to
n
∑
i=1
w
i
μ^
i
= m
0
n
∑
i=1
w
i
= 1
1
T
54 / 68
Reformulation
Introduce a new set of variables
y
t
=
∣
∣
n
∑
i=1
w
i
(r
it
−
^
μ
i
)
∣
∣
55 / 68
The optimization problem is equivalent to
This is a linear optimization problem!
minimize
T
∑
t=1
y
t
subject to y
t
≥
n
∑
i=1
w
i
(r
it
− μ^
i
), ∀t
y
t
≥ −
n
∑
i=1
w
i
(r
it
− μ^
i
), ∀t
n
∑
i=1
w
i
μ^
i
= m
0
n
∑
i=1
w
i
= 1
1
T
56 / 68
# compute absolute deviations
mu = ret[stocks].mean()
AD = ret[stocks] - mu
# create the model
model_MAD = pe.ConcreteModel()
# define the decision variables
T = nDays - 1
n = len(stocks)
model_MAD.w = pe.Var(range(n), domain = pe.Reals)
model_MAD.y = pe.Var(range(T), domain = pe.Reals)
#declare the objective function
model_MAD.MAD = pe.Objective(
expr = sum(model_MAD.y[t] for t in range(T))/T,
sense = pe.minimize)
57 / 68
# target mean daily return is the same as MSFT
# target mean daily return is the same as MSFT
s = 'MSFT'
mu_targ = mu[s]
# declare constraints
model_MAD.ret = pe.Constraint(
expr = sum(model_MAD.w[i]mu[i] for i in range(n)) mu_targ)
model_MAD.frac = pe.Constraint(
expr = sum(model_MAD.w[i] for i in range(n)) 1)
model_MAD.AD1 = pe.Constraint(
range(T), rule = lambda mod, t: mod.y[t] sum(
mod.w[i]*AD.iloc[t,i] for i in range(n)))
model_MAD.AD2 = pe.Constraint(
range(T), rule = lambda mod, t: mod.y[t] sum(
mod.w[i]*AD.iloc[t,i] for i in range(n)))
# solve the model
solver = pe.SolverFactory('gurobi')
solver.solve(model_MAD);
58 / 68
for i in range(n):
model.w[i].domain = pe.Reals
model.ret.set_value(sum(model.w[i]mu[i] for i in range(n)) mu_targ)
solver.solve(model);
59 / 68
weight = pd.DataFrame({
'Stock': stocks,
'MV': [model.w[i]() for i in range(n)],
'MAD': [model_MAD.w[i]() for i in range(n)]})
fig = px.bar(
weight.melt(id_vars = ['Stock']),
x = 'Stock', y = 'value', color = 'variable', barmode = 'group')
fig.update_layout(
xaxis_title = '', yaxis_title = 'Weight',
title = 'Target Daily Mean Return: {00.4f}'.format(mu_targ))
60 / 68
61 / 68
MAD = AD.abs().mean()
MAD_port = model_MAD.MAD()
stats = pd.concat([mu, MAD], axis = 1)
stats.columns = ['Mean Return', 'Absolute Deviation']
stats.reset_index(inplace = True)
stats.rename(columns = {'index':'Stock'}, inplace = True)
62 / 68
fig = px.scatter(
stats, x = 'Absolute Deviation', y = 'Mean Return',
text = 'Stock', color = 'Stock',
range_x = [0, 0.02], range_y = [-0.002, 0.002])
fig.add_scatter(
x = [MAD_port], y = [mu_targ], text = ['Min-AD Portfolio'],
mode = 'markerstext', marker = {'size':20})
s = 'MSFT'
fig.add_scatter(
x = [MAD_port, MAD[s]], y = [mu_targ, mu[s]],
mode = 'lines', line = {'dash': 'dot'})
fig.update_layout(
title = 'Risk/Return for Minimum-AD Portfolio', showlegend = False)
63 / 68
64 / 68
Efcient frontier of MAD portfolio
Similar to Markowitz's mean-variance framework, one can dene the efcient frontier
for the MAD framework
Given an absolute deviation, there exists a portfolio that has the highest mean rate of
return
The efcient frontier is then constructed by varying the absolute deviation
65 / 68
ret_targ = np.linspace(-0.004, 0.008, 100)
min_AD = np.zeros(ret_targ.shape)
for (i, mu_targ) in enumerate(ret_targ):
model_MAD.ret.set_value(
sum(model_MAD.w[i]mu[i] for i in range(n)) mu_targ)
results = solver.solve(model_MAD)
min_AD[i] = model_MAD.MAD()
66 / 68
fig = px.scatter(
stats, x = 'Absolute Deviation', y = 'Mean Return',
text = 'Stock', color = 'Stock', range_x = [0, 0.03],
range_y = [ret_targ.min(), ret_targ.max()])
# minimumabsolute deviation point (MADP)
MADP_idx = np.argmin(min_AD)
MADP = (min_AD[MADP_idx], ret_targ[MADP_idx])
fig.add_scatter(
x = [MADP[0]], y = [MADP[1]], mode = 'markers', text = ['MADP'],
marker = {'size':12, 'symbol':'diamond', 'color':'gold'})
fig.add_scatter(
x = min_AD[MADP_idx:], y = ret_targ[MADP_idx:],
mode = 'lines', line = {'dash':'dot'}, name = '')
fig.update_layout(title = 'Efficient Frontier', showlegend = False)
67 / 68
68 / 68