Mean-Variance Optimization in Practice: Well Diversified (Near) Efficient Portfolios

5 minute read

One well-known stylized fact of the Markowitz’s mean-variance framework is that, irrespective of the quality of the estimates of asset returns and (co)variances, efficient portfolios are concentrated in a very few assets1.

From a practitioner’s perspective, this has always been a problem12.

In this blog post, I will describe a solution proposed in Corvalan3, which is to replace efficient portfolios by near efficient portfolios equivalent in terms of risk-return but less concentrated in terms of asset weights.

Notes:

  • A Google sheet corresponding to this post is available here

Near optimal mean-variance portfolios

Definition

The output of a mean-variance portfolio optimization procedure is usually a unique4 optimal portfolio.

The neighborhood of such a portfolio in the (V,E) plane is called the region of near optimal portfolios5 associated to the original optimal portfolio, as illustrated in figure 1 taken from Chopra5.

Region of near optimal portfolios
Figure 1. Region of near optimal portfolios within a 10% window of risk-return around an optimal portfolio. Source: Chopra

Notes:

  • While figure 1 is representative of the general idea of neighborhood, the exact definition depends on the application.

Empirical properties

Several empirical properties of near optimal portfolios have been established in the literature:

  • Near optimal portfolios share similar risk-return characteristics but differ in asset composition. This is illustrated in Chopra5, from which figure 2 is reproduced.
Near optimal portfolios with extreme weights
Figure 2. Example of near optimal portfolios belonging to a 10% window of risk-return centered around an optimal portfolio. Source: Chopra
  • The region of near optimal portfolios is the most dense region in the (V,E) plane in terms of the number of existing portfolios. This is formally demonstrated in Corvalan3, from which figure 3 is reproduced.
The region of near optimal portfolios is dense
Figure 3. Holding the level of risk fixed and varying the level of return, the region in the (V,E) plane with the highest number of portfolios (~75) is the region close to the optimal portfolio whose return is equal to 8.97%. Source: Corvalan
  • Near optimal portfolios are significanlty more robust to errors in the estimates of asset returns and (co)variances than the original optimal portfolio. This is analyzed by Van Der Schans and De Graaf6, from which figure 4 is reproduced.
The region of near optimal portfolios is robust
Figure 4. The top yellow curve shows that between 60% and 90% of near optimal portfolios remain near optimal after a slight perturbation of asset returns and (co)variances used in the mean-variance optimization procedure. Source: Van Der Schans and De Graaf

Computing diversified near optimal mean-variance portfolios

Building upon the empirical properties of near optimal portfolios, Corvalan3 proposes a two step optimization procedure to compute what he calls well diversified efficient portfolios7.

Step 1: Mean-variance optimization

Let be:

  • $n$, the number of assets
  • $\mu \in \mathbb{R}^{n}$, the vector of the assets arithmetic returns
  • $\Sigma \in \mathcal{M}(\mathbb{R}^{n \times n})$, the asset covariance matrix

The first step consists in computing the asset weights $w^* \in \mathbb{R}^{n}$ of an optimal mean-variance portfolio, with return $\mu^{*} = w^* {}^t \mu$ and variance ${\sigma^{*}}^2 = w^* {}^t \Sigma w^* $.

Step 2: Diversification optimization

Let now be:

  • $\delta_{\mu} \in \mathbb{R}^{+}$ a relative tolerance over the mean-variance optimal portfolio return $\mu^*$
  • $\delta_{\sigma} \in \mathbb{R}^{+}$ a relative tolerance over the mean-variance optimal portfolio standard deviation $\sigma^*$

The second step consists in computing the most diversified8 portfolio belonging to the region of near optimal portfolios illustrated in figure 5.

The region of near optimal portfolios in Corvalan
Figure 5. Region of near optimal portfolios considered in the diversification optimization step.

More formally, and for example if the diversification measure is the Herfindahl-Hirschman Index, the asset weights $w^{**} \in \mathbb{R}^{n}$ of the most diversified portfolio are the solution of the following optimization program:

\[w^{**} = \operatorname{argmin} \sum_{i=1}^{n} w_i^2 \newline \textrm{s.t. } \begin{cases} w {}^t \Sigma w \leqslant {\sigma^*}^2 (1 + \delta_{\sigma})^2 \newline \mu^* (1 - \delta_{\mu}) \leqslant w {}^t \mu \newline ... \end{cases}\]

Notes:

  • The size of the region of near optimal portfolios is controlled by the parameters $\delta_{\mu}$ and $\delta_{\sigma}$. Intuitively, the higher the uncertainty on the estimates of the asset returns and (co)variances, the greater these parameters should be.
  • The Herfindahl-Hirschman Index has been chosen as the portfolio diversification measure in Portfolio Optimizer because it is related to the information content of the portfolio in the sense of information theory, as described in Bouchaud and al.9.

Conclusion - Optimal portfolio v.s. diversified near optimal portfolio

Now the one million dollar question.

Should optimal portfolios be replaced altogether by diversified near optimal portfolios?

While anyone will certainly have his own opinion, I would tend to agree.

I will illustrate my reasoning thanks to a simulation made with Portfolio Optimizer and using the weekly closing prices10 of the four ETFs SPY, TLT, GLD and SHY over the period 2015 - 2021.

At the end of every 12 weeks (~1 quarter):

The cumulative profit associated to these two strategies is depicted in figure 6.

Optimal v.s. near optimal maximum Sharpe ratio portfolio
Figure 6. Maximum Sharpe ratio portfolio v.s. diversified near maximum Sharpe ratio portfolio.

It is quite apparent that little harm is done over the studied ~15 years period by systematically replacing the maximum Sharpe ratio portfolio with its diversified counterpart…

So, if this experiment is representative of a “worst case harm”, and I think it is, then, because of the uncertainty in the estimation of future asset returns and (co)variances, there must be value in replacing optimal portfolios by diversified near optimal portfolios.

As Bouchaud and al. put it9:

The problem [with mean-variance optimization] is that one usually has only partial information: past data have a finite length, therefore the determination of statistical parameters is imperfect; predictions of future volatilities and returns are obviously not entirely trustworthy – this is why the optimal portfolio must reflect this lack of information and keep a minimal degree of diversification.

Happy diversification :smiley:!