# The Turbulence Index: Regime-based Partitioning of Asset Returns

The turbulence index, introduced in the previous blog post, is a measure of statistical unusualness of asset returns popularized by Kritzman and Li1. It provides a way to measure how much the behavior of a group of assets differs from its historical pattern.

In this post, based on the paper Optimal Portfolios in Good Times and Bad by Chow et al.2, I will describe how the turbulence index can be used to partition a set of asset returns into different subsets, each of them corresponding to a specific market risk regime.

I will also provide two examples of usage, one in portfolio optimization and one in the modeling of asset returns.

## Mathematical preliminaries

### Definitions

Let be:

• $n$, the number of assets in a given universe of assets
• $T$, the number of time periods
• $y_t \in \mathbb{R}^{n}$, the asset returns observed over the time period $t$, $t=1..T$
• $\mu \in \mathbb{R}^{n}$, the mean vector of the asset returns over the $T$ periods
• $\Sigma \in \mathcal{M}(\mathbb{R}^{n \times n})$, the covariance matrix of the asset returns over the $T$ periods

The (raw) turbulence index $d(y_t)$ for the universe of assets and for a given period $t=1..T$ is defined as the squared Mahalanobis distance2:

$d(y_t) = (y_t - \mu) {}^t \Sigma^{-1} (y_t - \mu)$

### Mathematical properties

The main mathematical property of the turbulence index relevant for this post3 is the following2:

Property 1: If the asset returns $y_t$ follow a multivariate Gaussian distribution, that is, $y_t \sim \mathcal{N} \left( \mu, \Sigma \right)$, the turbulence index $d(y_t)$ follows a chi-square distribution with $n$ degrees of freedom, that is, $d(y_t) \sim \mathcal{X}^2(n)$.

## A method to partition asset returns by regimes based on the turbulence index

### Description

Chow et al.2 describe how to use the turbulence index to identify multivariate outliers from a series of asset returns. These outliers, that are characterized by the unusual performance of an individual asset or from the unusual interaction of a combination of assets, none of which are necessarily unusual in isolation2 are representative of a [turbulent4 market] risk regime while the inliers are representative of a quiet market risk regime2.

In more details, the method of Chow et al.2 partitions a set of multivariate asset returns $y_t \in \mathbb{R}^{n}, t=1..T$ into two subsets corresponding to these two regimes as follows:

• Compute the mean vector $\mu$ of the asset returns

• Compute the covariance matrix $\Sigma$ of the asset returns

• Choose a turbulence threshold $tt$%, which represents the percentage of asset returns desired to be classified as quiet, with typical value 70%, 80%, 95%5

• Convert the turbulence threshold $tt$% into a turbulence score $ts$

• For each asset return vector $y_t, t=1..T$:

• Compute the turbulence index value $d(y_t)$
• If $d(y_t) \leq ts$, $y_t$ is classified as belonging to the quiet regime; otherwise, if $d(y_t) > ts$, $y_t$ is classified as belonging to the turbulent regime

### How to convert the turbulence threshold into a turbulence score?

The turbulence threshold $tt$% is not directly comparable to the turbulence index values $d(y_t), t=1..T$ because they are not expressed in the same units6.

As a consequence, $tt$% needs to be converted into a turbulence score $ts$.

Under the assumption that the asset returns $y_t$ follow a multivariate Gaussian distribution, and based on Property 1, this conversion can be done by thanks to the computation of the $tt$-th percentile of the chi-square distribution with $n$ degrees of freedom, that is,

$ts = \left( \mathcal{X}^2(n) \right)^{-1} (tt)$

Nevertheless, because asset returns do not follow a multivariate Gaussian distribution in practice7, this conversion will result in a proportion of asset returns classified as quiet that is different from the desired proportion.

This problem is highlighted in Chow et al.2, in which a turbulence threshold of 75% is used to separate outliers from inliers while the actual proportion of asset returns classified as quiet v.s. turbulent is equal to 79.1%.

A possible solution is to convert the turbulence threshold $tt$% into a turbulence score thanks to the computation of the $tt$-th empirical percentile of the turbulence index distribution89, with the caveat that this solution requires a long enough series of asset returns.

### Illustration

I will illustrate the method of Chow et al.2 with a simple two-asset universe made of:

• U.S. stocks (SPY ETF)
• U.S. 20+ year Treasuries (TLT ETF)

The turbulence index for this universe of assets, computed using monthly returns over the period August 2002 - January 202310, is represented in Figure 1.

Then, using for example a turbulence threshold $tt$% of 80%, converted into a turbulence score $ts$ of ~3.2211:

• All pairs of SPY-TLT returns whose turbulence index is below the red line depicted in Figure 2 are classified as belonging to the quiet regime
• All pairs of SPY-TLT returns whose turbulence index is strictly above the red line depicted in Figure 2 are classified as belonging to the turbulent regime

This partitioning of the SPY-TLT returns seems to make some sense, as several periods of market stress are identifiable within the turbulent regime: the Global Financial Crisis, the COVID-19 pandemic, the Russian invasion of Ukraine

### Implementation in Portfolio Optimizer

Portfolio Optimizer implements the method from Chow et al.2 through the endpoint /assets/returns/turbulence-partitioned, with two extensions:

• Several turbulence thresholds $tt_i$%, $i=1..m$, $m \geq 1$ can be provided, in which case the initial set of asset returns is partitioned into (at most) $m+1$ subsets12
• Turbulence threshold(s) can be converted into turbulence score(s) thanks to the computation of the empirical percentiles of the turbulence index distribution, as described in Kritzman et al.89

## Examples of application

Being able to partition asset returns into different market risk regimes has several applications.

For example, it allows to analyze the potential behavior of a given portfolio during periods of market stress, which is of utmost importance for long-term investing.

As Chow et al.2 puts it:

[a] portfolio may not survive to generate long-term performance if [it] cannot withstand exceptional periods of market turbulence.

I will not insist on this specific example, though, but I will provide one example related to portfolio optimization and one example related to the modeling of asset returns.

### Building mean-variance optimal and regime-sensitive portfolios

Under the Markowitz’s mean-variance framework, building an optimal portfolio within a universe of assets requires an estimation of the asset covariance matrix.

Because [the] typical risk-estimation procedure […] is to weight a sample [of asset returns]’ observations equally in order to estimate risk parameters2, the expected volatility of such a portfolio during periods of market stress, when asset returns typically become more volatile and more correlated, will be underestimated.

This situation is illustrated in Figure 3, taken from Chow et al.2, in the case of a universe of assets made of eight distinct asset classes13.

In this figure, it can be seen that an optimal portfolio whose asset covariance matrix is estimated from a non-partitioned set of asset returns (Full-Sample Optimal Mix) sees its volatility skyrocket during periods of market stress (Stressful environment) when compared to periods of market stability (Normal environment).

One solution to this issue is to estimate the asset covariance matrix from only the subset of asset returns corresponding to periods of market stress, but in this case, the expected portfolio return might be negatively impacted.

Indeed, as can also be seen in Figure 3, an optimal portfolio whose asset covariance matrix is estimated from only the subset of asset returns corresponding to periods of market stress (Outlier-Sample Optimal Mix) sees its expected return diminished by $\approx 1.24$% compared to the previous optimal portfolio (Full-Sample Optimal Mix).

Another solution, suggested by Chow et al2 and Kritzman et al.9, is to estimate a blended asset covariance matrix from both14:

• The subset of asset returns corresponding to periods of market stress
• The subset of asset returns corresponding to periods of market stability

More specifically, these papers suggest to estimate an asset covariance matrix $\Sigma^*$ equal to

$\Sigma^* = \lambda_{i}^* p_{i} \Sigma_{i} + \lambda_{o}^* \left( 1 - p_{i} \right) \Sigma_{o}$

, where:

• $\lambda_i^*$ (resp. $\lambda_o^*$) is the relative15 risk aversion of investors to the quiet (resp. turbulent) regime
• $p_{i}$ (resp. $1 - p_{i}$16) is the probability that the next period’s regime is quiet (resp. turbulent)
• $\Sigma_{i}$ is the quiet regime asset covariance matrix, estimated from the subset of asset returns corresponding to the quiet regime
• $\Sigma_{o}$ is the turbulent regime asset covariance matrix, estimated from the subset of asset returns corresponding to the turbulent regime

Such a blended covariance matrix enables investors to express their views about the likelihood of each risk regime and to differentiate their aversion to the regimes2.

Now, in practice, the computation of $\Sigma^*$ requires to estimate the probability $p_{i}$.

This can be done in an ad-hoc fashion, or using one’s preferred forecast technique, like the hidden Markov model used in Kritzman et al.9.

A couple of remarks to finish:

• Under the Markowitz’s mean-variance framework, building an optimal portfolio within a universe of assets also requires an estimation of the expected asset returns.

They are assumed to be regime-independent in Chow et al.2, but they could perfectly be made conditional on the regime like in Bruder et al.17.

• In the specific case where investors are equally averse to both the quiet and the turbulent regime, the formula for $\Sigma^*$ simplifies to

$\Sigma^* = p_{i} \Sigma_{i} + \left( 1 - p_{i} \right) \Sigma_{o}$

### Fitting the parameters of a multivariate Gaussian mixture distribution

It has been known since the early 1960s that the (marginal) statistical distribution of asset returns is neither normal nor lognormal7, but more than sixty years later it is still an open question in financial mathematics to determine its exact nature.

Empirically, though, it has been demonstrated that several distributions are able to capture most of the stylized facts18 of asset returns.

One such distribution is the Gaussian mixture distribution, which is a convex combination of Gaussian distributions with different means and variances in the univariate case and a a convex combination of multivariate Gaussian distributions with different mean vectors and covariance matrices in the multivariate case.

The main advantages of this distribution over other alternatives like the multivariate t distribution is that it is a non-elliptical distribution19 both numerically tractable20 and extremely flexible.

For example, a univariate Gaussian mixture distribution can be unimodal, symmetric, skewed, multimodal, leptokurtic…21.

As another example, a multivariate Gaussian mixture distribution with two components allows to approximate a multivariate jump-diffusion model driven by a standard Lévy process17.

In this context, the method of Chow et al.2 can be applied to fit the parameters of a two-component22 multivariate Gaussian mixture distribution as detailed below23:

• Set the weights of the two components respectively to the turbulence threshold $tt$%24 and its complement $1 - tt$%
• Estimate the mean vector and the covariance matrix of the multivariate Gaussian distribution with component weight $tt$% by their sample counterparts computed on the quiet subset of asset returns
• Estimate the mean vector and the covariance matrix of the multivariate Gaussian distribution with component weight $1 - tt$% by their sample counterparts computed on the turbulent subset of asset returns

In order to illustrate the validity of this approach on the two-asset SPY-TLT universe introduced in the previous section, Figure 4 through Figure 6 compare the empirical distribution of the monthly SPY (log) returns with the first marginal of:

• A multivariate Gaussian mixture distribution fitted to the pairs of SPY-TLT returns with the turbulence index-based methodology above using a turbulence threshold of 80%25

It is clearly visible on these figures that both marginal Gaussian mixture distributions are much more appropriate than the marginal Gaussian distribution to model the SPY returns, with a slightly better fit obtained with the expectation–maximization algorithm.

This is confirmed numerically by the Kolmogorov-Smirnov goodness of fit test27.

Going beyond univariate marginals, a 2D Kolmogorov-Smirnov goodness of fit test28 also confirms that both multivariate Gaussian mixture distributions are much more appropriate than the multivariate Gaussian distribution to model the joint SPY-TLT returns29, with a slightly better fit again obtained with the expectation–maximization algorithm.

This example shows that it is possible to fit the parameters of a multivariate Gaussian mixture distribution modeling joint asset returns through an easily interpretable procedure, with no local optima and no convergence issue to worry about30.

## Conclusion

This concludes this second post on the turbulence index.

As usual, feel free to connect with me on LinkedIn or follow me on Twitter to discuss about Portfolio Optimizer or quantitative finance in general.

1. For other properties, c.f. the first blog post of this series

2. The turbulent regime is called stressful in Chow et al.2

3. To be noted that $1 - tt$% is used in Chow et al.2 as the turbulence threshold, and not directly $tt$%.

4. The turbulence threshold is expressed as a percentage, while the turbulence index is expressed as a squared Mahalanobis distance.

5. I retrieved the monthly adjusted ETF prices over the period July 2002 - January 2023 using Tiingo

6. Using for example the Matlab function chi2inv, chi2inv(0.80,2) = 3.218875824868201.

7. For example, Kinlaw et al.31 use, although in a slightly different context, three subsets corresponding to the three market risk regimes calm, moderate and turbulent

8. The eight asset classes are Domestic equities, Foreign equities, Emerging market, Domestic bonds, Foreign bonds, High-yield bonds, Commodities and Cash.

9. Or from all the subsets of asset returns corresponding to all the market regimes in case more than two turbulence thresholds are used.

10. In Chow et al2, the relative risk aversion parameters are actually rescaled so that they sum to 2, that is, they must verify $\lambda_{i}^* + \lambda_{p}^* = 2$.

11. Because Chow et al2 consider only two regimes, $p_{o} = 1 - p_{i}$.

12. Because it is “just” an extension of the Gaussian model, calculations with this distribution are usually similar to those using the Gaussian distribution.

13. Or of a multivariate Gaussian mixture distribution with more than two components in case more than two turbulence thresholds are used.

14. This approach is similar in spirit to the thresholding method of Bruder et al.17

15. Or, for more robustness in case the chi-square distribution is used to convert the turbulence threshold into a turbulence score, to the actual proportion of asset returns qualified as quiet v.s. turbulent

16. This turbulence threshold results in an actual proportion of asset returns that qualify as quiet v.s. turbulent equal to ~0.83%.

17. Thanks to the Python Scikit-Learn package.

18. The Kolmogorov-Smirnov statistic (resp. p-values) for the three marginal distributions are, in order ~0.0892, ~0.0544, ~0.0527 (resp. ~0.0373, ~0.4437, ~0.4852).

19. Using the Python library https://github.com/syrte/ndtest

20. The 2-sample 2D Kolmogorov-Smirnov p-values28 are usually ~< 0.01 for the multivariate Gaussian distribution and much greater than ~0.20 for both multivariate Gaussian mixture distributions, indicating that the joint SPY-TLT returns distribution is significantly different from the former and not significantly different from any of the latter.

Tags:

Updated: