Basics of Multivariate Student's t distribution
I am a fan of the Student's t distribution -- it is almost as easy to handle as the normal distribution but has an additional flexibility with respect to the heaviness of the tails. In fact, the normal distribution is a special case and so is the Cauchy distribution.
The density of a (non-degenerate) multivariate normal distribution with zero mean is given by
\[ \frac{1}{\sqrt{(2\pi)^n |\Sigma|}} \exp \Big( -\frac{1}{2} x^\top \Sigma^{-1} x \Big), \]
where \(\Sigma\) is a (positive-semidefinite) \(n\times n\) covariance matrix.
For comparison, the density of the multivariate student t distribution with zero mean is
\[ \frac{\Gamma[(\nu + n)/2]}{\Gamma(\nu/2) \sqrt{v^{n} \pi^{n} |\Sigma|}} \Big[ 1 + \frac{1}{\nu} x^\top \Sigma^{-1} x \Big]^{-(\nu + n)/2}. \]
Compare this with the univariate version of the t density:
\[ \frac{\Gamma{(\nu +1)/2}}{\Gamma(\nu/2)\sqrt{\nu\pi}} \Big[ 1 + \frac{x^2}{2} \Big]^{-(\nu+1)/2}.\]
In both cases, \(\nu>2\) is the degrees of freedom parameter.
A standard way to construct a random variable \(Z\) with t-distribution is to use a normal random variable \(X\) and an independent chi-square random variable \(Y\) and set
\[ Z = \frac{X}{\sqrt{Y/\nu}}.\]
This formula works for both the univariate and the multivariate case and can be used to draw samples from the t distribution using independent samples of \(X\) and \(Y\). Also, we get
\[ \text{Cov} Z = \frac{\nu}{\nu-2} \text{Cov} X. \]
This is because the univariate \(\chi^2\) distribution with \(\nu\) degrees of freedom is really a \(\text{Gamma}(\frac{\nu}{2}, \frac{1}{2})\) distribution. This can be seen by comparing a \(\chi^2\) density with the \(\text{Gamma}(\alpha, \beta)\) density
\[ p_{\alpha,\beta}(x) = \frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x} 1_{[0, \infty)}(x).\]
The parameters \(\alpha\) and \(\beta\) are both positive. Moreover, \(1/Y\) is inverse-gamma distributed and thus \(\mathbb E \frac{1}{Y} = \frac{1}{\nu -2}.\)
References
- Michael Roth. On the Multivariate t Distribution. pdf
- Gregory Grundsen. A Python Implementation of the Multivariate t-distribution. (This code is now a part of sklearn)
- Wikipedia contributors, "Inverse-gamma distribution," Wikipedia, The Free Encyclopedia (accessed January 9, 2021).
- Wikipedia entry about the Inverse-gamma distribution
- Liu, Chuanhai, and Donald B. Rubin. "ML estimation of the t distribution using EM and its extensions, ECM and ECME." Statistica Sinica (1995): 19-39. pdf
- Nadarajah, Saralees, and Samuel Kotz. "Estimation methods for the multivariate t distribution." Acta Applicandae Mathematicae 102, no. 1 (2008): 99-118. pdf
- Martin Haugh. "The EM Algorithm." 2015. pdf