Two Continuous Random Variables Ar3 Said to Have Bivariate Normal Distribution

Multivariate Normal Distribution

J. Tacq , in International Encyclopedia of Education (Third Edition), 2010

Bivariate Normal Distribution

A special case of the multivariate normal distribution is the bivariate normal distribution with only two variables, so that we can show many of its aspects geometrically. (For more than two variables it becomes impossible to draw figures.) The probability density function of the univariate normal distribution contained two parameters: μ and σ. With two variables, say X ₁ and X ₂, the function will contain five parameters: two means μ ₁ and μ ₂, two standard deviations σ ₁ and σ ₂ and the product moment correlation between the two variables, ρ. The probability density function (pdf) of the bivariate normal distribution is given by

$f\left(x_1,x_2\right)=\frac{1}{2\pi {\sigma }_1{\sigma }_2\sqrt{1-{\rho }^2}}\text{exp}\left[-\frac{1}{2}\left(\mathbf{x}-\mathbf{\mu }\right)\prime {\mathbf{\Sigma }}^{-1}\left(\mathbf{x}-\mathbf{\mu }\right)\right].$

The constant term can be written in a more compact notation, if we notice that the determinant of the covariance matrix, |Σ|, simplifies to ${\sigma }_1^2{\sigma }_2^2\left(1-{\rho }^2\right)$ . Indeed, Σ contains on its principal diagonal the variances ${\sigma }_1^2$ and ${\sigma }_2^2$ and on its off-diagonal the covariance $\rho {\sigma }_1{\sigma }_2$ , where its determinant is equal to ${\sigma }_1^2{\sigma }_2^2-{\rho }^2{\sigma }_1^2{\sigma }_2^2={\sigma }_1^2{\sigma }_2^2\left(1-{\rho }^2\right)$ :

$\begin{array}{cc}\mathbf{\Sigma }=\left[\begin{array}{cc}{\sigma }_1^2& \rho {\sigma }_1{\sigma }_2\\ \rho {\sigma }_1{\sigma }_2& {\sigma }_2^2\end{array}\right],& \left|\mathbf{\Sigma }\right|={\sigma }_1^2{\sigma }_2^2\end{array}\left(1-{\rho }^2\right)$

Thus, the pdf of the bivariate normal distribution can also be expressed as

$f\left(x_1,x_2\right)={\left(2\pi \right)}^{-1}|\mathbf{\Sigma }{|}^{-1/2}\text{exp}\left[-\frac{1}{2}\left(\mathbf{x}-\mathbf{\mu }\right)\text{'}{\mathbf{\Sigma }}^{-1}\left(\mathbf{x}-\mathbf{\mu }\right)\right]$

We can see that this pdf displays a general bell-shaped appearance. It looks like a mountain of normal distribution curves. The surface is centered at the point (μ₁, μ₂), that is, the centroid. For each point on the bottom X ₁, X ₂ plane, we have a point f  (X ₁, X ₂) lying on the surface of the bell-shaped mountain ( Figure 1 ).

Figure 1. Joint bivariate normal density function.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780080448947013518

Some Multivariate Methods

Rand Wilcox , in Introduction to Robust Estimation and Hypothesis Testing (Fourth Edition), 2017

6.16 Multivariate Discriminate Analysis

Roughly, multivariate discriminate analysis, or classification analysis, deals with the following problem. Imagine that an individual belongs to one of G groups. For example, the groups might correspond to different diseases and the p measures might be symptoms associated with a given individual. The goal is to find an effective rule for classifying an individual as belonging to one of the G groups. The data used to determine a classification rule is typically called the training set. There are classic methods for addressing this issue (e.g., Mardia, Kent, & Bibby, 1979; Huberty, 1994) that assume multivariate normality. This section summarizes a more robust approach. For other methods worth considering, see for example Schapire and Fruend (2012) and Breiman (2001).

The basic strategy stems from Li, Cuesta-Albertos, and Liu (2012) who suggest transforming the data to some measure of depth, such as halfspace depth. For two groups, they search for the best separating polynomial based on the transformed data. Hubert, Rousseeuw, and Segaert (2015) suggest some modifications of the approach used by Li et al. First, they suggest using a distance measure that is based in part on halfspace depth, but which avoids measures of depth that become zero outside the convex hull of the data. (They focus on a particular measure of distance, which they all bagdistance.) They then use the kNN classification rule (Fix & Hodges, 1951) to classify individuals, rather then a separating polynomial, which was found to perform well and it is computationally easier to use. Briefly, for each new observation, the kNN (k nearest neighbor) rule looks up the k training data points closest to it (typically using Euclidean distance), and then assigns it to the most prevalent group among those neighbors. The value of k is typically chosen by cross-validation to minimize the misclassification rate. Here, the default measure of depth is the projection-based depth computed via the R function prodepth in Section 6.2.8 rather than the distance used by Hubert et al. Currently, an R function for computing bagdistance is not readily available. But the R function in the next section, which applies the method described here, can be used with any measure of depth, as will be seen.

For every vector x in the training data, transform it to the G-variate point

$(\mathrm{dist}(\mathbf{x},P_1),\dots ,\mathrm{dist}(\mathbf{x},P_G)),$

where $\mathrm{dist}(\mathbf{x},P_g)$ is some depth measure associated with x and based on the training data in group $P_g$ ( $g=1,\dots ,G$ ). Based on these distance measures, classify some future x based on the kNN rule. (For a possible way of improving the correct classification rate, see Croux, Joossens, & Lemmens, 2007.)

6.16.1 R Function KNNdist

The R function

KNNdist(train,test,g, k=1, prob=TRUE, plotit=FALSE, xlab='Group 1', ylab='Group 2',
depthfun=prodepth, ...)

applies the classification method described in the previous section. The argument train is a matrix with n rows and p columns that contains the training data. The argument test contains the data to be classified and the argument g, having length n, contains the labels for the training set. For example, g[1]=3 means that the first row vector in train belongs to group 3 and g[2]=1 means the second row vector in train belongs to group 1. The argument k corresponds to the k used by the kNN classification rule as described in the previous section. For two groups, plotit=TRUE will create a scatterplot of the distances. The argument depthfun indicates the distance measure that will be used, which defaults to Zou's projection-based measure of depth. The function applies the kNN rule via the R function knn, which belongs to the R library class. (For a description of the argument prob=TRUE, use the R command ?knn.)

Example

The following R commands illustrate the function KNNdist with data generated from a bivariate normal distribution:

set.seed(54)
x=rmul(100)
x[51:100,]=x[51:100,]+3
g=c(rep(1,50),rep(2,50))
test=rmul(10)
test[5:10,]=test[5:10,]+3
KNNdist(x,test,g)

That is, the training set has a sample of size $n=100$ and was generated from a bivariate normal distribution for which the first half has mean (0, 0) and the second half has mean (3, 3). The first five vectors for the test set come from the first group and the remaining five come from the second group. The output is:

[1]   1   1   1   1   2   2   2   2   2   2

This indicates that the first four vectors in the test set were classified as coming from group 1 and the remaining were classified as coming from group 2. So the fifth vector was misclassified.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128047330000068

Joint and conditional p.d.f.'s, conditional expectation and variance, moment generating function, covariance, and correlation coefficient

George G. Roussas , in An Introduction to Probability and Statistical Inference (Second Edition), 2015

4.5.2 Bivariate Normal Distribution

The joint distribution of the r.v.'s X and Y is said to be the Bivariate Normal distribution with parameters μ ₁, μ ₂ in ℜ, σ ₁, σ ₂ positive, and ρ ∈ [−1, 1], if the joint p.d.f. is given by the formula:

(50) $\begin{array}{cc}{f}_{X,Y}(x,y)=\frac{1}{2\pi {\sigma }_1{\sigma }_2\sqrt{1-{\rho }^2}}{\text{e}}^{-q/2},& x,y\in \Re ,\end{array}$

where

(51) $q=\frac{1}{1-{\rho }^2}\left[{\left(\frac{x-{\mu }_1}{{\sigma }_1}\right)}^2-2\rho \left(\frac{x-{\mu }_1}{{\sigma }_1}\right)\left(\frac{y-{\mu }_2}{{\sigma }_2}\right)+\left(\frac{y-{\mu }_2}{{\sigma }_2}\right)\right].$

This distribution is also referred to as two-dimensional Normal. The shape of f _X,Y looks like a bell sitting on the xy-plane and whose highest point is located at the point $({\mu }_1,{\mu }_2,1/(2\pi {\sigma }_1{\sigma }_2\sqrt{1-{\rho }^2}))$ (see Figure 4.7).

Figure 4.7. Graphs of the p.d.f. of the Bivariate Normal distribution: (a) Centered at the origin; (b) Centered elsewhere in the (x, y)-plane.

That f _X,Y integrates to 1 and therefore is a p.d.f. is seen by rewriting it in a convenient way. Specifically,

(52) $\begin{array}{l}{\left(\frac{x-{\mu }_1}{{\sigma }_1}\right)}^2-2\rho \left(\frac{x-{\mu }_1}{{\sigma }_1}\right)\left(\frac{y-{\mu }_2}{{\sigma }_2}\right)+{\left(\frac{y-{\mu }_2}{{\sigma }_2}\right)}^2\hfill \\ ={\left(\frac{y-{\mu }_2}{{\sigma }_2}\right)}^2-2\left(\rho \frac{x-{\mu }_1}{{\sigma }_1}\right)\left(\frac{y-{\mu }_2}{{\sigma }_2}\right)+{\left(\rho \frac{x-{\mu }_1}{{\sigma }_1}\right)}^2+(1-{\rho }^2){\left(\frac{x-{\mu }_1}{{\sigma }_1}\right)}^2\hfill \\ ={\left[\left(\frac{y-{\mu }_2}{{\sigma }_2}\right)-\left(\rho \frac{x-{\mu }_1}{{\sigma }_1}\right)\right]}^2+(1-{\rho }^2){\left(\frac{x-{\mu }_1}{{\sigma }_1}\right)}^2.\hfill \end{array}$

Furthermore,

$\begin{array}{ll}\frac{y-{\mu }_2}{{\sigma }_2}-\rho \frac{x-{\mu }_1}{{\sigma }_1}\hfill & =\frac{y-{\mu }_2}{{\sigma }_2}-\frac{1}{{\sigma }^2}\times \rho {\sigma }_2\frac{x-{\mu }_1}{{\sigma }_1}\hfill \\ \hfill & =\frac{1}{{\sigma }_2}\left\{y-\left[{\mu }_2+\frac{\rho {\sigma }_2}{{\sigma }_1}(x-{\mu }_1)\right]\right\}\hfill \\ \hfill & =\frac{y-b_x}{{\sigma }_2},\text{where}{b}_x={\mu }_2+\frac{\rho {\sigma }_2}{{\sigma }_1}(x-{\mu }_1)\hfill \end{array}$

(see also Exercise 5.6).

Therefore, the right-hand side of (52) is equal to:

${\left(\frac{y-b_x}{{\sigma }_2}\right)}^2+(1-{\rho }^2){\left(\frac{x-{\mu }_1}{{\sigma }_1}\right)}^2,$

and hence the exponent becomes

$-\frac{{(x-{\mu }_1)}^2}{2{\sigma }_1^2}-\frac{{(y-b_x)}^2}{2{({\sigma }_2\sqrt{1-{\rho }^2})}^2}.$

Then the joint p.d.f. may be rewritten as follows:

(53) $f_{X,Y}(x,y)=\frac{1}{\sqrt{2\pi }{\sigma }_1}{\text{e}}^{-{(x-{\mu }_1)}^2/2{\sigma }_1^2}\times \frac{1}{\sqrt{2\pi }({\sigma }_2\sqrt{1-{\rho }^2})}{\text{e}}^{-{(y-b_x)}^2/2{({\sigma }_2\sqrt{1-{\rho }^2})}^2}.$

The first factor on the right-hand side of (53) is the p.d.f. of $N({\mu }_1,{\sigma }_1^2)$ and the second factor is the p.d.f. of $N(b_x,{({\sigma }_2\sqrt{1-{\rho }^2})}^2)$ . Therefore, integration with respect to y produces the marginal $N({\mu }_1,{\sigma }_1^2)$ distribution, which, of course, integrates to 1. So, we have established the following two facts: ${\displaystyle {\int }_{-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }{f}_{X,Y}(x,y)\text{d}x\text{d}y}}=1$ , and

(54) $\begin{array}{cc}X\sim N\left({\mu }_1,{\sigma }_1^2\right),& \text{and,}\text{by}\text{symmetry,}Y\sim N\left({\mu }_2,{\sigma }_2^2\right).\end{array}$

The results recorded in (54) also reveal the special significance of the parameters ${\mu }_1,{\sigma }_1^2$ and ${\mu }_2,{\sigma }_2^2$ . Namely, they are the means and the variances of the (Normally distributed) r.v.'s X and Y, respectively. Relations (53) and (54) also provide immediately the conditional p.d.f. f _{Y | X} ; namely,

$f_{Y|X}\left(y|x\right)=\frac{1}{\sqrt{2\pi }{({\sigma }_2\sqrt{1-{\rho }^2})}^2}\exp \left[-\frac{{(y-b_x)}^2}{2{({\sigma }_2\sqrt{1-{\rho }^2})}^2}\right].$

Thus, in obvious notation:

(55) $\begin{array}{cc}Y|X=x\sim N(b_x,{({\sigma }_2\sqrt{1-{\rho }^2})}^2),& b_x={\mu }_2+\frac{\rho {\sigma }_2}{{\sigma }_1}(x-{\mu }_1),\end{array}$

and, by symmetry:

(56) $\begin{array}{cc}X|Y=y\sim N(b_y,{({\sigma }_1\sqrt{1-{\rho }^2})}^2),& b_y={\mu }_1+\frac{\rho {\sigma }_1}{{\sigma }_2}(y-{\mu }_2).\end{array}$

In Figure 4.8, the conditional p.d.f. f _Y|X (∙ | x) is depicted for three values of x : x = 5,10, and 15.

Figure 4.8. Conditional probability density functions of the bivariate Normal distribution.

Formulas (53), (54), and (56) also allow us to calculate easily the covariance and the correlation coefficient of X and Y. Indeed, by (53):

$\begin{array}{ll}E(XY)\hfill & ={\displaystyle {\int }_{-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }xy{f}_{X,Y}(x,y)\text{d}x\text{d}y}={\displaystyle {\int }_{-\infty }^{\infty }x{f}_X(x)}\left[{\displaystyle {\int }_{-\infty }^{\infty }y{f}_{Y|X}(y|x)\text{d}y}\right]}\text{d}x\hfill \\ \hfill & ={\displaystyle {\int }_{-\infty }^{\infty }x{f}_X(x)b_x\text{d}x}={\displaystyle {\int }_{-\infty }^{\infty }x{f}_X(x)}\left[{\mu }_2+\frac{\rho {\sigma }_2}{{\sigma }_1}(x-{\mu }_1)\right]\text{d}x\hfill \\ \hfill & ={\mu }_1{\mu }_2+\rho {\sigma }_1{\sigma }_2\hfill \end{array}$

(see also Exercise 5.7). Since we already know that EX = μ ₁, EY = μ ₂, and $\text{Var}(X)={\sigma }_1^2,\text{Var}(Y)={\sigma }_2^2$ , we obtain:

$\text{Cov}(X,Y)=E(XY)-(EX)(EY)={\mu }_1{\mu }_2+\rho {\sigma }_1{\sigma }_2-{\mu }_1{\mu }_2=\rho {\sigma }_1{\sigma }_2,$

and therefore $\rho (X,Y)=\frac{\rho {\sigma }_1{\sigma }_2}{{\sigma }_1{\sigma }_2}=\rho$ . Thus, we have:

(57) $\begin{array}{ccc}\text{Cov}(X,Y)=\rho {\sigma }_1{\sigma }_2& \text{and}& \rho (X,Y)=\rho .\end{array}$

Relation (57) reveals that the parameter ρ in (50) is, actually, the correlation coefficient of the r.v.'s X and Y.

Example 27

If the r.v.'s X ₁ and X ₂ have the Bivariate Normal distribution with parameters ${\mu }_1,{\mu }_2,{\sigma }_1^2,{\sigma }_2^2$ , and ρ:

(i)

Calculate the quantities: E(c ₁ X ₁ + c ₂ X ₂), Var(c ₁ X ₁ + c ₂ X ₂), where c ₁, c ₂ are constants.

(ii)

How the expression in part (i) becomes for: ${\mu }_1=-1,{\mu }_2=3,{\sigma }_1^2=4,{\sigma }_2^2=9$ , and $\rho =\frac{1}{2}$ ?

Discussion

(i)

E(c ₁ X ₁ + c ₂ X ₂) = c ₁ EX ₁ + c ₂ EX ₂ = c ₁ μ ₁ + c ₂ μ ₂, since $X_i\sim N({\mu }_i,{\sigma }_i^2)$ , so that EX _i = μ _i , i= 1, 2. Also,

$\begin{array}{ll}\text{Var}(c_1{X}_1+c_2{X}_2)\hfill & =c_1^2{\sigma }_{X_1}^2+c_2^2{\sigma }_{X_2}^2+2c_1{c}_2{\sigma }_{X_1}{\sigma }_{X_2}\rho (X_1,X_2)(\text{by}\text{39})\hfill \\ \hfill & =c_1^2{\sigma }_1^2+c_2^2{\sigma }_2^2+2c_1{c}_2{\sigma }_1{\sigma }_2\rho ,\hfill \end{array}$

since $X_i~N({\mu }_i,{\sigma }_i^2)$ , so that $\text{Var}(X_i)={\sigma }_i^2,i=1,2$ , and ρ(X ₁,X ₂) = ρ, by (57).

(ii)

Here, E(c ₁ X ₁ + c ₂ X ₂) = −c ₁ + 3c ₂, and $\text{Var}(c_1{X}_1+c_2{X}_2)=4c_1+9c_2+2c_1{c}_2\times 2\times 3\times \frac{1}{2}=4c_1+9c_2+6c_1{c}_2$ .

Example 28

Suppose that the heights of fathers and sons are r.v.'s X and Y, respectively, having (approximately) Bivariate Normal distribution with parameters (expressed in inches) μ ₁ = 70, σ ₁ = 2, μ ₂ = 71, σ ₂ = 2 and ρ = 0.90. If for a given pair (father, son) it is observed that X = x = 69, determine:

(i)

The conditional distribution of the height of the son.

(ii)

The expected height of the son.

(iii)

The probability of the height of the son to be more than 72 in.

Discussion

(i)

According to (55), $Y|X=x\sim N(b_x,{({\sigma }_2\sqrt{1-{\rho }^2})}^2),$ where

$\begin{array}{ll}\hfill b_x& ={\mu }_2+\frac{\rho {\sigma }_2}{{\sigma }_1}(x-{\mu }_1)=71+0.90\times (69-70)=70.1,\text{and}\hfill \\ {\sigma }_2\sqrt{1-{\rho }^2}\hfill & =2\times \sqrt{1-{0.90}^2}\simeq \mathrm{0.87.}\hfill \end{array}$

That is, YǀX = 69 is distributed as N(70.1, (0.87)²).

(ii)

The (conditional) expectation of Y, given X = 69, is equal to b ₆₉ = 70.1.

(iii)

The required (conditional) probability is

$\begin{array}{ll}P(Y>72|X=69)\hfill & =P\left(\frac{Y-b_{69}}{{\sigma }_2\sqrt{1-{\rho }^2}}>\frac{72-70.1}{0.87}\right)\simeq P(Z>2.18)\hfill \\ \hfill & =1-\Phi (2.18)=1-0.985371=\mathrm{0.014629.}\hfill \end{array}$

Finally, it can be seen by integration that the joint m.g.f. of X and Y is given by the formula:

(58) $\begin{array}{cc}{M}_{X,Y}(t_1,t_2)=\exp \left[{\mu }_1{t}_1+{\mu }_2{t}_2+\frac{1}{2}\left({\sigma }_1^2{t}_1^2+2\rho {\sigma }_1{\sigma }_2{t}_1{t}_2+{\sigma }_2^2{t}_2^2\right)\right],& t_1{t}_2\in \Re ;\end{array}$

we choose not to pursue its justification (which can be found, e.g., in pages 158–159, in the book A Course in Mathematical Statistics, 2nd edition (1997), Academic Press, by G.G. Roussas). We see, however, easily that:

$\frac{\partial }{\partial t_1}{M}_{X,Y}(t_1,t_2)=\left({\mu }_1+{\sigma }_1^2{t}_1+\rho {\sigma }_1{\sigma }_2{t}_2\right)M_{X,Y}(t_1,t_2),$

and hence:

$\begin{array}{ll}\frac{{\partial }^2}{\partial t_1\partial t_2}{M}_{X,Y}(t_1,t_2)=\hfill & \rho {\sigma }_1{\sigma }_2{M}_{X,Y}(t_1,t_2)+\left({\mu }_1+{\sigma }_1^2{t}_1+\rho {\sigma }_1{\sigma }_2{t}_2\right)\hfill \\ \hfill & \times \left({\mu }_2+{\sigma }_2^2{t}_2+\rho {\sigma }_1{\sigma }_2{t}_1\right)M_{X,Y}(t_1,t_2),\hfill \end{array}$

which, evaluated at t ₁ = t ₂ = 0, yields ρσ ₁ σ ₂ + μ ₁ μ ₂ = E(XY), as we have already seen.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128001141000044

Assessing structural relationships between distributions - a quantile process approach based on Mallows distance

G. Freitag , ... M. Vogt , in Recent Advances and Trends in Nonparametric Statistics, 2003

4.2 Acceleration model

For this model we used the symmetrized version of the test statistics, as indicated in (6) . Several bivariate normal distribution settings were considered, and each test was performed with Δ ² ₀  =   1. We found that in general trimming improves the performance of the BC _a test. For smaller variances, it turns out to be liberal in case of no trimming and conservative in case of trimming. For larger variances, the BC _a method is always liberal, although trimming reduces the liberality. It was seen that an increase in location difference (while keeping the true distance Γ _A,β (F, G) constant) results in more liberal tests for β =   0, and in less liberal tests for β =   0.05.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780444513786500090

Ratio and Product Type Exponential Estimators for Population Mean Using Ranked Set Sampling

Gajendra K. Vishwakarma , ... Carlos N. Bouza-Herrera , in Ranked Set Sampling, 2019

18.4 A Simulation Study

To illustrate how one can gain an insight into the application or properties of the proposed estimator, a computer simulation was conducted. Bivariate random observations were generated from a bivariate normal distribution with parameters ${\mu }_y$ , ${\mu }_x$ , ${\sigma }_x$ , ${\sigma }_y$ and correlation coefficient $\rho$ . The sampling method explained above is used to pick RSS data with sets of size $m$ and after $r$ repeated cycles to get an RSS of size $\mathit{mr}$ . A sample of size $\mathit{mr}$ bivariate units is randomly chosen from the population (we refer to these data as SRS data). The simulation was performed with $m$ = 3, 4, 5 and with $\mathit{r}$ = 3 and 6 (i.e., with total sample sizes of 9, 12, 15, 18, 24, and 30) for the RSS and SRS data sets. Here, we have ranked the auxiliary variate X which induces ranking in study variate Y (ranking on Y will be perfect if $\rho =1$ or will be with errors in ranking if $\rho <1$ ). Using R software we have conducted 5,000 replications for estimates of the means and mean square errors. The results of these simulations are summarized by the percentage relative efficiencies of the estimators using the formula.

(18.28) $\mathrm{PRE}\left[{*,{\hat{\overline{Y}}}_R}^{\mathrm{rss}}\right]=\frac{\mathrm{MSE}\left({\hat{\overline{Y}}}_R^{\mathrm{rss}}\right)}{\mathrm{MSE}\left(*\right)}\times 100$

(18.29) $\mathrm{PRE}\left[*,{{\hat{\overline{Y}}}_P}^{\mathrm{rss}}\right]=\frac{\mathrm{MSE}\left({\hat{\overline{Y}}}_P^{\mathrm{rss}}\right)}{\mathrm{MSE}\left(*\right)}\times 100$

where, $*={\hat{\overline{Y}}}_{\mathit{Re}}^{\mathrm{rss}},{\hat{\overline{Y}}}_{\mathit{Pe}}^{\mathrm{rss}},{\hat{\overline{Y}}}_G^{\mathrm{rss}}$ .

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128150443000186

Some Generalizations to k Random Variables, and Three Multivariate Distributions

George Roussas , in Introduction to Probability (Second Edition), 2014

9.4 Multivariate Normal Distribution

This chapter is concluded with a brief reference to the multivariate normal distribution without entering into any details. A relevant reference is given for the interested reader.

The multivariate normal distribution is the generalization of the bivariate normal distribution and can be defined in a number of ways; we choose the one given here. To this end, for $k\ge 2$ , let $\mathit{\mu }=({\mu }_1\text{,}\dots \text{,}{\mu }_k)$ be a vector of constants, and let $\mathbf{\Sigma }$ be a $k\times k$ nonsingular matrix, so that the inverse ${\mathbf{\Sigma }}^{-\mathbf{1}}$ exists and the determinant $\mid \mathbf{\Sigma }\mid \ne 0$ . Finally, set $\mathbf{X}$ for the vector of r.v.'s $X_1\text{,}\dots \text{,}{X}_k$ ; i.e., $\mathbf{X}=({\mathrm{X}}_1\text{,}\dots \text{,}{\mathrm{X}}_{\mathrm{k}})$ and $\mathbf{x}=({\mathrm{x}}_1\text{,}\dots \text{,}{\mathrm{x}}_{\mathrm{k}})$ for any point in ${\mathfrak{R}}^k$ . Then, the joint p.d.f. of the $X_i$ 's, or the p.d.f. of the random vector $\mathbf{X}$ , is said to be multivariate normal, or k-variate normal, if it is given by the formula:

$f_{\mathbf{X}}(\mathbf{x})=\frac{1}{{(2\pi )}^{k/2}\mid \mathbf{\Sigma }{\mid }^{1/2}}\exp \left[-\frac{1}{2}(\mathbf{x}-\mathit{\mu }){\mathbf{\Sigma }}^{-1}{(\mathbf{x}-\mathit{\mu })}^{\prime }\right]\text{,}\mathbf{x}\in {\mathfrak{R}}^k\text{,}$

where, it is to be recalled that " $\prime$ " stands for transpose.

It can be seen that ${\mathit{EX}}_i={\mu }_i\text{,}\mathit{Var}(X_i)={\sigma }_i^2$ is the $(i\text{,}i)$ th element of $\mathbf{\Sigma }$ , and $\mathit{Cov}(X_i\text{,}{X}_j)$ is the $(i\text{,}j)$ th element of $\mathbf{\Sigma }$ , so that $\mathit{\mu }=({\mathit{EX}}_1\text{,}\dots \text{,}{\mathit{EX}}_k)$ and $\mathbf{\Sigma }=(\mathit{Cov}(X_i\text{,}{X}_j))\text{,}i\text{,}j=1\text{,}\dots \text{,}k$ . The quantities $\mathit{\mu }$ and $\mathbf{\Sigma }$ are called the parameters of the distribution. It can also be seen that the joint m.g.f. of the $X_i$ 's, or the m.g.f. of the random vector $\mathbf{X}$ , is given by:

$M_{\mathbf{X}}(\mathbf{t})=\exp \left(\mathit{\mu }{\mathbf{t}}^{\prime }+\frac{1}{2}\mathbf{t}\mathbf{\Sigma }{\mathbf{t}}^{\prime }\right)\text{,}\mathbf{t}\in {\mathfrak{R}}^k\text{.}$

The k-variate normal distribution has properties similar to those of the 2-dimensional normal distribution, and the latter is obtained from the former by taking $\mathit{\mu }=({\mu }_1\text{,}{\mu }_2)$ and $\mathbf{\Sigma }=\left(\begin{array}{cc}\hfill {\sigma }_1^2\hfill & \hfill \rho {\sigma }_1{\sigma }_2\hfill \\ \hfill \rho {\sigma }_1{\sigma }_2\hfill & \hfill {\sigma }_2^2\hfill \end{array}\right)$ , where $\rho =\rho (X_1\text{,}{X}_2)$ .

More relevant information can be found, for example, in Chapter 18 of the reference cited in Exercise 3.4.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128000410000092

Quality of Analytical Measurements: Univariate Regression

M.C. Ortiz , ... L.A. Sarabia , in Comprehensive Chemometrics, 2009

1.05.4.4 Confidence Interval for the Prediction

For a concentration x ₀ where no experiments were carried out, the response is estimated by

(57) ${\hat{y}}_0=b_0+b_1{x}_0$

However, as b ₀ and b ₁ are random variables that follow a bivariate normal distribution, ${\hat{y}}_0$ is also a random variable that follows a normal distribution whose mean and variance as computed with Equations (11)–(15) are

(58) $E\left({\hat{y}}_0\right)={\beta }_0+{\beta }_1{x}_0$

(59) $\text{Var}\left({\hat{y}}_0\right)=\left(\frac{1}{N}+\frac{{\left(x_0-\overline{x}\right)}^2}{{\displaystyle {\sum }_{i=1}^N{\left(x_i-\overline{x}\right)}^2}}\right){\sigma }^2$

$\text{Var}\left({\hat{y}}_0\right)$ has a minimum when $x_0=\overline{x}$ and increases as x ₀ separates from $\overline{x}$ in either direction. In other words, the most precise prediction is expected to be in the mean of the calibration range and precision is lost as we separate from $\overline{x}$ .

The confidence interval at level (1   −   α)100% for the true mean value of the response for a given x ₀ is then computed as

(60) $\begin{array}{c}\left({\hat{y}}_0-t_{\alpha /2,N-2}{s}_{y_0},{\hat{y}}_0+t_{\alpha /2,N-2}{s}_{y_0}\right)\\ s_{y_0}=\sqrt{\widehat{\text{Var}}\left({\hat{y}}_0\right)}=s_{\text{<mi mathvariant="italic" is="true">yx</mi>}}\sqrt{\frac{1}{N}+\frac{{\left(x_0-\overline{x}\right)}^2}{{\displaystyle {\sum }_{i=1}^N{\left(x_i-\overline{x}\right)}^2}}}\end{array}$

If we join up all the lower endpoints and all the upper endpoints of intervals defined in Equation (60) as x ₀ changes, we would obtain the two dotted hyperbolas as shown in Figure 7 for the data in Example 1, Table 3 .

Figure 7. Confidence intervals at 95% for the calibration data of Example 1, Table 3 . Dotted hyperbolas are for the true mean. Continuous hyperbolas are for a new prediction with q  =   1 in Equation (63). The squares are the experimental points.

The individual values of the random variable ${\hat{y}}_0$ are distributed around its mean with variance σ² independent of $\text{Var}\left({\hat{y}}_0\right)$ . Therefore, the variance of the prediction of an individual observation would be

(61) ${\sigma }^2\left(1+\frac{1}{N}+\frac{{\left(x_0-\overline{x}\right)}^2}{{\displaystyle {\sum }_{i=1}^N{\left(x_i-\overline{x}\right)}^2}}\right)$

and the corresponding estimated value is obtained by substituting σ² by $s_{yx}^2$ in Equation (61).

The confidence interval for a new observation is then

(62) ${\hat{y}}_0\pm t_{\alpha /2,N-2}{s}_{\text{<mi mathvariant="italic" is="true">yx</mi>}}\sqrt{1+\frac{1}{N}+\frac{{\left(x_0-\overline{x}\right)}^2}{{\displaystyle {\sum }_{i=1}^N{\left(x_i-\overline{x}\right)}^2}}}$

A confidence interval for the average of q new observations is obtained similarly as follows:

(63) ${\hat{y}}_0\pm t_{\alpha /2,N-2}{s}_{\text{<mi mathvariant="italic" is="true">yx</mi>}}\sqrt{\frac{1}{q}+\frac{1}{N}+\frac{{\left(x_0-\overline{x}\right)}^2}{{\displaystyle {\sum }_{i=1}^N{\left(x_i-\overline{x}\right)}^2}}}$

Again, by joining the corresponding endpoints of the confidence intervals for one new observation, we obtain two hyperbolas (those drawn with a continuous line in Figure 7 ). The rest of the hyperbolas that would be obtained for q  >   1 would be located between the two 'limiting' hyperbolas (the dotted and the continuous lines in Figure 7 ).

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780444527011000910

Correlation and Tests of Independence

Rand Wilcox , in Introduction to Robust Estimation and Hypothesis Testing (Third Edition), 2012

9.4.3 The OP Correlation

The so-called OP correlation coefficient begins by eliminating any outliers using the projection method in Section 6.4.9. Then it merely computes some correlation coefficient with the data that remain. Pearson's correlation is assumed unless stated otherwise.

Imagine that data are randomly sampled from some bivariate normal distribution. If the goal is to use a skipped correlation coefficient that gives a reasonably accurate estimate of Pearson's correlation, ρ, relative to r, then the OP estimator is the only skipped estimator known to be reasonably satisfactory.

Let r_p represent the skipped correlation coefficient and let m be the number of pairs of points left after outliers are removed. A seemingly simple method for testing the hypothesis of independence is to apply the usual T test for Pearson's correlation but with r replaced by r_p and n replaced by m. But this simple solution fails because it does not take into account the dependence among the points remaining after outliers are removed. If this problem is ignored, unsatisfactory control over the probability of a type I error results (Wilcox, 2010f). However, let

$T_p=r_p\sqrt{\frac{n-2}{1-r_p^2}}$

and suppose the hypothesis of independence is rejected at the α = 0.05 level if |T_p | ≥ c, where

$c=\frac{6.947}{n}+\mathrm{2.3197.}$

The critical value c was determined via simulations under normality by determining an appropriate critical value for n ranging between 10 and 200, and then a least squares regression line was fit to the data. For nonnormal distributions, all indications are that this hypothesis testing method has an actual type I error probability reasonably close to the nominal 0.05 level.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123869838000093

Copulas and quasi-copulas: An introduction to their properties and applications

Roger B. Nelsen , in Logical, Algebraic, Analytic and Probabilistic Aspects of Triangular Norms, 2005

14.3.3 Normal copulas

Let N _ρ(x,y) denote the standard bivariate normal joint distribution function with correlation coefficient ρ. Then C _ρ, the copula corresponding to N _ρ, is given by C _ρ(u,v)   = N _ρ(Φ^{−   1}(u),Φ^{−   1} (v)) [where Φ denotes the standard normal distribution function]. Since there is no closed form expression for Φ^{−   1}, there is no closed form expression for N _ρ. However, N _ρ can be evaluated approximately in order to construct bivariate distribution functions with the same dependence structure as the standard bivariate normal distribution function but with non-normal marginals.

14.3.1 Definition

For a pair X,Y of random variables with marginal distribution functions F,G, respectively, and joint distribution function H, the corresponding marginal survival functions $\overline{F},\overline{G}$ , and joint survival function $\overline{H}$ are given by $\overline{F}\left(x\right)=P\left[X>x\right],\overline{G}\left(y\right)=p\left[y>y\right]$ and $\overline{H}\left(x,y\right)=p\left[X>x,Y>y\right]$ , respectively. The function Ĉ which couples the joint survival function to its marginal survival functions is called a survival copula:

$\overline{H}\left(x,y\right)=\widehat{C}\left(\overline{F}\left(x\right),\overline{G}\left(y\right)\right).$

It is easy to show that Ĉ is a copula, and is related to the (ordinary) copula C of X and Y via the equation Ĉ(u,v)   = u  + v  −   1   + C(1   − u,1   − v). See [29] for details.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780444518149500148

Markov Chain Monte Carlo

William L. Dunn , J. Kenneth Shultis , in Exploring Monte Carlo Methods, 2012

Example 6.3 A Modified Buffon's Needle Problem

Consider a modified Buffon's needle game in which needles of length L are dropped by a biased machine. In the original Buffon's needle problem, the PDF for the position of the needle endpoints is f (x) = 1/D, 0 ≤ x ≤ D, the PDF for the orientation angle is g(θ) = 1/π, 0 ≤ θ ≤ π, and P_cut is given by Eq. (1.7). The biased machine, however, drops needles such that the endpoints x and the orientation angles θ are distributed, within a unit cell, according to a bivariate normal distribution (see Appendix A.3.1), i.e.,

$f(x,\theta )=\frac{K}{2\pi {\sigma }_1{\sigma }_2\sqrt{1-{\rho }^2}}exp\left[-\frac{z(x,\theta )}{2(1-{\rho }^2)}\right],0\le x<D,0\le \theta <\pi ,$

where K is a normalization constant,

$z(x,\theta )=\frac{{(x-{\mu }_1)}^2}{{\sigma }_1^2}-\frac{2\rho (x-{\mu }_1)(\theta -{\mu }_2)}{{\sigma }_1{\sigma }_2}+\frac{{(\theta -{\mu }_2)}^2}{{\sigma }_2^2},$

and

$\rho =\frac{{\sigma }_{12}}{{\sigma }_1{\sigma }_2}.$

Employing an analysis similar to that taken in Example 1.1, the cut probability for this machine dropping needles on a grid with D ≥ L can be expressed as

$P_{cut}={\int }_0^{\pi }{\int }_0^D\frac{Lsin\theta }{D}f(x,\theta )dxd\theta .$

How might one estimate this cut probability for a given set of parameters?

Consider the case where D = 5, L = 2, μ₁ = 3, μ₂ = 1, σ₁ = 1, σ₂ = 1, and σ₁₂ = 0.1. One could drop n needles into the machine and observe k crossings on the grid below, and approximate P_cut as k/n. However, it is much more efficient to run an experiment on a computer using MCMC.

Choose as proposal distributions functions that are easy to sample, say h ₁(u) = 1/D and h ₂(v) = 1/π. Then, sample u and v from these proposal functions. In this case, the Metropolis ratio is simply

$R=\frac{f(u,v)}{f(x,\theta )}$

because the proposals are both constants. After running 10⁶ histories an estimate of P_cut = 0.2837 ± 0.0001 was obtained (a value statistically slightly higher than for uniform drops). Notice that the two-dimensional integral for P_cut is evaluated without having to sample from the bivariate normal distribution or having to determine the normalization constant K. All that is required is that the ratio of bivariate normals be evaluated at both current and candidate points sampled from the simple proposal functions.

Results generated by global MCMC sampling from a multidimensional distribution are shown in Figs. 6.9 to 6.11, in which a bivariate normal PDF (see Appendix A.3.1) is sampled by MCMC for three different values of N. The large spikes in the tails evident for 10³ histories are "softened" by 10⁵ histories and almost completely removed by N = 10⁷ histories.

Figure 6.9. Bivariate normal PDF sampled using MCMC after 10³ histories.

Figure 6.10. Bivariate normal PDF sampled using MCMC after 10⁵ histories.

Figure 6.11. Bivariate normal PDF sampled using MCMC after 10⁷ histories.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780444515759000063

meyertistonedide.blogspot.com

Source: https://www.sciencedirect.com/topics/mathematics/bivariate-normal-distribution

Share this post