Thursday, April 05, 2007

Geometry of correlated Gaussians

In class we were discussing the geometry of ρ-correlated, n-dimensional Gaussians; call them g, h.

We have that g2=i=1ngi2. This is the sum of n very well-behaved independent random variables with mean 1 and variance 2, so its distribution value will be n±O(n) with very high probability. (To be more accurate, say n±O(nlogn) with probability at least 1-n-O(1).) Hence g will be n(1+O(1/n) with high probability. Since h is also distributed as a standard n-dimensional Gaussian, the same is true of h.

Now imagine g is fixed and we choose h to be ρ-correlated to g. Then gh=i=1ngihi=i=1n(ρgi2+1-ρ2gigiʹ), where the giʹ random variables are independent standard normals. I.e., the dot product is ρg2 plus an independent one-dimensional normal, 1-ρ2N(0,igi2). Since g2=igi2=n±O(n) with high probability, the dot product is ρn±O((ρ+1-ρ2)n) with high probability.

So the cosine of the angle between g and h will be (ρn±O(n))/(n±O(n))=ρ±O(1/n), and hence the angle will be arccosρ±O(1/n) with high probability (assuming ρ is treated as a constant in (0,1)).

Now this doesn't really yet prove that g and h are distributed like a random pair on the surface of the n-radius sphere with angle arccosρ. We just know that their angle will be arccosρ and they will both be on the sphere. We should really look at the distribution of h-ρg. But this is just a scaled n-dimensional Gaussian, so its distribution is spherically symmetric. I guess this more or less justifies the overall claim.
Actually, I believe that h-ρg will be close to orthogonal to ρg with high probability. The dot product between the two will be distributed like 1-ρ2N(0,igi2), which will probably be on the order of 1-ρ2n. However the product of the lengths of the two vectors will be like ρ1-ρ2n. So the cosine of the angle between them will be like 1/(ρn), and the angle will be close to 90 degrees.

No comments: