Correlation

Section 2.3 Correlation

You can plot points and plot the resulting best-fit line determined in the previous section but the question remains whether the line is any good. In particular, the real use of the line often is to subsequently predict y-values for a given x-value. However, it is very likely that the best-fit line does not even pass through any of the provided data points. So, how can something that misses every marker still be considered a good fit. To quantify this, we first need to discuss a way to measure how well two variables vary with each other.

🔗

Checkpoint 2.3.1.

Determine the best-fit line for the following data:

(0, 1), (1, 1), (1, 0), (0, 0)

and then again for

(0, 1), (1, 1), (1, 2), (2, 2) .

Plot both data sets and plot their corresponding best-fit lines.

Comment on how well each line seems to describe the data sets.
Might you be more likely to use the best-fit line to predict an outcome when the first coordinate is 1/2? Why or why not?
Are you willing to say one line is "better" than the other?

🔗

Definition 2.3.2. Sample Covariance.

🔗

Given paired data

(x_{1}, y_{1}), (x_{2}, y_{2}), . . ., (x_{n}, y_{n})

🔗

with corresponding means

\overset{―}{x}

and

\overset{―}{y},

the sample covariance is given by

s_{x y} = \sum_{k = 1}^{n} \frac{(x_{k} - \overset{―}{x}) (y_{k} - \overset{―}{y})}{n - 1} .

🔗

This general definition provides a general measure which is a second order term (like variance) but also maintains "units". To provide a unit-less metric, consider the following measure.

🔗

Definition 2.3.3. Spearman’s Rho (Correlation Coefficient).

🔗

Given a collection of data points, the Spearman’s Rho correlation coefficient is given by

r = \frac{s_{x y}}{s_{x} s_{y}}

🔗

where

s_{x}

is the standard deviation of the x-values only and

s_{y}

is the standard deviation of the y-values only.


    
        
xxxxxxxxxx
 
1
pretty_print("Here we create the best-fit line and correlation coefficient by first creating the normal equations and solving")
2
var('u')
3
4
# Enter the data points here with (x,y) values paired in like positions in the list
5
x = [-1.0,3,4,6]
6
y = [1.0,1,3,4]
7
n = len(x)
8
9
Points = [(x[k],y[k]) for k in range(n)]   # Create graph of data points
10
G = point(Points)
11
12
xsq = []
13
ysq = []
14
xy = []
15
for k in range(n):
16
    xsq.append(x[k]^2)
17
    ysq.append(y[k]^2)
18
    xy.append(x[k]*y[k])
19
20
# Put everything in the right place for the normal equations    
21
a11 = sum(xsq)
22
a12 = sum(x)
23
a21 = a12
24
a22 = n
25
rhs1 = sum(xy)
26
rhs2 = sum(y)
27
ysqsum = sum(ysq)
28
A = matrix([[a11,a12],[a21,a22]])
29
rhs = matrix([[rhs1],[rhs2]])
30
show([A,rhs])
31
32
# and then solve
33
coefs = A.inverse()*rhs
34
m = coefs[0][0]
35
b = coefs[1][0]
36
show(['m=',m,'  b=',b])
37
38
# while at it, create the standard deviations, covariance, and correlation coefficient
39
sx = sqrt((n/(n-1))*(a11/n-(a12/n)^2))
40
sy = sqrt((n/(n-1))*(ysqsum/n-(rhs2/n)^2))
41
sxy = sqrt((n/(n-1))*(rhs1/n-(a12/n)*(rhs2/n)))
42
r = sxy/(sx*sy)
43
show(['r=',r])
44
45
G += plot(m*u+1,(u,min(x),max(x)),color='red')
46
show(G,figsize=[5,4])

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

Theorem 2.3.4. Interpretation of the Spearman’s Rho Correlation Coefficient.

If the points are co-linear with a positive slope then r = 1 and if the points are co-linear with a negative slope then r = -1.

🔗

Proof.

Assume the data points are co-linear with a positive slope. Then the

T S E (m_{0}, b_{0}) = 0

for some

m_{0}

and

b_{0} .

For this line notice that

f (x_{k}) = y_{k}

exactly for all data points. It is easy to show then that

\overset{―}{y} = m_{0} \overset{―}{x} + b_{0}

and

s_{y} = | m_{0} | s_{x} .

Therefore,

s_{x y} = \sum_{k = 1}^{n} \frac{(x_{k} - \overset{―}{x}) (m_{0} x_{k} + b_{0} - (m_{0} \overset{―}{x} + b_{0}))}{n - 1} = m_{0} s_{x}^{2}

Putting these together gives correlation coefficient

r = \frac{m_{0} s_{x}^{2}}{s_{x} m_{0} s_{x}} = 1.

A similar proof follows in the second case by noting that

m_{0} / | m_{0} | = - 1.

🔗

Theorem 2.3.5. Alternate formula for r.

🔗

s_{x y} = \frac{n}{n - 1} [\frac{\sum_{k = 1}^{n} x_{k} y_{k}}{n} - \overset{―}{x} \overset{―}{y}] .

🔗

Proof.

Expand

(x_{k} - \overset{―}{x}) (y_{k} - \overset{―}{y})

and notice that the sample means are actually constants with respect to the summation.

🔗

To compute the best-fit line and correlation coefficient by hand, it is often easiest to construct a table in a manner similar to how you might have computed the mean, variance, skewness and kurtosis in the previous chapter. Indeed,

Table 2.3.6. Computing best fit line and correlation coefficient by hand

$x_{k}^{2}$	$x_{k}$	$y_{k}$	$y_{k}^{2}$	$x_{k} \cdot y_{k}$
4	-2	3	9	-6
0	0	1	1	0
0	0	-1	1	0
1	1	1	1	1
9	3	1	1	3
9	3	-1	1	-3
16	4	0	0	0
39	9	4	14	7

Therefore

\overset{―}{x} = \frac{9}{7}

\overset{―}{y} = \frac{4}{7}

s_{x}^{2} = \frac{7}{6} \cdot [\frac{39}{7} - {(\frac{9}{7})}^{2}] = \frac{32}{7}

s_{y}^{2} = \frac{7}{6} \cdot [\frac{14}{7} - {(\frac{4}{7})}^{2}] = \frac{41}{21}

s_{x y} = \frac{7}{6} \cdot [\frac{7}{7} - \frac{9}{7} \cdot \frac{4}{7}] = \frac{13}{42}

r = \frac{\frac{13}{42}}{\sqrt{\frac{32}{7}} \sqrt{\frac{41}{21}}}

🔗

and the normal equations to find the actual line are

39 m + 9 b = 7 and 9 m + 7 b = 4.

🔗

You should go ahead and simplify these values, solve the system of equations, plot the points and draw the line to complete the creation and evaluation of this best-fit line.

🔗

Checkpoint 2.3.7. WeBWorK - Calculating correlation coefficients..

Consider the following small data set.

Subject	x	y
$1$	$6$	$19$
$2$	$8$	$17$
$3$	$14$	$21$
$4$	$8$	$31$
$5$	$6$	$17$

Find the linear correlation coefficient.

r =

Answer.

0.156556072771287

🔗

Checkpoint 2.3.8. WeBWorK - Interpreting correlation coefficients..

A study found a correlation of r = -0.33 between the gender of a worker and that worker’s net pay. From the following statements, which is true?

Women earn less than men on average
Women earn more than men on average
Correlation can’t be negative in this instance
r makes no sense in this context
None of the above

Enter the letter of the correct statement:

A science project considered the possible relationship between the weight in grams to the length of the tail in a group of mice. The correlation between these two measurements is r = 0.5. If the weights had instead been measured in kilograms instead of grams, what would be the resulting correlation?

0.5
0.5/1000
0.5(1000)
r makes no sense in this context
None of the above

Enter the letter of the correct statement:

Answer 1.

d

Answer 2.

a

Solution.

In the first case, r makes no sense since one can only create perform linear regression on numerical ordered paired data. As stated, the notion of gender is not a numerical quantity and thus cannot (for example) create a scatter plot. One might however “randomly” assign a numerical value to gender (such as 1 = woman and 0 = man) but that is not part of this exercise.

For the second case, adjusting the scale of the data points will not affect the correlation coefficient. Indeed, suppose all the x’s are scaled larger by a factor of 1000. Then, the mean and the standard deviation of x-values will also be 1000 times larger (convince yourself of this) and the covariance will be 1000 times larger. Indeed, for covariance

s_{x y} = \frac{n}{n - 1} (\sum \frac{1000 x_{k} \cdot y_{k}}{n} - (1000 \overset{―}{x}) (\overset{―}{y}))

and one can easily see that the 1000 factors out. So, 1000 will factor out of both numerator and denominator and hence cancel in the covariance formula.

🔗

Once again, we can use R to determine and display the data points and best-fit line with one of the built-in data sets. To do this, that data set needs to have the opportunity to utilize paired data. Below, we use the ’cars’ data set since that set has speed paired with a corresponding stopping distance from some experiment. See if you can use one of the other data sets to create a best-fit line and determine if the correlation coefficient is sufficiently ’good’ or not.


    
        
xxxxxxxxxx
 
1
data(cars)
2
x <- cars[,1]
3
y <- cars[,2]
4
head = paste("Scatter Plot and Best-Fit Line for
5
       Speed vs Stoping Distance \n Correlation Coefficient = ",cor(x,y))   
6
pts = data.frame(x, y)
7
plot(pts,pch = 16, cex = 1.0, col = "blue", main = head, xlab = "x", ylab = "y")
8
lm(y ~ x)
9
abline(lm(y ~ x))

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

🔗

Repeat the above R code but instead use two of the columns from EuStockMarkets for the data. Consider why you might get a relatively high correlation coefficient with this data.

🔗

Again, repeat the above R code by instead use columns 2 and 4 from USJudgeRatings. Notice the relatively large correlation coefficient but ponder if there is really any connection between these two columns. Repeat for columns 1 and 4 and interpret the results.

Essentials of Mathematical Probability and Statistics

Search Results:

Section 2.3 Correlation

Checkpoint 2.3.1.

Definition 2.3.2. Sample Covariance.

Definition 2.3.3. Spearman’s Rho (Correlation Coefficient).

Theorem 2.3.4. Interpretation of the Spearman’s Rho Correlation Coefficient.

Proof.

Theorem 2.3.5. Alternate formula for r.

Proof.

Checkpoint 2.3.7. WeBWorK - Calculating correlation coefficients..

Checkpoint 2.3.8. WeBWorK - Interpreting correlation coefficients..