Higher Degree Regression

Section 2.4 Higher Degree Regression

Continuing in a similar fashion to the previous section, consider now an approximation using a quadratic function

f (x) = a x^{2} + b x + c .

In this case, the total squared error would be of the form

T S E (a, b, c) = \sum_{k = 0}^{n} (a x_{k}^{2} + b x_{k} + c - y_{k})^{2} .

🔗

Taking all three partials gives

T S E_{a} = \sum_{k = 1}^{n} 2 (a x_{k}^{2} + b x_{k} + c - y_{k}) \cdot x_{k}^{2}

T S E_{b} = \sum_{k = 1}^{n} 2 (a x_{k}^{2} + b x_{k} + c - y_{k}) \cdot x_{k}

T S E_{c} = \sum_{k = 1}^{n} 2 (a x_{k}^{2} + b x_{k} + c - y_{k}) \cdot 1 .

🔗

Once again, setting equal to zero and solving gives the normal equations for the best-fit quadratic

a \sum_{k = 1}^{n} x_{k}^{4} + b \sum_{k = 1}^{n} x_{k}^{3} + c \sum_{k = 1}^{n} x_{k}^{2} = \sum_{k = 1}^{n} x_{k}^{2} y_{k}

a \sum_{k = 1}^{n} x_{k}^{3} + b \sum_{k = 1}^{n} x_{k}^{2} + c \sum_{k = 1}^{n} x_{k} = \sum_{k = 1}^{n} x_{k} y_{k}

a \sum_{k = 1}^{n} x_{k}^{2} + b \sum_{k = 1}^{n} x_{k} + c \sum_{k = 1}^{n} 1 = \sum_{k = 1}^{n} y_{k} .

🔗

One can easily use software to perform these calculations of course. Further, you can extend the ideas from above and derivate formulas for a best-fit cubic. The interactive cell below determines the optimal quadratic polynomial for a given set of data points and by commenting and uncommenting as suggested will also determine the best fit cubic.


    
        
xxxxxxxxxx
 
1
var('x')
2
xpts = vector([-1, 0, 1, 3, 5, 5])
3
ypts = vector([5, 3, 0, -1, 3, 6])
4
ones = vector([1, 1, 1, 1, 1, 1])
5
xpts3 = []
6
xpts2 = []
7
pts = []
8
for k in range(len(xpts)):
9
#    xpts3.append(xpts[k]^3)   # uncomment this for best cubic
10
    xpts2.append(xpts[k]^2)
11
    pts.append((xpts[k],ypts[k]))
12
# xpts3 = vector(xpts3)        # uncomment this for best cubic
13
xpts2 = vector(xpts2)
14
15
# For best cubic, instead uncomment the first and comment the second
16
# X = matrix([xpts3]).stack(xpts2).stack(xpts).stack(ones).transpose()
17
X = matrix([xpts2]).stack(xpts).stack(ones).transpose()
18
19
Y = matrix(ypts).transpose()
20
Xt = X.transpose()
21
print(X)
22
23
A = (Xt*X).inverse()*Xt*Y
24
25
# For best cubic, instead uncomment the first and comment the second
26
# [a,b,c,d] = [A[0][0],A[1][0],A[2][0],A[3][0]]
27
[a,b,c] = [A[0][0],A[1][0],A[2][0]]
28
29
# For best cubic, instead uncomment the first and comment the second
30
# f = a*x^3 + b*x^2 + c*x + d
31
f = a*x^2 + b*x + c
32
33
banner = "The interpolant is given by $%s$"%str(latex(f))
34
G = points(pts,size=20)
35
H = plot(f,x,min(xpts),max(xpts),title=banner)
36
show(G+H)

    
    
    
    
        
            
                Language:
                
            
        
    
    




    
    
        
        Messages

Essentials of Mathematical Probability and Statistics

Search Results:

Section 2.4 Higher Degree Regression