Local Regression
Description for Local Regression notebook.
Notebook Contents
This notebook covers:
- Topic 1
- Topic 2
- Topic 3
Use the buttons above to download the notebook or open it in your preferred environment.
š Notebook Preview
M4b: Splines and Generalized Additive ModelsĀ¶
InĀ [2]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.interpolate import LSQUnivariateSpline, UnivariateSpline, BSpline
import statsmodels.api as sm
from statsmodels.gam.tests.test_penalized import df_autos
from statsmodels.gam.smooth_basis import BSplines
from statsmodels.gam.generalized_additive_model import GLMGam
from pygam import LinearGAM, LogisticGAM, GAM, s, l, te, f
from sklearn.preprocessing import scale
from pygam.datasets import wage, default
InĀ [Ā ]:
df = pd.read_csv('mcycle.csv',index_col=0)
InĀ [3]:
df
Out[3]:
InĀ [4]:
#sns.set_theme()
ax = sns.scatterplot(x = df.times, y = df.accel)
ax.set_xlabel('Time after impact (ms)')
ax.set_ylabel('Acceleration (g)')
Out[4]:
Revisiting the spline fit using statsmodelsĀ¶
InĀ [24]:
df_autos
len(df_autos.weight.unique())
Out[24]:
InĀ [25]:
plt.scatter(df_autos.weight, df_autos.city_mpg)
Out[25]:
InĀ [19]:
plt.scatter(df_autos.hp, df_autos.city_mpg)
Out[19]:
InĀ [23]:
x_spline = df_autos[['weight', 'hp']]#scale(df_autos[['weight', 'hp']])
bs = BSplines(x_spline, df=[12, 10], degree=[3, 3])
alpha = np.array([21833888.8, 6460.38479])
InĀ [59]:
gam_bs = GLMGam.from_formula('city_mpg ~ weight + hp', data=df_autos, smoother=bs, alpha=alpha)
InĀ [60]:
res_bs = gam_bs.fit()
res_bs.summary()
Out[60]:
InĀ [61]:
res_bs.plot_partial(0, cpr=True)
Out[61]:
InĀ [176]:
res_bs.plot_partial(1, cpr=True)
Out[176]:
Introduction to pyGAMĀ¶
In pyGAM, the following functional forms are specified using "terms"
- l() linear terms: for terms like Xi
- s() spline terms
- f() factor terms
- te() tensor products
- intercept
GAM with single predictorĀ¶
The following functional forms are specified:
- intercept (default)
l(): linears(): splinef(): factorte(): tensor products
We revisit fitting the motorcycle helmet crash data using a smoothing spline, but now using the pyGAM library
InĀ [27]:
#Set knots
df2 = df.sort_values(by='times')
InĀ [28]:
df2.head()
Out[28]:
InĀ [47]:
gam0 = GAM(s(0), distribution='normal', link='identity').fit(scale(df2.times), df2.accel) # .fit(X,y)
#gam0 = LinearGAM(s(0)).fit(scale(df2.times), df2.accel)
InĀ [48]:
gam0.summary()
InĀ [49]:
gam0.coef_
Out[49]:
Model statisticsĀ¶
InĀ [50]:
list(gam0.statistics_.keys())
Out[50]:
InĀ [51]:
gam0.statistics_['m_features']
Out[51]:
InĀ [52]:
gam0.terms
Out[52]:
InĀ [53]:
XX = gam0.generate_X_grid(term=0)
pdep, confi = gam0.partial_dependence(term=0, X=XX, width=0.99)
InĀ [54]:
plt.plot(scale(df2.times), df2.accel, '.', label = 'Data') #, df2.times, ss(df2.times), '-')
plt.plot(XX[:, gam0.terms[0].feature], pdep, label='Spline fit')
plt.plot(XX[:, gam0.terms[0].feature], confi, c ='r', ls='--', label = '95% CI')
plt.legend()
Out[54]:
GAM with multiple predictorsĀ¶
InĀ [56]:
X, y = wage() # variables in wage are: ['year', 'age', 'education']
gam = LinearGAM(s(0) + s(1) + f(2)).fit(X, y) # we use f(2) because the 3rd variable (education) is qualitative (i.e. a factor)
gam.summary()
InĀ [57]:
gam.terms
Out[57]:
InĀ [58]:
titles = ['year', 'age', 'education']
for i, term in enumerate(gam.terms):
if term.isintercept:
continue
XX = gam.generate_X_grid(term=i)
pdep, confi = gam.partial_dependence(term=i, X=XX, width=0.95)
plt.figure()
plt.plot(XX[:, term.feature], pdep)
plt.plot(XX[:, term.feature], confi, c='r', ls='--')
plt.title(titles[i])# repr(term))
plt.show()
