Generalized linear models in R

www.nr.no
Generalized linear models in R
Magne Aldrin, Norwegian Computing Center and the
University of Oslo
Sharp workshop, Copenhagen, October 2012
Linear regression
Model
y = β 0 + β 1 x 1 + β 2 x2 + · · · + β p x p + ε
•
•
•
•
•
y: response variable, continious
x1, · · · xp : p explanatory variables, predictor variables, covariates
β0: intercept
β1, · · · βp: regressions coeffisients
ε: error term, E(ε) = 0, V ar(ε) = σ 2
1
Estimation
• The β’s are unknown, and must be estimated from observations
• n observations of yi, x1i, . . . xpi
• Estimation by Ordinary Least Squares
P
? Minimize i(yi − ŷi)2,
? where ŷi = βˆ0 + βˆ1x1i + βˆ2x2i + · · · + βˆpxpi
• If ε is normal distributed (Gaussian), then OLS = Maximum
Likelihood (ML)
2
Generalized Linear Models - GLM
•
•
•
•
•
•
y ∼ Distributed with mean µ and perhaps an additional parameter
h(µ) = η
η = β 0 + β 1 x 1 + β 2 x2 + · · · + β p x p
η: linear predictor
h: link fuction
Estimation by maximum likelihood
3
Gaussian regression/linear regression
• Continious, symmetric response
• y ∼ N (µ, σ 2)
• µ = η : identity link
4
Gamma regression
•
•
•
•
Continous, positive, skew response
y ∼ Gamma(expectation µ, parameter2)
log(µ) = η : log link
µ = exp(η)
5
Binary logistic regression
•
•
•
•
•
Binary response, take values 0 or 1, binomial with n = 1 trial
P (y = 1) = p = E(y) = µ
y ∼ Bin(n = 1,p)
logit(p) = log(p/(1 − p)) = η : logit link
p = exp(η)/(1 + exp(η)) : logistic inverse link
6
Binomial logistic regression
•
•
•
•
Binomial response, n trials, take values 0, 1, . . . , n
y ∼ Bin(n,p)
logit(p) = log(p/(1 − p)) = η : logit link
p = exp(η)/(1 + exp(η)) : logistic inverse link
7
Poisson regression
•
•
•
•
Count response, take values 0, 1, 2, . . .
y ∼ Poisson(λ = µ)
log(µ) = η : log link
µ = exp(η)
8
Ex: Ozone data - Gaussian regression
• Ozone - O3 is the response variable
• 9 expanatory variables (wind speed, humidity, temperature ...)
• 330 observation
> require(faraway)
> data(ozone)
>
> summary(ozone)
O3
vh
Min.
: 1.00
Min.
:5320
1st Qu.: 5.00
1st Qu.:5690
Median :10.00
Median :5760
Mean
:11.78
Mean
:5750
3rd Qu.:17.00
3rd Qu.:5830
Max.
:38.00
Max.
:5950
wind
Min.
: 0.000
1st Qu.: 3.000
Median : 5.000
Mean
: 4.848
3rd Qu.: 6.000
Max.
:11.000
humidity
Min.
:19.00
1st Qu.:47.00
Median :64.00
Mean
:58.13
3rd Qu.:73.00
Max.
:93.00
9
temp
Min.
:25.00
1st Qu.:51.00
Median :62.00
Mean
:61.75
3rd Qu.:72.00
Max.
:93.00
vis
Min.
: 0.0
1st Qu.: 70.0
Median :120.0
Mean
:124.5
3rd Qu.:150.0
Max.
:350.0
ibh
Min.
: 111.0
1st Qu.: 877.5
Median :2112.5
Mean
:2572.9
3rd Qu.:5000.0
Max.
:5000.0
doy
Min.
: 33.0
1st Qu.:120.2
Median :205.5
Mean
:209.4
3rd Qu.:301.8
Max.
:390.0
dpg
Min.
:-69.00
1st Qu.: -9.00
Median : 24.00
Mean
: 17.37
3rd Qu.: 44.75
Max.
:107.00
ibt
Min.
:-25.0
1st Qu.:107.0
Median :167.5
Mean
:161.2
3rd Qu.:214.0
Max.
:332.0
10
> pdf("PAIRSozone.pdf")
> pairs(ozone)
> dev.off()
11
5300 5800
20 60
0 3000
200
0
300
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●● ●
●
● ●
●●
●●
●●
●●
●●
●●
●●
●
●
●
●●
●●
●●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●●
●● ●
●●
●●
●
●
●
●
●●
●●●
●
●●●
●
●
●
●
● ● ●●
●●
●
●●●
● ●
● ●
●
● ●● ●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
● ●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●●●
●
●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●●
●
●
●●●● ●
●●●
●●
● ● ●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●●
● ●● ●
●
●
●●● ●●
●
●●
●
●
●
●●
●
●
●●
● ●●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●●●●●●
●
●
●●
●
●
●
●
●●
●
●
●
● ●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
● ●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●● ●● ● ●●●
●●
●●●●●●
●● ●
●●● ●
●
●
●●●●● ●
●● ●●
●
●
●
●
●● ●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●●●
●
●●
●●●●
●
●
●●
●
●
●
●
●
●● ●
●●
●
●
●●●●●●
●● ●
●●
●●
●
●
●
●●
●●
●
●●
●
●●●
●● ●
●●
●●
●
●
●●
●
●●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●●● ●
●
●●
●●● ● ●
●●●● ●
●
●
●
●
●
●
●
●● ●
●●●
●
● ●●●●
●
●●●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●●
●●
●●
●●● ●
●
●
●●
●●
● ●
●●●●
●
●●
●
●
●●●●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
● ●●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●●
● ●●
●
●
●●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●●●
●●●●
●
●
●
●
●
●
●●●●
●●●●
●
●
●
●
●
●
●
●
●●
●●
●●
●●●●
●●
●
●
●
●●●●
●●●●
●
●
●
●
●
●
●
●●●●
●●●●
●●
●
●
● ●●
●
●
●
●
●
●●●●
●
●
●
●
●●
●●
●●●●
●● ● ●
●●● ●●
●●
●
●●●
●
●●
●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●●●
●
●
●●
●
●
●●
●●
●●
●●
●●●●
●●
●●
●
●
●●
●●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●●●
●●
●● ●
●● ●
●
●
●●
●
●●● ●
●
● ● ●● ●
humidity
●● ●
●
●●●
●
● ●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●●●● ●
●●
● ●
●
●
●
●
● ●●
● ●
●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●● ● ● ●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●●●●●
●● ●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
● ●
●●
●●●
●
●●
●
●● ●
●●●●
●
●
●●
●
●●●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●●●●
●●
● ●
●
●●
●
●
●●●
●
●
● ●● ●
●●●
●● ●
●●●
●
●
●
●●●● ●
●●
●●
● ●●
●●●
●
●●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
● ● ●●●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●●
●●
●
●
●●●●●●●●
●
●● ●
●
●●
●
●●
●●
●
●
●●
●●
●●●
●●●
●
●●
●●
●●
●●
●●
●
●
●●
●
●●
●●
●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●●
●●
●●● ●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●●●
●●●
●●
●● ●
●
●
●
●● ●●●
●
●
●
●●
●
●
●
●
●● ●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●●●
●
●
●
● ●●
●
●
● ●
●
temp
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●● ●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●● ●
●●
●
●
●●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●●
●●
●●●
●●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
● ●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●●●●●●●●●
●
●
●●
●●
●●
●
●●
●
●●
●●
●
●●●
●
●●
●
●●
●
●●
●●●
● ●
●
●
●●
●●●
●●●●
●
●
●
●
●
●
●●●
●
●●●
●●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●
●●
●●
●
●
●
●
●
● ● ●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●● ●●
●
●
●●
●
●
●
●
● ●●●●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
ibh
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●●●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●●
●●
●●●
●●
●●
●
●
●●●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●●
●●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●● ●
●
●●
●
●
●
●●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●●
●
●
●●●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●
●
●
●
●
●
●
●
●
●●●
●●
●●●
●
●
● ●●●●●●
●
●
●
●●
●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●●●●
●●●
●●
●
●●
●
●
●
●
●
●
●● ●
●●●
●
●
●●
●
●
●
●
●●●● ●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ● ● ● ●
●
●
●
dpg
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ●●●
●●●●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●● ●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●● ●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●
●
●
●
●●
●●
●●
●●
●●
●
●●
●
●
●●●●
●●
●
●
●●
●
●
●
●
●●
●●
●
●●
●●
●●
●
●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●●
● ●●● ●
●●
●
●
●
●
●
●●
●●
●●●
●●
●●
●●
●
●●●
●
●●●
●
●
●●● ●●
●●
●
●
●
●
●
●
●
● ● ●●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ●●●●●
●
●
●
●
●
●●
●●●●●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ●
● ●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
● ●
●
ibt
●●
●
●
●●
●●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
● ●●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●●
●
●●
●●
●
●● ●●●●●
● ● ●●●●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●●●
●
●
●
●●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
● ●●●●●● ●●
●●●●●
●●●●●● ●
● ●●●●●●●●
● ●
●●●
●●
●●
●●●
●
● ●●
●●
●●
●●●
●●
●
●
●
●●
●●
●●
●●
●●●
●
●
●●
●●
●
●
●
●●● ●●●
●
●
●●●
●●
●
●● ●
●●●
●●●●
●
●
●
●
●
●
●
●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
● ● ●●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
● ●●
●
●●
●
●●
●
●●
●●
●
●
●
●●
●● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
● ●●●
●
●●● ●● ●● ●
●
●
●●
●
● ●●●● ●●
●●
●
●
●
●
●
●●●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●● ●
●
●
●●
●
●
●●
●●
●●
●●
●●
●
●
●●
●●
●●
●
●●
●
●●
●
●
● ●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●●●
●●●
●
●
●●●●●●
●●
●
●
●●●●●
●
●
●
●
●
●●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
vis
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●
●●
●●
● ●●●
●
●
●
●●
●
●
●
●
●
●● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●●
●●
●●●
●●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●●
●
●●
●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
● ●●●●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●● ●
●●
●
●
●●
●
●●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●●●
●
●●●
●●●●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●●●●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●●
●●●
●●●●
●●
●
●
●
●
●
●
●●
●● ●
●●
●●● ●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●●
●●●
●
●
●
●
●●
●
●
●●
●
●●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
● ●●
●●
●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●●
●●
●●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●●●
●
●
●●
●
●
● ●●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●●
●
●
●
● ●
●
●●●●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●●●●
●
●
●
●
●●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●●
●
●
●● ●
● ●
●●●●
●
●
●
●
●●
●
●
●
●
●●
● ●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●●●
●●●●
●
●
●
●
●
●
●
●
●●●
●●
●
●●●
●●
●●
●●
●●●
●
●●●
●
●
●
●
●
●
●●
●
● ●
●●
●●
●
●●
●●
●
●●
●
doy
wind
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 4 8
30
70
−50
100
0
20
●
●● ●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
20
●
●
●●
●
●
● ●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●●●●
●
● ●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●● ●
0
● ●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●● ●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
● ●●●●
●
●
●
●
●
●
●
●●●●
● ●●
●
●
●
●●●●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●●●●
0
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
0 4 8
● ● ●●
●●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
300
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
● ●●
70
● ●●
●
●
●
●●●● ●●
●
●
●
●●●
●
●
●●
●●
●●●●●
●
●●
●●
●
●
●
●
●
●●
●●
● ●
●
●
●●●
●
●●
●
●●
●
●
●●
●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●●●
●●●●●
●●
50
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
30
●
●
●●
●
●●● ●
●
●
●
●
●●●●
●●●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●●●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●● ●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●●●
●●●●●
●
●
●
●
●
●
●●
●● ●
200
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
100
vh
●
●
●
●
●
●
●
0
●
●●●
●
●●
●●●
●●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●● ●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●●
●
−50
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 3000
●
●
●●
●●
●●●
●●
●
●
●●
●●
●●
●●
●●
●
●
●●●
●●
●
●
●●●
●●
●●
●●
●
●
●
●
●
●
●●●●●
●●●
●
●
●
●
●
●
●
●
●●●● ●
●●
●●●
●●
●●
●●●
●● ●●●●●
200
●
●
●
●
●
●
●
O3
50
20 60
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●●
●
●
●
●
●
● ●●
●
0
5300 5800
200
12
> lmobj<-lm(log(O3)~vh+wind+humidity+temp+ibh+dpg+ibt+vis+doy,
data=ozone)
> glmobj<-glm(log(O3)~vh+wind+humidity+temp+ibh+dpg+ibt+vis+doy,
family=gaussian(link=identity),data=ozone)
> require(mgcv)
> glmobj<-gam(log(O3)~vh+wind+humidity+temp+ibh+dpg+ibt+vis+doy,
family=gaussian(link=identity),data=ozone)
> summary(glmobj)
Family: gaussian
Link function: identity
Formula:
log(O3) ~ vh + wind + humidity + temp + ibh + dpg + ibt + vis +
doy
13
Parametric coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.301e-01 2.684e+00
0.235 0.81453
vh
-1.244e-05 4.908e-04 -0.025 0.97980
wind
-8.403e-03 1.226e-02 -0.685 0.49357
humidity
5.084e-03 1.713e-03
2.968 0.00322 **
temp
2.993e-02 4.527e-03
6.611 1.60e-10 ***
ibh
-8.495e-05 2.687e-05 -3.161 0.00172 **
dpg
5.470e-04 1.026e-03
0.533 0.59414
ibt
4.902e-04 1.237e-03
0.396 0.69221
vis
-8.395e-04 3.410e-04 -2.462 0.01434 *
doy
-1.021e-03 2.522e-04 -4.049 6.47e-05 ***
--Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
R-sq.(adj) = 0.709
GCV score = 0.16808
Deviance explained = 71.7\%
Scale est. = 0.16298
n = 330
14
If y is skew and positive, alternative 1
Take the logarithm of y, log(y) is more symmetric
log(y) = η + ε
y = exp (η + ε) = exp (η) · exp (ε)
E(y) = exp (η) · E(exp (ε))
E(exp (ε)) = exp (σ 2/2) if ε ∼ N (0, σ 2)
15
If y is skew and positive, alternative 2
E(y) = µ
log(µ) = η
y ∼ Gamma(µ, parameter2)
16
Ex: Ozone data - Gamma regression
>
> ### GLM, O3 ~ Gamma-distributed with identity link
> glmobj<-gam(O3~vh+wind+humidity+temp+ibh+dpg+ibt+vis+doy,
family=Gamma(link=log),data=ozone)
> summary(glmobj)
Family: Gamma
Link function: log
Formula:
O3 ~ vh + wind + humidity + temp + ibh + dpg + ibt + vis + doy
17
Parametric coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 3.805e-01 2.583e+00
0.147 0.882957
vh
6.323e-05 4.723e-04
0.134 0.893591
wind
-8.965e-03 1.180e-02 -0.760 0.447870
humidity
5.752e-03 1.648e-03
3.490 0.000551 ***
temp
2.382e-02 4.356e-03
5.469 9.15e-08 ***
ibh
-6.420e-05 2.586e-05 -2.483 0.013548 *
dpg
9.455e-04 9.869e-04
0.958 0.338775
ibt
1.527e-03 1.191e-03
1.283 0.200561
vis
-8.488e-04 3.281e-04 -2.587 0.010126 *
doy
-1.011e-03 2.427e-04 -4.165 4.01e-05 ***
--Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
R-sq.(adj) = 0.736
GCV score = 0.15567
Deviance explained = 71.1\%
Scale est. = 0.15095
n = 330
18
Ex: Precipitation - binary logistic regression
• 10 years data with daily precipitation at Tryvann, Oslo
• Response is P01: 1 if precipitation, 0 if not, at day t
• 6 predictors
? P01.L1 - 1 if precipitation, 0 if not, at day t − 1
? P01.L2 - 1 if precipitation, 0 if not, at day t − 2
? P01.L3 - 1 if precipitation, 0 if not, at day t − 3
? Prep.L1 - amount of precipitation (mm) day t − 1
? Prep.L2 - amount of precipitation (mm) day t − 2
? Prep.L3 - amount of precipitation (mm) day t − 3
19
R code
### Note: Linux style on directory structure used here
> Tryvann.dat<-read.table("/nr/user/aldrin/Sharp/data/Tryvann.dat")
>
> ### load the mgcv library
> require(mgcv)
>
> ### Logistic regression with binary precipitation (0=no, 1=yes) as r
> glmobj<-gam(P01~P01.L1+P01.L2+P01.L3+Prep.L1+Prep.L2+Prep.L3,
data=Tryvann.dat,family=binomial(link=logit))
> summary(glmobj)
Family: binomial
Link function: logit
Formula:
P01 ~ P01.L1 + P01.L2 + P01.L3 + Prep.L1 + Prep.L2 + Prep.L3
20
Parametric coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.774966
0.068825 -11.260 < 2e-16 ***
P01.L1
1.064480
0.083536 12.743 < 2e-16 ***
P01.L2
0.322184
0.085679
3.760 0.00017 ***
P01.L3
0.411833
0.084189
4.892 1.00e-06 ***
Prep.L1
0.032718
0.007428
4.405 1.06e-05 ***
Prep.L2
-0.009255
0.006212 -1.490 0.13626
Prep.L3
-0.004704
0.006028 -0.780 0.43523
--Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
R-sq.(adj) = 0.125
UBRE score = 0.24364
Deviance explained = 9.43\%
Scale est. = 1
n = 3420
21
Ex: Simulated data - Poisson regression
n<-200
> x1<-rnorm(n)
# N(0,1)
> x2<-runif(n,-10,10) # Uniform(-10,10)
> x3<-rnorm(n)
>
> eta<- 1 + 2*x1 - 0.2*x2^2 + 0*x3
> mu<-exp(eta)
>
> y<-rpois(n=n,lambda=mu)
>
> data.obj<-data.frame(y=y,x1=x1,x2=x2,x3=x3)
>
> ### load the mgcv library
> require(mgcv)
Loading required package: mgcv
This is mgcv 1.7-6. For overview type ’help("mgcv-package")’.
>
22
> ### Estimation
> glmobj<-gam(y~x1+x2+x3,family=poisson(link=log),data=data.obj)
> summary(glmobj)
Family: poisson
Link function: log
Formula:
y ~ x1 + x2 + x3
23
Parametric coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.154044
0.071537
2.153
0.0313 *
x1
0.802362
0.051533 15.570 < 2e-16 ***
x2
-0.038901
0.009412 -4.133 3.58e-05 ***
x3
-0.107352
0.059222 -1.813
0.0699 .
--Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
R-sq.(adj) = -0.00555
Deviance explained = 21.1\%
UBRE score = 3.6933 Scale est. = 1
n = 200
24
Prediction for new data points
> new.data.obj<-data.frame(x1=c(0,1),x2=c(0,0),x3=c(0,0))
> predict(glmobj,newdata=new.data.obj,type="response")
1
2
1.166542 2.602326
25
Model selection
• Overfitting: The model has too many parameters relative to the
number of observations
? large estimation uncertainty
? poor predictions
• Underfitting: The model has too few parameters relative to the
number of observations
? poor fit to data
? does not use all information in the data
? poor predictions
• Methods for model selection
? Cross-validation - mentioned in space-time modeling course
? AIC
? BIC
26
Model selection by AIC and BIC
• Want as best fit as possible
with as few parameters as possible
• p=number of parameters and n=number of observations
• AIC = - 2 log likelihood + 2 p
• BIC = - 2 log likelihood + log(n) p
27
Ex: Model selection using BIC
> glmobj1<-gam(y~x1,family=poisson(link=log),data=data.obj)
> glmobj2<-gam(y~x2,family=poisson(link=log),data=data.obj)
> glmobj12<-gam(y~x1+x2,family=poisson(link=log),data=data.obj)
> glmobj123<-gam(y~x1+x2+x3,family=poisson(link=log),data=data.obj)
>
> bic<-rep(NA,4)
> bic[1]<-BIC(glmobj1)
> bic[2]<-BIC(glmobj2)
> bic[3]<-BIC(glmobj12)
> bic[4]<-BIC(glmobj123)
>
> bic
[1] 1113.910 1339.247 1104.025 1106.019
• The model with x1 and x2 has the lowest BIC value
and is selected
28