gam                   package:mgcv                   R Documentation

_G_e_n_e_r_a_l_i_z_e_d _A_d_d_i_t_i_v_e _M_o_d_e_l_s _u_s_i_n_g _p_e_n_a_l_i_z_e_d _r_e_g_r_e_s_s_i_o_n _s_p_l_i_n_e_s _a_n_d 
_G_C_V

_D_e_s_c_r_i_p_t_i_o_n:

     Fits the specified  generalized additive model (GAM) to data.
     'gam()' is not a clone of what Splus provides. Smooth terms are
     represented using penalized regression splines with smoothing
     parameters selected by GCV/UBRE or by regression splines with
     fixed degrees of freedom (mixtures of the two are permitted).
     Multi-dimensional smooths are available using penalized thin plate
     regression splines, but the user must make sure that covariates
     are sensibly scaled relative to each other when using such terms.
     For a general overview see Wood (2001). For more on specifying
     models see 'gam.models'. For more on model  selection see
     'gam.selection'.

_U_s_a_g_e:

     gam(formula,family=gaussian(),data=list(),weights=NULL,subset=NULL,
         na.action,control=gam.control(),scale=0,knots=NULL,sp=NULL,
         min.sp=NULL,H=NULL,gamma=1,...)

_A_r_g_u_m_e_n_t_s:

 formula: A GAM formula (see also 'gam.models'). This is exactly like
          the formula for a glm except that smooth terms can be added
          to the right hand side of the formula (and a formula of the
          form 'y ~ .' is not allowed). Smooth terms are specified by
          expressions of the form: 
           's(var1,var2,...,k=12,fx=FALSE,bs="tp",by=a.var)' where
          'var1', 'var2', etc. are the covariates which the smooth is a
          function of and 'k' is the dimension of the basis used to
          represent the smooth term. If 'k' is not specified then
          'k=10*3^(d-1)' is used where 'd' is the number of covariates
          for this term. 'fx' is used to indicate whether or not this
          term has a fixed muber of degrees of freedom ('fx=FALSE' to
          select d.f. by GCV/UBRE). 'bs' indicates the basis to use,
          with '"cr"' indicating cubic regression spline, and '"tp"'
          indicating thin plate regression spline: '"cr"' can only be
          used with 1-d smooths. 'by' can be used to specify a variable
          by which the smooth should be multiplied. For example
          'gam(y~z+s(x,by=z))' would specify a model E(y)=f(x)z where
          f(.) is a smooth function (the formula is 'y~x+s(x,by=z)'
          rather than 'y~s(x,by=z)' because in GAMs the smooths are
          always set up to sum to zero over the covariate values). The
          'by' option is particularly useful for models in which
          different functions of the same variable are required for
          each level of a factor: see 's'.

          For backwards compatibility the formula may also include
          terms like  's(x,12|f)', which specifies a regression spline
          which is not to be penalized and has 12 knots, or 's(x,z,25)'
          indicating a rank 25 penalized t.p.r.s. In such cases
          arguements 'k', 'fx' and 'bs' are ignored if supplied and a
          one dimensional term will always use a cubic regression
          spline basis. Note that a term of the form 's(x)' will result
          in a term with a '"tp"' basis.

          Formulae can involve nested or "overlapping" terms such as 
           'y~s(x)+s(z)+s(x,z)' or 'y~s(x,z)+s(z,v)': see
          'gam.side.conditions' for further details and examples. 

  family: This is a family object specifying the distribution and link
          to use in fitting etc. See 'glm' and 'family' for more
          details. The negative binomial families provided by the MASS
          library  can be used, with or without known theta parameter:
          see 'gam.neg.bin' for details. 

    data: A data frame containing the model response variable and
          covariates required by the formula. If this is missing then
          the frame from which 'gam' was called is searched for the
          variables specified in the formula.

 weights: prior weights on the data.

  subset: an optional vector specifying a subset of observations to be
          used in the fitting process.

na.action: a function which indicates what should happen when the data
          contain `NA's.  The default is set by the `na.action' setting
          of `options', and is `na.fail' if that is unset.  The
          ``factory-fresh'' default is `na.omit'.

 control: A list of fit control parameters returned by  'gam.control'.

   scale: If this is zero then GCV is used for all distributions except
          Poisson and binomial where UBRE is used with scale parameter
          assumed to be 1. If this is greater than 1 it is assumed to
          be the scale parameter/variance and UBRE is used: to use the
          negative binomial in this case theta must be known. If
          'scale' is negative  GCV  is always used, which means that
          the scale parameter will be estimated by GCV and the Pearson 
          estimator, or in the case of the negative binomial theta will
          be estimated  in order to force the GCV/Pearson scale
          estimate to unity (if this is possible). For binomial models
          in  particular, it is probably worth  comparing UBRE and GCV
          results; for ``over-dispersed Poisson'' GCV is probably more
          appropriate than UBRE.

   knots: this is an optional list containing user specified knot
          values to be used for basis construction.  For the 'cr' basis
          the user simply supplies the knots to be used, and there must
          be the same number as the basis dimension, 'k', for the
          smooth concerned. For the 'tp' basis 'knots' has two uses.
          Firstly, for large datasets  the calculation of the 'tp'
          basis can be time-consuming. The user can retain most of the
          advantages of the t.p.r.s.  approach by supplying  a reduced
          set of covariate values from which to obtain the basis - 
          typically the number of covariate values used will be
          substantially  smaller than the number of data, and
          substantially larger than the basis dimension, 'k'. The
          second possibility  is to avoid the eigen-decomposition used
          to find the t.p.r.s. basis altogether and simply use  the
          basis implied by the chosen knots: this will happen if the
          number of knots supplied matches the  basis dimension, 'k'.
          For a given basis dimension the second option is  faster, but
          gives poorer results (and the user must be quite careful in
          choosing knot locations).  Different terms can use different 
          numbers of knots, unless they share a covariate. 

      sp: A vector of smoothing parameters for each term can be
          provided here. Smoothing parameters must  be supplied in the
          order that the smooth terms appear in the model  formula.
          With fit method '"magic"' (see 'gam.control'  and 'magic')
          then negative elements indicate that the  parameter should be
          estimated, and hence a mixture of fixed and estimated 
          parameters is possible. With fit method '"mgcv"', if 'sp' is 
          supplied then all its elements must be positive.  

  min.sp: for fit method '"magic"' only, lower bounds can be  supplied
          for the smoothing parameters. Note that if this option is
          used then the smoothing parameters, 'sp', in the returned
          object will need to be added to what is supplied here to get
          the actual smoothing parameters. Lower bounds on the
          smoothing  parameters can sometimes help stabilize otherwise
          divergent P-IRLS iterations.

       H: a user supplied fixed quadratic penalty on the parameters of
          the  GAM can be supplied, with this as its coefficient
          matrix. A common use of this term is  to add a ridge penalty
          to the parameters of the GAM in circumstances in which the
          model is close to un-identifiable on the scale of the linear
          predictor, but perfectly well defined on the response scale.

   gamma: It is sometimes useful to inflate the model degrees of 
          freedom in the GCV or UBRE score by a constant multiplier.
          This allows  such a multiplier to be supplied if fit method
          is '"magic"'.

     ...: further arguments for  passing on e.g. to 'gam.fit'

_D_e_t_a_i_l_s:

     A generalized additive model (GAM) is a generalized linear model
     (GLM) in which the linear  predictor is given by a user specified
     sum of smooth functions of the covariates plus a  conventional
     parametric component of the linear predictor. A simple example is:

                   log(E(y_i))=f_1(x_1i)+f_2(x_2i)

     where the (idependent) response variables y_i~Poi, and f_1 and f_2
     are smooth functions of covariates x_1 and  x_2. The log is an
     example of a link function. 

     If absolutely any smooth functions were allowed in model fitting
     then maximum likelihood  estimation of such models would
     invariably result in complex overfitting estimates of  f_1  and
     f_2. For this reason the models are usually fit by  penalized
     likelihood  maximization, in which the model (negative log)
     likelihood is modified by the addition of  a penalty for each
     smooth function, penalizing its `wiggliness'. To control the
     tradeoff  between penalizing wiggliness and penalizing badness of
     fit each penalty is multiplied by  an associated smoothing
     parameter: how to estimate these parameters, and  how to
     practically represent the smooth functions are the main
     statistical questions  introduced by moving from GLMs to GAMs. 

     The 'mgcv' implementation of 'gam' represents the smooth functions
     using  penalized regression splines, and by default uses basis
     functions for these splines that  are designed to be optimal,
     given the number basis functions used. The smooth terms can be 
     functions of any number of covariates and the user has some
     control over how smoothness of  the functions is measured. 

     'gam' in 'mgcv' solves the smoothing parameter estimation problem
     by using the  Generalized Cross Validation (GCV) criterion or an
     Un-Biased Risk Estimator criterion  (UBRE) which is in practice an
     approximation to AIC. Smoothing parameters are chosen to  minimize
     the GCV or UBRE score for the model, and the main computational
     challenge solved  by the 'mgcv' package is to do this efficiently
     and reliably. Two alternative  numerical methods are provided, see
     'mgcv', 'magic' and  'gam.control'. 

     Broadly 'gam' works by first constructing basis functions and a
     quadratic penalty  coefficient matrix for each smooth term in the
     model formula, obtaining a model matrix for  the strictly
     parametric part of the model formula, and combining these to
     obtain a  complete model matrix (/design matrix) and a set of
     penalty matrices for the smooth terms.  Some linear
     identifiability constraints are also obtained at this point. The
     model is  fit using 'gam.fit', a modification of 'glm.fit'. The
     GAM  penalized likelihood maximization problem is solved by
     penalized Iteratively  Reweighted  Least Squares (IRLS) (see e.g.
     Wood 2000). At each iteration a penalized  weighted least squares
     problem is solved, and the smoothing parameters of that problem
     are  estimated by GCV or UBRE. Eventually both model parameter
     estimates and smoothing  parameter estimates converge. 

     The fitting approach just described, in which the smoothing
     parameters are estimated for  each approximating linear model of
     the IRLS process was suggested by Chong Gu (see, e.g.  Gu 2002),
     and is very computationally efficient. However, because the
     approach neglects  the dependence of the iterative weights on the
     smoothing parameters, it is usually  possible to find smoothing
     parameters which actually yield slightly lower GCV/UBRE score 
     estimates than those resulting from this `performance iteration'.
     'gam' therefore also  allows the user to `improve' the smoothing
     parameter estimates, by using O'Sullivan's  (1986) suggested
     method, in which for each trial set of smoothing parameters the
     IRLS is  iterated to convergance before the UBRE/GCV score is
     evaluated. This requires much less  efficient minimisation of the
     power iteration based on 'nlm', and is  therefore quite slow. 

     Two alternative bases are available for representing model terms.
     Univariate smooth terms can be represented using conventional
     cubic regression splines - which are very efficient
     computationally - or thin plate regression splines. Multivariate
     terms must be represented using thin plate regression splines. For
     either basis the user specifies the dimension of the basis for
     each smooth term. The dimension of the basis is one more than the
     maximum degrees of freedom that the  term can have, but usually
     the term will be fitted by penalized maximum likelihood estimation
     and the actual degrees of freedom will be chosen by GCV. However,
     the user can choose to fix the degrees of freedom of a term, in
     which case the actual degrees of freedom will be one less than the
     basis dimension.

     Thin plate regression splines are constructed by starting with the
     basis for a full thin plate spline and then truncating this basis
     in an optimal manner, to obtain a low rank smoother. Details are
     given in Wood (2003). One key advantage of the approach is that it
     avoids the knot placement problems of conventional regression
     spline modelling, but it also has the advantage that smooths of
     lower rank are nested within smooths of higher rank, so that it is
     legitimate to use conventional hypothesis testing methods to
     compare models based on pure regression splines. The t.p.r.s.
     basis can become expensive to calculate for large datasets. In
     this case the user can supply a reduced  set of knots to use in
     basis construction (see knots, in the argument list).

     In the case of the cubic regression spline basis, knots  of the
     spline are placed evenly throughout the covariate values to which
     the term refers:  For example, if fitting 101 data with an 11 knot
     spline of 'x' then there would be a knot at every 10th (ordered) 
     'x' value. The parameterization used represents the spline in
     terms of its values at the knots. The values at neighbouring knots
     are connected by sections of  cubic polynomial constrainted to be 
     continuous up to and including second derivative at the knots. The
     resulting curve is a natural cubic  spline through the values at
     the knots (given two extra conditions specifying  that the second
     derivative of the curve should be zero at the two end  knots).
     This parameterization gives the parameters a nice
     interpretability. 

     Details of '"mgcv"' GCV/UBRE minimization method are given in Wood
     (2000):  the basis of the approach  is to alternate efficient
     global optimization with respect to one overall smoothing 
     parameter with Newton updates of a set of relative smoothing
     parameters for each smooth term. The Newton updates are backed up
     by steepest descent, since the GCV/UBRE score functions are not
     positive definite everywhere.

_V_a_l_u_e:

     The function returns an object of class '"gam"' which has the
     following elements: 

boundary: did parameters end up at boundary of parameter space?

      by: a 2-d array of 'by' variables (i.e. covariates that multiply
          a smooth term) 'by[i,j]' is the jth value for the ith 'by'
          variable. There are only as many rows of this array as there
          are 'by' variables in the model (often 0). The rownames of
          'by' give the 'by' variable names.

by.exists: an array of logicals: 'by.exists[i]' is 'TRUE' if the ith
          smooth has a 'by' variable associated with it, 'FALSE'
          otherwise.

    call: the matched call (allows 'update' to be used with 'gam'
          objects, for example). 

coefficients: the coefficients of the fitted model. Parametric
          coefficients are  first, followed  by coefficients for each
          spline term in turn.

converged: indicates whether or not the iterative fitting method
          converged.

covariate.shift: covariates get shifted so that they are centred around
          zero - this is by how much.

deviance: (unpenalized)

      df: The maximum degrees of freedom for each smooth term plus one.
          In the '"cr"' basis case  this is the number of knots used.
          For the '"tp"' basis it is the rank of the smooth (one
          greater than the  maximum degrees of freedom because of the
          centering constraint).

 df.null: null degrees of freedom.

     dim: number of covariates of which term is a function.

     edf: estimated degrees of freedom for each smooth.

  family: family object specifying distribution and link used.

fit.method: The underlying multiple GCV/UBRE method used: '"magic"' 
          for the new more stable method, '"mgcv"' for the Wood (2000)
          method.

fitted.values: fitted model predictions of expected value for each
          datum.

 formula: the model formula.

full.formula: the model formula with each smooth term fully expanded
          and with option arguments given explicitly (i.e. not with
          reference to other variables) - useful for later prediction
          from the model.

gcv.ubre: The minimized GCV or UBRE score.

gcv.used: TRUE if GCV used for smoothing parameter selection, FALSE if
          UBRE used.

     hat: array of elements from the leading diagonal of the `hat' (or
          `influence') matrix. Same length as response data vector.

    iter: number of iterations of P-IRLS taken to get convergence.

linear.predictor: fitted model prediction of link function of expected
          value for  each datum.

mgcv.conv: A list of covergence diagnostics relating to smoothing
          parameter estimation. Differs for method '"magic"' and
          '"mgcv"'. Here is  the '"mgcv"' version:

     _s_c_o_r_e corresponding to edf, an array of GCV or UBRE scores for the
          model given the final  estimated relative smoothing
          parameters.

     _g the gradient of the GCV/UBRE score w.r.t. the relative smoothing
          parameters at termination.

     _h the second derivatives corresponding to 'g' above - i.e. the
          leading diagonal of the Hessian.

     _e the eigen-values of the Hessian. All non-negative indicates a
          positive definite Hessian.

     _i_t_e_r the number of iterations taken.

     _i_n._o_k 'TRUE' if the second smoothing parameter guess improved the
          GCV/UBRE score.

     _s_t_e_p._f_a_i_l 'TRUE' if the algorithm terminated by failing to improve
          the GCV/UBRE score rather than by "converging".  Not
          necessarily a problem, but check the above derivative
          information quite carefully.

          In the case of '"magic"' the items are:

     _f_u_l_l._r_a_n_k The apparent rank of the problem given the model matrix
          and  constraints.

     _r_a_n_k The numerical rank of the problem.

     _f_u_l_l_y._c_o_n_v_e_r_g_e_d 'TRUE' is multiple GCV/UBRE converged by meeting 
          convergence criteria. 'FALSE' if method stopped with a
          steepest descent step  failure.

     _h_e_s_s._p_o_s._d_e_f Was the hessian of the gCV/UBRE score positive
          definite at  smoothing parameter estimation convergence?

     _i_t_e_r How many iterations were required to find the smoothing
          parameters?

     _s_c_o_r_e._c_a_l_l_s and how many times did the GCV/UBRE score have to be
          evaluated?

     _r_m_s._g_r_a_d root mean square of the gradient of the GCV/UBRE score at
           convergence.

 min.edf: Minimum possible degrees of freedom for whole model.

   model: model frame containing all variables needed in original model
          fit.

    nsdf: number of parametric, non-smooth, model terms including the
          intercept.

null.deviance: deviance for single parameter model.

 p.order: the order of the penalty used for each term. 0 signals
          auto-selection.

prior.weights: prior weights on observations.

residuals: the deviance residuals for the fitted model.

    sig2: estimated or supplied variance/scale parameter.

      sp: smoothing parameter for each smooth.

  s.type: type of spline basis used: 0 for conventional cubic
          regression spline, 1 for t.p.r.s.

      UZ: array storing the matrices for transforming from t.p.r.s.
          basis to equivalent t.p.s. basis - see GAMsetup for details
          of how the matrices are packed in this array.

      Vp: estimated covariance matrix for parameter estimators.

 weights: final weights used in IRLS iteration.

      xp: knot locations for each cubic regression spline based smooth.
          'xp[i,]+covariate.shift[i]' are the locations for the ith
          smooth.

      Xu: The set of unique covariate locations used to define t.p.s.
          from which t.p.r.s. basis was derived. Again see GAMsetup for
          details of the packing algorithm.

xu.length: The number of unique covariate combinations in the data. 

       y: response data.

_W_A_R_N_I_N_G_S:

     The '"mgcv"' code does not check for rank defficiency of the model
     matrix that may result from lack of identifiability between the
     parametric and smooth components of the model. 

     You must have more unique combinations of covariates than the
     model has total parameters. (Total parameters is sum of basis
     dimensions plus sum of non-spline  terms less the number of spline
     terms). 

     Automatic smoothing parameter selection is not likely to work well
     when  fitting models to very few response data.

     Relative scaling of covariates to a multi-dimensional smooth term
     has an affect on the results: make sure that relative scalings are
     sensible. For example, measuring one spatial co-ordinate in
     millimetres and the other in lightyears will usually produce poor
     results.   

     With large datasets (more than a few thousand data) the '"tp"'
     basis gets very slow to use: use the 'knots' argument as discussed
     above and  shown in the examples. Alternatively, for 1-d smooths 
     you can use the '"cr"' basis.

     For data with many  zeroes clustered together in the covariate
     space it is quite easy to set up  GAMs which suffer from
     identifiability problems, particularly when using poisson or
     binomial families. The problem is that with e.g. log or logit
     links, mean value zero corresponds to an infinite range on the
     linear predictor scale. Some regularization is possible in such
     cases: see  'gam.control' for details.

_A_u_t_h_o_r(_s):

     Simon N. Wood simon@stats.gla.ac.uk

_R_e_f_e_r_e_n_c_e_s:

     Key References:

     Wood, S.N. (2000)  Modelling and Smoothing Parameter Estimation
     with Multiple  Quadratic Penalties. J.R.Statist.Soc.B
     62(2):413-428

     Wood, S.N. (2003) Thin plate regression splines. J.R.Statist.Soc.B
     65(1):95-114

     Background References:

     Green and Silverman (1994) Nonparametric Regression and
     Generalized  Linear Models. Chapman and Hall.

     Gu and Wahba (1991) Minimizing GCV/GML scores with multiple
     smoothing parameters via the Newton method. SIAM J. Sci. Statist.
     Comput. 12:383-398

     Gu (2002) Smoothing Spline ANOVA Models, Springer.

     Hastie and Tibshirani (1990) Generalized Additive Models. Chapman
     and Hall.

     O'Sullivan, Yandall and Raynor (1986) Automatic smoothing of
     regression functions in generalized linear models. J. Am.
     Statist.Ass. 81:96-103 

     Wahba (1990) Spline Models of Observational Data. SIAM 

     Wood (2001) mgcv:GAMs and Generalized Ridge Regression for R. R
     News 1(2):20-25

     Wood and Augustin (2002) GAMs with integrated model selection
     using penalized regression splines and applications  to
     environmental modelling. Ecological Modelling 157:157-177

     <URL: http://www.stats.gla.ac.uk/~simon/>

_S_e_e _A_l_s_o:

     'gam.models', 's', 'predict.gam', 'plot.gam', 'summary.gam',
     'gam.side.conditions', 'gam.selection','mgcv', 'gam.control'
     'gam.check', 'gam.neg.bin', 'magic','vis.gam'

_E_x_a_m_p_l_e_s:

     library(mgcv)
     set.seed(0) 
     n<-400
     sig2<-4
     x0 <- runif(n, 0, 1)
     x1 <- runif(n, 0, 1)
     x2 <- runif(n, 0, 1)
     x3 <- runif(n, 0, 1)
     pi <- asin(1) * 2
     f <- 2 * sin(pi * x0)
     f <- f + exp(2 * x1) - 3.75887
     f <- f+0.2*x2^11*(10*(1-x2))^6+10*(10*x2)^3*(1-x2)^10-1.396
     e <- rnorm(n, 0, sqrt(abs(sig2)))
     y <- f + e
     b<-gam(y~s(x0)+s(x1)+s(x2)+s(x3))
     summary(b)
     plot(b,pages=1)
     # an extra ridge penalty (useful with convergence problems) ....
     bp<-gam(y~s(x0)+s(x1)+s(x2)+s(x3),H=diag(0.5,41)) 
     print(b);print(bp);rm(bp)
     # set the smoothing parameter for the first term, estimate rest ...
     bp<-gam(y~s(x0)+s(x1)+s(x2)+s(x3),sp=c(0.01,-1,-1,-1))
     plot(bp,pages=1);rm(bp)
     # set lower bounds on smoothing parameters ....
     bp<-gam(y~s(x0)+s(x1)+s(x2)+s(x3),min.sp=c(0.001,0.01,0,10)) 
     print(b);print(bp);rm(bp)

     # now a GAM with 3df regression spline term & 2 penalized terms
     b0<-gam(y~s(x0,k=4,fx=TRUE,bs="tp")+s(x1,k=12)+s(x2,15))
     plot(b0,pages=1)
     # now fit a 2-d term to x0,x1
     b1<-gam(y~s(x0,x1)+s(x2)+s(x3))
     par(mfrow=c(2,2))
     plot(b1)
     par(mfrow=c(1,1))
     # now simulate poisson data
     g<-exp(f/5)
     y<-rpois(rep(1,n),g)
     b2<-gam(y~s(x0)+s(x1)+s(x2)+s(x3),family=poisson)
     plot(b2,pages=1)
     # and a pretty 2-d smoothing example....
     test1<-function(x,z,sx=0.3,sz=0.4)  
     { (pi**sx*sz)*(1.2*exp(-(x-0.2)^2/sx^2-(z-0.3)^2/sz^2)+
       0.8*exp(-(x-0.7)^2/sx^2-(z-0.8)^2/sz^2))
     }
     n<-500
     old.par<-par(mfrow=c(2,2))
     x<-runif(n);z<-runif(n);
     xs<-seq(0,1,length=30);zs<-seq(0,1,length=30)
     pr<-data.frame(x=rep(xs,30),z=rep(zs,rep(30,30)))
     truth<-matrix(test1(pr$x,pr$z),30,30)
     contour(xs,zs,truth)
     y<-test1(x,z)+rnorm(n)*0.1
     b4<-gam(y~s(x,z))
     fit1<-matrix(predict.gam(b4,pr,se=FALSE),30,30)
     contour(xs,zs,fit1)
     persp(xs,zs,truth)
     vis.gam(b4)
     par(old.par)
     # very large dataset example using knots
     n<-10000
     x<-runif(n);z<-runif(n);
     y<-test1(x,z)+rnorm(n)
     ind<-sample(1:n,1000,replace=FALSE)
     b5<-gam(y~s(x,z,k=50),knots=list(x=x[ind],z=z[ind]))
     vis.gam(b5)
     # and a pure "knot based" spline of the same data
     b6<-gam(y~s(x,z,k=100),knots=list(x= rep((1:10-0.5)/10,10),
             z=rep((1:10-0.5)/10,rep(10,10))))
     vis.gam(b6,color="heat")

