Alternating Least Squares in Generalized Linear Models

We derived a convergence result for a sequential procedure known as alternating maximization (minimization) to the maximum likelihood estimator for a pretty large family of models ——- Generalized Linear Models (GLMs). Alternating procedure for linear regression becomes to the well-known algorithm of Alternating Least Squares (ALS), because of the quadraticity of log-likelihood function $L(\bm{v})$. In GLMs framework we lose quadraticity of $L(\bm{v})$, but still have concavity due to the fact that error-distribution is from exponential family (EF). Concentration property makes the Taylor approximation of $L(\bm{v})$ up to the second order accurate and makes possible the use of ALS technique. Moreover, finite sample Fisher expansion was proved for GLMs. Examples and experiments confirm convergence result followed by the discussion of the importance of initial guess.