Link Functions
Understanding how link functions connect your linear predictor to the response variable.
What is a Link Function?
In generalized linear models (GLMs), a link function is a mathematical transformation that connects the linear predictor $\eta_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \cdots$ to the expected value of the response variable.
The Core Idea
Your linear predictor $\eta_i$ can take any value from $-\infty$ to $+\infty$. But many response variables have natural constraints:
- Counts must be non-negative $(0, 1, 2, \ldots)$
- Probabilities must be between 0 and 1
- Durations/times must be positive
The link function transforms between these constrained values and the unconstrained linear predictor.
Mathematically, if $\mu_i = \mathbb{E}[y_i]$ is the expected value of the response, then:
where $g(\cdot)$ is the link function and $g^{-1}(\cdot)$ is its inverse (the "mean function").
Why Do We Need Link Functions?
Example 1: Modeling Counts with Poisson
Suppose you're modeling the number of accidents per day. Your linear predictor might be:
Without a link function, if $x = 10$, you'd predict $\eta = 2.5 - 3 = -0.5$ accidents, which is impossible!
The log link solves this by setting $\log(\mu_i) = \eta_i$, so $\mu_i = e^{\eta_i} = e^{-0.5} \approx 0.61$ accidents. The mean is always positive, as required.
Example 2: Modeling Probabilities with Binomial
For binary outcomes (success/failure), probabilities must satisfy $0 < p < 1$. The linear predictor could give any value, but the logit link constrains the probability:
This ensures probabilities are always between 0 and 1, regardless of the predictor values.
Common Link Functions
Identity Link
No transformation. The mean equals the linear predictor directly.
Used for: Gaussian, Logistic, Student's t, Skew Normal, Log-Normal
Log Link
Ensures the mean is always positive. Coefficients have multiplicative interpretation.
Used for: Poisson, Gamma, Negative Binomial, Exponential, Weibull, Log-Logistic
Logit Link
Maps probabilities $(0,1)$ to the real line. Coefficients are log-odds ratios.
Used for: Binomial, Beta, Beta-Binomial, Negative Binomial (type 2)
Probit Link
Uses the standard normal CDF. Similar to logit but with lighter tails.
Used for: Binomial, Beta, Beta-Binomial (alternative)
Complementary Log-Log (cloglog)
Asymmetric link, useful when probability of success approaches 1 faster than 0.
Used for: Binomial, Beta, Beta-Binomial (rare events)
Negative Log (neglog)
Used in accelerated failure time (AFT) survival models. Positive coefficients increase survival time.
Used for: Weibull, Gamma, Exponential, Log-Normal (survival)
Link Functions by Likelihood Family
| Family | Default Link | Available Links | Why This Default? |
|---|---|---|---|
| Gaussian | identity | identity | Response is unbounded; no transformation needed |
| Logistic | identity | identity | Unbounded response with heavier tails than Gaussian |
| Student's t | identity | identity | Robust to outliers; models the location parameter |
| Skew Normal | identity | identity | Asymmetric but still unbounded response |
| Family | Default Link | Available Links | Why This Default? |
|---|---|---|---|
| Gamma | log | log, identity, neglog, quantile | Log ensures positive mean; canonical for GLM |
| Exponential | log | log, neglog | Positive rate parameter; neglog for AFT survival |
| Log-Normal | identity | identity, neglog | Models $\log(y) \sim N(\mu, \sigma^2)$; $\mu$ is unbounded |
| Weibull | log | log, neglog, quantile | Log for positive scale; neglog for AFT interpretation |
| Log-Logistic | log | log, neglog | Positive scale parameter; common in survival analysis |
| Family | Default Link | Available Links | Why This Default? |
|---|---|---|---|
| Poisson | log | log | Canonical link; ensures positive mean rate |
| Negative Binomial | log | log, logoffset, quantile | Same as Poisson; handles overdispersion |
| Negative Binomial (type 2) | logit | logit, loga, cauchit, probit, cloglog, ccloglog, loglog | Models success probability $p \in (0,1)$ |
| Family | Default Link | Available Links | Why This Default? |
|---|---|---|---|
| Binomial | logit | logit, probit, cloglog, ccloglog, loglog, loga, cauchit, robit, sn | Canonical link; log-odds interpretation |
| Beta | logit | logit, probit, cloglog, ccloglog, loglog, loga, cauchit | Maps $(0,1)$ to real line; intuitive for proportions |
| Beta-Binomial | logit | logit, probit, cloglog, ccloglog, loglog, loga, cauchit, robit, sn | Same as binomial; probability parameter in $(0,1)$ |
Special Link Functions
Log-Offset (logoffset)
Includes an exposure or offset term $E_i$ in count models. Useful when modeling rates with varying observation periods or population sizes.
Used for: Negative Binomial with known exposure
Quantile Link
Allows regression on quantiles (e.g., median) instead of the expected value. Useful for asymmetric distributions.
Used for: Gamma, Weibull, Negative Binomial
Cauchit Link
Based on the Cauchy distribution. Has heavier tails than logit or probit, making it robust to extreme observations.
Used for: Binomial, Beta, Beta-Binomial
Robit Link
Uses the Student's t distribution quantile function. Provides a robust alternative to probit with heavier tails.
Used for: Binomial, Beta-Binomial
Skew Normal (sn) Link
Uses the skew-normal quantile function. Allows asymmetric response curves for probability models.
Used for: Binomial, Beta-Binomial
Log-A (loga) Link
Generalization of logit allowing an upper bound different from 1. Useful for truncated proportions.
Used for: Binomial, Beta
Choosing a Link Function
General Guidelines
- Start with the default. Canonical links have nice statistical properties and are well-tested.
- Consider interpretability. Logit gives odds ratios, log gives multiplicative effects.
- Match the domain. Ensure the link maps to the correct range for your response.
- Check model fit. Compare DIC, WAIC, or CPO across different links if unsure.
When to Use Alternative Links
| Situation | Consider | Reason |
|---|---|---|
| Extreme probabilities (rare events) | cloglog or loglog | Better behavior near 0 or 1 |
| Outliers in probability models | cauchit or robit | Heavier tails provide robustness |
| Survival analysis (AFT) | neglog | Positive coefficients mean longer survival |
| Asymmetric probability response | sn (skew normal) | Allows asymmetric dose-response curves |
| Known exposure in count data | logoffset | Properly accounts for varying observation periods |
Interpreting Coefficients by Link Function
Each coefficient $\beta$ tells you: when a covariate increases by one unit, how does the response change? The link function determines the scale on which this change is expressed. To convert back to the natural scale of the response, you often exponentiate.
| Link | When covariate increases by 1 unit... | exp($\beta$) gives | Example |
|---|---|---|---|
| identity | The mean response changes by $\beta$ (directly) | Not needed | $\beta = 0.5$ → mean response increases by 0.5 units |
| log | $\log(\text{mean response})$ changes by $\beta$ | Rate ratio: the mean response is multiplied by exp($\beta$) | $\beta = 0.3$ → $e^{0.3} = 1.35$ → mean response increases by 35% |
| logit | The log-odds of the event change by $\beta$ | Odds ratio: the odds are multiplied by exp($\beta$) | $\beta = -1.4$ → $e^{-1.4} = 0.25$ → odds decrease by 75% |
| probit | $\Phi^{-1}(p)$ changes by $\beta$ | No simple exp() interpretation | $\beta = 0.5$ → probability increases, but the amount depends on the baseline $p$ |
| cloglog | $\log(-\log(1-p))$ changes by $\beta$ | Hazard ratio (in discrete-time survival models) | $\beta = 0.7$ → $e^{0.7} = 2.01$ → hazard doubles |
| neglog | $-\log(\text{survival time})$ changes by $\beta$ (AFT models) | Time ratio (inverse direction) | $\beta = 0.5$ → $e^{-0.5} = 0.61$ → survival time multiplied by 0.61 |
The General Rule
For logarithmic links (log, logit, cloglog), exponentiating the coefficient gives a multiplicative effect on the natural scale:
- exp($\beta$) > 1 → increase. Percentage increase = (exp($\beta$) − 1) × 100
- exp($\beta$) < 1 → decrease. Percentage decrease = (1 − exp($\beta$)) × 100
- exp($\beta$) = 1 (i.e., $\beta = 0$) → no effect
For the identity link (Gaussian), coefficients are directly interpretable without any transformation.
For the probit link, there is no simple exp() interpretation. Use marginal effects or convert to approximate odds ratios by multiplying probit coefficients by 1.6.
Concrete Examples
| Model | Coefficient | Interpretation |
|---|---|---|
| Gaussian (identity link) Predicting income |
$\beta_{\text{education}} = 3200$ | Each additional year of education increases income by $3,200 |
| Poisson (log link) Counting hospital visits |
$\beta_{\text{age}} = 0.02$ | $e^{0.02} = 1.02$ → each year of age increases visits by 2% |
| Binomial (logit link) Modeling vote choice |
$\beta_{\text{female}} = -0.18$ | $e^{-0.18} = 0.84$ → women have 16% lower odds of voting Bush |
| Poisson (log link) Disease counts with offset |
$\beta_{\text{pollution}} = 0.15$ | $e^{0.15} = 1.16$ → 16% higher disease rate per unit pollution increase |
Quick Reference: All Link Functions
| Link | Formula $g(\mu)$ | Inverse $\mu = g^{-1}(\eta)$ | Domain |
|---|---|---|---|
| identity | $\mu$ | $\eta$ | $(-\infty, \infty)$ |
| log | $\log(\mu)$ | $e^\eta$ | $(0, \infty)$ |
| neglog | $-\log(\mu)$ | $e^{-\eta}$ | $(0, \infty)$ |
| logit | $\log(\mu/(1-\mu))$ | $e^\eta/(1+e^\eta)$ | $(0, 1)$ |
| probit | $\Phi^{-1}(\mu)$ | $\Phi(\eta)$ | $(0, 1)$ |
| cloglog | $\log(-\log(1-\mu))$ | $1 - e^{-e^\eta}$ | $(0, 1)$ |
| ccloglog | $\log(-\log(\mu))$ | $e^{-e^\eta}$ | $(0, 1)$ |
| loglog | $-\log(-\log(\mu))$ | $e^{-e^{-\eta}}$ | $(0, 1)$ |
| cauchit | $\tan(\pi(\mu-0.5))$ | $0.5 + \arctan(\eta)/\pi$ | $(0, 1)$ |