library(lme4) # example ---- df <- read.csv("https://raw.githubusercontent.com/roualdes/data/refs/heads/master/carnivora.csv") %>% mutate(brain_weight = log10(SB), body_weight = log10(SW)) %>% select(Family, body_weight, brain_weight) %>% na.omit fit_np <- lm(brain_weight ~ Family + Family : body_weight, data = df) # lm(brain_weight ~ Family * body_weight) fit_pp <- lmer(brain_weight ~ Family + (body_weight | Family), data = df, control = lmerControl(optimizer = "bobyqa")) fit_cp <- lm(brain_weight ~ body_weight, data = df) AIC(fit_np, fit_pp, fit_cp) # smaller better => fit_np best, with regard to AIC BIC(fit_np, fit_pp, fit_cpp) # smaller better => fit_cp best, with regard to BIC # Notes from comparison above # * AIC favors prediction accuracy. # * BIC favors models with fewer parameters. # * df corresponds to the number of parameters. # AIC/BIC Summary ---- # 1. AIC focuses on out-of-sample prediction. # 2. BIC focuses on most probable model, if one of the candidates is true (likely none are true). # 3. Neither can compare models that have different response variables; # even y ~ ... and log(y) ~ ..., i.e. transformation of the same variable y # are considered different response variables. # 4. Neither considers other things you might want your model to do, # like make predictions/confidence intervals for Families that aren't in your data set, as hierarchical models do. # 5. Other Information Criteria exist, say *IC. I know of at least * \in {CA, F, T, WA, WB, D} # 6. The PRESS statistic mentioned in References 1. and 3. is from my PhD advisor, Dr. David Allen. # References ---- # OER: Open Educational Resources are both free and publicly accessible # 1. OER AIC derivation https://iowabiostat.github.io/research-highlights/joe/Cavanaugh_Neath_2019.pdf # 2. OER BIC derivation https://faculty.ucmerced.edu/hbhat/BICderivation.pdf # 3. OER A/BIC practically https://online.stat.psu.edu/stat462/node/199/ # 4. ($94) All of Statistics https://www.amazon.com/All-Nonparametric-Statistics-Springer-Texts/dp/0387251456/ref=pd_bxgy_b_img_b # 5. ($62) Mathematical Theory of Bayesian Statistics https://www.routledge.com/Mathematical-Theory-of-Bayesian-Statistics/Watanabe/p/book/9781315373010 # 6. OER KL Divergence visualization https://roualdes.us/lecturenotes/klhr#kl-divergence