---
# Copyright (c) 2026 Xiang Chen. Part of the spDBL R package.
# SPDX-License-Identifier: MIT
title: "Analytical and Sample-Based MNIW Model Comparison"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Analytical and Sample-Based MNIW Model Comparison}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

This vignette adapts Sudipto Banerjee's development example on analytical WAIC
for matrix-normal inverse-Wishart (MNIW) multivariate regression. It uses the
installed package and generated data, so it can run without the development
`material/` directory or external data files.

The workflow fits a conjugate regression, compares analytical scores with
posterior-draw estimates, optionally checks the latter against `loo`, and
compares a sequence of covariate subsets.

## Likelihood and posterior

For an $n\times q$ response matrix $Y$ and an $n\times p$ design matrix $X$,
the observations are **independent rows**:

$$
y_i\mid B,\Sigma \sim N_q(x_i B,\Sigma),\qquad
B\mid\Sigma,D\sim MN(m_n,M_n,\Sigma),\qquad
\Sigma\mid D\sim IW(\nu_n,\Psi_n).
$$

Here $m_n$ is the actual posterior coefficient mean. In a precision-based
derivation it might instead be written $M_n a_n$; supply that product as
`mn`, rather than the unscaled parameter $a_n$.

Each response row is one WAIC observation: its $q$ response components are
scored jointly. These helpers do not directly score an emulator whose
likelihood has correlated spatial GP rows. Such a model needs its own
likelihood factorization and definition of an observation.

## Fit a multivariate regression

Generate two correlated responses. Only the first covariate has a nonzero
coefficient in the generating model.

```{r regression-data}
library(spDBL)
set.seed(42)
n <- 120L
X <- cbind(intercept = 1, x1 = seq(-1, 1, length.out = n),
           x2 = rnorm(n), x3 = rnorm(n))
B <- rbind(c(0.8, -0.3), c(1.2, 0.7), c(0, 0), c(0, 0))
Sigma <- matrix(c(0.7, 0.25, 0.25, 0.5), 2)
Y <- X %*% B + matrix(rnorm(2 * n), n, 2) %*% chol(Sigma)
```

Use a zero coefficient prior mean, identity coefficient row covariance,
identity inverse-Wishart scale, and $q+2$ prior degrees of freedom.
`MNIW_R()` returns all four posterior parameters needed for analytical WAIC.
The identity `H` specifies independent likelihood rows.

```{r posterior}
p <- ncol(X)
q <- ncol(Y)
post <- MNIW_R(X, Y, H = diag(n), v = q + 2, S = diag(q),
               C = matrix(0, p, q), Vb = diag(p))
analytical <- compute_analytical_waic(
  Y, X, Mn = post$Vbnew, mn = post$Cnew,
  Psi_n = post$Snew, nu_n = post$vnew)
unlist(analytical[c("WAIC", "lppd", "p_waic")])
head(analytical$pointwise)
```

## Analytical calculation

Define $h_i=x_iM_nx_i^T$, $c_i=1+h_i$, and
$d_i=(y_i-x_im_n)\Psi_n^{-1}(y_i-x_im_n)^T$.
The marginal predictive distribution for one row is multivariate Student t
with $\nu_n-q+1$ degrees of freedom and scale
$c_i\Psi_n/(\nu_n-q+1)$. Its log density gives

$$
\operatorname{lppd}_i =
\log\Gamma((\nu_n+1)/2)-\log\Gamma((\nu_n-q+1)/2)
-\frac q2\log\pi-\frac12\{q\log c_i+\log|\Psi_n|\}
-\frac{\nu_n+1}{2}\log(1+d_i/c_i).
$$

The posterior variance of the row log likelihood is

$$
p_i=\frac14\left[
\sum_{j=1}^{q}\psi_1((\nu_n-j+1)/2)
+2qh_i^2+4h_i\nu_nd_i+2\nu_nd_i^2-4d_i\right],
$$

where $\psi_1$ is the trigamma function. The returned totals are

$$
\operatorname{lppd}=\sum_i\operatorname{lppd}_i,\qquad
p_{\rm WAIC}=\sum_i p_i,\qquad
\operatorname{WAIC}=-2(\operatorname{lppd}-p_{\rm WAIC}).
$$

The implementation uses Cholesky factors and triangular solves. Posterior
simulation is unnecessary for these analytical quantities.

## Compare with posterior draws

The imported `mniw` package supplies paired coefficient and covariance draws.
`rMNIW()` uses the coefficient **covariance** `Sigma = post$Vbnew` with
`prec = FALSE`. Its `X` and `V` components are arrays with dimensions
`p x q x S` and `q x q x S`.

```{r sampled-waic}
set.seed(123)
draws <- mniw::rMNIW(4000L, Lambda = post$Cnew, Sigma = post$Vbnew,
                     Psi = post$Snew, nu = post$vnew, prec = FALSE)
sampled <- compute_sample_waic(Y, X, draws$X, draws$V,
                               return_log_lik = TRUE)
comparison <- data.frame(
  Method = c("Analytical", "Posterior draws"),
  lppd = c(analytical$lppd, sampled$lppd),
  p_waic = c(analytical$p_waic, sampled$p_waic),
  WAIC = c(analytical$WAIC, sampled$WAIC))
knitr::kable(comparison, digits = 3)
```

The analytical calculation is exact for the supplied MNIW posterior.
Draw-based estimates have Monte Carlo error. Both functions use the same
row-level likelihood, and `compute_sample_waic()` uses sample variances with
denominator $S-1$.

### Optional comparison with loo

The returned `log_lik` matrix has draws in rows and observations in columns.
`loo` is optional; it is not needed to compute either spDBL score.

```{r loo-comparison}
if (requireNamespace("loo", quietly = TRUE)) {
  reference <- loo::waic(sampled$log_lik)
  comparison <- rbind(comparison, data.frame(
    Method = "loo (same draws)",
    lppd = reference$estimates["elpd_waic", "Estimate"] +
      reference$estimates["p_waic", "Estimate"],
    p_waic = reference$estimates["p_waic", "Estimate"],
    WAIC = reference$estimates["waic", "Estimate"]))
  knitr::kable(comparison, digits = 3)
}
```

The two calculations using identical draws should agree up to numerical
precision. `loo` can warn about large pointwise WAIC penalties. Its authors
recommend PSIS leave-one-out cross-validation for the additional diagnostics;
the same `sampled$log_lik` can be supplied to `loo::loo()`.

## Compare sequential covariate subsets

Refit each subset using its corresponding prior. Keep the responses and the
definition of an observation fixed across comparisons.

```{r sequential-comparison}
profile <- do.call(rbind, lapply(seq_len(p), function(k) {
  X_k <- X[, seq_len(k), drop = FALSE]
  fit <- MNIW_R(X_k, Y, H = diag(n), v = q + 2, S = diag(q),
                C = matrix(0, k, q), Vb = diag(k))
  score <- compute_analytical_waic(Y, X_k, fit$Vbnew, fit$Cnew,
                                   fit$Snew, fit$vnew)
  data.frame(Model = if (k == 1L) "Intercept" else
    paste0("Intercept + x1:", k - 1L),
    lppd = score$lppd, p_waic = score$p_waic, WAIC = score$WAIC)
}))
knitr::kable(profile, digits = 3)
```

```{r comparison-profile, fig.width=8, fig.height=7, eval=requireNamespace("ggplot2", quietly = TRUE)}
metrics <- c("WAIC", "lppd", "p_waic")
labels <- c("WAIC (lower is better)", "lppd", "p_WAIC")
long <- do.call(rbind, lapply(seq_along(metrics), function(j)
  data.frame(Model = profile$Model, Metric = labels[j],
             Value = profile[[metrics[j]]])))
long$Model <- factor(long$Model, levels = profile$Model)
long$Metric <- factor(long$Metric, levels = labels)
colors <- stats::setNames(c("#c85200", "#2b5c8f", "#7570b3"), labels)
ggplot2::ggplot(long, ggplot2::aes(Model, Value, color = Metric, group = Metric)) +
  ggplot2::geom_line(linewidth = 1.1) +
  ggplot2::geom_point(size = 3) +
  ggplot2::facet_wrap(~Metric, ncol = 1, scales = "free_y") +
  ggplot2::scale_color_manual(values = colors) +
  ggplot2::theme_minimal(base_size = 12) +
  ggplot2::theme(legend.position = "none",
    axis.text.x = ggplot2::element_text(angle = 30, hjust = 1),
    panel.grid.minor = ggplot2::element_blank()) +
  ggplot2::labs(title = "Sequential MNIW model comparison",
                x = "Covariates", y = NULL)
```

Lower WAIC favors a model's estimated predictive performance among these
candidates. Neither lppd nor the effective-parameter penalty is guaranteed to
increase monotonically with added covariates. The minimum is a comparative
estimate, not a guarantee that the selected subset is the true model.

## References

Watanabe, S. (2010). Asymptotic equivalence of Bayes cross validation and
widely applicable information criterion in singular learning theory.
*Journal of Machine Learning Research*, **11**, 3571--3594.

Vehtari, A., Gelman, A., and Gabry, J. (2017). Practical Bayesian model
evaluation using leave-one-out cross-validation and WAIC.
*Statistics and Computing*, **27**, 1413--1432.

The [loo WAIC documentation](https://mc-stan.org/loo/reference/waic.html)
describes its likelihood-array conventions and interpretation.
