A STEM model imposes a single vector of regression coefficients on the whole monitoring network. Over a large and heterogeneous domain this is often not credible: the way concentrations respond to altitude or to a co-pollutant may differ between the center of a plain and the areas nearer the reliefs.
The spatially-clustered STEM (SC-STEM) model relaxes the assumption. Each location \(i\) belongs to one of \(K\) latent regimes, and conditionally on regime \(k\)
\[ z_{it} = x_{it}' \beta_k + A_i y^{(k)}_t + e_{it}, \qquad y^{(k)}_t = G_k y^{(k)}_{t-1} + \eta^{(k)}_t, \]
so that coefficients, variance components and latent dynamics are all regime-specific. Setting \(K = 1\) returns the pooled fit.
Labels and parameters are estimated jointly by maximizing a Potts-penalized log-likelihood
\[ Q = \sum_{i=1}^{d} \ell_i(k_i) + \phi\, c \sum_{(i,j) \in E} I(k_i = k_j), \]
where \(E\) is the edge set of the symmetrized \(k\)-nearest-neighbor graph, each unordered pair counted once, and \(\phi \ge 0\) controls how strongly neighboring locations are pushed into the same regime. This is the penalty of Sugasawa and Murakami (2021), in the form used for spatially-clustered Fay–Herriot and spatial autoregressive models.
data(povalley)
Tn <- 150L # a five-month slice, to keep the vignette quick
Tfull <- nrow(povalley$z)
d <- ncol(povalley$z)
keep <- as.vector(outer(seq_len(Tn), (seq_len(d) - 1L) * Tfull, "+"))
z <- povalley$z[seq_len(Tn), ]
X <- povalley$covariates[keep, ]
dim(z)
#> [1] 150 36
head(round(povalley$coords, 3), 3)
#> Longitude Latitude
#> IT0470A 7.695 45.104
#> IT0594A 10.224 45.541
#> IT0692A 8.906 45.102plot(povalley$coords, pch = 19, cex = 0.9,
xlab = "Longitude", ylab = "Latitude",
main = "Po Valley background stations")phi <- list(
beta = matrix(c(1.25, -0.00003, 0.64), 3, 1),
sigma2eps = 18.66,
sigma2omega = 1e-06,
theta = 2e-06,
G = matrix(0.59, 1, 1),
Sigmaeta = matrix(4.25, 1, 1),
m0 = as.matrix(0),
C0 = as.matrix(1)
)
mod <- STEM_Model(z = z, covariates = X, coordinates = povalley$coords,
phi = phi, A = matrix(1, d, 1))Coordinates are longitude and latitude, so every call below uses
distance = "geo".
fit <- SCSTEM_Estimation(mod, K = 3, phi_penalty = 0.05, distance = "geo",
precision = 0.05)
fit
#> Spatially-clustered STEM model
#> regimes requested : K = 3
#> regimes estimated : K_eff = 3
#> spatial penalty : phi = 0.05 (auto, knn = 5)
#> label update : ICM
#> convergence : Convergence reached (labels stable)
#> cluster sizes : 7, 11, 18
#>
#> Information criteria
#> loglik df AIC BIC KIC
#> -16843.06 27.00 33740.12 33918.16 33767.12
#>
#> Cluster-wise estimates
#> sigma2omega beta1 beta2 beta3 G Sigmaeta m0 C0 theta
#> cluster 1 0 6.3554 0.0069 0.5185 0.9175 5.7784 -1.7945 1 0
#> cluster 2 0 2.5502 -0.0019 0.4989 0.9752 0.3453 6.6109 1 0
#> cluster 3 0 0.7476 0.0010 0.7153 0.8210 1.8112 1.1891 1 0
#> sigma2eps
#> cluster 1 49.8337
#> cluster 2 34.3333
#> cluster 3 19.9660The algorithm alternates two steps. Given the labels, a STEM model is
estimated in each regime. Given the parameters, the labels are updated
by a sequential ICM sweep (Besag, 1986): locations are
visited in turn and each label maximizes its own penalized contribution
given the current labels of all the others. Because the penalty is
recomputed on the fly, the sweep cannot decrease \(Q\) at fixed parameters and cannot
cycle - unlike the simultaneous update, still available through
label_update = "simultaneous".
Where the alternation starts depends on the penalty. An unpenalized
fit starts from the k-means of how each location departs from the pooled
model - the mean of its residual, the slopes of the residual on the
covariates, and the autocorrelation and variance of what is left
(init_method = "departures", the default). A penalized fit
such as the one above fits \(\phi = 0\)
first and starts from its solution.
The alternation as a whole, however, is not guaranteed to be
monotone, and the trace below may well show a dip. The reason is
structural: \(\ell_{ik}\) is a
conditional pseudo-likelihood, because the spatial covariance
couples the locations and the exact marginal likelihood does not
factorize across them, while the parameter step maximizes the exact
within-cluster likelihood through the EM algorithm. A parameter update
can therefore lower \(Q\) while raising
the exact likelihood. This is a property of the spatio-temporal
specification, not a numerical artifact. To keep the answer well
defined, SCSTEM_Estimation() always returns the best
partition visited, and reports both objectives
(best_objective, last_objective).
plot(povalley$coords, col = fit$group, pch = 19, cex = 1.2,
xlab = "Longitude", ylab = "Latitude", main = "Estimated regimes")
legend("topright", legend = paste("regime", seq_len(3)),
col = seq_len(3), pch = 19, bty = "n")Within-cluster homogeneity is exactly what the assignment step pursues, so the regime-specific variances shrink as locations are reallocated, and the regime with the smallest residual variance tends to attract everything. This is the clusterwise counterpart of the degenerate-likelihood problem of Gaussian mixtures. Two defaults keep it under control:
enforce_min_size = TRUE forbids a move that would take
a regime below min_cluster_size;swap_pass = TRUE then restores mobility by exchanging
the labels of two locations whenever that increases \(Q\), which leaves the regime sizes - and
hence feasibility - untouched.Each location contributes \(T\)
observations to the likelihood, so a value of \(\phi\) borrowed from a cross-sectional
clustered model would be far too weak. The default
phi_scale = "auto" rescales the penalty by the typical
spread of the location-wise log-likelihood contributions, so that \(\phi \approx 1\) is the point where
agreeing with one’s neighbors is worth about as much as picking the
best-fitting regime.
A fit with \(\phi > 0\) starts
from the partition of the unpenalized fit at the same \(K\), which SCSTEM_Estimation()
computes first, and the spread is measured there, on fitted regimes. The
scale is then the same for every \(\phi\) at that \(K\) and does not depend on the
initialization, and it is large: it is of the order of the whole gain of
the right regime over the wrong ones, so the useful values of \(\phi\) are a few hundredths.
SCSTEM_Infocrit() fits the whole grid and reports the
criteria together with an admissibility flag.
ic <- SCSTEM_Infocrit(mod, K_grid = 1:3, phi_grid = c(0, 0.025, 0.05, 0.1, 0.2),
distance = "geo", precision = 0.05)
ic$table[, c("K", "phi", "K_eff", "admissible", "loglik", "BIC", "min_size")]
#> K phi K_eff admissible loglik BIC min_size
#> 1 1 0.000 1 TRUE -17469.92 35017.19 36
#> 2 2 0.000 2 TRUE -17051.58 34257.85 15
#> 3 2 0.025 2 TRUE -17051.58 34257.85 15
#> 4 2 0.050 2 TRUE -17051.58 34257.85 15
#> 5 2 0.100 2 TRUE -17051.58 34257.85 15
#> 6 2 0.200 2 TRUE -17051.58 34257.85 15
#> 7 3 0.000 3 TRUE -16843.06 33918.16 7
#> 8 3 0.025 3 TRUE -16843.06 33918.16 7
#> 9 3 0.050 3 TRUE -16843.06 33918.16 7
#> 10 3 0.100 3 TRUE -16843.06 33918.16 7
#> 11 3 0.200 3 TRUE -16843.06 33918.16 7Two caveats make a naive “minimize the BIC” rule unsafe here. The
partition is itself optimized on the data, so the maximized likelihood
keeps an optimism bias that the parameter count does not fully correct;
and the shrinking variance components inflate the likelihood of the
least regularized configurations. Both effects are strongest at \(\phi \approx 0\), where in-sample criteria
overselect \(K\) - and where fits often
collapse a regime altogether, which is what the admissible
column is for.
SCSTEM_Select() therefore applies a two-step rule: pick
\(K\) by BIC inside a moderate
band of penalties, then pick the \(\phi\) with the smallest BIC at that \(K\) over the whole grid. Every penalized
fit starts from the unpenalized one, so a penalty is chosen only when it
leads to a partition with a higher likelihood. The default grid of
SCSTEM_Infocrit() adds the strong values \(0.5\) and \(1\), which compete in the second step only;
this example uses the moderate values to keep it fast.
sel <- SCSTEM_Select(ic, band = c(0.025, 0.2))
sel
#> Two-step selection of the SC-STEM hyperparameters
#> moderate-penalty band : [0.025, 0.2]
#> criterion : BIC
#>
#> (S1) criterion-minimizing K within the band
#> phi K_best value
#> 0.025 3 33918.16
#> 0.050 3 33918.16
#> 0.100 3 33918.16
#> 0.200 3 33918.16
#>
#> (S2) criterion over the penalties at K = 3
#> phi value selected
#> 0.000 33918.16 TRUE
#> 0.025 33918.16 FALSE
#> 0.050 33918.16 FALSE
#> 0.100 33918.16 FALSE
#> 0.200 33918.16 FALSE
#>
#> Selected configuration: K = 3 , phi = 0
#> loglik = -16843.06 | BIC = 33918.16 | cluster sizes >= 7
#> pooled reference (K = 1): BIC = 35017.19The pooled \(K = 1\) model competes
in the first step: when no partition improves on it,
SCSTEM_Select() returns it and the second step does not
apply.
Conditioning on the estimated partition would understate the
uncertainty, because the partition is itself an estimate.
SCSTEM_Bootstrap() therefore generates data regime by
regime from the fitted model and re-runs the entire procedure -
clustering included - on every draw.
best <- sel$fit
boot <- SCSTEM_Bootstrap(best, B = 12, seed = 1)
boot
#> Refit-with-clustering parametric bootstrap for SC-STEM
#> replicates requested : 12
#> usable replicates : 12 (100%)
#> clusters : 3
#> spatial penalty : phi = 0SCSTEM_BootInference() relocates each refit onto the
original regimes by the majority rule and summarizes the draws.
inf <- SCSTEM_BootInference(boot)
inf
#> SC-STEM bootstrap inference (12 usable draws, level = 0.95)
#>
#> cluster parameter estimate se t p_value signif
#> 1 sigma2omega 0.00000 0.00000 29.55975 0.00000 ***
#> 1 beta1 6.35542 2.79392 2.27473 0.02292 *
#> 1 beta2 0.00686 0.00380 1.80396 0.07124 .
#> 1 beta3 0.51850 0.02204 23.52840 0.00000 ***
#> 1 G 0.91754 0.05388 17.02812 0.00000 ***
#> 1 Sigmaeta 5.77836 1.35676 4.25893 0.00002 ***
#> 1 m0 -1.79451 4.09425 -0.43830 0.66117
#> 1 C0 1.00000 0.00000 NA NA
#> 1 theta 0.00000 0.00000 NA NA
#> 1 sigma2eps 49.83369 1.68586 29.55975 0.00000 ***
#> 2 sigma2omega 0.00000 0.00000 28.46119 0.00000 ***
#> 2 beta1 2.55019 2.37703 1.07285 0.28334
#> 2 beta2 -0.00187 0.00191 -0.98086 0.32666
#> 2 beta3 0.49891 0.01102 45.28169 0.00000 ***
#> 2 G 0.97516 0.01790 54.46294 0.00000 ***
#> 2 Sigmaeta 0.34531 0.10398 3.32106 0.00090 ***
#> 2 m0 6.61086 2.53331 2.60957 0.00907 **
#> 2 C0 1.00000 0.00000 NA NA
#> 2 theta 0.00000 0.00000 2.43113 0.01505 *
#> 2 sigma2eps 34.33331 1.20632 28.46118 0.00000 ***
#> 3 sigma2omega 0.00000 0.00000 44.37988 0.00000 ***
#> 3 beta1 0.74755 0.68480 1.09164 0.27499
#> 3 beta2 0.00101 0.00108 0.92925 0.35276
#> 3 beta3 0.71528 0.00917 77.98751 0.00000 ***
#> 3 G 0.82099 0.08235 9.96932 0.00000 ***
#> 3 Sigmaeta 1.81122 0.19079 9.49345 0.00000 ***
#> 3 m0 1.18910 3.19826 0.37180 0.71004
#> 3 C0 1.00000 0.00000 NA NA
#> 3 theta 0.00000 0.00000 2.79644 0.00517 **
#> 3 sigma2eps 19.96603 0.44989 44.37988 0.00000 ***
#>
#> Median ARI of the refit partitions against the original: 1
#> Use $summary for the confidence intervals, $tests for the between-cluster
#> comparisons and $coclustering for the stability of the partition.Percentile intervals for the regression coefficients:
s <- inf$summary
s[grepl("beta", s$parameter),
c("cluster", "parameter", "estimate", "se",
grep("^perc_", names(s), value = TRUE))]
#> cluster parameter estimate se perc_lower_95_CI perc_upper_95_CI
#> 2 1 beta1 6.35542 2.79392 1.04044 9.38772
#> 3 1 beta2 0.00686 0.00380 0.00079 0.01205
#> 4 1 beta3 0.51850 0.02204 0.47930 0.54382
#> 12 2 beta1 2.55019 2.37703 -0.12659 7.00009
#> 13 2 beta2 -0.00187 0.00191 -0.00423 0.00177
#> 14 2 beta3 0.49891 0.01102 0.48807 0.51843
#> 22 3 beta1 0.74755 0.68480 -0.27528 1.60202
#> 23 3 beta2 0.00101 0.00108 0.00032 0.00326
#> 24 3 beta3 0.71528 0.00917 0.70304 0.72897And the question that matters substantively - do the regimes really differ?
inf$tests[grepl("beta", inf$tests$parameter), ]
#> parameter cluster_a cluster_b diff_estimate diff_boot_mean lower_95_CI
#> 2 beta1 1 2 3.80523 3.17927 -5.95965
#> 3 beta2 1 2 0.00873 0.00677 0.00211
#> 4 beta3 1 2 0.01959 0.01580 -0.01561
#> 12 beta1 1 3 5.60787 5.90285 0.44710
#> 13 beta2 1 3 0.00585 0.00386 -0.00110
#> 14 beta3 1 3 -0.19678 -0.20280 -0.23195
#> 22 beta1 2 3 1.80264 2.72358 -0.44656
#> 23 beta2 2 3 -0.00288 -0.00292 -0.00584
#> 24 beta3 2 3 -0.21637 -0.21860 -0.23645
#> upper_95_CI p_value n_draws signif
#> 2 7.55962 0.33333 12
#> 3 0.01175 0.00000 12 ***
#> 4 0.04310 0.50000 12
#> 12 9.04202 0.00000 12 ***
#> 13 0.01156 0.33333 12
#> 14 -0.16771 0.00000 12 ***
#> 22 6.40676 0.16667 12
#> 23 0.00031 0.50000 12
#> 24 -0.18949 0.00000 12 ***The co-clustering matrix records how often two locations end up in the same regime across the refits, and is a direct measure of how reproducible the partition is:
Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society, Series B, 48, 259–302.
Cerqueti, R., Maranzano, P. and Mattera, R. (2025). Spatially-clustered spatial autoregressive models with application to agricultural market concentration in Europe. Journal of Agricultural, Biological and Environmental Statistics.
Sugasawa, S. and Murakami, D. (2021). Spatially clustered regression. Spatial Statistics, 44, 100525.