2. Spatially-clustered STEM models

library(Stem)

Why cluster the regression

A STEM model imposes a single vector of regression coefficients on the whole monitoring network. Over a large and heterogeneous domain this is often not credible: the way concentrations respond to altitude or to a co-pollutant may differ between the center of a plain and the areas nearer the reliefs.

The spatially-clustered STEM (SC-STEM) model relaxes the assumption. Each location \(i\) belongs to one of \(K\) latent regimes, and conditionally on regime \(k\)

\[ z_{it} = x_{it}' \beta_k + A_i y^{(k)}_t + e_{it}, \qquad y^{(k)}_t = G_k y^{(k)}_{t-1} + \eta^{(k)}_t, \]

so that coefficients, variance components and latent dynamics are all regime-specific. Setting \(K = 1\) returns the pooled fit.

Labels and parameters are estimated jointly by maximizing a Potts-penalized log-likelihood

\[ Q = \sum_{i=1}^{d} \ell_i(k_i) + \phi\, c \sum_{(i,j) \in E} I(k_i = k_j), \]

where \(E\) is the edge set of the symmetrized \(k\)-nearest-neighbor graph, each unordered pair counted once, and \(\phi \ge 0\) controls how strongly neighboring locations are pushed into the same regime. This is the penalty of Sugasawa and Murakami (2021), in the form used for spatially-clustered Fay–Herriot and spatial autoregressive models.

The data

data(povalley)

Tn <- 150L                       # a five-month slice, to keep the vignette quick
Tfull <- nrow(povalley$z)
d <- ncol(povalley$z)
keep <- as.vector(outer(seq_len(Tn), (seq_len(d) - 1L) * Tfull, "+"))

z <- povalley$z[seq_len(Tn), ]
X <- povalley$covariates[keep, ]

dim(z)
#> [1] 150  36
head(round(povalley$coords, 3), 3)
#>         Longitude Latitude
#> IT0470A     7.695   45.104
#> IT0594A    10.224   45.541
#> IT0692A     8.906   45.102
plot(povalley$coords, pch = 19, cex = 0.9,
     xlab = "Longitude", ylab = "Latitude",
     main = "Po Valley background stations")

Location of the 36 monitoring stations

phi <- list(
  beta        = matrix(c(1.25, -0.00003, 0.64), 3, 1),
  sigma2eps   = 18.66,
  sigma2omega = 1e-06,
  theta       = 2e-06,
  G           = matrix(0.59, 1, 1),
  Sigmaeta    = matrix(4.25, 1, 1),
  m0          = as.matrix(0),
  C0          = as.matrix(1)
)

mod <- STEM_Model(z = z, covariates = X, coordinates = povalley$coords,
                  phi = phi, A = matrix(1, d, 1))

Coordinates are longitude and latitude, so every call below uses distance = "geo".

Fitting an SC-STEM model

fit <- SCSTEM_Estimation(mod, K = 3, phi_penalty = 0.05, distance = "geo",
                    precision = 0.05)
fit
#> Spatially-clustered STEM model
#>   regimes requested  : K = 3
#>   regimes estimated  : K_eff = 3
#>   spatial penalty    : phi = 0.05 (auto, knn = 5)
#>   label update       : ICM
#>   convergence        : Convergence reached (labels stable)
#>   cluster sizes      : 7, 11, 18
#> 
#> Information criteria
#>    loglik        df       AIC       BIC       KIC 
#> -16843.06     27.00  33740.12  33918.16  33767.12 
#> 
#> Cluster-wise estimates
#>           sigma2omega  beta1   beta2  beta3      G Sigmaeta      m0 C0 theta
#> cluster 1           0 6.3554  0.0069 0.5185 0.9175   5.7784 -1.7945  1     0
#> cluster 2           0 2.5502 -0.0019 0.4989 0.9752   0.3453  6.6109  1     0
#> cluster 3           0 0.7476  0.0010 0.7153 0.8210   1.8112  1.1891  1     0
#>           sigma2eps
#> cluster 1   49.8337
#> cluster 2   34.3333
#> cluster 3   19.9660

The algorithm alternates two steps. Given the labels, a STEM model is estimated in each regime. Given the parameters, the labels are updated by a sequential ICM sweep (Besag, 1986): locations are visited in turn and each label maximizes its own penalized contribution given the current labels of all the others. Because the penalty is recomputed on the fly, the sweep cannot decrease \(Q\) at fixed parameters and cannot cycle - unlike the simultaneous update, still available through label_update = "simultaneous".

Where the alternation starts depends on the penalty. An unpenalized fit starts from the k-means of how each location departs from the pooled model - the mean of its residual, the slopes of the residual on the covariates, and the autocorrelation and variance of what is left (init_method = "departures", the default). A penalized fit such as the one above fits \(\phi = 0\) first and starts from its solution.

The alternation as a whole, however, is not guaranteed to be monotone, and the trace below may well show a dip. The reason is structural: \(\ell_{ik}\) is a conditional pseudo-likelihood, because the spatial covariance couples the locations and the exact marginal likelihood does not factorize across them, while the parameter step maximizes the exact within-cluster likelihood through the EM algorithm. A parameter update can therefore lower \(Q\) while raising the exact likelihood. This is a property of the spatio-temporal specification, not a numerical artifact. To keep the answer well defined, SCSTEM_Estimation() always returns the best partition visited, and reports both objectives (best_objective, last_objective).

fit$obj_trace
#>   iter objective label_changes swaps min_cluster
#> 1    1  -16561.6             0     0           7
plot(povalley$coords, col = fit$group, pch = 19, cex = 1.2,
     xlab = "Longitude", ylab = "Latitude", main = "Estimated regimes")
legend("topright", legend = paste("regime", seq_len(3)),
       col = seq_len(3), pch = 19, bty = "n")

Estimated spatial regimes

Two safeguards worth knowing about

Within-cluster homogeneity is exactly what the assignment step pursues, so the regime-specific variances shrink as locations are reallocated, and the regime with the smallest residual variance tends to attract everything. This is the clusterwise counterpart of the degenerate-likelihood problem of Gaussian mixtures. Two defaults keep it under control:

The scale of the penalty

Each location contributes \(T\) observations to the likelihood, so a value of \(\phi\) borrowed from a cross-sectional clustered model would be far too weak. The default phi_scale = "auto" rescales the penalty by the typical spread of the location-wise log-likelihood contributions, so that \(\phi \approx 1\) is the point where agreeing with one’s neighbors is worth about as much as picking the best-fitting regime.

A fit with \(\phi > 0\) starts from the partition of the unpenalized fit at the same \(K\), which SCSTEM_Estimation() computes first, and the spread is measured there, on fitted regimes. The scale is then the same for every \(\phi\) at that \(K\) and does not depend on the initialization, and it is large: it is of the order of the whole gain of the right regime over the wrong ones, so the useful values of \(\phi\) are a few hundredths.

c(multiplier = fit$phi_multiplier, effective_phi = fit$phi_effective)
#>    multiplier effective_phi 
#>    13.4485055     0.6724253

Choosing \(K\) and \(\phi\)

SCSTEM_Infocrit() fits the whole grid and reports the criteria together with an admissibility flag.

ic <- SCSTEM_Infocrit(mod, K_grid = 1:3, phi_grid = c(0, 0.025, 0.05, 0.1, 0.2),
                      distance = "geo", precision = 0.05)
ic$table[, c("K", "phi", "K_eff", "admissible", "loglik", "BIC", "min_size")]
#>    K   phi K_eff admissible    loglik      BIC min_size
#> 1  1 0.000     1       TRUE -17469.92 35017.19       36
#> 2  2 0.000     2       TRUE -17051.58 34257.85       15
#> 3  2 0.025     2       TRUE -17051.58 34257.85       15
#> 4  2 0.050     2       TRUE -17051.58 34257.85       15
#> 5  2 0.100     2       TRUE -17051.58 34257.85       15
#> 6  2 0.200     2       TRUE -17051.58 34257.85       15
#> 7  3 0.000     3       TRUE -16843.06 33918.16        7
#> 8  3 0.025     3       TRUE -16843.06 33918.16        7
#> 9  3 0.050     3       TRUE -16843.06 33918.16        7
#> 10 3 0.100     3       TRUE -16843.06 33918.16        7
#> 11 3 0.200     3       TRUE -16843.06 33918.16        7

Two caveats make a naive “minimize the BIC” rule unsafe here. The partition is itself optimized on the data, so the maximized likelihood keeps an optimism bias that the parameter count does not fully correct; and the shrinking variance components inflate the likelihood of the least regularized configurations. Both effects are strongest at \(\phi \approx 0\), where in-sample criteria overselect \(K\) - and where fits often collapse a regime altogether, which is what the admissible column is for.

SCSTEM_Select() therefore applies a two-step rule: pick \(K\) by BIC inside a moderate band of penalties, then pick the \(\phi\) with the smallest BIC at that \(K\) over the whole grid. Every penalized fit starts from the unpenalized one, so a penalty is chosen only when it leads to a partition with a higher likelihood. The default grid of SCSTEM_Infocrit() adds the strong values \(0.5\) and \(1\), which compete in the second step only; this example uses the moderate values to keep it fast.

sel <- SCSTEM_Select(ic, band = c(0.025, 0.2))
sel
#> Two-step selection of the SC-STEM hyperparameters
#>   moderate-penalty band : [0.025, 0.2]
#>   criterion             : BIC
#> 
#> (S1) criterion-minimizing K within the band
#>    phi K_best    value
#>  0.025      3 33918.16
#>  0.050      3 33918.16
#>  0.100      3 33918.16
#>  0.200      3 33918.16
#> 
#> (S2) criterion over the penalties at K = 3
#>    phi    value selected
#>  0.000 33918.16     TRUE
#>  0.025 33918.16    FALSE
#>  0.050 33918.16    FALSE
#>  0.100 33918.16    FALSE
#>  0.200 33918.16    FALSE
#> 
#> Selected configuration: K = 3 , phi = 0
#>   loglik = -16843.06 | BIC = 33918.16 | cluster sizes >= 7
#>   pooled reference (K = 1): BIC = 35017.19

The pooled \(K = 1\) model competes in the first step: when no partition improves on it, SCSTEM_Select() returns it and the second step does not apply.

Uncertainty: the refit-with-clustering bootstrap

Conditioning on the estimated partition would understate the uncertainty, because the partition is itself an estimate. SCSTEM_Bootstrap() therefore generates data regime by regime from the fitted model and re-runs the entire procedure - clustering included - on every draw.

best <- sel$fit
boot <- SCSTEM_Bootstrap(best, B = 12, seed = 1)
boot
#> Refit-with-clustering parametric bootstrap for SC-STEM
#>   replicates requested : 12
#>   usable replicates    : 12 (100%)
#>   clusters             : 3
#>   spatial penalty      : phi = 0

SCSTEM_BootInference() relocates each refit onto the original regimes by the majority rule and summarizes the draws.

inf <- SCSTEM_BootInference(boot)
inf
#> SC-STEM bootstrap inference (12 usable draws, level = 0.95)
#> 
#>  cluster   parameter estimate      se        t p_value signif
#>        1 sigma2omega  0.00000 0.00000 29.55975 0.00000    ***
#>        1       beta1  6.35542 2.79392  2.27473 0.02292      *
#>        1       beta2  0.00686 0.00380  1.80396 0.07124      .
#>        1       beta3  0.51850 0.02204 23.52840 0.00000    ***
#>        1           G  0.91754 0.05388 17.02812 0.00000    ***
#>        1    Sigmaeta  5.77836 1.35676  4.25893 0.00002    ***
#>        1          m0 -1.79451 4.09425 -0.43830 0.66117       
#>        1          C0  1.00000 0.00000       NA      NA       
#>        1       theta  0.00000 0.00000       NA      NA       
#>        1   sigma2eps 49.83369 1.68586 29.55975 0.00000    ***
#>        2 sigma2omega  0.00000 0.00000 28.46119 0.00000    ***
#>        2       beta1  2.55019 2.37703  1.07285 0.28334       
#>        2       beta2 -0.00187 0.00191 -0.98086 0.32666       
#>        2       beta3  0.49891 0.01102 45.28169 0.00000    ***
#>        2           G  0.97516 0.01790 54.46294 0.00000    ***
#>        2    Sigmaeta  0.34531 0.10398  3.32106 0.00090    ***
#>        2          m0  6.61086 2.53331  2.60957 0.00907     **
#>        2          C0  1.00000 0.00000       NA      NA       
#>        2       theta  0.00000 0.00000  2.43113 0.01505      *
#>        2   sigma2eps 34.33331 1.20632 28.46118 0.00000    ***
#>        3 sigma2omega  0.00000 0.00000 44.37988 0.00000    ***
#>        3       beta1  0.74755 0.68480  1.09164 0.27499       
#>        3       beta2  0.00101 0.00108  0.92925 0.35276       
#>        3       beta3  0.71528 0.00917 77.98751 0.00000    ***
#>        3           G  0.82099 0.08235  9.96932 0.00000    ***
#>        3    Sigmaeta  1.81122 0.19079  9.49345 0.00000    ***
#>        3          m0  1.18910 3.19826  0.37180 0.71004       
#>        3          C0  1.00000 0.00000       NA      NA       
#>        3       theta  0.00000 0.00000  2.79644 0.00517     **
#>        3   sigma2eps 19.96603 0.44989 44.37988 0.00000    ***
#> 
#> Median ARI of the refit partitions against the original: 1
#> Use $summary for the confidence intervals, $tests for the between-cluster
#> comparisons and $coclustering for the stability of the partition.

Percentile intervals for the regression coefficients:

s <- inf$summary
s[grepl("beta", s$parameter),
  c("cluster", "parameter", "estimate", "se",
    grep("^perc_", names(s), value = TRUE))]
#>    cluster parameter estimate      se perc_lower_95_CI perc_upper_95_CI
#> 2        1     beta1  6.35542 2.79392          1.04044          9.38772
#> 3        1     beta2  0.00686 0.00380          0.00079          0.01205
#> 4        1     beta3  0.51850 0.02204          0.47930          0.54382
#> 12       2     beta1  2.55019 2.37703         -0.12659          7.00009
#> 13       2     beta2 -0.00187 0.00191         -0.00423          0.00177
#> 14       2     beta3  0.49891 0.01102          0.48807          0.51843
#> 22       3     beta1  0.74755 0.68480         -0.27528          1.60202
#> 23       3     beta2  0.00101 0.00108          0.00032          0.00326
#> 24       3     beta3  0.71528 0.00917          0.70304          0.72897

And the question that matters substantively - do the regimes really differ?

inf$tests[grepl("beta", inf$tests$parameter), ]
#>    parameter cluster_a cluster_b diff_estimate diff_boot_mean lower_95_CI
#> 2      beta1         1         2       3.80523        3.17927    -5.95965
#> 3      beta2         1         2       0.00873        0.00677     0.00211
#> 4      beta3         1         2       0.01959        0.01580    -0.01561
#> 12     beta1         1         3       5.60787        5.90285     0.44710
#> 13     beta2         1         3       0.00585        0.00386    -0.00110
#> 14     beta3         1         3      -0.19678       -0.20280    -0.23195
#> 22     beta1         2         3       1.80264        2.72358    -0.44656
#> 23     beta2         2         3      -0.00288       -0.00292    -0.00584
#> 24     beta3         2         3      -0.21637       -0.21860    -0.23645
#>    upper_95_CI p_value n_draws signif
#> 2      7.55962 0.33333      12       
#> 3      0.01175 0.00000      12    ***
#> 4      0.04310 0.50000      12       
#> 12     9.04202 0.00000      12    ***
#> 13     0.01156 0.33333      12       
#> 14    -0.16771 0.00000      12    ***
#> 22     6.40676 0.16667      12       
#> 23     0.00031 0.50000      12       
#> 24    -0.18949 0.00000      12    ***

The co-clustering matrix records how often two locations end up in the same regime across the refits, and is a direct measure of how reproducible the partition is:

round(inf$coclustering[1:6, 1:6], 2)
#>         IT0470A IT0594A IT0692A IT0707A IT0771A IT0778A
#> IT0470A       1       0       0       0       0       0
#> IT0594A       0       1       1       0       0       1
#> IT0692A       0       1       1       0       0       1
#> IT0707A       0       0       0       1       1       0
#> IT0771A       0       0       0       1       1       0
#> IT0778A       0       1       1       0       0       1

References

Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society, Series B, 48, 259–302.

Cerqueti, R., Maranzano, P. and Mattera, R. (2025). Spatially-clustered spatial autoregressive models with application to agricultural market concentration in Europe. Journal of Agricultural, Biological and Environmental Statistics.

Sugasawa, S. and Murakami, D. (2021). Spatially clustered regression. Spatial Statistics, 44, 100525.