We introduce a nonparametric model for inferring time-evolving, unobserved prob- ability distributions from discrete-time data consisting of unlabelled partitions. The latent process is a two-parameter Poisson–Dirichlet diffusion, and observations arise via exchangeable sampling. Applications include social and genetic data where only aggregate clustering summaries are observed. To address the intractable likelihood, we develop a tractable inferential framework that avoids label enumeration and direct simulation of the latent state. We exploit a duality between the diffusion and a pure-death process on partitions, together with coagulation operators that encode the effect of new data. These yield closed-form, recursive updates for forward and backward inference. We compute exact posterior distributions of the latent state at arbitrary times and predictive distributions of future or interpolated partitions. This enables online and offline inference and forecasting with full uncertainty quantification, bypassing MCMC and sequential Monte Carlo. Compared to particle filtering, our method achieves higher accuracy, lower variance, and substantial computational gains. We illustrate the methodology with synthetic experiments and a social network applica- tion, recovering interpretable patterns in time-varying heterozygosity.

Exact inference via quasi-conjugacy in two-parameter Poisson–Dirichlet hidden Markov models

Marco Dalla Pria
;
Matteo Ruggiero;
2026-01-01

Abstract

We introduce a nonparametric model for inferring time-evolving, unobserved prob- ability distributions from discrete-time data consisting of unlabelled partitions. The latent process is a two-parameter Poisson–Dirichlet diffusion, and observations arise via exchangeable sampling. Applications include social and genetic data where only aggregate clustering summaries are observed. To address the intractable likelihood, we develop a tractable inferential framework that avoids label enumeration and direct simulation of the latent state. We exploit a duality between the diffusion and a pure-death process on partitions, together with coagulation operators that encode the effect of new data. These yield closed-form, recursive updates for forward and backward inference. We compute exact posterior distributions of the latent state at arbitrary times and predictive distributions of future or interpolated partitions. This enables online and offline inference and forecasting with full uncertainty quantification, bypassing MCMC and sequential Monte Carlo. Compared to particle filtering, our method achieves higher accuracy, lower variance, and substantial computational gains. We illustrate the methodology with synthetic experiments and a social network applica- tion, recovering interpretable patterns in time-varying heterozygosity.
2026
1
61
https://www.tandfonline.com/doi/full/10.1080/01621459.2026.2676717
Bayesian nonparametrics; Chinese restaurant process; Filtering and smoothing; Markov duality; Partition structures; Young diagrams
Marco Dalla Pria; Matteo Ruggiero; Dario Spanò
File in questo prodotto:
File Dimensione Formato  
Dalla Pria, Ruggiero, Spanò - JASA.pdf

Accesso aperto

Descrizione: Dalla Pria, M., Ruggiero, M., Spanò, D. (2026) - Exact inference via quasi-conjugacy in two-parameter Poisson–Dirichlet hidden Markov models
Tipo di file: POSTPRINT (VERSIONE FINALE DELL’AUTORE)
Dimensione 6.45 MB
Formato Adobe PDF
6.45 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2318/2148510
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact