Bayesian nonparametric analysis of reversible Markov chains

We introduce a three parameter random walk with reinforcement, called the (t,a,b) scheme, which generalizes the linearly edge reinforced random walk to uncountable spaces. The parameter b smoothly tunes the (t,a,b) scheme between this edge reinforced random walk and the classical exchangeable two-parameter Hoppe urn scheme, while the parameters a and t modulate how many states are typically visited. Resorting to de Finetti’s theorem for Markov chains, we use the (t,a,b) scheme to define a nonparametric prior for Bayesian analysis of reversible Markov chains. The prior is applied in Bayesian nonparametric inference for species sampling problems with data generated from a reversible Markov chain with an unknown transition kernel. As a real example, we analyze data from molecular dynamics simulations of protein folding.