Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models.

3rd Symposium on Advances in Approximate Bayesian Inference, 2020 1{37
Transforming Worlds: Automated Involutive MCMC for
Open-Universe Probabilistic Models
George Matheos1;2;georgematheos@berkeley.edu
Alexander K. Lew1;alexlew@mit.edu
Matin Ghavamizadeh1;2mghavami@berkeley.edu
Stuart Russell2russell@berkeley.edu
Marco Cusumano-Towner1marcoct@mit.edu
Vikash K. Mansinghka1vkm@mit.edu
1MIT2UC BerkeleyEqual contribution
1. Introduction
Open-universe probabilistic models (OUPMs) enable Bayesian inference about the existence
and attributes of latent objects underlying observed data, as well as their interconnections.
Prominent applications include seismic monitoring for global nuclear safety (Arora et al.,
2013; Arora, 2012); information extraction from natural language documents (Russell et al.,
2016); generating realistic 3D scenes with varying numbers of objects (Yeh et al., 2012); \in-
verse graphics" approaches to breaking CAPTCHAs (Mansinghka et al., 2013) and inferring
3D scenes from 2D data (Kulkarni et al., 2015; Zinberg et al., 2019); and simultaneous de-
duplication, cleaning, and record linkage from real-world databases with millions of records
(Lew et al., 2020). Because the state space in OUPMs has a priori unknown dimension,
popular black-box inference algorithms such as Hamiltonian Monte Carlo cannot be applied.
Instead, applications rely on custom MCMC kernels. These kernels use application-specic,
data-driven heuristics to intelligently delete, split, or merge hypothesized objects, \birth"
new objects, and modify the properties of and relationships between objects. Although these
proposals can be designed and justied in the reversible-jump MCMC framework, this is
often challenging. As Brooks et al. (2003) noted, \the application of reversible jump...has
predominantly remained within the domain of the MCMC expert," due to the diculty of
deriving and implementing eective RJMCMC kernels.
This abstract introduces a new framework for OUPM inference that makes it easier to de-
sign and implement custom, data-driven kernels. We adapt involutive MCMC (Cusumano-
Towner et al., 2020; Neklyudov et al., 2020), a generalization of reversible-jump MCMC, to
the setting of OUPMs. This enables software to automatically generate correct and scalable
implementations of complex, application-specic kernels from high-level user specications.
Users design inference programs that propose incremental changes to possible worlds, creat-
ing, deleting, or modifying objects according to data-driven, application-specic heuristics;
these proposals are automatically converted into stationary MCMC kernels via an accep-
t/reject step. To automatically compute the acceptance probability, our approach leverages
program tracing and dependency tracking (for ecient computation of proposal and model
densities) and dierentiable programming (for the Jacobian determinant). Preliminary ex-
periments on (1) an auditory scene analysis application (Cusimano et al., 2018) (Figure 3)
and (2) Gaussian mixture modeling with an unknown number of components (Richard-
son and Green, 1997) (Figure 4) show that our approach enables data-driven kernels that
c
G. Matheos, A.K. Lew, M. Ghavamizadeh, S. Russell, M. Cusumano-Towner & V.K. Mansinghka.
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
are faster than generic probabilistic programming language (PPL) inference and generic
birth/death RJMCMC kernels without application-specic customizations.
Related Work. We build on involutive MCMC, introduced by Cusumano-Towner et al.
(2020) as a framework supporting higher degrees of automation for application-specic
MCMC algorithms, and (independently) as a mathematical generalization of many clas-
sic MCMC methods by Neklyudov et al. (2020). Our main theoretical contribution is a
generalization of involutive MCMC to OUPMs. We compare our approach to the generic
ancestral resampling MH algorithm (Wingate et al., 2011) used by many probabilistic pro-
gramming languages supporting OUPMs, such as BLOG (Milch et al., 2005a). Previous
work has explored using application-specic kernels for BLOG models (Milch and Russell,
2012), but these require modifying the BLOG inference engine's source code and man-
ually deriving parts of the acceptance ratio. The Gen (Cusumano-Towner et al., 2019)
and Stochaskell (Roberts et al., 2019) PPLs support forms of automated reversible-jump
MCMC, but neither features high-level OUPM inference constructs; custom kernels in both
languages operate on low-level program traces, not high-level object-based world represen-
tations, which can compromise both ease of use and asymptotic performance. Furthermore,
the automation technique proposed by Roberts et al. (2019) does not support unrestricted
use of auxiliary variables, and (e.g.) cannot handle the algorithms benchmarked in Figure 3.
2. Open Universe Probabilistic Models
Figure 1 shows an open-universe model for inferring the set of seismic events underlying
observed detections from seismic monitoring stations. A priori, it is unknown how many
events there are, and which detections correspond to the same event (or, in the case of
spurious false positives, to no event at all). Standard probabilistic graphical models are
ill-suited for encoding this sort of uncertainty about the number of objects re
ected in a
dataset and their causal relationships. Open-universe probabilistic models enable reasoning
about such problems, by dening generative processes over entire relational domains.
Model specication. An OUPM is fully specied by: (1) a nite set Tof object types
(in Figure 1, Event ,Station , and Detection ); (2) a nite set Nofpossible origins for
each object type (in Figure 1, a Detection may originate from a ( Station;Event ) pair,
or from a single ( Station ) that records a false positive); (3) a countable set Pof typed
object properties P(1;:::;n) (such as magnitude(Event) andreading(Detection) ); and
(4) a contingent Bayesian network Cover a set of random variables that encode the number
of objects that exist, as well as their properties and relationships (Milch et al., 2005b).
Contingent Bayesian networks. The probabilistic structure of an OUPM is given by a
contingent Bayesian network Cover an innite set Vofpossible variables . These variables
are dened in terms of objects : we write((o1;:::;on);i) to refer to the ithobject of type
with origin ( o1;:::;on). Then each node v2Vof the contingent Bayesian network corre-
sponds to either a number variable N(o1;:::;on), which represents the number of objects of
typewith origin ( o1;:::;on), or to a property variable P(o1;:::;on), which represents the
value of the property Pfor the given tuple of objects. Like an ordinary Bayesian network,
a contingent Bayesian network uses a directed graph to describe the probabilistic depen-
dencies among variables. However, the edges in a CBN's graph are contingent: they are
2
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
(a) A simple world model
(b) A region of the contingent Bayesian network for the model
Figure 1: An OUPM for seismic monitoring, inspired by Arora et al. (2013), in which Detection s
are received at seismic monitoring Station s. Each Station receives some number of false-positive
Detection s, and for each Event , each Station has probability 0.8 of detecting it. Each Event has
anamplitude , and a reading of a Richter-scale magnitude is recorded for each Detection , which is
distributed around the underlying Event 's magnitude when the Detection is not a false-positive.
Algorithm 1 Automated Involutive MCMC for OUPMs
procedure OUPM-IMCMC (p;q;f;x )
(y;qx(y)) SampleAndScore (q;x)
(U;M;y0;J) ProcessInvolution (f;(x;y))
(x0;p(x0)
p(x)) UpdateWorld (x;U;M )
qx0(y0) Score (q;x0;y0)
with probabilityp(x0)
p(x)qx0(y0)1
qx(y)Jreturnx0else return x
end procedure
procedure ProcessInvolution (f;(x;y))
.Track continuous reads and writes, manipulation moves M, and property updates U
(Rd;Wr;M;U;y0) fg;fg;[ ];fg;fg
Executef, but with
each manipulation command m(create ,delete ,change ,split ,merge )(M M[fmg)
"proposed [k]"(ify[k] is continuous: Rd  Rd[fy[k]g;y[k])
"set backward [k] =v"(ifvis continuous: Wr  Wr[fy[k]g;y0 y0[fk7!vg)
"get(P(o1;:::;on))"(v x[P(o1;:::;on)];ifvis continuous: Rd  Rd[fvg;v)
"setP(o1;:::;on) =v"(ifvis continuous: Wr  Wr[fvg;U[P(o1;:::;on)] v)
J uninitializedjRdjjRdjmatrix
foriinf1;:::;jWrjg:
.Executefwith reverse-mode AD to compute@Wr[i]
@Rd[j]for eachj2f1;:::;jRdjg
J[:;i] r Rd(Wr[i])
return (U;M;y0;jdetJj)
end procedure
labeled with predicates, involving the other possible variables, determining the conditions
under which they are active . The conditional distribution for a variable v,pv(![v]j!Pa!(v)),
may depend only on v'sactive parentsPa!(v) in a particular possible world !.
Induced probability distribution over minimal self-supporting instantiations.
Fixing a set UVofobserved variables (in Figure 1, fdetections()g), the CBNCcan be
used to dene a probability distribution over minimal self-supporting instantiations (MSSIs)
forU:nite assignments wto a subset of variables vars( w)V, whereUvars(w), such
that (1)wisself-supporting |ifv2vars(w) andu2Paw(v) thenu2vars(w); and (2)wis
minimal |if any variable v62Uwere removed from wit would no longer be self-supporting.
3
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
(b) An involutive MCMC kernel consisting of a proposal generative function  and an involution .Object origin relationships
before birth: Object origin relationships
after birth:
stations stations
1
21
Posit new event          to explain
Delete event        , making      into a false positive False positive
detections
          ,       1 2Dubious events Dubious events
1 1 1 2
1 21 1 1 1
1 21 2 3
False positive
detection
(a) A pair of corresponding forward and reverse moves made by the MCMC kernel defined in (b).
Figure 2: (a) An execution of a MCMC kernel transforming the state of the seismic monitoring model
on the left to that on the right. The move, with random choices shown above the right arrow, posits
a new event to explain a false positive detection. The involution sets the new event's amplitude
to 10r, converting Richter-scale magnitude rto an event amplitude, so the acceptance probability's
(d(f1)
d(x;y)) term is log(10)10r; our system calculates this via automatic dierentiation. The
kernel can also make the reverse move (  ). (b) The involutive MCMC kernel visualized in (a).
LetWbe the set of all possible minimal self-supporting instantiations for a xed set U
of observed variables, and let vars( W) =fvars(w)jw2Wg be the set of possible sets-
of-variables that can be jointly specied by a minimal self-supporting instantiation. For
v2vars(W), let vdbe the set of all possible joint assignments to the discrete variables in
v. Then we can dene a reference measure over the spaceWof MSSIs, given by
(E) =X
v2vars(W)X
wd2vdjvjjwdj(fw0
cjw02E^vars(w0) =v^8v2vars(wd);w0[v] =wd[v]g);
wherewcis the vector of values of the continuous variables in an MSSI w, andkis the
Lebesgue measure on Rk. With respect to , the probability density for the OUPM, as a
distribution over MSSIs, is given by p(w) =Q
v2vars(w)pv(w[v]jfu7!w[u]ju2Paw(v)g):1
1. Milch et al. (2005b)'s formulation of contingent Bayesian networks does not support continuous random
variables. See Appendix B for a more careful development of CBNs, and an extension (to our knowledge,
novel) to the continuous-variable case.
4
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
+
+
++
+
Figure 3: An OUPM for auditory perception. (a) A waveform (left) is observed as a spectrogram
in the frequency domain (center), from which we infer the number and type of constituent sound
sources that together explain the observed data (right). (b) Convergence proles of 4 MCMC kernels;
the two data-driven iMCMC kernels outperform generic PPL and generic RJMCMC kernels. (c)
Approximate posterior samples re
ecting typical failure modes of non-data-driven kernels.
3. Involutive MCMC for Open Universe Models
An involutive MCMC kernel (Cusumano-Towner et al., 2020; Neklyudov et al., 2020) for a
target density pon a measure space ( X;X;X) uses a three-step process to transform a
statexinto a next state x0. First, auxiliary variables yare sampled from a state-dependent
proposal with density qx, dened over a measure space ( Y;Y;Y). Second, an involution2
fis applied to ( x;y) to obtain ( x0;y0):x0is the proposed next state, and y0is an auxiliary
variable value that would cause the kernel to propose the old state xfrom the new state
x0(sincef(x0;y0) = (x;y)). Third, an acceptance probability (x;y;x0;y0) is computed.
With probability ,x0is the Markov chain's next state; otherwise the state xis repeated.
Cusumano-Towner et al. (2020) show that this process yields a stationary kernel for pwhen
(x;y;x0;y0) =p(x0)qx0(y0)
p(x)qx(y)d(f1)
d(x;y)
; (1)
whered(f1)
dis the Radon-Nikodym derivative of the pushforward of =XYthrough
fwith respect to . In general, this does not yield a constructive procedure for computing
this Radon-Nikodym derivative, but Cusumano-Towner et al. (2020) show that when pand
qare distributions over a space Dof nite dictionaries , i.e., nite sets of key-value pairs,
the Radon-Nikodym derivative can be computed as the determinant of a Jacobian matrix.
It turns out (Theorem 1) that a similar result holds when pis an open-universe model.
Supposefis an involution on the space WD , and that there is a countable partition
fCigofWD such that: (i) ( x;y)2Ci=)f(x;y)2Ci, (ii) for all ( x;y)2Ci, the total
number of continuous values in minimal self-supporting instantiation x, plus the number of
continuous values in dictionary y, is constant, and (iii) for each Ci, for (x;y)2Ci, there
2. An involution is a function that is its own inverse: f(f(x;y)) = (x;y).
5
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
is a dierentiable bijection hi:Rki!Rkimapping the continuous values in xandyto
continuous values in x0andy0(where (x0;y0) =f(x;y)). Then, we have:
Theorem 1 For any open universe model with density p, any collection of distributions
over nite dictionaries with densities qx, and any involution satisfying the above conditions,
involutive MCMC with (x;y;x0;y0) =p(x0)qx0(y0)
p(x)qx(y)jdet(Jhi(x;y))jis stationary for p, where
Jhi(x;y)is the Jacobian of hiat the continuous variables of (x;y), andiis s.t. (x;y)2Ci.
The proof involves a translation to the nite dictionary case, and exploits the structure of
minimal self-supporting instantiations to guarantee the necessary properties of f.
4. Automation of Acceptance Probability via PPLs and AD
We have extended the Gen probabilistic programming system (Cusumano-Towner et al.,
2019) with new constructs for OUPMs (sample code in Figure 1) and automated involutive
MCMC for these models (Figure 2, Algorithm 1). The contributions are: (1) a new model-
ing language for open-universe models, capable of eciently simulating MSSIs, evaluating
densities, and modifying world states; (2) a new DSL for dening object-based involutive
MCMC proposals (Appendix C); and (3) an automated implementation of involutive MCMC
for open-universe models, which automatically calculates acceptance ratios via probabilistic
and dierentiable programming techniques (Algorithm 1). See Appendix D for more details.
5. Experiments
Auditory scene analysis. We implemented a variant of Cusimano et al. (2018)'s model
of human auditory scene perception, which is based on an OUPM of sound sources with
properties including amplitude, type, pitch, and duration. See Figure 3 for an example
problem, and Appendix A.2 for details. We implemented two custom, data-driven kernels:
adata-driven sound detection kernel (Figure 11) that applies image-ltering and edge-
detection to frequency-domain spectrograms and proposes new audio sources to explain
sounds it detects in the data but not in the currently-hypothesized world, and a data-
driven sound splitting kernel (Figure 12) that splits long sounds into shorter ones at
endpoints detected in the observed soundwaves. We also implemented two baseline inference
strategies: an ancestral resampling kernel (Figure 9) based on generic PPL inference in
BLOG; and a generic birth/death RJMCMC kernel (Figure 10) that proposes to create a
sound with parameters generated from the prior, or delete a sound. Figure Figure 3(b) shows
quantitative performance comparisons indicating superior performance for custom data-
driven kernels. Common failure modes of the generic kernels are illustrated in Figure 3(c).
Open-universe Gaussian mixture modeling. We also implemented Richardson and
Green (1997)'s data-driven inference algorithm for clustering in a Gaussian mixture model
with an unknown number of components. See Appendix A.1 for details, including an illus-
tration in Figure 4. We have found that BLOG inference typically fails to accept moves to
world states with dierent numbers of clusters, whereas the data-driven algorithm rapidly
mixes. Note that Richardson and Green (1997)'s algorithm includes a split-merge kernel
with a Jacobian term that our approach calculates automatically.
6
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
References
Nimar Arora, Stuart Russell, and Erik Sudderth. Net-visa: Network processing vertically
integrated seismic analysis. The Bulletin of the Seismological Society of America , 103:
709{729, 04 2013. doi: 10.1785/0120120107.
Nimar S Arora. Model-based Bayesian Seismic Monitoring . PhD thesis, EECS Department,
University of California, Berkeley, May 2012. URL http://www2.eecs.berkeley.edu/
Pubs/TechRpts/2012/EECS-2012-125.html .
Albert S Bregman. Auditory scene analysis: The perceptual organization of sound . MIT
press, 1994.
Stephen Brooks, Paolo Giudici, and G.O. Roberts. Ecient construction of reversible jump
proposal distributions (with discussion). J. R. Statist. Soc. B , 65:3{55, 01 2003.
Maddie Cusimano, Luke B Hewitt, Josh Tenenbaum, and Josh H McDermott. Auditory
scene analysis as bayesian inference in sound source models. In CogSci , 2018.
Marco Cusumano-Towner, Alexander K. Lew, and Vikash K. Mansinghka. Automating
involutive mcmc using probabilistic and dierentiable programming. 2020.
Marco F Cusumano-Towner, Feras A Saad, Alexander K Lew, and Vikash K Mansinghka.
Gen: a general-purpose probabilistic programming system with programmable inference.
InProceedings of the 40th ACM SIGPLAN Conference on Programming Language Design
and Implementation , pages 221{236. ACM, 2019.
T. D. Kulkarni, P. Kohli, J. B. Tenenbaum, and V. Mansinghka. Picture: A probabilistic
programming language for scene perception. In 2015 IEEE Conference on Computer
Vision and Pattern Recognition (CVPR) , pages 4390{4399, 2015. doi: 10.1109/CVPR.
2015.7299068.
Alexander K. Lew, Monica Agrawal, David Sontag, and Vikash K. Mansinghka. Pclean:
Bayesian data cleaning at scale with domain-specic probabilistic programming, 2020.
Vikash K Mansinghka, Tejas D Kulkarni, Yura N Perov, and Josh Tenenbaum. Ap-
proximate bayesian image interpretation using generative probabilistic graphics
programs. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q.
Weinberger, editors, Advances in Neural Information Processing Systems 26 , pages
1520{1528. Curran Associates, Inc., 2013. URL http://papers.nips.cc/paper/
4881-approximate-bayesian-image-interpretation-using-generative-probabilistic-graphics-programs.
pdf.
Brian Milch and Stuart J. Russell. General-purpose MCMC inference over relational struc-
tures. CoRR , abs/1206.6849, 2012. URL http://arxiv.org/abs/1206.6849 .
Brian Milch, Bhaskara Marthi, Stuart Russell, David Sontag, Daniel L. Ong, and Andrey
Kolobov. BLOG: Probabilistic models with unknown objects. In Proceedings of the
Nineteenth International Joint Conference on Articial Intelligence , IJCAI 2005, pages
1352{1359. Morgan Kaufmann Publishers Inc., 2005a.
7
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Brian Milch, Bhaskara Marthi, David Sontag, Stuart Russell, Daniel L. Ong, and An-
drey Kolobov. Approximate inference for innite contingent bayesian networks. In
(AISTATS'05) 10th International Workshop on Articial Intelligence and Statistics ,
January 2005b. URL https://www.microsoft.com/en-us/research/publication/
approximate-inference-infinite-contingent-bayesian-networks/ .
Brian Christopher Milch. Probabilistic Models with Unknown Objects . PhD thesis, EECS
Department, University of California, Berkeley, Dec 2006. URL http://www2.eecs.
berkeley.edu/Pubs/TechRpts/2006/EECS-2006-174.html .
Kirill Neklyudov, Max Welling, Evgenii Egorov, and Dmitry Vetrov. Involutive MCMC: A
Unifying Framework. arXiv preprint arXiv:2006.16653 , 2020.
Sylvia. Richardson and Peter J. Green. On bayesian analysis of mixtures with an unknown
number of components (with discussion). Journal of the Royal Statistical Society: Series
B (Statistical Methodology) , 59(4):731{792, 1997. doi: 10.1111/1467-9868.00095. URL
https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/1467-9868.00095 .
David A. Roberts, Marcus Gallagher, and Thomas Taimre. Reversible jump probabilistic
programming. volume 89 of Proceedings of Machine Learning Research , pages 634{643.
PMLR, 16{18 Apr 2019. URL http://proceedings.mlr.press/v89/roberts19a.html .
S. Russell, O. Lassen, J. Uang, and W. Wang. The physics of text: Ontological realism in
information extraction. In AKBC@NAACL-HLT , 2016.
Wei Wang and Stuart Russell. A smart-dumb/dumb-smart algorithm for ecient split-
merge mcmc. In Proceedings of the Thirty-First Conference on Uncertainty in Articial
Intelligence , UAI'15, page 902{911, Arlington, Virginia, USA, 2015. AUAI Press. ISBN
9780996643108.
David Wingate, Andreas Stuhlmueller, and Noah Goodman. Lightweight implementations
of probabilistic programming languages via transformational compilation. In Georey
Gordon, David Dunson, and Miroslav Dud k, editors, Proceedings of the Fourteenth In-
ternational Conference on Articial Intelligence and Statistics , volume 15 of Proceedings
of Machine Learning Research , pages 770{778, Fort Lauderdale, FL, USA, 11{13 Apr
2011. JMLR Workshop and Conference Proceedings. URL http://proceedings.mlr.
press/v15/wingate11a.html .
Yi Wu, Siddharth Srivastava, Nicholas Hay, Simon Du, and Stuart J. Russell. Discrete-
continuous mixtures in probabilistic programming: Generalized semantics and inference
algorithms. CoRR , abs/1806.02027, 2018. URL http://arxiv.org/abs/1806.02027 .
Yi-Ting Yeh, Lingfeng Yang, Matthew Watson, Noah D. Goodman, and Pat Hanrahan.
Synthesizing open worlds with constraints using locally annealed reversible jump mcmc.
ACM Trans. Graph. , 31(4), July 2012. ISSN 0730-0301. doi: 10.1145/2185520.2185552.
URL https://doi.org/10.1145/2185520.2185552 .
Ben Zinberg, Marco Cusumano-Towner, and Vikash K. Mansinghka. Structured dier-
entiable models of 3D scenes via generative scene graphs. In Workshop on Perception
8
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
as Generative Reasoning , 2019. URL https://pgr-workshop.github.io/img/PGR025.
pdf.
9
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Appendix A. Experimental Details
A.1. Open-universe Gaussian Mixture Model.
Richardson and Green (1997) develop a series of highly data-driven MCMC kernels for
a Gaussian mixture model with an unknown number of mixture components. Here we
compare their algorithm, implemented using our system for involutive MCMC in OUPMs,
to the generic inference engine in BLOG.
The model is as follows. The number of mixture components, k, is a Poisson random
variable. The mixture weights and the parameters of each mixture component are drawn
from the appropriate conjugate priors:
w1;:::;wkDirichlet(1k)jN(;1=)2
jinverse-gamma( ;);
where 1kdenotes a vector of ones in Rk;wj,j, and2
jdenote the weight, mean, and
variance of the j-th mixture component respectively; and ,,,, andare xed hyper-
parameters. Lastly, for the i-th data point we sample a cluster allocation, and given the
cluster allocation, we sample a value from the assigned cluster; ie.
ziCategorical( w1;:::;wk)yiN(zi;2
zi)
Our involutive MCMC proposal consists of rst updating the weights, means, variances,
and allocations using Gibbs updates, and then performing a split or merge move to change
the number of clusters. The Gibbs updates are given by
w1;:::;wkj:::Dirichlet(w1+n1;:::;wk+nk)
jj:::N
+2
jP
zi=jyi
2
jnj+;
2
jnj+1!
2
jj:::inverse-gamma( +1
2nj;+X
zi=j(yii)2)
p(zi=jj:::)/N(j;2
j);
wherenjdenotes the number of data points allocated to the j-th cluster. Our merge moves
consist of choosing indices j16=j2andjuniformly at random and merging the j1-th and
j2-th clusters into the j-th cluster. We ensure that the zeroth, rst, and second moments
of the newly generated cluster match the corresponding moments in the mixture of the
original clusters; i.e.
w=w1+w2
w=w11+w22
w(+2
) =w1(1+2
1) +w2(2+2
2): (2)
To ensure ergodicity, our split procedure relies on three auxiliary variables
u1beta(2;2)u2beta(2;2)u3beta(1;1);
10
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
(a) Ground Truth Clusterings
(colored) and Data (black)
(b) Two Samples from the Inferred Posterior:
Richardson & Green’s Data-driven MCMC (top),
BLOG Ancestral Sampling (bottom) (c) Trace Likelihoods vs. Sample
number for Richardson &
Green’s algorithm
Figure 4: Gaussian Mixture Model Inference Experiment Results
and generates new clusters according to
w1=wu1 w2=w(1u1)
1=u2p
w2=w1 2=+u2p
w1=w2
2
1=u3(1u2
2)2
w=w1 2
2= (1u3)(1u2
2)2
w=w2: (3)
Using (2) and (3), we can derive our involution's Jacobian matrix (leaving out rows and
columns which merely copy a value):
J=2
6666666666664 2
 w u1 u2 u3
w1 0 0 u1 w 0 0
1 1u2qw2
w1
20u2q
u11
u1
2u1(u11)q
w2
w10
2
1 0u3w(u2
21)
w10u32
(u2
21)
u2
12u2u3w2

w1w2
(u2
21)
w1
w2 0 0 1 u1w 0 0
2 1u2qw1
w2
20u2q
u1
u11
2u1(u11)q
w1
w20
2
2 0w(u2
2u3u2
2u3+1)
w202
(u2
2u3u2
2u3+1)
(u11)22u2w2
(u31)
w2w2
(u2
21)
w23
7777777777775:
Given a specication of the inference kernel, our system automatically calculates
this Jacobian matrix and its determinant
jdetJj=wj12j2
12
2
u2(1u2
2)u3(1u3)2
without the user needing to work through this derivation.
Figure 4 shows representative results, in which the involutive MCMC proposal accurately
recovers the true latent clusters, whereas the ancestral sampling proposal only roughly
narrows down the component parameters.
11
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
type Cluster
type Datapoint
number Cluster() ~ poisson( λ) + 1
number Datapoint() = NUM_DATAPOINTS # global constant
property  mean(::Cluster) ~ normal( ξ,1/ κ)
property  var(::Cluster) ~ inverse_gamma( α_v, β_v)
property  unnormalized_weight(::Cluster) ~ gamma( α_w, β_w)
property  function  cluster(::Datapoint)
  clust_to_un_weight = Dict(c => get(unnormalized_weight(c))
                         for c in get_object_set (Cluster()))
  cluster ~ unnormalized_categorical(clust_to_un_weight)
  return cluster
end
property  function value(d::Datapoint)
  cluster = get(cluster(d))
  value ~ normal( get(mean(cluster)), get(var(cluster)))
  return value
end
observation_model  function get_datapoints()
  return [ get(value(Datapoint with index  i))
                         for i=1: get_number (Datapoint())]
end20 number function Station()
21  return 2
22 end
23 number function Event()
24   return num ~ poisson(5)
25 end
26 number function Detection(:: Station)
27    return num ~ poisson(4)
28 end
29 number function Detection(:: Station, :: Event)
30   return num ~ Int(bernoulli(0.8))
31 end
32 observation_model  function detections()
33  evts = [Event(i) for i=1: get number Event()]
34  sts = [Station(i) for i=1: get number Station()]
34  fp_dets = [Detection(st, j) for st in sts,
j=1:get number Detection(st) ]
35  real_dets = [Detection(st, evt, j) for
      st in sts, evt in evts, j=1: get
      number Detection(st, evt) ]
36  detections = union(fp_dets, real_dets)
37  return [ get reading[d] for d in detections]
38 end
detections() #Event()
#Detection(Station(1), Event(1))
#Detection(Station(1), Event(2)) magnitude(Event(1))
reading(Detection(Station(1), Event(1), 1))
reading(Detection(Station(2), Event(1), 1))
#Event() >= 1
#Event() >= 2 #Event() >= 1 #Event() >= 1
......
...(a) A simple world model
(b) A region of the contingent Bayesian network for the model
Figure 5: Gaussian mixture model with an unknown number of components. Instead of imperatively
sampling cluster weights w1;:::;wnDirichlet(1k), in our model, each cluster has an unnormalized
cluster weight property sampled from a Gamma distribution; normalizing these then sampling clus-
ters from the resulting distribution is equivalent to sampling clusters according to weights sampled
from a Dirichlet.
Figure 6: Data-driven Gaussian mixture model split/merge kernel from Richardson and Green
(1997).
12
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
A.2. Auditory Scene Interpretation.
Auditory scene analysis Bregman (1994) is a perception task that involves separating an au-
dio waveform into individual sound sources when the number and properties of the sources
are unknown a-priori (Figure 3a). Recently, Cusimano et al. (2018) proposed a compu-
tational model of human auditory scene analysis based on an open-universe probabilistic
model of sound sources, and showed that probabilistic inference in this model based on
MCMC is able to qualitatively reproduce human inferences about auditory scenes, includ-
ing reporting ambiguity in the same situations as humans.
The model. We developed a simplied version of the model of Cusimano et al. (2018)
that uses a prior distribution on the number of sounds and the type of each sound (`tone'
or `white noise'). Tones have an unknown frequency, onset time, and duration, and white
noise sounds have an unknown amplitude, onset time, and duration.
(a) A simple world model
(b) A region of the contingent Bayesian network for the model
Figure 7: Our OUPM for audio inference.
Figure 8: Performance of dierent inference ker-
nels, measured on 100 randomly-generated audio
scenes with 3 underlying sounds.
Inference Experiments. We implemented four involutive MCMC kernels for this model
in our extended version of Gen. We performed experiments applying these kernels to the
audio waveform shown in Figure 3a, which consists of a short tone, followed by a short period
of white noise, followed by a short tone at the same frequency as the rst tone (Cusimano
et al., 2018). When the noise is very loud, humans tend to hear one continuous tone
temporarily obscured by a loud noise, whereas when the noise is quiet, humans hear two
distinct short tones, with noise in the middle. Our example uses a softer noise, and our
best-performing inference programs, which use application-specic kernels implemented via
our framework, recover the true decomposition (two tones separated by noise). However,
generic inference kernels often incorrectly infer a single long tone (Figure 3). In addition to
testing our kernels on the specic audio scene in Figure 3a, we tested them on 100 audio
scenes generated from our model's prior distribution, with similar results (Figure 8).
Generic Kernels The rst two kernels we implemented are not application specic or
data driven. Our ancestral resampling kernel (Figure 9) resimulates a single variable
13
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
from the prior distribution (also ancestrally generating any additional variable values which
need to be added to the world due to the resimulated variable). Our generic birth-death
reversible-jump kernel (Figure 10) randomly proposes to create or delete individual sounds,
proposing new sounds' properties from the prior. We cycle this kernel with the ancestral
resampling kernel so that property values of existing sounds can more easily be updated.
Figure 3(b) shows that in our experiments, these two kernels converge to inferences of
latent waveforms under which the observed sounds are substantially less likely than under
the latent waveforms our data-driven kernels recover. Figure 3(c) visualizes two common
failure mode these kernels exhibit due to their inability to eciently incorporate evidence
from the data, and their inability to make proposals to split long sounds into short ones
(which require application-specic logic to calculate endpoints of the new sounds based on
the position of the sound being split). We found that the generic birth-death kernel often
failed to exit a local minimum in which a single continuous tone is inferred instead of two
temporally separated tones of the same frequency (Figure 3c, left). The generic kernel
performs worst|it usually infers one continuous tone, but often also infers spurious tones.
Aside: proposing from the prior in involutive MCMC. To facilitate proposing
values from the prior in involutive MCMC kernels, we have implemented a regenerate
command which can be called from within the involution DSL. (For examples, see Figures
9, 10, and 11.) The command regenerate properties of object samples a value from
the prior for each property of the given object (using the prior as it will appear in x0). This
complicates our picture of involutive MCMC slightly, as there are now random values being
sampled during the execution of the involution function, instead of only in the proposal
probabilistic program. We thus must view the proposal function as only implementing part
of theqdistribution and sampling part of y, with the rest of qimplemented by the involution
DSL's regenerate commands.
To ensure involutiveness, when a transition ( x;y)!(x0;y0) uses a regenerate com-
mand, the reversing move ( x0;y0)!(x;y) must specify in ythe value sampled by re-
generation during the forward move. To specify reverse-move values for non-regenerated
values, users use syntax backward[addr] = value , where addr is the address (variable
name) at which the proposal function should sample this value. Since users do not spec-
ify an address for the values in ywhich are regenerated (unlike values in ywhich are
sampled in the proposal function), we provide the following additional syntax syntax:
backward[property(object)] = regenerated sets the value in yfor regenerating the vari-
able property(object) to the value of property(object) inx, and backward[properties
of object] = regenerated does this for all properties of object . See Section E for more
details.
Custom kernels. Some specic domains, like data-clustering, have seen standard MCMC
kernels developed for data-driven inference, which are fairly conceptually simple but pre-
viously required fairly deriving fairly involved calculations (Richardson and Green, 1997;
Wang and Russell, 2015). In addition to facilitating the implementation of these types
of kernels and removing the need to manually derive acceptance probabilities, involutive
MCMC facilitates the development of data-driven kernels in more complicated domains
which involve complicated logic to use information from data eectively. For instance,
Cusimano et al. (2018)'s experiments used MCMC inference with a proposal based neural-
14
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Figure 9: Our ancestral resampling kernel, which selects a random variable and samples a new
value for it from the prior. The only complication is that when we sample the number variable
(changing the number of audio sources in the world), we must also generate new properties for any
newly created audio sources. (These properties are also generated from the prior.) For details on
generating values from the prior in involutive MCMC, see Section E.
Figure 10: The generic birth/death kernel either creates or deletes a random sound. When a new
sound is created, its properties are all generated from the prior. For details on generating values
from the prior in involutive MCMC, see Section E.
network transformations of observed soundwaves, implemented in an early version of our
inference programming system. For our experiments in this paper, we developed a roughly
250-LOC image-ltering and edge-detection algorithm to analyze observed soundwaves and
detect possible errors in the currently inferred latent sounds. We called this sound-analysis
program from within 2 involutive MCMC kernels we developed, which our system auto-
mated the implementation of, including eciently updating world states and calculating
the Jacobian correction term.
Ourdata-driven sound detection kernel (Figure 11) either proposes to delete a
sound, or to create a new sound which with high probability is proposed at a time and
frequency where our analysis program detected a sound in the data but not the currently
inferred latents. Figure 3(b) and Figure 8 show that this kernel both initially performs
better than the generic kernels, and ultimately converges to a better state.
We also add a data-driven sound splitting kernel (Figure 12), which either proposes
to merge two short tones/noises into a long one, or splits a long sound into two short ones,
15
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Figure 11: The data-driven sound detection kernel inference kernel used in the audio example.
The kernel uses an image-ltering edge-detection algorithm detect sounds (not shown) to detect
noises and tones in the observed sound waves which are not explained by the currently hypothesized
audio sources. Heuristic scoring functions birth score anddeath score (not shown) are used to
score the value of adding detected sounds to the latents, or deleting sounds in the current latents.
Based on these scores, either a currently inferred sound is selected to be deleted (a DeathAction ), or
a detected sound is selected to be created, with some noise added to the detected sound parameters (a
\birth" move). To ensure reversibility, death moves must select which detected sound will be sampled
during the reversing birth move. If no compatible sounds will be detected during a reversing birth,
the death move will specify that the reversing birth should be \dumb" (non-data-driven) and create
a sound generated from the prior.16
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Figure 12: The data-driven sound splitting kernel either chooses to split one long sound into
two short ones, or merge two short tones into a longer one. In the split move, we use a similar edge-
detection algorithm to what was used in the create detected sound kernel to detect changepoints
in the observed sound during the time at which the sound being split occurs. The kernel proposes
an endpoint for the rst new sound and a startpoint for the second new sound to be close to the
detected changepoints.
17
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
proposing endpoints for the new sounds based on the analysis of the sounds. Figure 3(b)
and Figure 8 show that the addition of this kernel substantially speeds up convergence.
This is to be expected, since this kernel can integrate information from the data about
another class of possible mistakes in inferred latents: when one sound has been mistakenly
interpreted as two, or vice versa. We cycle both of the data-driven kernels with Gaussian
drift kernels for the parameters of inferred audio sources.
A.2.1. The automated acceptance probability for the data-driven sound
detection kernel
To illustrate the calculations our system automates, we give the acceptance probability
calculation performed when running the data-driven sound detection kernel , including
calculating the Jacobian of the transformation.
Model density. We can factor the density of the model, p(x), asp(x) =p(xL)p(xOjxL),
wherexLis the collection of latent variable values in x(the number of underlying sound
sources, and the properties of each one), and xOis the observed sound waves variable in x.
For anyxwhere each variable is within its support, the probability of the latents is:
p(xL) =1
4xNSY
i=1(p(xD;i)p(x;i)p(x(jf);i)p(xn;ys))
=1
4xNSY
i=1((1
L0:1)(1
L0:1)((
N(x(jf);i;8;10)xn;i= 1
1
240:4otherwise)0:4xn;i(10:4)1xn;i)
Here,xNSis the number of sound sources in MSSI x(the value of the number variable
number Source() );xD;iis the duration of the ith audio source in x;x;iis the starttime
of theith audio source in x;xn;iis an indicator which is 1 if the ith source in xis a noise
and 0 otherwise; x(jf);idenotes the amplitude of the ith sound in xif it is a noise, and the
frequency of this sound if it is a tone. We let Ldenote the length of the rendered audio
tones (which is a global constant).
The probability of the observation given the latents is:
p(xOjxL) =N(xO;r(xL);Ijr(xL)j)
wherexOdenotes the observed soundwaves in x(from the observed sound property in
Figure 7) represented as a vector, r(xL) denotes the rendering of the latent audio sources
inxinto sound-waves, and Ikdenotes the k-dimensional identity matrix.
State-dependent proposal density. The state-dependent proposal distribution qx(y) is
parameterized by a current MSSI x. It is a distribution over nite dictionaries, implemented
as a Gen program. Using standard techniques from probabilistic programming, Gen can
automate the computation of densities of dictionaries sampled by the proposal:
18
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
qx(y) =8
>>>>>>><
>>>>>>>:(1(x))n(x)yn(x;yn)yi((x;yi)(x;yi;yn)ys0)y(1(x;yi))1y yb= 0
(x)(1(x))
xNS+1(0:4N(y;8;10))yn(0:6(370:4)1)1yn
(L0:1)2 yb= 1^y= 0
(x)(x)
xNS+1n(x)yn(x;yn)ysNT0
B@2
64yD
y
yv(yni)3
75;2
64RD
R
Rv(yn)3
75;2
64DD(x;ys;yn)
D(x;ys;yn)
Dv(yn)(x;ys;yn)3
75;2
640:1 0 0
0 0:1 0
0 0v(yn)3
751
CAyb= 1^y= 1
Here,xNSis the number of sound sources in MSSI x, andybdetermines whether a new
sound source will be proposed; if yb= 1,ydetermines whether the new source will be
proposed using the data-driven edge-detection proposal or from the prior. (In Figure 11, yb
corresponds to dobirth ;ytodosmart birth .)ynis an indicator for whether the source
being created/deleted is white noise, as opposed to a tone ( ynisobjtype in Figure 11
whenyb= 0_y= 1, and is proposed via regenerate whenyb= 1^y= 0). When
y= 1, an index ysis computed determining which of the sounds from the image-ltering
and edge-detection algorithm with the type indicated by ynwill be proposed (the index
of sample action in Figure 11). Based on ys, new sound onset and duration parameters
yandyD, as well as either a frequency yfor an amplitude y. When the data-driven
proposal is not used ( yb= 1^y= 0),y(oryf),yn, and other parameters are proposed
from the model's prior over sound sources. If yb= 0, a death move is proposed, to delete
soundyi. The variable yis introduced to enable the implementation of an involution:
it represents whether the move reversing a proposed death move will be data-driven. If
y= 1,ys0represents which heuristically detected sound source with the type indicated by
ynwill be used as the basis to repropose the deleted audio source, in the reverse proposal.
NT(v;R;; ) denotes the PDF of a multivariate truncated normal distribution with ranges
R, means, and covariance matrix , evaluated at v.
The proposal is based on a number of functions, as well as the scene length parameter
L, the standard deviations = 4 andf= 1, and the ranges RD= (0:1;1),R= (0;L),
Rf= (0:4;24), andR= (1;1). These functions, and some additional functions needed
for their denitions, are described below:
k(x;yn) is the number of possible sounds to create of type yndetected in state x
SB(x;i;yn) is a score for the action of creating a sound close to the ith detected sound
of typeyn, from a user-written scoring heuristic
SD(x;i) is a score for the action of deleting the ith sound in the current world, from
a user-written scoring heuristic (a dierent heuristic from SB)
B(x) =P1
yn=0Pk(x;yn)
i=1SB(x;i;yn)
P1
yn=0Pk(x;yn)
i=1SB(x;i;yn)+PxNS
i=1SD(x;i)is the total score of all detected birth
actions, divided by the total score of all detected birth and death actions

(x) =8
><
>:B(s)+
BR
1+RxNS2(0;4)
0 xNS= 4
1 xNS= 0
19
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
is the probability of yb= 1, givenx. Hyperparameter
B= 0:5 is a prior probability
of doing a birth move, not factoring in the scores assigned to detected actions, and
hyperparameter R= 0:1 is the probability of making a decision to do a birth vs
death move without considering the scores.
(x) = 10:05[j(x)j>0]is the probability of y= 1 given that yb= 1 for MSSI x.
(x;i) is 0, if removing sound ifromxand running image ltering and edge detection
does not nd any sounds of the same type as sound iinx, and 0:95 otherwise.
DD(x;i;yn) is the predicted duration of the ithdetected sound source of type ynfrom
the image ltering and edge detection algorithm applied to x. Similarly, Dgives the
predicted onset, Dthe predicted amplitude (if applicable), and Dfthe predicted
frequency (if applicable).
v(yn) is the symbol (amplitude) if yn= 1 and the symbol f(frequency) if yn= 0.
n(x) =f07!Pk(x;0)
i=1SB(x;i;0)
Pk(x;0)
i=1SB(x;i;0)+Pk(x;1)
i=1SB(x;i;1);17!Pk(x;1)
i=1SB(x;i;1)
Pk(x;0)
i=1SB(x;i;0)+Pk(x;1)
i=1SB(x;i;1)gis a
dictionary from values of ynto the probability of choosing to birth an object of type
ynbased on the heuristically-assigned SBscores.
n(x) =f07!P
i2N\(0;xNS):xn;i=0SD(x;i)
PxNS
i=1SD(x;i);17!P
i2N\(0;xNS):xn;i=1SD(x;i)
PxNS
i=1SD(x;i)ggives the prob-
ability of deleting a noise vs deleting a tone.
(x;yn) is a vector-valued function, taking values in the ( k(x;yn)1)-simplex. It
represents a distribution over more or less promising detections based on the SB
scores. In particular, (x;yn)j=SB(x;j;yn)Pk(x;yn)
i=1SB(x;i;yn).
(x;yn) is a vector-valued function representing a distribution over which existing
audio source of type ynto delete. In particular, (x;yn)j=SD(x;j)P
i2N\(0;xNS):xn;i=0SD(x;i).
(x;i;yn) is a vector-valued function, taking values in the ( k(x;i;yn)-1) simplex,
wherek(x;i;yn) is the number of detected sounds of type ynfrom the edge detection
routine, applied to xwithout sound source i. In particular,
(x;i;yn)j/NT0
@2
4xD;i
x;i
xv(yn);i3
5;2
4RD
R
Rv(yn))3
5;2
4D
D(x;i;j;yn)
D
(x;i;j;yn)
D
v(yn)(x;i;j;yn)3
5;2
40:1 0 0
0 0:1 0
0 0v(yn)3
51
A
whereD
D,D
,D
, andD
fare functions on ( x;i;j;yn) giving the duration, onset,
and amplitude/frequency of the jth sound of type yndetected via the detector applied
toxwithout source i.
20
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
The Jacobian. Thecreate detected sound kernel always either creates or deletes a
sound; we show the Jacobian of the matrix for a move which creates a sound (the transpose of
which is the Jacobian of the matrix for the corresponding deletion). The transition ( x;y)!
(x0;y0) during a creation copies over every continuous value in xandyintox0. These
values come from the properties starttime () and duration (D) of each source, and the
amplitude (D) property of each noise or frequency (f) property of each tone. Depending
on the index of the newly-created sound, the indices of each object may change, as is
described in . In particular, when we birth a new source with index i, we must increment
the index of each existing source with index i. We dene the following identiers for
whether the object at index ais moved to index b, and whether the birth index iequals a
given index a:
Ca!b=(
1 (i>a^a=b)_(ia^b=a+ 1)
0 otherwise
Ba=(
1i=a
0 otherwise
Using these denitions, we can write the Jacobian of the transformation which copies
starttime (), duration ( D), and amplitude/frequency ( jf) values for each object as:
J=0
BBBBBBBBBBBBBBBBB@x(jf);1x;1xD;1x(jf);2x;2xD;2::: xD;NS+1
y(jf)B1 0 0 B2 0 0 ::: 0
y 0B1 0 0 B2 0::: 0
yD 0 0 B1 0 0 B2::: B NS+1
x(jf);1C1!1 0 0 C1!2 0 0 ::: 0
x;1 0C1!1 0 0 C1!2 0::: 0
xD;1 0 0 C1!1 0 0 C1!2::: C 1!NS+1
x(jf);2C2!1 0 0 C2!2 0 0 ::: 0
x;2 0C2!1 0 0 C2!2 0::: 0
xD;2 0 0 C2!1 0 0 C2!2::: C 2!NS+1
........................
xD;NS0 0 C2!NS 0 0 C2!NS::: CNS!NS+11
CCCCCCCCCCCCCCCCCA
where
Ja;b=8
><
>:0a(mod 3)6=b(mod 3)
Bba3
Ca!ba>3
Since this is a permutation matrix, it always has determinant 1 or 1. Our system is
capable of computing this Jacobian using automatic dierentiation and discovering this.
As an optimization, in practice, our system can identify that this transformation only
copies values, and thus will have a permutation-matrix Jacobian which it does not need
to take the determinant of, since it will certainly have magnitude 1, as we describe in the
Jacobian sparsity paragraph in Section D. For an description of an inference kernel we
have implemented which has a Jacobian whose determinant's magnitude is (usually) not 1,
see Section A.1.
21
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Acceptance probability. When using the involutive MCMC kernel, our system auto-
matically calculates qx(y),qx0(y0),p(x),p(x0), andJ, to compute the acceptance probability
of the move ( x;y)!(x0;y0),
p(x0)
p(x)qx0(y0)
qx(y)jdetJj
These terms are all equivalent to the above expressions for qx(y),p(x), andJ(sometimes
substituting x0forxory0fory). In practice, the qterms are automatically calculated
using the given expressions, while computational optimizations let us avoid calculating the
determinant of the Jacobian matrix (which is just a permutation matrix), and let us avoid
fully calculating p(x0) andp(x), instead directly calculating
p(x0)
p(x)=p(x0
Ojx0
L)1
4Qx0
NS
i=1p(x0
D;i)p(x0
;i)p(x0
(jf);i)p(x0
n;i)
p(xOjxL)1
4QxNS
i=1p(xD;i)p(x;i)p(x(jf);i)p(xn;i)
=p(x0
Ojx0
L)
p(xOjxL)(
p(x0
D;ys)p(x0
;ys)p(x0
(jf);ys)p(x0
n;ys)yb= 1
1
p(xD;ys)p(x;ys)p(x(jf);ys)p(xn;ys)yb= 0
where, as above, ybis 1 whenyproposes to create a new source and 0 otherwise, and
ysis the index of the source being created/deleted. We can avoid evaluating the product
expressions because the majority of terms in the numerator and denominator cancel out,
since all audio sources except the one being created/deleted are unaected by the update.
22
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Appendix B. Open-Universe Models and Contingent Bayesian Networks
In this section, we give a more complete denition of open-universe probabilistic models
and contingent Bayesian networks (outlined in Section 2).
Open-universe probabilistic models are models of relational domains in which the num-
ber of objects, and their link structure, is not known a priori. The usual representations for
structured probability distributions|e.g., probabilistic graphical models|are insucient
for specifying OUPMs, which typically dene an unbounded number of random variables,
with conditional independence relationships that are themselves random. OUPMs can be
specied formally, however, as programs in suciently expressive probabilistic programming
languages (Milch et al., 2005a), or as contingent Bayesian networks (CBNs) with innitely
many nodes (Milch et al., 2005b). Here, we review the standard denition of OUPMs as
CBNs, and extend it to allow for objects with real-valued random properties.
Open-Universe Probabilistic Models. Anopen universe probabilistic model is a tuple
(T;N;P;C), where
 T is a nite3set of object types over which the model is dened (e.g., Person ).
 N is a nite3set of object origin declarations (;(1;:::;n)), with;j2T. An
object origin declaration species that a tuple of objects, of types ( 1;:::;n) respec-
tively, can be the origin of a separate object of type . For example, in a model
of genealogy, we might declare ( Person;(Person;Person ))2N, encoding that a pair
of people can be the origin (i.e., parents) of other people. A model's object origin
declarations determines sets of possible objectsO(N) for each type , which are -
nite expressions of the form ( ;(o1;:::;on);i), where (;(1;:::;n))2N,i2N, and
oj2Oj(N) for eachj. We call (;(o1;:::;on);i) theithobject of type with origin
(o1;:::;on).
 Pis a countable set of typed object property declarations (P;(1;:::;n)), withj2T,
together with value domains dom( P) for each property. A property represents data
concerning zero or more objects, of types ( 1;:::;n) respectively. For example, a
model might declare a property representing the number of
ights between two places.
 Cis acontinuously-extended contingent Bayesian network on the set of possible vari-
ables for (T;N;P),f(x;(o1;:::;on))j(x;(1;:::;n))2P[N^8 j;oj2Oj(N)g.
For every origin declaration ( ;(1;:::;n))2 N , and every tuple ( o1;:::;on)2N
jOj(N) of possible objects of the correct types, there is a possible number variable
(;(o1;:::;on)), with domain Z0, representing the number of objects of type that
have objects ( o1;:::;on) as their origin. For every property declaration ( P;(1;:::;n))
and object tuple ( o1;:::;on) of the correct types, there is a possible property variable
specifying the value of the property P(within dom( P)) of the object tuple ( o1;:::;on).
The contingent Bayesian network itself denes a distribution over assignments to these
innitely many possible variables, as we describe in the following section.
3. The set of types and the set of object origin declarations can be countable without compromising any
results, but existing formal languages for dening OUPMs, including ours, do not provide a way to
declare models where either of these are innite. It would be interesting to consider whether any natural
applications would require this
exibility, and how languages might be designed to expose it.
23
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Contingent Bayesian Networks. For a set of possible variables V, let
 =N
v2Vdom(v)
be the space of all possible assignments of the (possibly innitely many) variables to values
within their domains. When the domains are discrete, we can use a contingent Bayesian
network (CBN) (Milch et al., 2005b) to dene a distribution over such assignments. Anal-
ogously to a Bayesian network on a nite number of variables, a CBN on an innite set of
variablesVspecies an innite directed graph with Vas the nodes. But whereas traditional
Bayesian networks x the dependency structure, in a CBN the edges are contingent : each
edgeu!vis labeled with an event
 u!v
 within which the edge is active . As in
Bayesian networks, to each variable vwe associate a conditional probability distribution
(CPD)pv. But this conditional probability distribution is allowed to depend only on the
active parents ofv, which can be formalized as follows: for each variable v2V, the con-
tingent Bayesian network associates a partition  vof the space
, such that if !1and!2
dier only in their assignment to u, and the edge u!veither does not exist or is inactive
in!1, then!1and!2belong to the same component of  v. Thenpv(j!2) is allowed
to depend on which component of the partition !is in, but nothing else, preventing de-
pendencies on the values of inactive parents. We write Pa!(v) for the active parents of the
variablevin state!. These conditional probabilities together dene a probability measure
 on the space
, equipped with a non-standard -algebra; we refer the reader to Milch
et al. (2005b) for a detailed treatment.
Minimal Self-Supporting Instantiations. In open-universe models, possible worlds
!2
 are generally innite, because they specify assignments to all of the possible variables
v2V. But for inference, we must work with nite representations. If we can partition
into disjoint equivalence classes
 i, then we can work with the equivalence classes, using
the distribution p(
i) = (
i). One useful strategy for partitioning
 is to consider the
minimal self-supporting instantiations of a nite set of variables UV. Each component
wof the partition is labeled by a set of variables vars( w)V, withUvars(w), as well
as an assignment w[v]2dom(v) for eachv2vars(w). The component wcontains all full
possible worlds !2
 that agree with won the values of each v2vars(w). To ensure that
the components ware indeed disjoint, we only include was a component if: (1) it is self-
supporting , meaning that for all !2w, ifv2vars(w) and u2Pa!(v), thenu2vars(w);
and (2) it is minimal , meaning that removing any variable v62Ufrom vars(w) would cause
it to fail to be self-supporting. Together, these criteria ensure that for any UV, the
components ware disjoint, and so p(w) = (w) is a valid probability distribution over
minimal self-supporting instantiations w. Conveniently, the probability of each component
wis a simple product of the conditional distributions pv, as in a typical Bayesian network:
p(w) = (w) =Y
pv(w[v]jfw[u]ju2Paw(v)g): (4)
B.1. Contingent Bayesian Networks with Continuous Variables
We now extend the CBN framework to support models with some continuous object prop-
erty domains, so long as no discrete variables count continuous variables as parents.4(Wu
4. See the following section for a similar but more involved extension to a more general case, where discrete
variables can depend on continuous variables.
24
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
et al. (2018) give a semantics for open-universe models that interprets them as innite prod-
uct measures over assignments to all basic random variables. This allows them to interpret
OUPMs with continuous and discrete random variables, but does not serve our purposes:
it does not yield a formula for the density of an MSSI in an open-universe model, nor
can it be used to prove that incremental computation strategies for fast MCMC are cor-
rect, as|unlike in CBNs|it does not account for contingent dependency structure among
variables.)
As before, letVbe a collection of variables, but now suppose some variables VcV
have continuous domains. Then dene the surrogate discrete domain dom D(v) of a variable
vto be dom( v) ifv62Vc, andf?gotherwise. A continuously extended CBN denes (1) an
ordinary CBN on the variables V, treating them as having surrogate discrete domains dom D,
and (2) conditional probability densities pv(jfw[u]ju2Paw(v)g) for each continuous
variablev2Vc. The CBN induces a distribution  on minimal self-supporting worlds,
which we modify to dene a probability measure on the space Wofcontinuously extended
minimal self-supporting worlds : pairs (wd;wc) of a minimal self-supporting world wdfor
the discrete CBN, plus a real-valued assignment wcto all variables in vars( wd)\Vc. We let
w[v] denotewc[v] ifv2Vcandwd[v] otherwise. The distribution has density
p(w) = (wd)Y
v2Vc\vars(w)pv(wc[v]jfw[u]ju2Paw(v)g) (5)
with respect to the reference measure
(E) =X
wjwcj(fw0
cjw02E^vars(w0) = vars(w)^8v2vars(w0)nVc;w0[v] =w[v]g);
wherekdenotes the k-dimensional Lebesgue measure. Using Equation 4 for the rst term
ofp(w), we see that in the continuous case, we still have the ordinary Bayes net factorization
p(w) =Q
v2vars(w)pv(w[v]jfw[u]ju2Paw(v)g).
B.1.1. Continuously-Extended CBNs with Continuous Parents of Discrete
Variables
In the previous section, we dene continuously-extended contingent Bayesian networks with
the restriction that no discrete variables have continuous variables as parents. We now give
a more
exible development.
LetVbe a collection of variables, where Vc V have continuous domains. Dene
surrogate discrete domain dom D(v) to be dom( v) ifv =2 Vc, andf?gotherwise. Now
dene a continuously extended CBN to comprise: (1) an ordinary CBN on the variables
V, treating them as having surrogate discrete domains dom D, and (2) a conditional joint
probability density Pc(wcjw) on the values of the continuous variables in a minimal self-
supporting instantiation wgiven the values of all w's discrete variables. This CBN induces
distribution  on minimal self-supporting worlds; as before, we modify this to dene the
spaceWofcontinuously extended minimal self-supporting worlds , consisting of a minimal
self-supporting world wof the CBN, and an assignment w[v] to each continuous variable in
vars(v)\Vc. We usewcto denote the assignment of values w[v] to the continuous variables
v2Vc\vars(w) inw. The distribution then has density
p(w) = (f!j8v2vars(w)nVc;![v] =w[v]g)Pc(wcjw)
25
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
with respect to the same reference measure from Section 2:
(E) =X
wjwcj(fw0
cjw02E^vars(w0) = vars(w)^8v2vars(w0)nVc;w0[v] =w[v]g);
wherekdenotes the k-dimensional Lebesgue measure. We note that the density we give
ultimately factorizes, as before, as a marginal probability for the discrete variables and a
conditional density for the continuous variables. (In principle, this alone is not a restriction,
because any distribution over minimal self-supporting instantiations can be factored in this
way.) But in the previous formulation (from Section 2), this factorization was embedded into
the discrete CBN itself, aecting which worlds wcounted as minimal and self-supporting
in the rst place. The key dierence between this section's formulation and the one in
Section 2 is that here we divorce the factorization of the continuous density from the CBN's
structure; the edges involving continuous variables in the CBN serve only to determine
which continuous variables are required in a minimal, self-supporting instantiation for an
observed set of variables U.
26
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Appendix C. Modeling and Inference Programming Languages for
OUPMs
In this section, we brie
y outline our modeling language for OUPMs, and our DSL for
writing involutions for involutive MCMC on OUPMs.
OUPM Modeling Language We have developed a new modeling language that is in-
spired by BLOG (Milch, 2006) for use in the Gen probabilistic programming system; for
examples, refer to Figure 1, Figure 5, or Figure 7. Our language extends one of Gen's
built-in modeling languages in the following ways. We add type statements for declar-
ing object types, property declarations for adding properties, and number declarations for
adding possible origins. When writing world models with our system, users must dene
a special property O() with respect to which the model will take minimal self-supporting
instantiations; they do this using an observation model statement. Property and number
declarations are attached to function bodies which together dene the contingent Bayesian
network.
In particular, a property statement declares the existence of a property P(1;:::;n),
and denes a probabilistic generative process for the property's value. A number statement
(;(1;:::;n)) declares that ( 1;:::;n) is a permissible origin signature for objects of
type, and declares the distribution over how many objects of type there are with each
origin matching ( 1;:::;n). (So each property/number statement simultaneously serves
as an object property declaration/object origin declaration, and declares the distribution
over the corresponding property/number variables in the OUPM's CBN.) The bodies of
number and property declarations are specied in an imperative probabilistic programming
language that induces distributions over nite dictionaries, called traces. The language uses
Julia syntax for dening functions, but may sample random variable values using xd
statements. A program in the language denes a distribution over traces mapping each
keyxto the value sampled from the corresponding din an execution of the function.
The language is a built-in modeling language in Gen that was augmented with support
forget(P(o1;:::;on))statements to access the value of P(o1;:::;on) in the world; this
introduces an edge in the contingent Bayes net representing the model.
Represented as imperative probabilistic programs, number variables must always have
0 or 1 keys in their traces, corresponding to either deterministically returning a number,
or sampling a single random number. Properties may have more than 1 key in their trace
(so long as the set of possible keys is countable), but when they do, the the property
statementPmust be understood as dening a separate object property declaration P=>
kfor eachkwhich can appear as a key in the trace. For simplicity, in what follows, we will
assume that each P(o1;:::;on) is itself a variable with only 0 or 1 keys in its probabilistic
program's traces; however, the algorithms we give also hold when there can be multiple
keys. We represent MSSIs as sets of pairs ( P(o1;:::;on);t), wherePis a number or property
probabilistic program whose type signature matches ( o1;:::;on), andtis a trace for this
variable.
OUPM Involution Programming Language We implement a new OUPM involution
programming language to implement involutions on the space of pairs ( x;y), wherexis
a world represented as described above, and yis a trace from a imperative probabilistic
27
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
program. Involutions are written as imperative functions which may call special commands
to modify state xin order to produce x0, to access values in xandy, and to specify the new
values fory0. Specically, the DSL provides support for the following.
Querying the current model state ( x).Users may query the current model state
using the syntax get(P(o1;:::;on)[a]), which retrieves the sample at address aof the
trace for the property P(o1;:::;on). Similarly, getnumber((o1;:::;on)))retrieves
the number variable value in xdening how many objects of type exist with origin
(o1;:::;on). Several higher-level commands are also provided to access higher-level
discrtete information about the current state, including getobject set(), which
gets the set of all existing objects of type inx.
Querying the auxiliary randomness ( y).The syntax proposed[a]returnsy[a],
the value at address ain the random trace yof the proposal program Q.
Proposing the new model state ( x0).The new model state is proposed via
commands that imperatively modify the old state x. They can do this either by
changing the structure ofx(by adding or removing objects, and changing the origins
or indices of objects), or by writing new values to object properties.
The commands create ,delete ,split ,merge , and change add, remove, and change
the relationships of objects in the possible world, updating the values of number vari-
ables as necessary. The behavior of each detailed below, and illustrated and described
in Figure 13. Commands that remove objects have the eect of deleting any traces
for variables G(o1;:::;oi;:::on) of an object tuple including an oislated for deletion.
If the deleted object is the origin of other objects (e.g., if an Event with Detection s
is deleted), those objects are not automatically deleted: users may either manually
remove them or change their origins using change .
The command setG(o1;:::;on)[a] =vsets the sample at address aof a trace for
G(o1;:::;on) tovin the proposed state x0.
Transforming the auxiliary randomness ( y0).The user's code is responsible for
computing a new trace y0ofQthat makes finto an involution. Intuitively, y0must
specify the proposal choices that would reverse the current move; in Figure 2, the
forward move that posits a new Event to explain a false positive is reversed by a
move that deletes the newly created Event and reclassies its sole detection as a false
positive. Users can write to y0using the syntax backward[a] =v.
C.1. Object manipulation moves
The involution programming language includes commands to create, delete, split, merge,
and change the origins of objects in the world. It is clear that these moves should increment
or decrements number variables in the world to change the number of objects hypothesized
to exist. However, to enable us to delete any object with any index, or create an object at
any index (needed for reversibility), we need some scheme for reindexing the other existing
objects. (For example, when creating a new object 1, the system must \make room" for the
new object 1, for example by incrementing the index of every existing object.) We describe
the reindexing scheme we use.
28
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
create <type> with [origin <origin>] index <index>
create Event with index 2
delete
merge Detection with origin  ■ index 2,3 to index  2change     to origin ( ■, ⬤)  index  1Event()
1 2 3 1 2 3 4Event()
Event()
1 2 3 1 2 3 4Event()
Detection( ■)
1 2 1 2 3Detection( ■)
Detection( ■)
1 2 1 2 3Detection( ■)1 2Detection( ■)
1 2Detection( ■, ⬤)1Detection( ■)
2 3Detection( ■, ⬤)
1delete <object>
merge <object1>, <object2> to index  <index> [< changing  clauses>...] change <object> to origin <origin>  index <index>
split <object> to indices  <index1>, <index2> [< changing  clauses>...]
split    to indices 1, 2
  changing
    to origin (   , Event with index 2) index 1
Station() Detection(  ,  ) Event()
1
1 1Event()
2 1
Station() Detection(  ,  ) 1 1
 merge    ,     to index 1
  changing
     to origin (    , Event with index  1) index 1
  changing
     to origin (    , Event with index 1) index 2
Station() Detection(  ,  ) Event()
1
1 11Event()
2
Station()
Detection(  ,  ) 11
1Detection(  ,  )
2Inserts a new object <type>(<origin>, <index>) .
Deletes <object>.
Deletes <object>  with type <type> and inserts  new=<type>(<origin>, <index>) ,
changing all references to <object>  to become references to new.
Deletes <object>=<type>(<origin>, <index>)  and inserts  <type>(<origin>, <index1>) , and <type>(<origin>, <index2>) .
Performs the change commands specified by the changing  clauses, to change the origins of children of <object>.
Deletes <object1>=<type>(<origin>, <index1>)  and <object2>=<type>(<origin>, <index2>)  and inserts <type>(<origin>, <index>) .
Performs the change commands specified by the changing  clauses, to change origins of children of the deleted objects .
<type> with [origin <origin>], index <index>
Constructor for the object <type>(<origin>, <index>) .
      = Station with index 2
      = Detection  with origin    index 11
Station() 1
Detection( ■)
Figure 13: The object manipulation moves, with examples illustrating how they change the space of
objects in the seismic monitoring example. To construct references objects, for use in the commands,
illustrated with the shapes in this diagram, users use the syntax <type> with origin <origin>
index <index>.
Reindexing. For any given origin tuple of objects, origin , a world contains Norigin
Tob-
jects of type T:fT(origin;1);:::;T (origin;Norigin
T)g. Deleting the object T(origin;i)
corresponds to a \vector deletion" which decrements the value of Norigin
Tand decrements
the index of every object T(origin;j) forj > i . Likewise, adding an object T(origin;i)
is a \vector insertion" which increments Norigin
Tand increments the index of every object
T(origin;j) forji. In a world model, \changing the index" of an object T(origin;j)
corresponds to updating the world so that the trace for any property or number vari-
ableP[o1;:::;T (origin;j);:::;on] becomes the trace for P[o1;:::;T (origin;j+ 1);:::;on] or
29
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
P[o1;:::;T (origin;j1);:::;on]. Likewise, the update must change any object which has
T(origin;j) in its origin to instead have T(origin;j+ 1) orT(origin;j1) in its ori-
gin. While in a naive implementation, simple reindexing operations require copying many
traces and greatly perturbing the model state, which causes extreme ineciency when im-
plementing split/merge and birth/death moves in PPLs like vanilla Gen or Stochaskell, the
use of explicit open universe modeling constructs encodes enough structure for us to imple-
ment these moves very eciently without copying data, using the object aliasing technique
presented in Appendix D.
30
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Appendix D. Automatically Implementing Inference Kernels Eciently
In this section, we describe in more detail the technique by which we automate the imple-
mentation of inference kernels, and the optimizations we use to produce implementations
performant enough to scale to real-world data sizes. This technique extends upon an ap-
proach for automating involutive MCMC over traces of probabilistic programs described
in Cusumano-Towner et al. (2020). In particular, we are able to exploit sparsity in the
calculated Jacobian to avoid calculating the determinant of a large matrix, and exploit the
conditional-independence relationships encoded in the contingent Bayesian network under-
lying the OUPM to eciently calculate the state ratio p(x0)=p(x). The performance enabled
by the latter is one of the key benets of introducing rst-class support for involutive MCMC
on OUPMs.
To implement the inference kernel specied by a given proposal qand involution f,
we must be able to sample and score auxiliary randomness yqx, produce transitions
(x;y)!(x0;y0), and calculate acceptance probabilities by calculating qx(y),qx0(y0),p(x0)
p(x),
andjdet@(x0;y0)
@(x;y)j5. Sinceqis a probabilistic program, scoring and sampling are simple;
furthermore, yonly contains the minimal number of random choices needed to specify the
transitionx!x0, and thus is cheap to score. On the other hand, naive calculation of the
Jacobian term and state ratio may be extremely expensive, since xmay be large, for instance
containing a number of variables polynomial in dataset size. As shown in algorithm 3, after
samplingyqx, we factorize the move into 2 parts, both of which we provide ecient
algorithms for:
1. Running fin an instrumented manner to compute the Jacobian term, y0, and an
update specication Uforx
2. Computing x0and the state ratio from xandU
D.1. Eciently calculating the Jacobian term
The involutive acceptance probability must include the absolute value of the determinant
of the Jacobian matrix for the transformation of continuous values in ( x;y) into those in
(x0;y0). Naively, this can be extremely expensive to calculate, since jxjmay be very large,
and determinant calculation is O(n3) in matrix dimension.
Jacobian sparsity. In practice, most transformations ( x;y)!(x0;y0) only modify the
values of a small number of continuous values; all other continuous values are simply copied
from (x;y) to (x0;y0) (either at the same address or a new one). Such a copy introduces a
column into the Jacobian matrix which contains all zeros except for one 1 in the position
the value was copied to. Such a column cannot eect the absolute value of the matrix's
determinant, so we do not need to include rows and columns corresponding to copying in the
matrix. What this means in practice is that in algorithm 2, we only track transformations
of the values which are explicitly read from xoryinto values explicitly written into the next
5. In this section, we switch to using the notation jdet@(x0;y0)
@(x;y)jto mean the same thing as Jhi(x;y)) from
Theorem 1, to avoid the dependency on the symbol hi, and emphasize the idea of dierentiating contin-
uous values in ( x0;y0) with respect to those in ( x;y).
31
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Algorithm 2 Non-standard interpretation of ffor Jacobian calculation
procedure ProcessInvolution (f;(x;y))
.Track continuous reads and writes, manipulation moves M, and property updates U
(Rd;Wr;M;U;y0) fg;fg;[ ];fg;fg
Executef, but with
each manipulation command m(create ,delete ,change ,split ,merge )(M M[fmg)
"proposed [k]"(ify[k] is continuous: Rd  Rd[fy[k]g;y[k])
"set backward [k] =v"(ifvis continuous: Wr  Wr[fy[k]g;y0 y0[fk7!vg)
"get(P(o1;:::;on)[k])"(v x[P(o1;:::;on)][k];ifvis continuous: Rd  Rd[fvg;v)
"setP(o1;:::;on)[k] =v"(ifvis continuous: Wr  Wr[fvg;U[P(o1;:::;on)][k] v)
J uninitializedjRdjjRdjmatrix
foriinf1;:::;jWrjg:
.Executefwith reverse-mode AD to compute@Wr[i]
@Rd[j]for eachj2f1;:::;jRdjg
J[:;i] r Rd(Wr[i])
U CompileManipulationMoves( U;M;x ).Get variable updates to implement manipulation moves.
return (U;y0;jdetJj)
end procedure
statex0or reversing randomness y0; all unmentioned continuous values are simply copied
without transformation. This enables us to calculate the determinant of the Jacobian matrix
inO(1) in the size of xfor many updates, rather than O(jxj3).
Implementation via nonstandard interpretation. Algorithm 2 shows how to use
nonstandard interpretation of a world involution fto calculate the Jacobian correction
term. Additionally, this interpretation tracks all object manipulation moves which are
called, and all setcommands. These are compiled into an object Uwhich fully spec-
ies the update x!x0.Uis a mapping from variable names G(o1;:::;on) to partial
tracesU[G(o1;:::;on)], which map addresses to values. Uspecies to update each trace
x[G(o1;:::;on)] by overwriting addresses ain this trace with U[G(o1;:::;on)][a]. We do
not detail the process of compiling object manipulation moves into update specications; it
is a fairly straightforward process of determining how much each number variable must be
incremented, and changing object alias associations (which are described below).
D.2. Eciently calculating the state ratio
Because we use world models written using declarative getcommands to mediate depen-
dencies among variables in each world, our backend has access to information about which
variables are conditionally independent in each state for any world model. Unlike in existing
systems for programmable inference, which can only track dependency information in very
simple models, this in principle enables us to eciently calculate the state ratio
p(x0)
p(x)=Q
v2vars(x0)p(x0[v]jx0
Pax0(v))
Q
v2vars(x)p(x[v]jxPax(v))(6)
To do this, we only need to consider variables which introduce terms to this ratio that
do not cancel out. These are the variables which have been added to or deleted from x0,
variables whose traces have changed between xandx0, and variables whose dependencies'
values have changed.
Complications vs updating regular Bayesian networks. While world models are
similar to Bayesian networks in that they contain a dependency graph among variables,
they are far more general, and present novel challenges for producing ecient updates. In
32
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Algorithm 3 Ecient Involutive MCMC for World Models
procedure OUPM-IMCMC (p;q;f;x )
(y;qx(y)) SampleAndScore (q;x)
(U;y0;J) ProcessInvolution (f;(x;y))
(x0;p(x0)
p(x)) UpdateWorld (x;U)
qx0(y0) Score (q;x0;y0)
 p(x0)
p(x)qx0(y0)1
qx(y)J
with probability returnx0else return x
end procedure
.Update a trace by executing the corresponding
imperative probabilistic program.
procedure Execute (v=G(o1;:::;on);~x;U;x )
r 1,C get(U;v;fg)
t TraceG(o1;:::;on), but with
"ad"ifa2vars(C):C[a];else:x[v][a]
"getG0(o0
1;:::;o0
m)(
if(u=G0(o0
1;:::o0
m))=2vars(~x):
.If we might have to update u, we must do
it before accessing its value in v.
(~x;r0) Execute(G0(o0
1;:::o0
m);~x;U;x )
r rr0
~x[u] )
ifp~x(t) = 0:.In this case, vwill be pruned.
~x x[fx[v]g
else:
~x ~x[fv7!tg
ifv2vars(x):
r rp~x(t)
px(x[v])
else:
r rp~x(t)
return (~x;r)
end procedure.UpdatexusingUgiven ord = topologicalorderx.
procedure UpdateWorld (x;U; ord;childrenx)
~x fg.New world.
r 1.State ratio.
pos 0.Invariant: variables topologically be-
fore pos have been updated.
Q priorityqueue(ord ;default =1)
forv2vars(U):
enqueue!(Q;v)
while:isempty(Q):
v pop!(Q)
ifv =2vars(~x):
.Copy variables between pos and ord[ v] to
~x.
~x ~x[fu7!x[u] : ord[u]2[pos;ord[v])^
u =2vars(~x)g
pos ord[v]
(~x;r0) Execute(v;~x;U;x ).Updatev.
r rr0
ifv2vars(x)and ret~x(~x[v])6= retx(x[v]):
foru2childrenx(v):
enqueue!(Q;u)
.Remove variables the observation model no
longer depends on.
(x0;deleted) prune(~x;U)
forv2deleted:
r r=p~x(~x[v])
~x ~x
vars(~x)nv
return (x0;r)
end procedure
OUPMs, dependencies among variables are not xed, and may change from state to state
in a model, because the underlying Bayesian networks are contingent. While Milch et al.
(2005b) introduced the contingent Bayesian network formalism to handle such models, they
did not develop algorithms for performing arbitrary updates to these networks. Another
complication is that in a world model, the values of variables in the world may determin-
istically depend on other variables. For instance, it would be valid for the reading of a
seismic Detection to be deterministically equivalent to the magnitude of the correspond-
ingEvent . In these situations, the values of variables ( reading s) may change even if no
update is specied for them, since their values are deterministic functions of other updated
variables ( magnitude s). Hence, we cannot compile an update which only visit variables
in the Markov blankets of variables updates are explicitly specied for, unlike in Bayesian
networks. Instead, we introduce algorithm 3, which dynamically tracks changes to the de-
pendency graph and changes to variables which require new variables to be updated. One
important insight is that when we update a variable a, we must visit every variable bwhich
depends on ato correctly calculate the state ratio. At this time, we can check whether
b's value is deterministically changed by the update to a, and if so, enqueue the depen-
dencies ofbto be updated. This is implemented by procedure UpdateWorld , which pulls
variables relevant to the state ratio or update from a queue, adding new variables to the
33
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
queue as needed. However, complications arise due to the necessity of updating variables
in a topological order of the dependency graph.
Update order. The order in which variables are updated is critical for the correctness
ofUpdateWorld . For instance, say b2Pax0(a) anda;b2vars(U). If we update bbefore
we have updated a, it is impossible to calculate the factor p(x0[b]jx0
Pax0(b)) =p(x0[b]jx0[a])
in the state ratio, because the value of x0[a] is not yet known. Thus, variables must be
updated in the topological order induced by the graph of dependencies in the state x0.
However, we cannot know the topological order of x0before the update, so we instead use
the topological ordering of the old state x, with special-case logic to handle places where
this order diers from that in x0. This is justied by the fact that ecient updates are only
helpful in transitions x!x0producing fairly local changes, and thus we expect that updates
we could implement eciently will only change the world's topological order minorly. To
implement this, the update queue QinUpdateWorld is a priority queue indexed by the
topological order of x. In practice, UpdateWorld must also update the topological order
ofxto one forx0; we elide these details, along with details of maintaining the children x
lookup table (which gives the children of each variable in the CBN graph) and the details
of implementing the prune function which removes variables which are not needed for a
minimal self-supporting instantiation.
We handle changes in variable order between xandx0in the Execute procedure, which
updates variable traces. When Execute observes that a variable vgets the value of a
variableuwhich has not yet been updated, this means that in x,uwas topologically later
thanv, but inx0, it is topologically before v. In this situation, we recursively call Execute( u)
to ensure that we have updated uto the proper value before it is used to update v.
Runtime. The full algorithm which accounts for topological order updating runs in
O((N+M)(log(N+M) +T)), whereNin the number of variables which are updated
or impact the state ratio, Mis the number of variables whose indices in the topological
ordering change during the update, and Tis the maximum runtime of updating a single
variable in the world.6So long as N <<jvars(x0)jandM <<jvars(x0)j(both of which
are true for most local updates), this algorithm is much faster than generating a new state
x0from scratch, which takes (jvars(x0)jT).
Aliased object representation. To eciently implement object manipulation moves,
which may change the origins and indices of objects, we use a special internal representation
of worlds. If we stored a mapping from the string \ G(o1;:::;on)" to traces in the world,
then whenever an object ok=T(origin;j) had its origin or index change, we would need to
update the entry G(o1;:::;T (origin;j);:::;on) for every property of this object. Instead,
we assign each object a unique identier by storing a lookup table from identier ikto
objectok=T(origin;j) with each world state. We then store traces in a dictionary from
keysG(i1;:::;in), where each ikis an identier. Index and origin changes can thus be
implemented in O(1) by updating entries in the lookup table, rather than O(n) in the
number of properties of the updated object.
6. The log terms come from the need to sort variables to update in the priority queue. Future work on
static analysis of inference programs may be able to perform this sorting at compile-time in some cases.
34
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Appendix E. Involutive MCMC with Ancestral Regeneration.
A key benet of involutive MCMC is that it makes it feasible to rapidly prototype in-
ference kernels, because the acceptance-probability calculations are automated. A useful
development pattern is to begin with a kernel which uses generic inference techniques, like
ancestral regeneration, for proposal values for most variables, and then iterate on this ker-
nel by adding more data-driven proposals for variables for which the generic kernels are
ineective.
To facilitate this, in the involution programming language presented in Section C, we
add a command regenerate P(o1;:::;on) to resample property Pof (o1;:::;on) from the
prior distribution
pP(o1;:::;on)(jxPax(P(o1;:::;on))) (7)
wherePax(u) denotes the parents of variable uin the contingent Bayesian network for the
open universe model the kernel operates on (with density p), in world x.
To support this command, we must modify our formulation of automated involutive
MCMC. The reason for this is that in involutive MCMC, all randomness in an inference
kernel must come from the proposal yqx;fmust be a deterministic function. Therefore,
we must understand the execution of regenerate lines in the involution program as in fact
being part of the qxproposal which produces auxiliary randomness y.
The ancestral sampling distribution. Given a partial world Pcontaining values for
some of the variables over which a contingent Bayesian network is dened, we dene the
ancestral sampling distribution rPover the spaceWof MSSIs, which completes Pinto a
full MSSIw. IfwandPdisagree on any variable v(sow[v]6=P[v]), thenrP(w) = 0, and
if they agree on all variables,
rP(w) =Y
v2vars(w)nvars(P)pv(w[v]jwPaw(v)): (8)
An extended proposal and involution for ancestral regeneration. Say we have an
involutive MCMC kernel dened by a partial proposal distribution ~ qxand an \involution"
including regeneration commands ~f. To explain this as involutive MCMC, we now dene a
distribution qxusing ~qxwhich includes the randomness from regeneration. We will describe
qxas producing tuples (~ y;w), where ~y~qx, andwis sampled from the ancestral resampling
distribution r. Dene the function ~f1by~f(x;y) = ( ~f1(x;y);). We require ~fto be such that
~f(x;y) = ~x0, where ~x0is a partial world. (This comes from the user-written \involution"
~fapplying changes to world x, based on y, to produce a partial world, in which other
variables may then be ancestrally sampled to produce a complete world.) We can then say
wr~f1(x;~y). We dene:
qx((~y;w)) = ~qx(~y)r~f1(x;~y)(w) (9)
We now dene an involution f(x;(~y;w)) = (x0;(~y0;w0)). For this, we require that ~fhas
the property that if ~f(x;y) = (~x0;~y0), for any x02W which agrees with ~ x0,~f(x0;~y0) = (~x;~y0)
for some ~xagreeing with x. (This is an involution in the case where ~ x;~x02W.) Given ~f,
we dene
f(x;(~y;w)) = (w;(~y0;w0)) (10)
35
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
where
(~x0;~y0) =~f(x;~y);w0xvars(x)nvars( ~f1(~x0;~y0))
The acceptance ratio. Using the above distribution, for ~fsatisfying similar parti-
tioning properties as we require for involutions fin Theorem 1, the acceptance probability
(x;(~y;w);x0;(~y0;w0)) is
p(x0)
p(x)~qx0(~y0)r~f1(x0;~y0)(x)
~qx(~y)r~f1(x;~y)(x0)jdet@~f(x;~y)
@(x;~y)j (11)
wherejdet@~f(x;~y)
@(x;~y)jis the determinant of the Jacobian of the transformation of continuous
values under ~fat (x;~y).
E.1. Automating involutive MCMC with ancestral resampling.
In section C, we introduced a language for specifying involutions on open-universe models.
An involution in the language takes as arguments the current world state xand auxiliary
randomness ~ y~qx, and produces a sequence Mof object manipulation moves, and a set
Uof tuples (v;u), wherevis a variable to update the trace for, and uspecies the updated
value. It also produces the reversing randomness ~ y0and the Jacobian correction term J.
A new state x0is constructed by applying Mto the current state x, then applying all the
specied updates from U. However, it is possible that the produced state x0is not a valid
world model state, either because
1.x0is not minimal, in which case variables in x0are automatically deleted by our system
to ensure minimality; or
2.x0is not self-supporting, in which case additional variables needed for it to be self
supporting are automatically generated from the prior by our system.
Automatic deletions impose no semantic diculties; however, when adding variables auto-
matically, the above formulation of involutive MCMC with ancestral regeneration is needed
to correctly calculate acceptance ratios.
Accumulating the state ratio. To implement an MCMC kernel, after sampling
~y~qxand calculating ( M;U; ~y0;J), we construct state x0using a function update( x;M;U )
(Algorithm 3). To calculate the acceptance ratio, we can calculate ~ qx0(~y) and ~qx(~y0) directly,
and we know J. Thus, what remains is to calculatep(x0)
p(x)r~f1(x0;~y0)(x)
r~f1(x;~y)(x0). When no values are
generated via ancestral resampling, this reduces to the termp(x0)
p(x). Our implementation has
the update procedure return this term as well, as it can be calculated by accumulating values
while updating xtox0. We accumulatep(x0[v]jx0
Pax0(v))
p(x[v]jxPax(v))every time the trace for variable vis
updated from tvtot0
v,1
p(x[v]jxPax(v))for every variable vwhich is deleted, and p(x0[v]jx0
Pax0(v))
for every variable which is added. In the case where some values are produced via ancestral
sampling rather than the external proposal, we must account for ther~f1(x0;~y0)(x)
r~f1(x;~y)(x0)term. By
equation 8, the denominator of this is the product of terms pv(x0[v]jx0
Pax0(v)) for variables
36
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
vwhich are ancestral sampled in this move from xtox0. This is straightforward for the
update procedure to accumulate as it generates each of these values. The numerator of
this fraction is the product of pv(x[v]jxPax(v)) terms for each variable vwhich is ancestral-
sampled in the reverse move. To properly account for this, then, the update procedure
needs to know which properties will be ancestral-sampled in the reversing move. We can
thus dene this procedure's signature as ( x0;r~f(x0;~y0)(x)
r~f(x;~y)(x0)) = update( x;M;U;R ) whereRis
the set of variables which will be regenerated in the reversing move.
Involution language extension. To provide access to R, we extend the language from
section C, adding the syntax backward[P(o1;:::;on)] = regenerated to specify that the
reverse-direction proposal ancestral samples a value for P(o1;:::;on). One can also write
backward[properties of (o1;:::;on)] = regenerated to specify that all properties for
any object tuple including oare regenerated by the reverse move. We also add a command
regenerate Pofo, to specify that the value for P(o) should be ancestral-sampled to
producex0, rather than copying the value for this property currently in x. (Copying from x
is the default behavior when P(o1;:::;on) is present in x; ancestral sampling is the default
behavior when it is not present in x.)
One way to think about the need to specify the values ancestrally sampled in the reverse
move is as follows. Although users do not specify a dictionary key for values in the full
auxiliary randomness ywhich are generated by ancestral resampling, to ensure that the
fullfis a valid involution, fmust still write the values of these variables to y0in the
reversing move. Because users don't specify a dictionary key, we provide a special syntax,
set backward[ P(o1;:::;on)] = regenerated to automatically set the regenerated value
iny0of variable P(o1;:::;on) to equal this variable's current value in x.
While the need to specify when a reverse move uses ancestral sampling adds a small
cognitive burden to the user of our inference programming language, we provide dynamic
error checks to ensure it is not a source of incorrectness. If users forget to include the
proper backward[:::] = regenerated statements, we can dynamically detect this error
by running the reverse move and seeing which values are actually regenerated. It is thus
simple to report when users have forgotten to include these commands, making it a fairly
painless debugging procedure to add the proper reverse-regenerate specications.
37

George Matheos
Alexander K
Lew
Matin Ghavamizadeh
Stuart Russell
Marco Cusumano-Towner
Vikash K
Mansinghka