Real-time approximate inference for scene understanding with generative models

Real-time approximate inference for scene understanding with generative models. Felip, J., Ahuja, N., Gómez-Gutiérrez, D., Tickoo, O., & Mansinghka, V. In Workshop on Perceptions as Generative Reasoning (co-located with NeurIPS 2019), 2019.

Paper abstract bibtex 4 downloads

Consider scene understanding problems such as predicting where a person is probably reaching, or inferring the pose of 3D objects from depth images, or inferring the probable street crossings of pedestrians. This paper shows how to solve these problems using inference in generative models, by introducing new techniques for real-time inference. The underlying generative models are built from realistic simulation software, wrapped in a Bayesian error model for the gap between simulation outputs and real data. This paper shows that it is possible to perform approximately Bayesian inference in these models, obtaining high-quality approximations to the full posterior over scenes, without using bottom-up networks. Instead, this paper introduces two general real-time inference techniques. The first is to train neural surrogates of the simulators. The second is to adaptively discretize the latent variables using a Tree-Pyramid (TP) approach. THis paper also shows that by combining these techniques, it is possible to perform accurate, approximately Bayesian inference in realistic generative models, in real time.

@inproceedings{felip2019real,
title                 = {Real-time approximate inference for scene understanding with generative models},
author                = {Felip, Javier and Ahuja, Nilesh and G{\'o}mez-Guti{\'e}rrez, David and Tickoo, Omesh and Mansinghka, Vikash},
booktitle             = {Workshop on Perceptions as Generative Reasoning (co-located with NeurIPS 2019)},
year                  = 2019,
url_paper             = {https://pgr-workshop.github.io/img/PGR019.pdf},
abstract              = {Consider scene understanding problems such as predicting where a person is probably reaching, or inferring the pose of 3D objects from depth images, or inferring the probable street crossings of pedestrians. This paper shows how to solve these problems using inference in generative models, by introducing new techniques for real-time inference.  The underlying generative models are built from realistic simulation software, wrapped in a Bayesian error model for the gap between simulation outputs and real data. This paper shows that it is possible to perform approximately Bayesian inference in these models, obtaining high-quality approximations to the full posterior over scenes, without using bottom-up networks.  Instead, this paper introduces two general real-time inference techniques.  The first is to train neural surrogates of the simulators.  The second is to adaptively discretize the latent variables using a Tree-Pyramid (TP) approach.  THis paper also shows that by combining these techniques, it is possible to perform accurate, approximately Bayesian inference in realistic generative models, in real time.}
}

Downloads: 4