Independent finite approximations for Bayesian nonparametric inference: construction, error bounds, and practical implications

Files
2009.10780.pdf(1.82 MB)
First author draft
Date
2020-09-22
DOI
Authors
Nguyen, Tin
Huggins, Jonathan
Masoero, Lorenzo
Mackey, Lester
Broderick, Tamara
Version
First author draft
OA Version
Citation
Tin Nguyen, Jonathan Huggins, Lorenzo Masoero, Lester Mackey, Tamara Broderick. 2020. "Independent finite approximations for Bayesian nonparametric inference: construction, error bounds, and practical implications." arXiv.org, Volume arXiv:2009.10780 [stat.ME],
Abstract
Bayesian nonparametrics based on completely random measures (CRMs)offers a flexible modeling approach when the number of clusters or latent componentsin a dataset is unknown. However, managing the infinite dimensionality of CRMs oftenleads to slow computation. Practical inference typically relies on either integrating outthe infinite-dimensional parameter or using afinite approximation: a truncated finiteapproximation (TFA) or an independent finite approximation (IFA). The atom weightsof TFAs are constructed sequentially, while the atoms of IFAs are independent, which(1) make them well-suited for parallel and distributed computation and (2) facilitatesmore convenient inference schemes. While IFAs have been developed in certain spe-cial cases in the past, there has not yet been a general template for construction or asystematic comparison to TFAs. We show how to construct IFAs for approximating dis-tributions in a large family of CRMs, encompassing all those typically used in practice.We quantify the approximation error between IFAs and the target nonparametric prior,and prove that, in the worst-case, TFAs provide more component-efficient approxima-tions than IFAs. However, in experiments on image denoising and topic modeling taskswith real data, we find that the error of Bayesian approximation methods overwhelmsany finite approximation error, and IFAs perform very similarly to TFAs.
Description
License
Copyright © 2020 Elizabeth Coppock, Elizabeth Bogal-Allbritten and Golsa Nouri-Hosseini