Show simple item record

dc.contributor.authorDhaka, Akash Kumaren_US
dc.contributor.authorCatalina, Alejandroen_US
dc.contributor.authorAndersen, Michael Riisen_US
dc.contributor.authorMagnusson, Månsen_US
dc.contributor.authorHuggins, Jonathan H.en_US
dc.contributor.authorVehtari, Akien_US
dc.date.accessioned2021-04-07T13:05:33Z
dc.date.available2021-04-07T13:05:33Z
dc.date.issued2020-12
dc.identifier.citationAkash Kumar Dhaka, Alejandro Catalina, Michael Riis Andersen, Måns Magnusson, Jonathan H Huggins, Aki Vehtari. 2020. "Robust, Accurate Stochastic Optimization for Variational Inference." Advances in Neural Information Processing Systems.
dc.identifier.urihttps://hdl.handle.net/2144/42359
dc.description.abstractWe consider the problem of fitting variational posterior approximations using stochastic optimization methods. The performance of these approximations depends on (1) how well the variational family matches the true posterior distribution, (2) the choice of divergence, and (3) the optimization of the variational objective. We show that even in the best-case scenario when the exact posterior belongs to the assumed variational family, common stochastic optimization methods lead to poor variational approximations if the problem dimension is moderately large. We also demonstrate that these methods are not robust across diverse model types. Motivated by these findings, we develop a more robust and accurate stochastic optimization framework by viewing the underlying optimization algorithm as producing a Markov chain. Our approach is theoretically motivated and includes a diagnostic for convergence and a novel stopping rule, both of which are robust to noisy evaluations of the objective function. We show empirically that the proposed framework works well on a diverse set of models: it can automatically detect stochastic optimization failure or inaccurate variational approximation.en_US
dc.description.urihttps://papers.nips.cc/paper/2020/file/7cac11e2f46ed46c339ec3d569853759-Paper.pdf
dc.language.isoen_US
dc.relation.ispartofAdvances in Neural Information Processing Systems
dc.titleRobust, accurate stochastic optimization for variational inferenceen_US
dc.typeConference materialsen_US
pubs.elements-sourcemanual-entryen_US
pubs.notesEmbargo: Not knownen_US
pubs.organisational-groupBoston Universityen_US
pubs.organisational-groupBoston University, College of Arts & Sciencesen_US
pubs.organisational-groupBoston University, College of Arts & Sciences, Department of Mathematics & Statisticsen_US
pubs.publication-statusPublisheden_US
dc.description.oaversionPublished version
dc.identifier.mycv592279


This item appears in the following Collection(s)

Show simple item record