Neural network optimization with biologically inspired low-dimensional manifold learning
Files
Accepted manuscript
Date
DOI
Authors
Chin, Sang
Le, Hieu
Wood, Andrew
Dandekar, Sylee
Version
Accepted manuscript
OA Version
Citation
S. Chin, H. Le, A. Wood, S. Dandaker. "Neural Network Optimization with Biologically Inspired Low-Dimensional Manifold Learning,." International Conference on Computational Science and Computational Intelligence, 2021.
Abstract
Neural Networks learn to recognize and leverage
patterns in data. In most cases, while data is represented in
a high-dimensional space, the patterns within the data exist
along a manifold in a small subset of those dimensions. In this
paper, we show that by using a biologically inspired algorithm
called Geometric Multi-Resolution Analysis (GMRA), these low dimensional
manifolds can be computed and can be used to
convert datasets into more useful forms for learning. We also
show that, thanks to the lower-dimensional representation of the
converted datasets, that smaller networks can achieve state-of-the-art performance while using significantly fewer parameters.