Accelerating large scale Graph Neural Network training using SmartSSD
OA Version
Citation
Abstract
Graph Neural Network (GNN) is a type of neural network that processes data structured as graphs. It excels at capturing complex relationships and dependencies between nodes within the graph. GNNs are increasingly significant, with numerous applications in the real world, such as social network analysis, recommendation systems, and node or graph classification. However, as graphs grow in size, they can exceed available memory, necessitating on-disk sampling for large-scale GNNs. In this work, we propose a novel system using SmartSSD to perform sampling for large-scale Graph Neural Networks. We designed three different SmartSSD-based kernels to perform neighborhood sampling. The Random Read Sampler simulates the random read process of in-memory sampling on the SSD, aggregates the sampled results on an FPGA, and then sends them back to the host in batches. The Sequential Read Sampler transfers the edge file in chunks to the FPGA memory and scans the chunk to draw samples.
The Streaming Sampler, an optimization of the Sequential Read Sampler, employs an improved edge file chunk format to enable parallel sampling on the FPGA, thereby accelerating the sampling process. We discuss the design and implementation of all three samplers and demonstrate their sampling time consumption in the experiment. For the largest graph dataset, Yahoo, that we tested, our optimized kernels achieve a speedup factor of 3.6 over the baseline.
Description
2024
License
Attribution 4.0 International