Person re-identification using fisheye cameras with application to occupancy analysis
OA Version
Citation
Abstract
Person re-identification (PRID), the problem of matching identity of a person between images, finds applications in video surveillance, sports analytics, and, more recently, in spatial analytics (e.g., retail). PRID has been extensively studied for the case of standard surveillance cameras equipped with rectilinear lens. However, their narrow field of view (FOV) severely limits the indoor area each camera can monitor. Recently, fisheye cameras that capture 360° FOV have penetrated the video surveillance market, but little attention has been paid in the literature to PRID for such cameras. This dissertation focuses on fisheye-camera PRID and demonstrates its effectiveness in occupancy estimation in large indoor spaces.
Since no fisheye PRID datasets were publicly available, we created one using 3 ceiling-mounted fisheye cameras in a large classroom and published it on-line (63 downloads to date). Subsequently, we evaluated 6 state-of-the-art PRID methods on our dataset and concluded that such methods, developed for rectilinear cameras, do not perform well on fisheye images due to potential dramatic body-viewpoint and body-size differences between different camera views, and fisheye-lens distortions. To address these challenges, we developed a novel approach to PRID that relies on occupant location in the room instead of appearance. This approach is possible in our scenario since overhead fisheye cameras have overlapping FOVs; knowing location of a person in one camera view, we map this location to another camera view with knowledge of the person's approximate height. The distance between a mapped and current location allows to match identities, and we develop 4 distance metrics for this purpose using a range of typical human heights. Evaluated on our dataset, the location-based approach outperforms the 6 state-of-the-art PRID methods that use appearance by at least 10% points in accuracy, but struggles when people are very close to one another. To address this challenge, we proposed combining location-based methodology with appearance features (deep-learning embedding and color histogram) by means of a Naive Bayes method. The additional appearance features improve the location-based re-identification accuracy by at least 2% points.
To demonstrate the practical importance of fisheye PRID, we evaluated its potential for accurate people counting in a large space with high occupancy. Firstly, we assessed the performance of occupancy sensing using single fisheye camera and concluded that high counting accuracy is possible only in small-to-medium size spaces. We then proposed a two-camera system, that employs fisheye PRID to avoid overcounting, and demonstrated an up to 20%-point accuracy boost compared to single-camera approaches. To support even larger spaces, we proposed and evaluated two extensions of PRID to N cameras. Overall, our results show that the proposed fisheye PRID methods enable high-accuracy people counting in large indoor environments, and have a great potential for improving people tracking and activity analysis.
Description
License
Attribution 4.0 International