The rapid increase in the senior population is posing serious challenges to national healthcare systems. Hence, innovative tools are needed to early detect health issues, including cognitive decline. Several clinical studies show that it is possible to identify cognitive impairment based on the locomotion patterns of the elderly. In this work, we investigate the use of sensor data and deep learning to recognize those patterns in instrumented smart-homes. In order to get rid of the noise introduced by indoor constraints and activity execution, we introduce novel visual feature extraction methods for locomotion data. Our solution relies on locomotion trace segmentation, image-based extraction of salient features from locomotion segments, and vision-based deep learning. We carried out extensive experiments with a large dataset acquired in a smart-home test bed from 153 seniors, including people with cognitive diseases. Results show that our system can accurately recognize the cognitive status of the senior, reaching a macro- score of 0.873 for the three categories that we target: cognitive health, mild cognitive impairment, and dementia. Moreover, an experimental comparison shows that our system outperforms state-of-the-art methods.