Early detection of aging-related diseases requires a model of the underlying biological aging process. In this paper, we develop a brain-age predictor by using structural magnetic resonance imaging (SMRI) and deep learning and evaluate the predicted brain age as a marker of brain- aging. Our approach does not require any domain knowledge in that it uses a transfer-learning paradigm and has been validated on real SMRI data collected from elderly subjects. We developed two different predictive models by using convolutional neural network (CNN) based regression and bucket classification to predict brain ages from SMRI images. Our models achieved root mean squared errors (RMSE) of 5.54 and 6.44 (years) in predicting brain ages of cognitively normal subjects. Further analysis showed that there is a substantial difference between the predicted brain ages of cognitively impaired subjects and normal subjects within the same chronological age group.