It is important for a construction and property development company to know weather conditions in their daily operation. In this paper, a deep learning-based approach is investigated to detect snow and rain conditions in construction sites using drone imagery. A Convolutional Neural Network (CNN) is developed for the feature extraction and performing classification on those features using machine learning (ML) algorithms. Well-known existing deep learning algorithms AlexNet and VGG16 models are also deployed and tested on the dataset. Results show that smaller CNN architecture with three convolutional layers was sufficient at extracting relevant features to the classification task at hand compared to the larger state-of-the-art architectures. The proposed model reached a top accuracy of 97.3% in binary classification and 96.5% while also taking rain conditions into consideration. It was also found that ML algorithms,i.e., support vector machine (SVM), logistic regression and k-nearest neighbors could be used as classifiers using feature maps extracted from CNNs and a top accuracy of 90% was obtained using SVM algorithms.