The need for monitoring and surveillance systems for safe access in private or public places is increasing nowadays and becoming very necessary. In the current research paper, we introduced a monitoring agent system that helps in recognition of suspicious human behavior. The paper focuses on the classification task of the agent rather than on its architecture that can be a conventional one. Based on RGB images provided by the camera of the agent, an extraction process extracts silhouette images from RGB ones. The classification agent used the images of the silhouettes to classify human behavior into normal or abnormal. Two classification algorithms are implemented: Convolutional Neural Network (CNN) and a Support Vector Machine. The CNN algorithm implicitly extracts the required features from the silhouette images to perform two steps of classification. However, before SVM perform classification step, appropriate features are extracted. These features correspond to interest points (corners) that describe, the GAIT (walking manner). The results of performance, which are reported in this research showed that a very close to 93% classification rate, is achieved by CNN. The SVM classification rate is near to 75%. CNN obtained the classification model, has an average complexity as only two convolutional layers founded it