Breast cancer is one of the dangerous cancers among women. Due to this, the rate of death increases every year. In order to ease the radiologist task and early detection of breast cancer, multilayer architecture based on dyadic wavelet transform and Gaussian Mixture Model (GMM) is proposed in this paper. The chain of processes includes; preprocessing, feature extraction and classification. The need for preprocessing is to remove the noise such as background and patient information in the digital mammograms that affects the classification accuracy of the proposed system. In the feature extraction stage, the textural properties of mammograms are extracted by dyadic wavelet transform in various scale of decomposition. In order to reduce the redundancy of dyadic wavelet coefficients, an efficient averaged sub-band concept is developed. Then the features energy and entropy are extracted from the averaged sub-band and fed into the classifier. The classification consists of series of components; the first layer classifies the given mammogram into either normal or abnormal, the second layer decides the type of abnormalities either mass or microcalcification and the final layer classifies the severity of the abnormality into benign or malignant using GMM classifier. The results show that the average classification accuracy obtained at each layer is more than 95% when using the Digital Database for Screening Mammography (DDSM) database.