Image processing widely used in agriculture sector for finding problems like disease identification, weed detection, fruit grading etc., In this paper, work is carried out based on automatic deficiency detection of Paddy leaves. Early detection of deficiency is most important to avoid qualitative and quantitative loss. Among all the grains rice is one of the most consumed grain in south India, but is easily affected by the nutrition deficiency. To increase the yield early identification of nutrient deficiencies of paddy crop is very essential. Paddy leaves color plays an important role in identifying micro deficiencies such as CSM (Calcium, Sulfur and Magnesium) during middle stage of its growth. Database of healthy, calcium defected, sulfur defected and magnesium defected leaves is created to identify deficient paddy leaves. HSV color model is used to extract Color features of both healthy and defected paddy leaves. Color features of test image is also extracted and compared against database properties. Comparison results are checked with the rules set to decide the specific deficiency. The rules are framed based on thorough experiment.