This paper attempts to introduce the concept of moment invariance into the classification algorithm based on the morphological boundary extraction and generalized skeleton transform, and tries to propose a new method about shape classification in this field. Firstly, the method extracts boundary of the object, secondly, skeleton using main skeleton extraction algorithm based on visual important parts, and then improves Hu moment which is traditionally used to region while in this paper is extended to calculate the invariants of the skeleton. The present paper describes respective theories and application of the method from four aspects, including the boundary extraction, skeleton extraction, calculation of moment invariance and classification algorithm.