Image Classification by Spatial Pooling of Surf

Today, there are many multimedia applications based on image understanding and processing, such as image retrieval, image classification, scene understanding, and so on. In image classification tasks, one of the most successful algorithms is the Bag-of-Features(BOF) model. Although the BOF model has many advantages, such as simplicity, generality, and scalability, it still suffers from several drawbacks, including the limited semantic description of local descriptors, lack of robust structures upon single visual words, and missing of efficient spatial weighting. To address these problems, the traditional BOF model is expanded on three aspects. First, a new scheme for combining texture and edge-based local features together at the descriptor extraction level. Next, to build geometric visual phrases to model spatial context upon complementary features for midlevel image representation. Finally, based on a smoothed edge map, a simple and effective spatial weighting scheme is performed to count occurrences of gradient orientation in localized portions of an image. Then test the framework by comparing the image with Fused Descriptors, which has the combination of SIFT, Edge-SIFT and Histogram of Oriented Gradient (HOG) descriptors for image classification and retrieval.