C++/OpenCV:如何使用BOWImgDescriptorExtractor来确定哪些集群与词汇表中的哪些图像相关？

Question

我的目标是将图像作为查询并在图像库中找到它的最佳匹配.我在openCV 3.0.0中使用SURF功能,并使用Bag of Words方法找到匹配项.我需要一种方法来查明查询图像是否在库中匹配.如果是这样,我想知道最接近匹配的图像的索引.

这是我的所有图像读取代码(图像库中共300个)并提取和聚类功能:

Mat training_descriptors(1, extractor->descriptorSize(), extractor->descriptorType());
//read in all images and set to binary
char filepath[1000];
for (int i = 1; i < trainingSetSize; i++){
    cout << "in for loop, iteration: " << i << endl;
    _snprintf_s(filepath, 100, "C:/Users/Randal/Desktop/TestCase1Training/%d.bmp", i);
    Mat temp = imread(filepath, CV_LOAD_IMAGE_GRAYSCALE);
    Mat tempBW;
    adaptiveThreshold(temp, tempBW, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 11, 2);
    detector->detect(tempBW, keypoints1);
    extractor->compute(tempBW, keypoints1, descriptors1);
    training_descriptors.push_back(descriptors1);
    cout << "descriptors added" << endl;

}
cout << "Total descriptors: " << training_descriptors.rows << endl;
trainer.add(training_descriptors);

Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("FlannBased");
BOWImgDescriptorExtractor BOW(extractor, matcher);
Mat library = trainer.cluster();
BOW.setVocabulary(library);

我写了下面的代码,试图找到一个匹配.问题是BOW.compute只返回图像和图像库中存在的簇(单词)的索引.imgQ是查询图像.

Mat output;
Mat imgQBW;
adaptiveThreshold(imgQ, imgQBW, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 11, 2);
imshow("query image", imgQBW);
detector->detect(imgQBW, keypoints2);
extractor->compute(imgQBW, keypoints2, descriptors2);

BOW.compute(imgQBW, keypoints1, output);
cout << output.row(0) << endl;

我需要知道BoW中哪些簇对应哪些图像.我现在的输出 - output.row(0) - 只是一个数组,其中包含库中找到的所有簇索引.我误解了这个输出吗？有没有办法确定哪个图像具有最匹配的聚类？

Answer 1

我也根据这段代码做了类似的事情：

https://github.com/royshil/FoodcamClassifier/blob/master/training_common.cpp

但上面的部分是在聚类完成之后。你所要做的就是使用你的 ML（我使用了 SVM）和你的聚类中心（你拥有的视觉词袋）进行训练。此外，您需要找到所有与聚类点“最接近”的点，并使用直方图训练它们。接下来，您将获得需要训练的频率直方图（关键点包）。

Ptr<ifstream> ifs(new ifstream("training.txt"));
int total_samples_in_file = 0;
vector<string> classes_names;
vector<string> lines; 

//read from the file - ifs and put into a vector
for(int i=0;i<lines.size();i++) {

    vector<KeyPoint> keypoints;
    Mat response_hist;
    Mat img;
    string filepath;

    string line(lines[i]);
    istringstream iss(line);

    iss >> filepath;

    string class_to_train; 
    iss >> class_to_train; 
    class_ml = "class_" + class_to_train;
    if(class_ml.size() == 0) continue;

    img = imread(filepath);

    detector->detect(img,keypoints);
    bowide.compute(img, keypoints, response_hist);

    cout << "."; cout.flush();
    //here create the logic for the class to train(class_0, e.g) and the data you need to train.
}

您可以在此 git 项目中找到更多信息：
https://github.com/royshil/FoodcamClassifier
此处的文档： http:
//www.morethantechnical.com/2011/08/25/a-simple-object-classifier-with-bag -of-words-using-opencv-2-3-w-code/