在一组cv :: Point上执行cv :: warpPerspective以进行伪偏移

Question

我正在尝试对一组点进行透视变换以实现偏斜效果:

http://nuigroup.com/?ACT=28&fid=27&aid=1892_H6eNAaign4Mrnn30Au8d

我正在使用下面的图像进行测试,绿色矩形显示感兴趣的区域.

我在想,如果有可能实现,我希望使用的简单组合的效果cv::getPerspectiveTransform和cv::warpPerspective.我正在分享我到目前为止所写的源代码,但它不起作用.这是结果图像:

因此,有一个vector<cv::Point>是定义感兴趣的区域,但点不存储在任何特定的顺序载体内,这件事情我不能在检测过程中发生改变.无论如何,稍后,向量中的点用于定义a RotatedRect,而这又用于组装cv::Point2f src_vertices[4];,所需的变量之一cv::getPerspectiveTransform().

我对顶点及其组织方式的理解可能是其中一个问题.我还认为使用a RotatedRect不是存储ROI原始点的最佳方法,因为坐标会稍微改变以适应旋转的矩形,这并不是很酷.

#include <cv.h>
#include <highgui.h>
#include <iostream>

using namespace std;
using namespace cv;

int main(int argc, char* argv[])
{
    cv::Mat src = cv::imread(argv[1], 1);

    // After some magical procedure, these are points detect that represent 
    // the corners of the paper in the picture: 
    // [408, 69] [72, 2186] [1584, 2426] [1912, 291]
    vector<Point> not_a_rect_shape;
    not_a_rect_shape.push_back(Point(408, 69));
    not_a_rect_shape.push_back(Point(72, 2186));
    not_a_rect_shape.push_back(Point(1584, 2426));
    not_a_rect_shape.push_back(Point(1912, 291));

    // For debugging purposes, draw green lines connecting those points 
    // and save it on disk
    const Point* point = &not_a_rect_shape[0];
    int n = (int)not_a_rect_shape.size();
    Mat draw = src.clone();
    polylines(draw, &point, &n, 1, true, Scalar(0, 255, 0), 3, CV_AA);
    imwrite("draw.jpg", draw);

    // Assemble a rotated rectangle out of that info
    RotatedRect box = minAreaRect(cv::Mat(not_a_rect_shape));
    std::cout << "Rotated box set to (" << box.boundingRect().x << "," << box.boundingRect().y << ") " << box.size.width << "x" << box.size.height << std::endl;

    // Does the order of the points matter? I assume they do NOT.
    // But if it does, is there an easy way to identify and order 
    // them as topLeft, topRight, bottomRight, bottomLeft?
    cv::Point2f src_vertices[4];
    src_vertices[0] = not_a_rect_shape[0];
    src_vertices[1] = not_a_rect_shape[1];
    src_vertices[2] = not_a_rect_shape[2];
    src_vertices[3] = not_a_rect_shape[3];

    Point2f dst_vertices[4];
    dst_vertices[0] = Point(0, 0);
    dst_vertices[1] = Point(0, box.boundingRect().width-1);
    dst_vertices[2] = Point(0, box.boundingRect().height-1);
    dst_vertices[3] = Point(box.boundingRect().width-1, box.boundingRect().height-1);

    Mat warpMatrix = getPerspectiveTransform(src_vertices, dst_vertices);

    cv::Mat rotated;
    warpPerspective(src, rotated, warpMatrix, rotated.size(), INTER_LINEAR, BORDER_CONSTANT);

    imwrite("rotated.jpg", rotated);

    return 0;
}

有人可以帮我解决这个问题吗？

Answer 1

所以,第一个问题是拐角顺序.它们在两个向量中必须处于相同的顺序.因此,如果在第一个向量中您的顺序是:(左上角,左下角,右下角,右上角),它们必须在另一个向量中的顺序相同.

其次,要使结果图像仅包含感兴趣的对象,必须将其宽度和高度设置为与生成的矩形宽度和高度相同.别担心,warpPerspective中的src和dst图像可以有不同的大小.

第三,表现关注.虽然你的方法是绝对准确的,因为你只做仿射变换(旋转,调整大小,去歪斜),在数学上,你可以使用你的函数的仿射对应.它们要快得多.

• getAffineTransform()

• warpAffine().

重要提示:getAffine转换需要并且只需要3个点,结果矩阵是2乘3,而不是3乘3.

如何使结果图像具有与输入不同的大小:

cv::warpPerspective(src, dst, dst.size(), ... );

使用

cv::Mat rotated;
cv::Size size(box.boundingRect().width, box.boundingRect().height);
cv::warpPerspective(src, dst, size, ... );

所以你在这里,你的编程任务结束了.

void main()
{
    cv::Mat src = cv::imread("r8fmh.jpg", 1);


    // After some magical procedure, these are points detect that represent 
    // the corners of the paper in the picture: 
    // [408, 69] [72, 2186] [1584, 2426] [1912, 291]

    vector<Point> not_a_rect_shape;
    not_a_rect_shape.push_back(Point(408, 69));
    not_a_rect_shape.push_back(Point(72, 2186));
    not_a_rect_shape.push_back(Point(1584, 2426));
    not_a_rect_shape.push_back(Point(1912, 291));

    // For debugging purposes, draw green lines connecting those points 
    // and save it on disk
    const Point* point = &not_a_rect_shape[0];
    int n = (int)not_a_rect_shape.size();
    Mat draw = src.clone();
    polylines(draw, &point, &n, 1, true, Scalar(0, 255, 0), 3, CV_AA);
    imwrite("draw.jpg", draw);

    // Assemble a rotated rectangle out of that info
    RotatedRect box = minAreaRect(cv::Mat(not_a_rect_shape));
    std::cout << "Rotated box set to (" << box.boundingRect().x << "," << box.boundingRect().y << ") " << box.size.width << "x" << box.size.height << std::endl;

    Point2f pts[4];

    box.points(pts);

    // Does the order of the points matter? I assume they do NOT.
    // But if it does, is there an easy way to identify and order 
    // them as topLeft, topRight, bottomRight, bottomLeft?

    cv::Point2f src_vertices[3];
    src_vertices[0] = pts[0];
    src_vertices[1] = pts[1];
    src_vertices[2] = pts[3];
    //src_vertices[3] = not_a_rect_shape[3];

    Point2f dst_vertices[3];
    dst_vertices[0] = Point(0, 0);
    dst_vertices[1] = Point(box.boundingRect().width-1, 0); 
    dst_vertices[2] = Point(0, box.boundingRect().height-1);

   /* Mat warpMatrix = getPerspectiveTransform(src_vertices, dst_vertices);

    cv::Mat rotated;
    cv::Size size(box.boundingRect().width, box.boundingRect().height);
    warpPerspective(src, rotated, warpMatrix, size, INTER_LINEAR, BORDER_CONSTANT);*/
    Mat warpAffineMatrix = getAffineTransform(src_vertices, dst_vertices);

    cv::Mat rotated;
    cv::Size size(box.boundingRect().width, box.boundingRect().height);
    warpAffine(src, rotated, warpAffineMatrix, size, INTER_LINEAR, BORDER_CONSTANT);

    imwrite("rotated.jpg", rotated);
}

Answer 2

问题是在向量中声明点的顺序,然后在定义上还有另一个与此相关的问题dst_vertices.

该点的顺序关系到getPerspectiveTransform(),必须按以下顺序指定:

1st-------2nd
 |         |
 |         |
 |         |
3rd-------4th

因此,原产地需要重新订购:

vector<Point> not_a_rect_shape;
not_a_rect_shape.push_back(Point(408, 69));
not_a_rect_shape.push_back(Point(1912, 291));
not_a_rect_shape.push_back(Point(72, 2186));
not_a_rect_shape.push_back(Point(1584, 2426));

和目的地:

Point2f dst_vertices[4];
dst_vertices[0] = Point(0, 0);
dst_vertices[1] = Point(box.boundingRect().width-1, 0); // Bug was: had mistakenly switched these 2 parameters
dst_vertices[2] = Point(0, box.boundingRect().height-1);
dst_vertices[3] = Point(box.boundingRect().width-1, box.boundingRect().height-1);

在此之后,需要进行一些裁剪,因为生成的图像不仅仅是绿色矩形内的区域,我认为它将是:

我不知道这是不是OpenCV的错误,或者我错过了什么,但主要问题已经解决了.

Answer 3

使用四边形时,OpenCV并不是你的朋友.RotatedRect会给你不正确的结果.此外,你需要一个透视投影,而不是像这里提到的其他仿射投影.

基本上必须做的是:

• 遍历所有多边形段并连接几乎相同的多边形段.
• 对它们进行排序,使您拥有4个最大的线段.
• 相交这些线,你有4个最可能的角点.
• 在从角点和已知对象的纵横比收集的透视图上变换矩阵.

我实现了一个Quadrangle处理轮廓到四边形转换的类,并且还将在正确的视角上对其进行转换.

请参阅此处的工作实现: Java OpenCV纠正轮廓

Answer 4

更新：已解决

我几乎有这个工作。如此接近可用。它校正正确，但我似乎有比例或翻译问题。我已将锚点设置为零，并尝试更改缩放模式（aspectFill、缩放以适应等）。

设置纠偏点（红色使它们难以看到）：

应用计算的变换：

现在它歪斜了。这看起来很不错，只是它不在屏幕上居中。通过向图像视图添加平移手势，我可以将其拖过并验证它是否对齐：

这不像通过 -0.5, -0.5 转换那么简单，因为原始图像变成了一个延伸非常远（可能）的多边形，因此它的边界矩形比屏幕框架大得多。

有没有人看到我能做些什么来解决这个问题？我想把它提交并在这里分享。这是一个热门话题，但我还没有找到像复制/粘贴一样简单的解决方案。

完整的源代码在这里：

git 克隆 https://github.com/zakkhoyt/Quadrilateral.git

git 结帐演示

但是，我将在此处粘贴相关部分。第一种方法是我的，也是我获得纠偏点的地方。

- (IBAction)buttonAction:(id)sender {

    Quadrilateral quadFrom;
    float scale = 1.0;
    quadFrom.topLeft.x = self.topLeftView.center.x / scale;
    quadFrom.topLeft.y = self.topLeftView.center.y / scale;
    quadFrom.topRight.x = self.topRightView.center.x / scale;
    quadFrom.topRight.y = self.topRightView.center.y / scale;
    quadFrom.bottomLeft.x = self.bottomLeftView.center.x / scale;
    quadFrom.bottomLeft.y = self.bottomLeftView.center.y / scale;
    quadFrom.bottomRight.x = self.bottomRightView.center.x / scale;
    quadFrom.bottomRight.y = self.bottomRightView.center.y / scale;

    Quadrilateral quadTo;
    quadTo.topLeft.x = self.view.bounds.origin.x;
    quadTo.topLeft.y = self.view.bounds.origin.y;
    quadTo.topRight.x = self.view.bounds.origin.x + self.view.bounds.size.width;
    quadTo.topRight.y = self.view.bounds.origin.y;
    quadTo.bottomLeft.x = self.view.bounds.origin.x;
    quadTo.bottomLeft.y = self.view.bounds.origin.y + self.view.bounds.size.height;
    quadTo.bottomRight.x = self.view.bounds.origin.x + self.view.bounds.size.width;
    quadTo.bottomRight.y = self.view.bounds.origin.y + self.view.bounds.size.height;

    CATransform3D t = [self transformQuadrilateral:quadFrom toQuadrilateral:quadTo];
//    t = CATransform3DScale(t, 0.5, 0.5, 1.0);
    self.imageView.layer.anchorPoint = CGPointZero;
    [UIView animateWithDuration:1.0 animations:^{
        self.imageView.layer.transform = t;
    }];

}


#pragma mark OpenCV stuff...
-(CATransform3D)transformQuadrilateral:(Quadrilateral)origin toQuadrilateral:(Quadrilateral)destination {

    CvPoint2D32f *cvsrc = [self openCVMatrixWithQuadrilateral:origin];
    CvMat *src_mat = cvCreateMat( 4, 2, CV_32FC1 );
    cvSetData(src_mat, cvsrc, sizeof(CvPoint2D32f));


    CvPoint2D32f *cvdst = [self openCVMatrixWithQuadrilateral:destination];
    CvMat *dst_mat = cvCreateMat( 4, 2, CV_32FC1 );
    cvSetData(dst_mat, cvdst, sizeof(CvPoint2D32f));

    CvMat *H = cvCreateMat(3,3,CV_32FC1);
    cvFindHomography(src_mat, dst_mat, H);
    cvReleaseMat(&src_mat);
    cvReleaseMat(&dst_mat);

    CATransform3D transform = [self transform3DWithCMatrix:H->data.fl];
    cvReleaseMat(&H);

    return transform;
}

- (CvPoint2D32f*)openCVMatrixWithQuadrilateral:(Quadrilateral)origin {

    CvPoint2D32f *cvsrc = (CvPoint2D32f *)malloc(4*sizeof(CvPoint2D32f));
    cvsrc[0].x = origin.topLeft.x;
    cvsrc[0].y = origin.topLeft.y;
    cvsrc[1].x = origin.topRight.x;
    cvsrc[1].y = origin.topRight.y;
    cvsrc[2].x = origin.bottomRight.x;
    cvsrc[2].y = origin.bottomRight.y;
    cvsrc[3].x = origin.bottomLeft.x;
    cvsrc[3].y = origin.bottomLeft.y;

    return cvsrc;
}

-(CATransform3D)transform3DWithCMatrix:(float *)matrix {
    CATransform3D transform = CATransform3DIdentity;

    transform.m11 = matrix[0];
    transform.m21 = matrix[1];
    transform.m41 = matrix[2];

    transform.m12 = matrix[3];
    transform.m22 = matrix[4];
    transform.m42 = matrix[5];

    transform.m14 = matrix[6];
    transform.m24 = matrix[7];
    transform.m44 = matrix[8];

    return transform; 
}

更新：我让它正常工作。坐标需要是中心的原点，而不是左上角。我应用了 xOffset 和 yOffset 以及中提琴。上面提到的位置的演示代码（“演示”分支）