如何在不包含(或少数几个)背景像素的情况下调整或调整对象内的矩形？

Question

应用阈值并找到对象的轮廓后,我使用以下代码获取对象周围的直线矩形(或旋转的矩形输入其指令):

img = cv2.imread('image.png')
imgray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(imgray,127,255,cv2.THRESH_BINARY)
# find contours 
contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
cnt = contours[0]
# straight rectangle
x,y,w,h = cv2.boundingRect(cnt)
img= cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)

看到图像

然后我使用以下代码计算了直线矩形内的对象和背景像素数:

# rectangle area (total number of object and background pixels inside the rectangle)
area_rect = w*h
# white or object pixels (inside the rectangle)
obj = cv2.countNonZero(imgray)
# background pixels (inside the rectangle)
bac = area_rect - obj

现在我想调整对象的矩形作为背景像素和对象的关系的函数,即,在没有背景像素或背景像素较少的情况下占据对象的较大部分的矩形,例如:

我该如何创建呢？

Answer 1

这个问题可以说是找到一个非凸多边形内接的最大矩形.

可以在此链接中找到近似解决方案.

这个问题也可以表述为:对于每个角度,找到矩阵中仅包含零的最大矩形,在此SO 问题中进行了探讨.

我的解决方案基于这个答案.这将只找到轴对齐的矩形,因此您可以轻松地按给定角度旋转图像,并为每个角度应用此解决方案.我的解决方案是C++,但您可以轻松地将其移植到Python,因为我主要使用OpenCV函数,或者在上面提到的答案中调整解决方案来解决轮换问题.

我们到了:

#include <opencv2\opencv.hpp>
#include <iostream>
using namespace cv;
using namespace std;


// https://stackoverflow.com/a/30418912/5008845
Rect findMinRect(const Mat1b& src)
{
    Mat1f W(src.rows, src.cols, float(0));
    Mat1f H(src.rows, src.cols, float(0));

    Rect maxRect(0,0,0,0);
    float maxArea = 0.f;

    for (int r = 0; r < src.rows; ++r)
    {
        for (int c = 0; c < src.cols; ++c)
        {
            if (src(r, c) == 0)
            {
                H(r, c) = 1.f + ((r>0) ? H(r-1, c) : 0);
                W(r, c) = 1.f + ((c>0) ? W(r, c-1) : 0);
            }

            float minw = W(r,c);
            for (int h = 0; h < H(r, c); ++h)
            {
                minw = min(minw, W(r-h, c));
                float area = (h+1) * minw;
                if (area > maxArea)
                {
                    maxArea = area;
                    maxRect = Rect(Point(c - minw + 1, r - h), Point(c+1, r+1));
                }
            }
        }
    }

    return maxRect;
}


RotatedRect largestRectInNonConvexPoly(const Mat1b& src)
{
    // Create a matrix big enough to not lose points during rotation
    vector<Point> ptz;
    findNonZero(src, ptz);
    Rect bbox = boundingRect(ptz); 
    int maxdim = max(bbox.width, bbox.height);
    Mat1b work(2*maxdim, 2*maxdim, uchar(0));
    src(bbox).copyTo(work(Rect(maxdim - bbox.width/2, maxdim - bbox.height / 2, bbox.width, bbox.height)));

    // Store best data
    Rect bestRect;
    int bestAngle = 0;

    // For each angle
    for (int angle = 0; angle < 90; angle += 1)
    {
        cout << angle << endl;

        // Rotate the image
        Mat R = getRotationMatrix2D(Point(maxdim,maxdim), angle, 1);
        Mat1b rotated;
        warpAffine(work, rotated, R, work.size());

        // Keep the crop with the polygon
        vector<Point> pts;
        findNonZero(rotated, pts);
        Rect box = boundingRect(pts);
        Mat1b crop = rotated(box).clone();

        // Invert colors
        crop = ~crop; 

        // Solve the problem: "Find largest rectangle containing only zeros in an binary matrix"
        // https://stackoverflow.com/questions/2478447/find-largest-rectangle-containing-only-zeros-in-an-n%C3%97n-binary-matrix
        Rect r = findMinRect(crop);

        // If best, save result
        if (r.area() > bestRect.area())
        {
            bestRect = r + box.tl();    // Correct the crop displacement
            bestAngle = angle;
        }
    }

    // Apply the inverse rotation
    Mat Rinv = getRotationMatrix2D(Point(maxdim, maxdim), -bestAngle, 1);
    vector<Point> rectPoints{bestRect.tl(), Point(bestRect.x + bestRect.width, bestRect.y), bestRect.br(), Point(bestRect.x, bestRect.y + bestRect.height)};
    vector<Point> rotatedRectPoints;
    transform(rectPoints, rotatedRectPoints, Rinv);

    // Apply the reverse translations
    for (int i = 0; i < rotatedRectPoints.size(); ++i)
    {
        rotatedRectPoints[i] += bbox.tl() - Point(maxdim - bbox.width / 2, maxdim - bbox.height / 2);
    }

    // Get the rotated rect
    RotatedRect rrect = minAreaRect(rotatedRectPoints);

    return rrect;
}



int main()
{
    Mat1b img = imread("path_to_image", IMREAD_GRAYSCALE);

    // Compute largest rect inside polygon
    RotatedRect r = largestRectInNonConvexPoly(img);

    // Show
    Mat3b res;
    cvtColor(img, res, COLOR_GRAY2BGR);

    Point2f points[4];
    r.points(points);

    for (int i = 0; i < 4; ++i)
    {
        line(res, points[i], points[(i + 1) % 4], Scalar(0, 0, 255), 2);
    }

    imshow("Result", res);
    waitKey();

    return 0;
}

结果图像是:

注意

我想指出这个代码没有优化,所以它可能表现得更好.有关近似解决方案,请参见此处,并在那里报告论文.

这回答一个相关的问题使我在正确的方向.