获取opencv中聚集在一起的多个近似行的单行表示

Question

我在一个图像中检测到了行,并使用HoughLinesP方法将它们绘制在OpenCv C++中的单独图像文件中.以下是该结果图像的一部分.实际上有数百条细线和细线形成一条大单线.

但我想要一些代表所有这些行数的行.较近的线应该合并在一起形成一条线.例如,上面的行集应该由下面的3个单独的行表示.

预期产量如上.如何完成这项任务.

---

---

到目前为止,akarsakov的答案取得了进展.

---

(产生的不同类别的行以不同的颜色绘制).请注意,此结果是我正在处理的原始完整图像,但不是我在问题中使用的示例部分

Answer 1

如果您不知道图像中的行数,可以使用该cv::partition函数在等效组上拆分行.

我建议你使用以下程序:

1. 使用分割您的线条cv::partition.您需要指定一个好的谓词函数.它真的取决于你从图像中提取的线条,但我认为它应该检查以下条件:

   • 线之间的角度应该非常小(例如,小于3度).使用点积来计算角度的余弦值.
   • 段中心之间的距离应小于两段最大长度的一半.

例如,它可以实现如下:

bool isEqual(const Vec4i& _l1, const Vec4i& _l2)
{
    Vec4i l1(_l1), l2(_l2);

    float length1 = sqrtf((l1[2] - l1[0])*(l1[2] - l1[0]) + (l1[3] - l1[1])*(l1[3] - l1[1]));
    float length2 = sqrtf((l2[2] - l2[0])*(l2[2] - l2[0]) + (l2[3] - l2[1])*(l2[3] - l2[1]));

    float product = (l1[2] - l1[0])*(l2[2] - l2[0]) + (l1[3] - l1[1])*(l2[3] - l2[1]);

    if (fabs(product / (length1 * length2)) < cos(CV_PI / 30))
        return false;

    float mx1 = (l1[0] + l1[2]) * 0.5f;
    float mx2 = (l2[0] + l2[2]) * 0.5f;

    float my1 = (l1[1] + l1[3]) * 0.5f;
    float my2 = (l2[1] + l2[3]) * 0.5f;
    float dist = sqrtf((mx1 - mx2)*(mx1 - mx2) + (my1 - my2)*(my1 - my2));

    if (dist > std::max(length1, length2) * 0.5f)
        return false;

    return true;
}

猜猜你有你的线路vector<Vec4i> lines;.接下来,您应该调用cv::partition如下:

vector<Vec4i> lines;
std::vector<int> labels;
int numberOfLines = cv::partition(lines, labels, isEqual);

你需要调用cv::partition一次,它会聚集所有行.Vector labels将为其所属的群集的每个行标签存储.见文档的cv::partition

2. 获得所有线组后,您应该合并它们.我建议计算组中所有线的平均角度并估计"边界"点.例如,如果角度为零(即所有线条几乎都是水平的),则它将是最左侧和最右侧的点.它仍然只是在这些点之间画一条线.

我注意到示例中的所有行都是水平或垂直的.在这种情况下,您可以计算所有段的中心和"边界"点的平均值,然后只绘制通过中心点的"边界"点限制的水平或垂直线.

请注意,cv::partition花费O(N ^ 2)时间,因此如果处理大量行,可能需要花费很多时间.

我希望它会有所帮助.我用这种方法进行类似的任务.

Answer 2

首先我要注意你的原始图像是一个微小的角度,所以你的预期输出对我来说似乎有些偏差.我假设您的输出中的线条不是100%垂直,因为它们在您的输入上略微偏离.

Mat image;
Mat binary = image > 125;  // Convert to binary image

// Combine similar lines
int size = 3;
Mat element = getStructuringElement( MORPH_ELLIPSE, Size( 2*size + 1, 2*size+1 ), Point( size, size ) );
morphologyEx( mask, mask, MORPH_CLOSE, element );

到目前为止,这产生了这个图像

这些线不是90度角,因为原始图像不是.

您还可以选择关闭线之间的间隙:

Mat out = Mat::zeros(mask.size(), mask.type());

vector<Vec4i> lines;
HoughLinesP(mask, lines, 1, CV_PI/2, 50, 50, 75);
for( size_t i = 0; i < lines.size(); i++ )
{
    Vec4i l = lines[i];
    line( out, Point(l[0], l[1]), Point(l[2], l[3]), Scalar(255), 5, CV_AA);
}

如果这些线太胖了,我已经成功地将它们变薄:

size = 15;
Mat eroded;
cv::Mat erodeElement = getStructuringElement( MORPH_ELLIPSE, cv::Size( size, size ) );
erode( mask, eroded, erodeElement );

Answer 3

Here is a refinement build upon @akarsakov answer. A basic issue with:

Distance between centers of segments should be less than half of maximum length of two segments.

is that parallel long lines that are visually far might end up in same equivalence class (as demonstrated in OP's edit).

Therefore the approach that I found working reasonable for me:

1. Construct a window (bounding rectangle) around a line1.
2. line2 angle is close enough to line1's and at least one point of line2 is inside line1's bounding rectangle

通常，图像中很弱的长线性特征最终会被一组线段识别（HoughP，LSD），它们之间有相当大的间隙。为了缓解这种情况，我们的边界矩形围绕在两个方向上延伸的线构建，其中延伸由原始线宽的一小部分定义。

bool extendedBoundingRectangleLineEquivalence(const Vec4i& _l1, const Vec4i& _l2, float extensionLengthFraction, float maxAngleDiff, float boundingRectangleThickness){

    Vec4i l1(_l1), l2(_l2);
    // extend lines by percentage of line width
    float len1 = sqrtf((l1[2] - l1[0])*(l1[2] - l1[0]) + (l1[3] - l1[1])*(l1[3] - l1[1]));
    float len2 = sqrtf((l2[2] - l2[0])*(l2[2] - l2[0]) + (l2[3] - l2[1])*(l2[3] - l2[1]));
    Vec4i el1 = extendedLine(l1, len1 * extensionLengthFraction);
    Vec4i el2 = extendedLine(l2, len2 * extensionLengthFraction);

    // reject the lines that have wide difference in angles
    float a1 = atan(linearParameters(el1)[0]);
    float a2 = atan(linearParameters(el2)[0]);
    if(fabs(a1 - a2) > maxAngleDiff * M_PI / 180.0){
        return false;
    }

    // calculate window around extended line
    // at least one point needs to inside extended bounding rectangle of other line,
    std::vector<Point2i> lineBoundingContour = boundingRectangleContour(el1, boundingRectangleThickness/2);
    return
        pointPolygonTest(lineBoundingContour, cv::Point(el2[0], el2[1]), false) == 1 ||
        pointPolygonTest(lineBoundingContour, cv::Point(el2[2], el2[3]), false) == 1;
}

其中linearParameters, extendedLine, boundingRectangleContour有以下几点：

Vec2d linearParameters(Vec4i line){
    Mat a = (Mat_<double>(2, 2) <<
                line[0], 1,
                line[2], 1);
    Mat y = (Mat_<double>(2, 1) <<
                line[1],
                line[3]);
    Vec2d mc; solve(a, y, mc);
    return mc;
}

Vec4i extendedLine(Vec4i line, double d){
    // oriented left-t-right
    Vec4d _line = line[2] - line[0] < 0 ? Vec4d(line[2], line[3], line[0], line[1]) : Vec4d(line[0], line[1], line[2], line[3]);
    double m = linearParameters(_line)[0];
    // solution of pythagorean theorem and m = yd/xd
    double xd = sqrt(d * d / (m * m + 1));
    double yd = xd * m;
    return Vec4d(_line[0] - xd, _line[1] - yd , _line[2] + xd, _line[3] + yd);
}

std::vector<Point2i> boundingRectangleContour(Vec4i line, float d){
    // finds coordinates of perpendicular lines with length d in both line points
    // https://math.stackexchange.com/a/2043065/183923

    Vec2f mc = linearParameters(line);
    float m = mc[0];
    float factor = sqrtf(
        (d * d) / (1 + (1 / (m * m)))
    );

    float x3, y3, x4, y4, x5, y5, x6, y6;
    // special case(vertical perpendicular line) when -1/m -> -infinity
    if(m == 0){
        x3 = line[0]; y3 = line[1] + d;
        x4 = line[0]; y4 = line[1] - d;
        x5 = line[2]; y5 = line[3] + d;
        x6 = line[2]; y6 = line[3] - d;
    } else {
        // slope of perpendicular lines
        float m_per = - 1/m;

        // y1 = m_per * x1 + c_per
        float c_per1 = line[1] - m_per * line[0];
        float c_per2 = line[3] - m_per * line[2];

        // coordinates of perpendicular lines
        x3 = line[0] + factor; y3 = m_per * x3 + c_per1;
        x4 = line[0] - factor; y4 = m_per * x4 + c_per1;
        x5 = line[2] + factor; y5 = m_per * x5 + c_per2;
        x6 = line[2] - factor; y6 = m_per * x6 + c_per2;
    }

    return std::vector<Point2i> {
        Point2i(x3, y3),
        Point2i(x4, y4),
        Point2i(x6, y6),
        Point2i(x5, y5)
    };
}

要分区，请致电：

std::vector<int> labels;
int equilavenceClassesCount = cv::partition(linesWithoutSmall, labels, [](const Vec4i l1, const Vec4i l2){
    return extendedBoundingRectangleLineEquivalence(
        l1, l2,
        // line extension length - as fraction of original line width
        0.2,
        // maximum allowed angle difference for lines to be considered in same equivalence class
        2.0,
        // thickness of bounding rectangle around each line
        10);
});

现在，为了将每个等价类减少为单线，我们从中构建点云并找到线拟合：

// fit line to each equivalence class point cloud
std::vector<Vec4i> reducedLines = std::accumulate(pointClouds.begin(), pointClouds.end(), std::vector<Vec4i>{}, [](std::vector<Vec4i> target, const std::vector<Point2i>& _pointCloud){
    std::vector<Point2i> pointCloud = _pointCloud;

    //lineParams: [vx,vy, x0,y0]: (normalized vector, point on our contour)
    // (x,y) = (x0,y0) + t*(vx,vy), t -> (-inf; inf)
    Vec4f lineParams; fitLine(pointCloud, lineParams, CV_DIST_L2, 0, 0.01, 0.01);

    // derive the bounding xs of point cloud
    decltype(pointCloud)::iterator minXP, maxXP;
    std::tie(minXP, maxXP) = std::minmax_element(pointCloud.begin(), pointCloud.end(), [](const Point2i& p1, const Point2i& p2){ return p1.x < p2.x; });

    // derive y coords of fitted line
    float m = lineParams[1] / lineParams[0];
    int y1 = ((minXP->x - lineParams[2]) * m) + lineParams[3];
    int y2 = ((maxXP->x - lineParams[2]) * m) + lineParams[3];

    target.push_back(Vec4i(minXP->x, y1, maxXP->x, y2));
    return target;
});

示范：

检测到的分区线（过滤掉小线）：

减少：

演示代码：

int main(int argc, const char* argv[]){

    if(argc < 2){
        std::cout << "img filepath should be present in args" << std::endl;
    }

    Mat image = imread(argv[1]);
    Mat smallerImage; resize(image, smallerImage, cv::Size(), 0.5, 0.5, INTER_CUBIC);
    Mat target = smallerImage.clone();

    namedWindow("Detected Lines", WINDOW_NORMAL);
    namedWindow("Reduced Lines", WINDOW_NORMAL);
    Mat detectedLinesImg = Mat::zeros(target.rows, target.cols, CV_8UC3);
    Mat reducedLinesImg = detectedLinesImg.clone();

    // delect lines in any reasonable way
    Mat grayscale; cvtColor(target, grayscale, CV_BGRA2GRAY);
    Ptr<LineSegmentDetector> detector = createLineSegmentDetector(LSD_REFINE_NONE);
    std::vector<Vec4i> lines; detector->detect(grayscale, lines);

    // remove small lines
    std::vector<Vec4i> linesWithoutSmall;
    std::copy_if (lines.begin(), lines.end(), std::back_inserter(linesWithoutSmall), [](Vec4f line){
        float length = sqrtf((line[2] - line[0]) * (line[2] - line[0])
                             + (line[3] - line[1]) * (line[3] - line[1]));
        return length > 30;
    });

    std::cout << "Detected: " << linesWithoutSmall.size() << std::endl;

    // partition via our partitioning function
    std::vector<int> labels;
    int equilavenceClassesCount = cv::partition(linesWithoutSmall, labels, [](const Vec4i l1, const Vec4i l2){
        return extendedBoundingRectangleLineEquivalence(
            l1, l2,
            // line extension length - as fraction of original line width
            0.2,
            // maximum allowed angle difference for lines to be considered in same equivalence class
            2.0,
            // thickness of bounding rectangle around each line
            10);
    });

    std::cout << "Equivalence classes: " << equilavenceClassesCount << std::endl;

    // grab a random colour for each equivalence class
    RNG rng(215526);
    std::vector<Scalar> colors(equilavenceClassesCount);
    for (int i = 0; i < equilavenceClassesCount; i++){
        colors[i] = Scalar(rng.uniform(30,255), rng.uniform(30, 255), rng.uniform(30, 255));;
    }

    // draw original detected lines
    for (int i = 0; i < linesWithoutSmall.size(); i++){
        Vec4i& detectedLine = linesWithoutSmall[i];
        line(detectedLinesImg,
             cv::Point(detectedLine[0], detectedLine[1]),
             cv::Point(detectedLine[2], detectedLine[3]), colors[labels[i]], 2);
    }

    // build point clouds out of each equivalence classes
    std::vector<std::vector<Point2i>> pointClouds(equilavenceClassesCount);
    for (int i = 0; i < linesWithoutSmall.size(); i++){
        Vec4i& detectedLine = linesWithoutSmall[i];
        pointClouds[labels[i]].push_back(Point2i(detectedLine[0], detectedLine[1]));
        pointClouds[labels[i]].push_back(Point2i(detectedLine[2], detectedLine[3]));
    }

    // fit line to each equivalence class point cloud
    std::vector<Vec4i> reducedLines = std::accumulate(pointClouds.begin(), pointClouds.end(), std::vector<Vec4i>{}, [](std::vector<Vec4i> target, const std::vector<Point2i>& _pointCloud){
        std::vector<Point2i> pointCloud = _pointCloud;

        //lineParams: [vx,vy, x0,y0]: (normalized vector, point on our contour)
        // (x,y) = (x0,y0) + t*(vx,vy), t -> (-inf; inf)
        Vec4f lineParams; fitLine(pointCloud, lineParams, CV_DIST_L2, 0, 0.01, 0.01);

        // derive the bounding xs of point cloud
        decltype(pointCloud)::iterator minXP, maxXP;
        std::tie(minXP, maxXP) = std::minmax_element(pointCloud.begin(), pointCloud.end(), [](const Point2i& p1, const Point2i& p2){ return p1.x < p2.x; });

        // derive y coords of fitted line
        float m = lineParams[1] / lineParams[0];
        int y1 = ((minXP->x - lineParams[2]) * m) + lineParams[3];
        int y2 = ((maxXP->x - lineParams[2]) * m) + lineParams[3];

        target.push_back(Vec4i(minXP->x, y1, maxXP->x, y2));
        return target;
    });

    for(Vec4i reduced: reducedLines){
        line(reducedLinesImg, Point(reduced[0], reduced[1]), Point(reduced[2], reduced[3]), Scalar(255, 255, 255), 2);
    }

    imshow("Detected Lines", detectedLinesImg);
    imshow("Reduced Lines", reducedLinesImg);
    waitKey();

    return 0;
}

Answer 4

我建议您使用 OpenCV 中的 HoughLines。

\n\n

void HoughLines(InputArray 图像、OutputArray 线、double rho、double theta、int 阈值、double srn=0、double stn=0 )

\n\n

您可以使用 rho 和 theta 调整要观察的线条的可能方向和位置。\n在您的情况下，theta = 90\xc2\xb0 就可以了（仅垂直和水平线条）。

\n\n

之后，您可以得到具有 Pl\xc3\xbccker 坐标的唯一直线方程。从那里您可以应用具有 3 个中心的 K 均值，该中心应该大约适合第二张图像中的 3 条线。

\n\n

PS：我会看看是否可以用你的图像测试整个过程

\n