适合四边形(Tetragon)到blob

Question

应用不同的过滤和分割技术后,我最终得到如下图像:

我可以访问一些轮廓检测函数,它们返回该对象边缘上的点列表,或者返回一个拟合多边形(虽然有很多边,但远远超过4).我想要一种方法来将四边形贴合到那个形状上,因为我知道它是一个鞋盒的正面,应该是一个四边形.由于透视图,并行性不守恒,因此我现在没有约束,只需要包含此框的四个线段.

到目前为止我能找到的只是矩形拟合,它并没有真正返回我需要的结果,因为它迫使拟合的四边形为矩形.

如果我可以访问相机与鞋盒的相对角度并知道鞋盒与相机之间的距离,我可以生成一个Homography矩阵并扭曲图像,使鞋盒再次显示为矩形,但是现在我无法访问这些信息并希望以纯粹的愿景为基础.

任何已知的解决此类问题的方法？

Answer 1

我建议采取以下步骤:

1. threshold() 图片
2. dilate()图像 - 这将删除分割顶部和底部的黑线以及下部的较暗的伪影
3. findContours()使用设置仅检索外部轮廓(RETR_EXTERNAL)并简化输出(CHAIN_APPROX_SIMPLE)
4. 进一步处理轮廓

第1步:阈值

# threshold image
ret,thresh = cv2.threshold(img,127,255,0)
cv2.imshow('threshold ',thresh)

第2步:扩张

# dilate thresholded image - merges top/bottom 
kernel = np.ones((3,3), np.uint8)
dilated = cv2.dilate(thresh, kernel, iterations=3)
cv2.imshow('threshold dilated',dilated)

第3步:找到轮廓

# find contours
contours, hierarchy = cv2.findContours(dilated,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(img, contours, 0, (255,255,255), 3)
print "contours:",len(contours)
print "largest contour has ",len(contours[0]),"points"

请注意,首先放大,然后使用简单的外部轮廓让你塑造你所追求的,但它仍然相当复杂(包含279个点)

从这一点开始,您可以进一步处理轮廓特征.有几个选项,例如:

a:得分.区域矩形

# minAreaRect
rect = cv2.minAreaRect(contours[0])
box = cv2.cv.BoxPoints(rect)
box = np.int0(box)
cv2.drawContours(img,[box],0,(255,255,255),3)

可能有用,但不完全是你需要的.

b:凸壳

# convexHull
hull = cv2.convexHull(contours[0])
cv2.drawContours(img, [hull], 0, (255,255,255), 3)
print "convex hull has ",len(hull),"points"

更好,但你还有22分要处理,并且它可能不紧张

c:简化轮廓

# simplify contours

epsilon = 0.1*cv2.arcLength(contours[0],True)
approx = cv2.approxPolyDP(contours[0],epsilon,True)
cv2.drawContours(img, [approx], 0, (255,255,255), 3)
print "simplified contour has",len(approx),"points"

这可能就是你所追求的:只需4分.如果您需要更多积分,可以使用epsilon值.

请记住,现在你有一个四边形,但它的图像变平了:没有关于透视/三维旋转的信息.

完整的OpenCV Python代码清单(根据需要注释/取消注释,使用引用来适应c ++/java /等):

import numpy as np
import cv2

img = cv2.imread('XwzWQ.png',0)

# threshold image
ret,thresh = cv2.threshold(img,127,255,0)
cv2.imshow('threshold ',thresh)

# dilate thresholded image - merges top/bottom 
kernel = np.ones((3,3), np.uint8)
dilated = cv2.dilate(thresh, kernel, iterations=3)
cv2.imshow('threshold dilated',dilated)

# find contours
contours, hierarchy = cv2.findContours(dilated,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
# cv2.drawContours(img, contours, 0, (255,255,255), 3)
print "contours:",len(contours)
print "largest contour has ",len(contours[0]),"points"

# minAreaRect
# rect = cv2.minAreaRect(contours[0])
# box = cv2.cv.BoxPoints(rect)
# box = np.int0(box)
# cv2.drawContours(img,[box],0,(255,255,255),3)

# convexHull
# hull = cv2.convexHull(contours[0])
# cv2.drawContours(img, [hull], 0, (255,255,255), 3)
# print "convex hull has ",len(hull),"points"

# simplify contours
epsilon = 0.1*cv2.arcLength(contours[0],True)
approx = cv2.approxPolyDP(contours[0],epsilon,True)
cv2.drawContours(img, [approx], 0, (255,255,255), 3)
print "simplified contour has",len(approx),"points"


# display output 
cv2.imshow('image',img)

cv2.waitKey(0)
cv2.destroyAllWindows()

Answer 2

将四边形适合任意蒙版的更准确方法是使用视图平截头体优化\n最大化对象\xe2\x80\x99s 图像区域中的技术中的技术。

\n\n

这是输出 -

\n

\n\n

import cv2\nimport numpy as np\nimport sympy\n\n\ndef appx_best_fit_ngon(mask_cv2, n: int = 4) -> list[(int, int)]:\n    # convex hull of the input mask\n    mask_cv2_gray = cv2.cvtColor(mask_cv2, cv2.COLOR_RGB2GRAY)\n    contours, _ = cv2.findContours(\n        mask_cv2_gray, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE\n    )\n    hull = cv2.convexHull(contours[0])\n    hull = np.array(hull).reshape((len(hull), 2))\n\n    # to sympy land\n    hull = [sympy.Point(*pt) for pt in hull]\n\n    # run until we cut down to n vertices\n    while len(hull) > n:\n        best_candidate = None\n\n        # for all edges in hull ( <edge_idx_1>, <edge_idx_2> ) ->\n        for edge_idx_1 in range(len(hull)):\n            edge_idx_2 = (edge_idx_1 + 1) % len(hull)\n\n            adj_idx_1 = (edge_idx_1 - 1) % len(hull)\n            adj_idx_2 = (edge_idx_1 + 2) % len(hull)\n\n            edge_pt_1 = sympy.Point(*hull[edge_idx_1])\n            edge_pt_2 = sympy.Point(*hull[edge_idx_2])\n            adj_pt_1 = sympy.Point(*hull[adj_idx_1])\n            adj_pt_2 = sympy.Point(*hull[adj_idx_2])\n\n            subpoly = sympy.Polygon(adj_pt_1, edge_pt_1, edge_pt_2, adj_pt_2)\n            angle1 = subpoly.angles[edge_pt_1]\n            angle2 = subpoly.angles[edge_pt_2]\n\n            # we need to first make sure that the sum of the interior angles the edge\n            # makes with the two adjacent edges is more than 180\xc2\xb0\n            if sympy.N(angle1 + angle2) <= sympy.pi:\n                continue\n\n            # find the new vertex if we delete this edge\n            adj_edge_1 = sympy.Line(adj_pt_1, edge_pt_1)\n            adj_edge_2 = sympy.Line(edge_pt_2, adj_pt_2)\n            intersect = adj_edge_1.intersection(adj_edge_2)[0]\n\n            # the area of the triangle we\'ll be adding\n            area = sympy.N(sympy.Triangle(edge_pt_1, intersect, edge_pt_2).area)\n            # should be the lowest\n            if best_candidate and best_candidate[1] < area:\n                continue\n\n            # delete the edge and add the intersection of adjacent edges to the hull\n            better_hull = list(hull)\n            better_hull[edge_idx_1] = intersect\n            del better_hull[edge_idx_2]\n            best_candidate = (better_hull, area)\n\n        if not best_candidate:\n            raise ValueError("Could not find the best fit n-gon!")\n\n        hull = best_candidate[0]\n\n    # back to python land\n    hull = [(int(x), int(y)) for x, y in hull]\n\n    return hull\n

\n

这是我用来生成此图像的代码 -

\n

hull = appx_best_fit_ngon(mask_cv2)\n\nfor idx in range(len(hull)):\n    next_idx = (idx + 1) % len(hull)\n    cv2.line(mask_cv2, hull[idx], hull[next_idx], (0, 255, 0), 1)\n\nfor pt in hull:\n    cv2.circle(mask_cv2, pt, 2, (255, 0, 0), 2)\n

\n