OpenCV数字合并到周围的框中

Question

我有一堆约会,我正在尝试使用tesseract进行OCR.但是,日期中的许多数字与日期框中的行合并为:

---

---

此外,这是一个很好的形象,我可以很好地测试:

---

这是我的代码:

import os
import cv2
from matplotlib import pyplot as plt
import subprocess
import numpy as np
from PIL import Image

def show(img):
    plt.figure(figsize=(20,20))
    plt.imshow(img,cmap='gray')
    plt.show()

def sort_contours(cnts, method="left-to-right"):
    # initialize the reverse flag and sort index
    reverse = False
    i = 0

    # handle if we need to sort in reverse
    if method == "right-to-left" or method == "bottom-to-top":
        reverse = True

    # handle if we are sorting against the y-coordinate rather than
    # the x-coordinate of the bounding box
    if method == "top-to-bottom" or method == "bottom-to-top":
        i = 1

    # construct the list of bounding boxes and sort them from top to
    # bottom
    boundingBoxes = [cv2.boundingRect(c) for c in cnts]

    cnts, boundingBoxes = zip(*sorted(zip(cnts, boundingBoxes),
        key=lambda b:b[1][i], reverse=reverse))

    # return the list of sorted contours and bounding boxes
    return cnts, boundingBoxes


def tesseract_it(contours,main_img, label,delete_last_contour=False):
    min_limit, max_limit = (1300,1700)
    idx =0 
    roi_list = []
    slist= set()
    for cnt in contours:
        idx += 1
        x,y,w,h = cv2.boundingRect(cnt)
        if label=='boxes':
            roi=main_img[y+2:y+h-2,x+2:x+w-2]
        else:
            roi=main_img[y:y+h,x:x+w]

        if w*h > min_limit and w*h < max_limit and w>10 and w< 50 and h>10 and h<50:
            if (x,y,w,h) not in slist: # Stops from identifying repeted contours

                roi = cv2.resize(roi,dsize=(45,45),fx=0 ,fy=0, interpolation = cv2.INTER_AREA)
                roi_list.append(roi)
                slist.add((x,y,w,h))

    if not delete_last_contour:
        vis = np.concatenate((roi_list),1)
    else:
        roi_list.pop(-1)
        vis = np.concatenate((roi_list),1)

    show(vis)

    # Tesseract the final image here
    # ...


image = 'bad_digit/1.jpg'
# image = 'bad_digit/good.jpg'
specimen_orig = cv2.imread(image,0)


specimen = cv2.fastNlMeansDenoising(specimen_orig)
#     show(specimen)
kernel = np.ones((3,3), np.uint8)

# Now we erode
specimen = cv2.erode(specimen, kernel, iterations = 1)
#     show(specimen)
_, specimen = cv2.threshold(specimen, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
#     show(specimen)
specimen_canny = cv2.Canny(specimen, 0, 0)
#     show(specimen_canny)

specimen_blank_image = np.zeros((specimen.shape[0], specimen.shape[1], 3))
_,specimen_contours, retr = cv2.findContours(specimen_canny.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE )
# print(len(specimen_contours))
cv2.drawContours(specimen_blank_image, specimen_contours, -1, 100, 2)
#     show(specimen_blank_image)
specimen_blank_image = np.zeros((specimen.shape[0], specimen.shape[1], 3))

specimen_sorted_contours, specimen_bounding_box = sort_contours(specimen_contours)

output_string = tesseract_it(specimen_sorted_contours,specimen_orig,label='boxes',)
# return output_string

---

附加的良好图像的输出是这样的:

---

测试这个图像确实给我准确的结果.

但是,对于行合并为数字的行,我的输出如下所示:

这些与Tesseract完全不兼容.我想知道是否有办法删除线条并只保留数字.

我也尝试了以下内容:https: //docs.opencv.org/3.2.0/d1/dee/tutorial_moprh_lines_detection.html

对于我附上的图像,这似乎并不是很好.

我也尝试过使用imagemagick:

convert original.jpg \
\( -clone 0 -threshold 50% -negate -statistic median 200x1 \)  \
-compose lighten -composite                                    \
\( -clone 0 -threshold 50% -negate -statistic median 1x200 \)  \
-composite output.jpg

它的结果是公平的,但删除的行有点削减数字如下:

有没有更好的方法来解决这个问题？我的最终目标是测试数字,因此最终图像确实需要非常清晰.

Answer 1

以下是一些似乎运行良好的代码.有两个阶段:

• 人们可以观察到数字比盒子略大.此外,整个图像具有强烈的水平状态.所以我们可以在水平方向上应用更强的扩张来摆脱大多数垂直线.
• 此时,OCR(例如Google的 OCR)可以检测到大多数数字.不幸的是,它有点太好了,并且看到了其他东西,所以我添加了另一个更复杂且与您的特定上下文相关的阶段.

这是第一阶段后的一张图片的结果:

以下是第二阶段后的所有结果:

正如你所看到的那样,它并不完美,8可以被看作是B(好吧,就像我这样的人把它视为B ......但如果你的世界中只有数字,它就可以轻松修复).还有一个":"字符(来自垂直线的遗留物已被删除),我无法摆脱过多调整代码......

C#代码:

static void Unbox(string inputFilePath, string outputFilePath)
{
    using (var orig = new Mat(inputFilePath))
    {
        using (var gray = orig.CvtColor(ColorConversionCodes.BGR2GRAY))
        {
            using (var dst = orig.EmptyClone())
            {
                // this is what I call the "horizontal shake" pass.
                // note I use the Rect shape here, this is important
                using (var dilate = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(4, 1)))
                {
                    Cv2.Dilate(gray, dst, dilate);
                }

                // erode just a bit to get back some numbers to life
                using (var erode = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(2, 1)))
                {
                    Cv2.Erode(dst, dst, erode);
                }

                // at this point, good OCR will see most numbers
                // but we want to remove surrounding artifacts

                // find countours
                using (var canny = dst.Canny(0, 400))
                {
                    var contours = canny.FindContoursAsArray(RetrievalModes.List, ContourApproximationModes.ApproxSimple);

                    // compute a bounding rect for all numbers w/o boxes and artifacts
                    // this is the tricky part where we try to discard what's not related exclusively to numbers
                    var boundingRect = Rect.Empty;
                    foreach (var contour in contours)
                    {
                        // discard some small and broken polygons
                        var polygon = Cv2.ApproxPolyDP(contour, 4, true);
                        if (polygon.Length < 3)
                            continue;

                        // we want only numbers, and boxes are approx 40px wide,
                        // so let's discard box-related polygons, if any
                        // and some other artifacts that passed previous checks
                        // this quite depends on some context knowledge...
                        var rect = Cv2.BoundingRect(polygon);
                        if (rect.Width > 40 || rect.Height < 15)
                            continue;

                        boundingRect = boundingRect.X == 0 ? rect : boundingRect.Union(rect);
                    }

                    using (var final = dst.Clone(boundingRect))
                    {
                        final.SaveImage(outputFilePath);
                    }
                }
            }
        }
    }
}