咖啡豆分离算法

Question

什么是在二进制图像上分离(计数)咖啡豆的适当算法？豆类可以触摸并部分重叠.

咖啡豆图片http://cmm.ensmp.fr/~beucher/ex2a.gif

我的工作实际上不是咖啡豆,但咖啡豆更容易描述.这是我计算所有现在的人并计算人们从超市监控视频中穿过一些想象线的任务中的子问题.我已经将移动的对象提取到二进制掩码中,现在我需要以某种方式将它们分开.

有人在评论中提到的两个有希望的算法:

• Wathershed + distancetransofrm +标签.这可能是我提出的这个问题的答案(bean分离).
• 跟踪视频序列中的移动对象(此算法的名称是什么？).它可以跟踪重叠的对象.这是更有希望的算法,可能正是我需要解决的任务(移动人员分离).

Answer 1

这种方法是从mmgp的答案中衍生出来的,该答案详细解释了分水岭算法的工作原理.因此,如果您需要对代码的作用进行一些解释,请检查他的答案.

可以播放代码以提高检测率.这里是:

import sys
import cv2
import numpy
from scipy.ndimage import label

def segment_on_dt(a, img):
    border = cv2.dilate(img, None, iterations=3)
    border = border - cv2.erode(border, None)
    cv2.imwrite("border.png", border)

    dt = cv2.distanceTransform(img, 2, 5)    
    dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(numpy.uint8)
    _, dt = cv2.threshold(dt, 135, 255, cv2.THRESH_BINARY)
    cv2.imwrite("dt_thres.png", dt)    

边框(左),dt(右):

    lbl, ncc = label(dt)
    lbl = lbl * (255/ncc)      
    # Completing the markers now. 
    lbl[border == 255] = 255

    lbl = lbl.astype(numpy.int32)
    cv2.imwrite("label.png", lbl)

lbl:

    cv2.watershed(a, lbl)

    lbl[lbl == -1] = 0
    lbl = lbl.astype(numpy.uint8)
    return 255 - lbl

# Application entry point
img = cv2.imread("beans.png")
if img == None:
    print("!!! Failed to open input image")
    sys.exit(0)

# Pre-processing.
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)    
_, img_bin = cv2.threshold(img_gray, 128, 255, cv2.THRESH_OTSU | cv2.THRESH_BINARY_INV)
cv2.imwrite("img_bin.png", img_bin)

img_bin = cv2.morphologyEx(img_bin, cv2.MORPH_OPEN, numpy.ones((3, 3), dtype=int))
cv2.imwrite("img_bin_morphoEx.png", img_bin)

img_bin(左)前后(右)形态学操作:

result = segment_on_dt(img, img_bin)
cv2.imwrite("result.png", result)

result[result != 255] = 0
result = cv2.dilate(result, None)
img[result == 255] = (0, 0, 255)
cv2.imwrite("output.png", img)

分水岭分割的结果(左),然后是输出(右):

Answer 2

下面介绍了一种查找每个bean中心的方法.分析不同但连续的时间帧中分段对象的中心位置,可以跟踪它们.保持视觉轮廓或分析其路径可以在对象穿过另一个或存在一些重叠的情况下提高跟踪算法的准确性.

我使用了Marvin Image Processing Framework和Java.

寻找中心方法

我使用了三种基本算法:阈值,形态侵蚀和填充分割.第一步是删除背景的阈值,如下所示.

下一步是应用形态侵蚀以分离豆类.在小内核矩阵的情况下,我可以分离小豆,但将较大的豆保持在一起,如下所示.使用每个独立段的质量(像素数)进行过滤,可以仅选择较小的段,如下所示.

使用大核矩阵我可以分离较大的核矩阵,小的核矩阵消失,如下所示.

结合两个结果 - 删除太靠近并且可能来自同一个bean的中心点 - 我得到了下面的结果.

即使没有每个bean的真实段,使用中心位置也可以计算和跟踪它们.这些中心也可用于查找每个bean段.

源代码

源代码是Java,但解决方案中使用的图像处理算法由大多数框架提供.

编辑:我编辑了源代码,以保存每个步骤的图像.可以优化源代码,删除这些调试步骤并创建重用代码的方法.创建一些objets和列表只是为了演示这些步骤,也可以删除.

import static marvin.MarvinPluginCollection.floodfillSegmentation;
import static marvin.MarvinPluginCollection.thresholding;
import marvin.image.MarvinColorModelConverter;
import marvin.image.MarvinImage;
import marvin.image.MarvinSegment;
import marvin.io.MarvinImageIO;
import marvin.math.MarvinMath;
import marvin.plugin.MarvinImagePlugin;
import marvin.util.MarvinPluginLoader;

public class CoffeeBeansSeparation {

    private MarvinImagePlugin erosion = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.morphological.erosion.jar");

    public CoffeeBeansSeparation(){

        // 1. Load Image 
        MarvinImage image = MarvinImageIO.loadImage("./res/coffee.png");
        MarvinImage result = image.clone();

        // 2. Threshold
        thresholding(image, 30);

        MarvinImageIO.saveImage(image, "./res/coffee_threshold.png");

        // 3. Segment using erosion and floodfill (kernel size == 8)
        List<MarvinSegment> listSegments = new ArrayList<MarvinSegment>();
        List<MarvinSegment> listSegmentsTmp = new ArrayList<MarvinSegment>();
        MarvinImage binImage = MarvinColorModelConverter.rgbToBinary(image, 127);

        erosion.setAttribute("matrix", MarvinMath.getTrueMatrix(8, 8));
        erosion.process(binImage.clone(), binImage);

        MarvinImageIO.saveImage(binImage, "./res/coffee_bin_8.png");
        MarvinImage binImageRGB = MarvinColorModelConverter.binaryToRgb(binImage);
        MarvinSegment[] segments =  floodfillSegmentation(binImageRGB);

        // 4. Just consider the smaller segments
        for(MarvinSegment s:segments){
            if(s.mass < 300){   
                listSegments.add(s);
            }
        }

        showSegments(listSegments, binImageRGB);
        MarvinImageIO.saveImage(binImageRGB, "./res/coffee_center_8.png");

        // 5. Segment using erosion and floodfill (kernel size == 18)
        listSegments = new ArrayList<MarvinSegment>();
        binImage = MarvinColorModelConverter.rgbToBinary(image, 127);

        erosion.setAttribute("matrix", MarvinMath.getTrueMatrix(18, 18));
        erosion.process(binImage.clone(), binImage);

        MarvinImageIO.saveImage(binImage, "./res/coffee_bin_8.png");
        binImageRGB = MarvinColorModelConverter.binaryToRgb(binImage);
        segments =  floodfillSegmentation(binImageRGB);

        for(MarvinSegment s:segments){
            listSegments.add(s);
            listSegmentsTmp.add(s);
        }

        showSegments(listSegmentsTmp, binImageRGB);
        MarvinImageIO.saveImage(binImageRGB, "./res/coffee_center_18.png");

        // 6. Remove segments that are too near.
        MarvinSegment.segmentMinDistance(listSegments, 10);

        // 7. Show Result
        showSegments(listSegments, result);
        MarvinImageIO.saveImage(result, "./res/coffee_result.png");
    }

    private void showSegments(List<MarvinSegment> segments, MarvinImage image){
        for(MarvinSegment s:segments){
            image.fillRect((s.x1+s.x2)/2, (s.y1+s.y2)/2, 5, 5, Color.red);
        }
    }

    public static void main(String[] args) {
        new CoffeeBeansSeparation();
    }
}

Answer 3

有一些优雅的答案,但我想分享我尝试的东西,因为它与其他方法有点不同.

在阈值处理并找到距离变换之后,我传播距离变换图像的局部最大值.通过调整最大值传播的范围,我对距离变换图像进行分割,然后按照它们的区域过滤这些分段,从而拒绝较小的分段.

这样我可以实现给定图像的合理良好分割,尽管它没有明确定义边界.对于给定的图像,我使用我在Matlab代码中使用的参数值来获得42的段计数,以控制最大值传播的范围和面积阈值.

结果:

这是Matlab代码:

clear all;
close all;

im = imread('ex2a.gif');
% threshold: coffee beans are black
bw = im2bw(im, graythresh(im));
% distance transform
di = bwdist(bw);
% mask for coffee beans
mask = double(1-bw);

% propagate the local maxima. depending on the extent of propagation, this
% will transform finer distance image to coarser segments 
se = ones(3);   % 8-neighbors
% this controls the extent of propagation. it's some fraction of the max
% distance of the distance transformed image (50% here)
mx = ceil(max(di(:))*.5);
peaks = di;
for r = 1:mx
    peaks = imdilate(peaks, se);
    peaks = peaks.*mask;
end

% how many different segments/levels we have in the final image
lvls = unique(peaks(:));
lvls(1) = []; % remove first, which is 0 that corresponds to background
% impose a min area constraint for segments. we can adjust this threshold
areaTh = pi*mx*mx*.7;
% number of segments after thresholding by area
nseg = 0;

% construct the final segmented image after thresholding segments by area
z = ones(size(bw));
lblid = 10;  % label id of a segment
for r = 1:length(lvls)
    lvl = peaks == lvls(r); % pixels having a certain value(level)
    props = regionprops(lvl, 'Area', 'PixelIdxList'); % get the area and the pixels
    % threshold area
    area = [props.Area];
    abw = area > areaTh;
    % take the count that passes the imposed area threshold
    nseg = nseg + sum(abw);
    % mark the segments that pass the imposed area threshold with a unique
    % id
    for i = 1:length(abw)
        if (1 == abw(i))
            idx = props(i).PixelIdxList;
            z(idx) = lblid; % assign id to the pixels
            lblid = lblid + 1; % increment id
        end
    end
end

figure,
subplot(1, 2, 1), imshow(di, []), title('distance transformed')
subplot(1, 2, 2), imshow(peaks, []), title('after propagating maxima'), colormap(jet)
figure,
subplot(1, 2, 1), imshow(label2rgb(z)), title('segmented')
subplot(1, 2, 2), imshow(im), title('original')