Python,OpenCV - 一个接一个地对齐和叠加多个图像

Question

我的项目是对齐航拍照片,制作马赛克地图.我的计划是从两张照片开始,将第二张与第一张对齐,然后从两张对齐的图像中创建一个"初始马赛克".完成后,我将第三张照片与初始马赛克对齐,然后将第四张照片与其结果对齐,从而逐步构建地图.

我有两种技术可以做到这一点,但使用的更准确的一种技术calcOpticalFlowPyrLK()仅适用于双图像阶段,因为两个输入图像的大小必须相同.因此,我尝试了一种新的解决方案,但它不太准确,并且每一步都会引入错误,最终会产生无意义的结果.

我的问题是双重的,但如果你知道一个人的答案,除非你愿意,否则你不必回答这两个问题.首先,有没有办法使用类似calcOpticalFlowPyrLK()但有两个不同大小的图像(这包括任何潜在的解决方法)？第二,有没有办法修改探测器/描述符解决方案,使其更准确？

这是仅适用于两个图像的准确版本:

# load images
base = cv2.imread("images/1.jpg")
curr = cv2.imread("images/2.jpg")

# convert to grayscale
base_gray = cv2.cvtColor(base, cv2.COLOR_BGR2GRAY)

# find the coordinates of good features to track  in base
base_features = cv2.goodFeaturesToTrack(base_gray, 3000, .01, 10)

# find corresponding features in current photo
curr_features = np.array([])
curr_features, pyr_stati, _ = cv2.calcOpticalFlowPyrLK(base, curr, base_features, curr_features, flags=1)

# only add features for which a match was found to the pruned arrays
base_features_pruned = []
curr_features_pruned = []
for index, status in enumerate(pyr_stati):
    if status == 1:
        base_features_pruned.append(base_features[index])
        curr_features_pruned.append(curr_features[index])

# convert lists to numpy arrays so they can be passed to opencv function
bf_final = np.asarray(base_features_pruned)
cf_final = np.asarray(curr_features_pruned)

# find perspective transformation using the arrays of corresponding points
transformation, hom_stati = cv2.findHomography(cf_final, bf_final, method=cv2.RANSAC, ransacReprojThreshold=1)

# transform the images and overlay them to see if they align properly
# not what I do in the actual program, just for use in the example code
# so that you can see how they align, if you decide to run it
height, width = curr.shape[:2]
mod_photo = cv2.warpPerspective(curr, transformation, (width, height))
new_image = cv2.addWeighted(mod_photo, .5, base, .5, 1)

这是适用于多个图像的不准确的图像(直到错误变得太大):

# load images
base = cv2.imread("images/1.jpg")
curr = cv2.imread("images/2.jpg")


# convert to grayscale
base_gray = cv2.cvtColor(self.base, cv2.COLOR_BGR2GRAY)

# DIFFERENCES START
curr_gray = cv2.cvtColor(self.curr_photo, cv2.COLOR_BGR2GRAY)

# create detector, get keypoints and descriptors
detector = cv2.ORB_create()
base_keys, base_desc = detector.detectAndCompute(base_gray, None)
curr_keys, curr_desc = detector.detectAndCompute(curr_gray, None)

matcher = cv2.DescriptorMatcher_create("BruteForce-Hamming")

max_dist = 0.0
min_dist = 100.0

for match in matches:
     dist = match.distance
     min_dist = dist if dist < min_dist else min_dist
     max_dist = dist if dist > max_dist else max_dist

good_matches = [match for match in matches if match.distance <= 3 * min_dist ]

base_matches = []
curr_matches = []
for match in good_matches:
    base_matches.append(base_keys[match.queryIdx].pt)
    curr_matches.append(curr_keys[match.trainIdx].pt)

bf_final = np.asarray(base_matches)
cf_final = np.asarray(curr_matches)

# SAME AS BEFORE

# find perspective transformation using the arrays of corresponding points
transformation, hom_stati = cv2.findHomography(cf_final, bf_final, method=cv2.RANSAC, ransacReprojThreshold=1)

# transform the images and overlay them to see if they align properly
# not what I do in the actual program, just for use in the example code
# so that you can see how they align, if you decide to run it
height, width = curr.shape[:2]
mod_photo = cv2.warpPerspective(curr, transformation, (width, height))
new_image = cv2.addWeighted(mod_photo, .5, base, .5, 1)

最后,这里有一些我正在使用的图片:

Answer 1

单应组成,所以如果你有间单应img1和img2之间,img2和img3那么这两个单应性的组合物提供之间的单应img1和img3.

您的尺寸当然不合适,因为您试图匹配img3包含img1和的拼接图像img2.但你不需要这样做.在每个连续图像之间有所有单应性之前,不要缝合它们.然后你可以采取两种方式之一; 从后面工作或从前面工作.我将用于例如h31指代扭曲img3成坐标的单应性img1.

从前面(伪代码):

warp img2 into coordinates of img1 with h21
warp img3 into coordinates of img1 with h31 = h32 @ h21
warp img4 into coordinates of img1 with h41 = h43 @ h31
...
stitch/blend images together

这@是矩阵乘法运算符,它将实现我们的单应性组合(请注意,最明确的是除以单应性中的最终条目以确保它们全部缩放相同).

从后面(伪代码):

...
warp prev stitched img into coordinates of img3 with h43
stitch warped stitched img with img3
warp prev stitched img into coordinates of img2 with h32
stitch warped stitched img with img2
warp prev stitched img into coordinates of img1 with h21
stitch warped stitched img with img1

这个想法要么是你从前面开始,要么把所有东西都扭曲到第一个图像坐标系,或者从后面开始,扭曲到前一个图像并缝合,然后将拼接后的图像扭曲成前一个图像,然后重复.我认为第一种方法可能更容易.在任何一种情况下,你都要担心单应性估计中误差的传播,因为它们会在多个组合的单应性上积累.

这是将多个图像与单应性混合在一起的天真方法.更复杂的方法是使用束调整,其考虑所有图像的特征点.然后为了良好的混合,步骤是增益补偿以消除相机增益调整和渐晕,然后多波段混合以防止模糊.见布朗和Lowe的开创性的论文在这里和一个光辉的榜样和免费demo软件在这里.