如何在 Python OpenCV 中使用立体图像对计算对极线

Question

如何从不同角度拍摄一个物体的两个图像，并根据另一个点在一个物体上绘制极线？

例如，我希望能够使用鼠标在左侧图片上选择一个点，用圆圈标记该点，然后在与标记点相对应的右侧图像上绘制一条极线。

我有 2 个 XML 文件，其中包含 3x3 相机矩阵和每张图片的 3x4 投影矩阵列表。相机矩阵为K。左图的投影矩阵为P_left。右图的投影矩阵是P_right。

我试过这种方法：

1. 在左侧图像中选择一个像素坐标 (x,y)（通过鼠标点击）

2. 计算p左图中的一个点K^-1 * (x,y,1)

3. Calulate伪逆矩阵P+的P_left（使用np.linalg.pinv）

4. 计算右图e'的对极：P_right * (0,0,0,1)

5. 计算反对称矩阵e'_skew的e'

6. 计算基本矩阵F：e'_skew * P_right * P+

7. 计算l'右图的极线：F * p

8. 计算p'右图中的一个点：P_right * P+ * p

9. 变换p'并l返回像素坐标

10. 使用cv2.line通过p'和绘制一条线l

Answer 1

我几天前刚刚这样做了，它工作得很好。这是我使用的方法：

1. 校准相机以获取相机矩阵和失真矩阵（使用 openCVgetCorners和calibrateCamera，您可以找到很多关于此的教程，但听起来您已经拥有此信息）
2. 使用 openCV 执行立体声校准stereoCalibrate()。它将所有相机和失真矩阵作为参数。您需要它来确定两个视野之间的相关性。你会得到几个矩阵，旋转矩阵 R、平移向量 T、基本矩阵 E 和基本矩阵 F。
3. 然后您想使用 openCVgetOptimalNewCameraMatrix和undistort(). 这将消除很多相机像差（它会给你更好的结果）
4. 最后，使用 openCVcomputeCorrespondEpilines来计算线条并绘制它们。我将在下面包含一些代码，您可以在 Python 中试用。当我运行它时，我可以得到这样的图像（彩色点在另一个图像中绘制了它们对应的外线）

这是一些代码（Python 3.0）。它使用两个静态图像和静态点，但您可以轻松地用光标选择这些点。您还可以在此处参考有关校准和立体声校准的 OpenCV 文档。

import cv2
import numpy as np

# find object corners from chessboard pattern  and create a correlation with image corners
def getCorners(images, chessboard_size, show=True):
    criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)

    # prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
    objp = np.zeros((chessboard_size[1] * chessboard_size[0], 3), np.float32)
    objp[:, :2] = np.mgrid[0:chessboard_size[0], 0:chessboard_size[1]].T.reshape(-1, 2)*3.88 # multiply by 3.88 for large chessboard squares

    # Arrays to store object points and image points from all the images.
    objpoints = [] # 3d point in real world space
    imgpoints = [] # 2d points in image plane.

    for image in images:
        frame = cv2.imread(image)
        # height, width, channels = frame.shape # get image parameters
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        ret, corners = cv2.findChessboardCorners(gray, chessboard_size, None)   # Find the chess board corners
        if ret:                                                                         # if corners were found
            objpoints.append(objp)
            corners2 = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)    # refine corners
            imgpoints.append(corners2)                                                  # add to corner array

            if show:
                # Draw and display the corners
                frame = cv2.drawChessboardCorners(frame, chessboard_size, corners2, ret)
                cv2.imshow('frame', frame)
                cv2.waitKey(100)

    cv2.destroyAllWindows()             # close open windows
    return objpoints, imgpoints, gray.shape[::-1]

# perform undistortion on provided image
def undistort(image, mtx, dist):
    img = cv2.imread(image, cv2.IMREAD_GRAYSCALE)
    image = os.path.splitext(image)[0]
    h, w = img.shape[:2]
    newcameramtx, _ = cv2.getOptimalNewCameraMatrix(mtx, dist, (w, h), 1, (w, h))
    dst = cv2.undistort(img, mtx, dist, None, newcameramtx)
    return dst

# draw the provided points on the image
def drawPoints(img, pts, colors):
    for pt, color in zip(pts, colors):
        cv2.circle(img, tuple(pt[0]), 5, color, -1)

# draw the provided lines on the image
def drawLines(img, lines, colors):
    _, c, _ = img.shape
    for r, color in zip(lines, colors):
        x0, y0 = map(int, [0, -r[2]/r[1]])
        x1, y1 = map(int, [c, -(r[2]+r[0]*c)/r[1]])
        cv2.line(img, (x0, y0), (x1, y1), color, 1)

if __name__ == '__main__':

 # undistort our chosen images using the left and right camera and distortion matricies
    imgL = undistort("2L/2L34.bmp", mtxL, distL)
    imgR = undistort("2R/2R34.bmp", mtxR, distR)
    imgL = cv2.cvtColor(imgL, cv2.COLOR_GRAY2BGR)
    imgR = cv2.cvtColor(imgR, cv2.COLOR_GRAY2BGR)

    # use get corners to get the new image locations of the checcboard corners (undistort will have moved them a little)
    _, imgpointsL, _ = getCorners(["2L34_undistorted.bmp"], chessboard_size, show=False)
    _, imgpointsR, _ = getCorners(["2R34_undistorted.bmp"], chessboard_size, show=False)

    # get 3 image points of interest from each image and draw them
    ptsL = np.asarray([imgpointsL[0][0], imgpointsL[0][10], imgpointsL[0][20]])
    ptsR = np.asarray([imgpointsR[0][5], imgpointsR[0][15], imgpointsR[0][25]])
    drawPoints(imgL, ptsL, colors[3:6])
    drawPoints(imgR, ptsR, colors[0:3])

    # find epilines corresponding to points in right image and draw them on the left image
    epilinesR = cv2.computeCorrespondEpilines(ptsR.reshape(-1, 1, 2), 2, F)
    epilinesR = epilinesR.reshape(-1, 3)
    drawLines(imgL, epilinesR, colors[0:3])

    # find epilines corresponding to points in left image and draw them on the right image
    epilinesL = cv2.computeCorrespondEpilines(ptsL.reshape(-1, 1, 2), 1, F)
    epilinesL = epilinesL.reshape(-1, 3)
    drawLines(imgR, epilinesL, colors[3:6])

    # combine the corresponding images into one and display them
    combineSideBySide(imgL, imgR, "epipolar_lines", save=True)

希望这会有所帮助！