Eli*_*TUM 3 matlab projection intrinsics computer-vision 3d-reconstruction
我正在尝试使用边条件(在MatLab中)从相机图片中的2D像素坐标重建3D坐标.我确实有外在和内在的相机参数.
使用同质变换,我可以将3D坐标从最初的世界坐标系转换为我的相机坐标系.所以我在变换矩阵R_world_to_Camera中有我的外部参数:
R_world_to_Camera = [ r_11, r_12, r_13, t1;
r_21, r_22, r_23, t2;
r_31, r_32, r_33, t3;
0, 0, 0, 1];
对于内在参数,我使用了加州理工学院的"MatLab相机校准工具箱"并获得了以下参数:
Calibration results (with uncertainties):
Focal Length: fc = [ 1017.21523 1012.54901 ] ± [ NaN NaN ]
Principal point: cc = [ 319.50000 239.50000 ] ± [ NaN NaN ]
Skew: alpha_c = [ 0.00000 ] ± [ NaN ] => angle of pixel axes = 90.00000 ± NaN degrees
Distortion: kc = [ 0.00000 0.00000 0.00000 0.00000 0.00000 ] ± [ NaN NaN NaN NaN NaN ]
Pixel error: err = [ 0.11596 0.14469 ]
Note: The numerical errors are approximately three times the standard deviations (for reference).
所以我得到了Camera-Calibration-Matrix K(3x3)
K = [1.017215234570303e+03, 0, 3.195000000000000e+02;
0, 1.012549014668498e+03,2.395000000000000e+02;
0, 0, 1.0000];
并使用这个我可以计算3D - > 2D - 投影 - 矩阵P(3x4):
P = K * [eye(3), zeros(3,1)];
在世界坐标[X,Y,Z] _World中转换点时,我首先将其转换为相机坐标,然后将其投影到2D:
% Transformation
P_world = [X; Y; Z; 1]; % homogenous coordinates in World coordinate System
P_camera = R_world_to_Camera * [X; Y; Z; 1];
% Projection
P_pixels = P * camera;
P_pixels = P_pixels / P_pixels(3); % normalize coordinates
所以我现在的问题是如何扭转这些步骤？作为边条件,我想设置Z坐标(世界坐标为零).我在Stackoverflow上尝试了这里提出的解决方案,但不知怎的,我得到了错误的坐标.任何的想法？每一个帮助都很受欢迎!!
您无法反转该步骤:当3D点投影到2D图像时,深度和比例信息会丢失.但是,如果您指出所有3D点都在Z = 0平面上,那么从它们的投影中取回它们是微不足道的:计算相机矩阵的逆Ki = K ^ -1,并将其应用于图像点在齐次坐标中.
P_camera = Ki*[u,v,1]'
其中[u,v]是图像坐标,撇号表示换位.您想要的3D点位于从摄像机中心到P_camera的光线上.在世界坐标中表达:
P_world = [R | t] _camera_to_world*[P_camera,1]'
C_world = [R | t] _camera_to_world*[0,0,0,1]'
其中[R | t]是4x4坐标变换.现在,每条射线上的点集表示为
P = C_world + lambda*P_world;
其中lambda是一个标量(沿着光线的坐标).您现在可以强加P(3)= 0的条件来查找将点放在Z = 0平面上的lambda值.
感谢一些红葡萄酒和彻底阅读我在(德国)硕士论文中找到答案:)我的"测试代码"如下:
% Clean-Up First
clear all;
close all;
clc;
% Caamera-Calibration-Matrix
K = [1.017215234570303e+03, 0, 3.195000000000000e+02, 0;
0, 1.012549014668498e+03,2.395000000000000e+02, 0;
0, 0, 1.0000, 0;
0, 0, 0, 0];
% Transforma Matrix from 3D-World-Coordinate System to 3D-Camera-Coordinate System (Origin on CCD-Chip)
%
% The Camera is oriented "looking" into positive X-Direction of the World-Coordinate-System. On the picture,
% positive Y-Direction will be to the left, positive Z-Direction to the top. (right hand coordinate system!)
R_World_to_Cam = [-0.0113242625465167 -0.999822053685344 0.0151163536128891 141.173585444427;
0.00842007509644635 -0.0152123858102325 -0.999848810645587 1611.96528372161;
0.999900032304804 -0.0111955728474261 0.00859117128537919 847.090629282911;
0 0 0 1];
% Projection- and Transforma Matrix P
P = K * R_World_to_Cam;
% arbitrary Points X_World in World-Coordinate-System [mm] (homogenous Coordinates)
% forming a square of size 10m x 4m
X_World_1 = [20000; 2000; 0; 1];
X_World_2 = [20000; -2000; 0; 1];
X_World_3 = [10000; 2000; 0; 1];
X_World_4 = [10000; -2000; 0; 1];
% Transform and Project from 3D-World -> 2D-Picture
X_Pic_1 = P * X_World_1;
X_Pic_2 = P * X_World_2;
X_Pic_3 = P * X_World_3;
X_Pic_4 = P * X_World_4;
% normalize homogenous Coordinates (3rd Element has to be 1!)
X_Pic_1 = X_Pic_1 / X_Pic_1(3);
X_Pic_2 = X_Pic_2 / X_Pic_2(3);
X_Pic_3 = X_Pic_3 / X_Pic_3(3);
X_Pic_4 = X_Pic_4 / X_Pic_4(3);
% Now for reverse procedure take arbitrary points in Camera-Picture...
% (for simplicity, take points from above and "go" 30px to the right and 40px down)
X_Pic_backtransform_1 = X_Pic_1(1:3) + [30; 40; 0];
X_Pic_backtransform_2 = X_Pic_2(1:3) + [30; 40; 0];
X_Pic_backtransform_3 = X_Pic_3(1:3) + [30; 40; 0];
X_Pic_backtransform_4 = X_Pic_4(1:3) + [30; 40; 0];
% ... and transform back following the formula from the Master Thesis (in German):
% Ilker Savas, "Entwicklung eines Systems zur visuellen Positionsbestimmung von Interaktionspartnern"
M_Mat = P(1:3,1:3); % Matrix M is the "top-front" 3x3 part
p_4 = P(1:3,4); % Vector p_4 is the "top-rear" 1x3 part
C_tilde = - inv( M_Mat ) * p_4; % calculate C_tilde
% Invert Projection with Side-Condition ( Z = 0 ) and Transform back to
% World-Coordinate-System
X_Tilde_1 = inv( M_Mat ) * X_Pic_backtransform_1;
X_Tilde_2 = inv( M_Mat ) * X_Pic_backtransform_2;
X_Tilde_3 = inv( M_Mat ) * X_Pic_backtransform_3;
X_Tilde_4 = inv( M_Mat ) * X_Pic_backtransform_4;
mue_N_1 = -C_tilde(3) / X_Tilde_1(3);
mue_N_2 = -C_tilde(3) / X_Tilde_2(3);
mue_N_3 = -C_tilde(3) / X_Tilde_3(3);
mue_N_4 = -C_tilde(3) / X_Tilde_4(3);
% Do the inversion of above steps...
X_World_backtransform_1 = mue_N_1 * inv( M_Mat ) * X_Pic_backtransform_1 + C_tilde;
X_World_backtransform_2 = mue_N_2 * inv( M_Mat ) * X_Pic_backtransform_2 + C_tilde;
X_World_backtransform_3 = mue_N_3 * inv( M_Mat ) * X_Pic_backtransform_3 + C_tilde;
X_World_backtransform_4 = mue_N_4 * inv( M_Mat ) * X_Pic_backtransform_4 + C_tilde;
% Plot everything
figure(1);
% First Bird Perspective of World-Coordinate System...
subplot(1,2,1);
xlabel('Y-World');
ylabel('X-World');
grid on;
axis([-3000 3000 0 22000]);
hold on;
plot( -X_World_1(2), X_World_1(1), 'bo' );
plot( -X_World_2(2), X_World_2(1), 'bo' );
plot( -X_World_3(2), X_World_3(1), 'bo' );
plot( -X_World_4(2), X_World_4(1), 'bo' );
line([-X_World_1(2) -X_World_2(2) -X_World_4(2) -X_World_3(2) -X_World_1(2)], [X_World_1(1) X_World_2(1) X_World_4(1) X_World_3(1) X_World_1(1)], 'Color', 'blue' );
plot( -X_World_backtransform_1(2), X_World_backtransform_1(1), 'ro' );
plot( -X_World_backtransform_2(2), X_World_backtransform_2(1), 'ro' );
plot( -X_World_backtransform_3(2), X_World_backtransform_3(1), 'ro' );
plot( -X_World_backtransform_4(2), X_World_backtransform_4(1), 'ro' );
line([-X_World_backtransform_1(2) -X_World_backtransform_2(2) -X_World_backtransform_4(2) -X_World_backtransform_3(2) -X_World_backtransform_1(2)], [X_World_backtransform_1(1) X_World_backtransform_2(1) X_World_backtransform_4(1) X_World_backtransform_3(1) X_World_backtransform_1(1)], 'Color', 'red' );
hold off;
% ...then the camera picture (perspective!)
subplot(1,2,2);
hold on;
image(ones(480,640).*255);
colormap(gray(256));
axis([0 640 -480 0]);
line([X_Pic_1(1) X_Pic_2(1) X_Pic_4(1) X_Pic_3(1) X_Pic_1(1)], -1*[X_Pic_1(2) X_Pic_2(2) X_Pic_4(2) X_Pic_3(2) X_Pic_1(2)], 'Color', 'blue' );
line([X_Pic_backtransform_1(1) X_Pic_backtransform_2(1) X_Pic_backtransform_4(1) X_Pic_backtransform_3(1) X_Pic_backtransform_1(1)], -1*[X_Pic_backtransform_1(2) X_Pic_backtransform_2(2) X_Pic_backtransform_4(2) X_Pic_backtransform_3(2) X_Pic_backtransform_1(2)], 'Color', 'red' );
hold off;