THREE.JS & Reality Capture - 3D 空间中摄影测量参考相机的旋转问题
Dia*_*ark 5 javascript photogrammetry point-clouds three.js typescript
感谢您花时间审阅我的帖子。我希望这篇文章不仅能为我自己带来成果，也能对其他人有所帮助！
介绍
目前我正在开展一个涉及摄影测量生成的点云的项目。它由照片与激光扫描相结合组成。用于制作点云的软件是 Reality Capture。除了点云导出之外，还可以导出“内部/外部相机参数”，从而提供检索用于构成点云中某个 3D 点的照片的能力。现实捕捉在网上并没有很好的记录，我也在他们的论坛上发布了有关相机变量的文章，也许它可以用于解决手头的问题？
只有相机参数文件中列出的几个变量（目前）与参考相机定位相关，例如文件名、x、y、alt（位置、航向、俯仰和滚动作为其旋转）。
目前，生成的点云被加载到与浏览器兼容的 THREE.JS 查看器中，之后加载相机参数 .csv 文件，并且对于每张已知照片，都会生成带有绿色立方体的“ PerspectiveCamera ”。一个例子如下所示：
挑战
事实上，根据上一张图片（当然，或者这篇文章的标题；P），您可能已经知道问题出在哪里，以防万一您没有发现它，相机的方向都是错误的。让我用破旧的自画矢量为您形象化它，这些矢量基本显示了它应该面向的方向（标记为红色）以及它当前的矢量方向（绿色）。
第 37 行，DJI_0176.jpg是最右边的相机，带有红色参考线第 38 行是 177 等。最后一张图片（第 48 行是 DJI_189.jpg）对应于聚类图像的最左边的图像（因为我没有在上图中绘制其他两个相机参考（我没有包括其他相机参考）。
当您将下面的数据复制到 Excel 工作表中时，它应该正确显示 ^^
#name x y alt heading pitch roll f px py k1 k2 k3 k4 t1 t2
DJI_0174.JPG 3.116820957 -44.25690188 14.05258109 -26.86297007 66.43104338 1.912026354 30.35179628 7.25E-03 1.45E-03 -4.02E-03 -2.04E-02 3.94E-02 0 0 0
DJI_0175.JPG -5.22E-02 -46.97266554 14.18056658 -16.2033133 66.11532302 3.552072396 30.28063771 4.93E-03 4.21E-04 1.38E-02 -0.108013599 0.183136287 0 0 0
DJI_0176.JPG -3.056586953 -49.00754998 14.3474763 4.270483155 65.35247679 5.816970677 30.50596933 -5.05E-03 -3.53E-03 -4.94E-03 3.24E-02 -3.84E-02 0 0 0
DJI_0177.JPG -6.909437337 -50.15910066 14.38391206 19.4459053 64.26828897 6.685020944 30.6994734 -1.40E-02 4.72E-03 -5.33E-04 1.90E-02 -1.74E-02 0 0 0
DJI_0178.JPG -11.23696688 -50.36025313 14.56924433 19.19192622 64.40188316 6.265995184 30.7665397 -1.26E-02 2.41E-03 1.24E-04 -4.63E-03 2.84E-02 0 0 0
DJI_0179.JPG -16.04060554 -49.92320365 14.69721478 19.39979452 64.85507307 6.224929846 30.93772566 -1.19E-02 -4.31E-03 -1.27E-02 4.62E-02 -4.48E-02 0 0 0
DJI_0180.JPG -20.95614556 -49.22915437 14.92273203 20.39327092 65.02028543 6.164031482 30.99807237 -1.02E-02 -7.70E-03 1.44E-03 -2.22E-02 3.94E-02 0 0 0
DJI_0181.JPG -25.9335097 -48.45330177 15.37330388 34.24388008 64.82707628 6.979877709 31.3534556 -1.06E-02 -1.19E-02 -5.44E-03 2.39E-02 -2.38E-02 0 0 0
DJI_0182.JPG -30.40507957 -47.21269946 15.67804925 49.98858409 64.29238807 7.449650513 31.6699868 -8.75E-03 -1.31E-02 -4.57E-03 2.31E-02 2.68E-03 0 0 0
DJI_0183.JPG -34.64277285 -44.84034207 15.89229254 65.84203906 62.9109777 7.065942792 31.78292476 -8.39E-03 -2.94E-03 -1.40E-02 8.96E-02 -0.11801932 0 0 0
DJI_0184.JPG -39.17179024 -40.22577764 16.28164396 65.53938063 63.2592604 6.676581293 31.79546988 -9.81E-03 -8.13E-03 1.01E-02 -8.44E-02 0.179931606 0 0 0
DJI_0185.JPG -43.549378 -33.09364534 16.64130671 68.61427166 63.15205908 6.258411625 31.75339036 -9.78E-03 -7.12E-03 4.75E-03 -6.25E-02 0.1541638 0 0 0
DJI_0186.JPG -46.5381556 -24.2992233 17.2286956 74.42382577 63.75110346 6.279208736 31.88862443 -1.01E-02 -1.73E-02 1.02E-02 -6.15E-02 4.89E-02 0 0 0
DJI_0187.JPG -48.18737751 -14.67333218 17.85446854 79.54477952 63.0503902 5.980759013 31.69602914 -8.83E-03 -1.01E-02 -7.63E-03 -7.49E-03 2.71E-02 0 0 0
DJI_0188.JPG -48.48581505 -13.79840485 17.84756621 93.43316271 61.87561678 5.110113503 31.6671977 1.99E-03 -9.40E-04 2.40E-02 -0.180515731 0.32814456 0 0 0
DJI_0189.JPG -48.32815991 -13.88055437 17.77818573 106.3277582 60.87171036 4.039469869 31.50757712 2.84E-03 4.12E-03 8.54E-03 -1.32E-02 3.89E-02 0 0 0
到目前为止尝试过的事情
我们发现导出的模型与现实是镜像的，但这并不影响相机参考的放置，因为它们完美对齐。我们尝试镜像引用的相机、点云和视口相机，但这似乎并没有解决当前的问题。（因此camera.applyMatrix4(new THREE.Matrix4().makeScale(-1, 1, 1));）
到目前为止，我们尝试加载欧拉角、直接设置角度或转换并应用四元数，遗憾的是没有任何好的结果。使用以下逻辑解析相机参考文件：
// Await the .csv file being parsed from the server
await new Promise((resolve) => {
(file as Blob).text().then((csvStr) => {
const rows = csvStr.split('\n');
for (const row of rows) {
const col = row.split(',');
if (col.length > 1) {
const suffixes = col[0].split('.');
const extension = suffixes[suffixes.length - 1].toLowerCase();
const validExtensions = ['jpeg', 'jpg', 'png'];
if (!validExtensions.includes(extension)) {
continue;
}
// == Parameter index by .csv column names ==
// 0: #name; 1: x; 2: y; 3: alt; 4: heading; 5: pitch; 6: roll; 7:f (focal);
// == Non .csv param ==
// 8: bool isRadianFormat default false
this.createCamera(col[0], parseFloat(col[1]), parseFloat(col[2]), parseFloat(col[3]), parseFloat(col[4]), parseFloat(col[5]), parseFloat(col[6]), parseFloat(col[7]));
}
}
resolve(true);
});
});
}
下面您将找到用于实例化相机及其位置和旋转的代码片段。我留下了一些额外的评论来进一步阐述它。我还留下了注释的代码行，看看我们还尝试过什么：
private createCamera(fileName: string, xPos: number, yPos: number, zPos: number, xDeg: number, yDeg: number, zDeg: number, f: number, isRadianFormat = false) : void {
// Set radials as THREE.JS explicitly only works in radians
const xRad = isRadianFormat ? xDeg : THREE.MathUtils.degToRad(xDeg);
const yRad = isRadianFormat ? yDeg : THREE.MathUtils.degToRad(yDeg)
const zRad = isRadianFormat ? zDeg : THREE.MathUtils.degToRad(zDeg)
// Create camera reference and extract frustum
// Statically set the FOV and aspectratio; Near is set to 0,1 by default and Far is dynamically set whenever a point is clicked in a 3D space.
const camera = new THREE.PerspectiveCamera(67, 5280 / 2970, 0.1, 1);
const pos = new THREE.Vector3(xPos, yPos, zPos); // Reality capture z = up; THREE y = up;
/* ===
In order to set an Euler angle one must provide the heading (x), pitch (y) and roll(z) as well as the order (variable four 'XYZ') in which the rotations will be applied
As a last resort we even tried switching the x,y and zRad variables as well as switching the orientation orders.
Possible orders:
XYZ
XZY
YZX
YXZ
ZYX
ZXY
=== */
const rot = new THREE.Euler(xRad, yRad, zRad, 'XYZ');
//camera.setRotationFromAxisAngle(new THREE.Vector3(0,))
//camera.applyMatrix4(new THREE.Matrix4().makeScale(-1, 1, 1));
// const rot = new THREE.Quaternion();
// rot.setFromAxisAngle(new THREE.Vector3(1, 0, 0), zRad);
// rot.setFromAxisAngle(new THREE.Vector3(0, 1, 0), xRad);
// rot.setFromAxisAngle(new THREE.Vector3(0, 0, 1), yRad);
// XYZ
// === Update camera frustum ===
camera.position.copy(pos);
// camera.applyQuaternion(rot);
camera.rotation.copy(rot);
camera.setRotationFromEuler(rot);
camera.updateProjectionMatrix(); // TODO: Assert whether projection update is required here
/* ===
The camera.applyMatrix listed below was an attempt in rotating several aspects of the 3D viewer.
An attempt was made to rotate each individual photo camera position, the pointcloud itself aswell as the viewport camera both separately
as well as solo. It made no difference however.
=== */
//camera.applyMatrix4(new THREE.Matrix4().makeScale(-1, 1, 1));
// Instantiate CameraPosition instance and push to array
const photo: PhotoPosition = {
file: fileName,
camera,
position: pos,
rotation: rot,
focal: f,
width: 5120, // Statically set for now
height: 5120, // Statically set for now
};
this.photos.push(photo);
}
然后，下一段代码会抓取上面代码片段中创建的相机，该代码将相机传递给相机管理器并绘制一个CameraHelper（显示在上面的两个 3D 查看器图片中）。它是在异步函数中编写的，等待 csv 文件加载，然后再继续初始化相机。
private initializeCameraPoses(url: string, csvLoader: CSVLoader) {
const absoluteUrl = url + '\\references.csv';
(async (scene, csvLoader, url, renderer) => {
await csvLoader.init(url);
const photos = csvLoader.getPhotos(); // The cameras created by the createCamera() method
this.inspectionRenderer = new InspectionRenderer(scene); // InspectionRenderer manages all further camera operations
this.inspectionRenderer.populateCameras(photos);
for (const photoData of photos) {
// Draw the green cube
const geometry = new THREE.BoxGeometry(0.5, 0.5, 0.5);
const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 });
const cube = new THREE.Mesh(geometry, material);
scene.add(cube);
cube.position.copy(photoData.position);
photoData.camera.updateProjectionMatrix();
// Draws the yellow camera viewport to the scene
const helper = new CameraHelper(photoData.camera);
renderer.render(scene, photoData.camera);
scene.add(helper);
}
})(this.scene, csvLoader, absoluteUrl, this.renderer);
}
Marquizzo 的代码片段
下面发布的 Marquizzo 代码片段似乎让我们更接近解决方案。摄像机的方向似乎是正确的。然而，不知怎的，球场似乎有点偏离。下面我将包含 DJI_0189.jpg 的源图像。请注意，对于此示例，当前未设置 FOV，因为当为每个相机位置渲染相机助手时，它看起来很混乱。对于此示例，我仅渲染了 DJI_0189 相机助手。
@Marquizzo 提供的编辑反转音高 ( const rotX = deg2rad(photo.pitch * -1);) 将导致中点交点始终按预期略低：
当间距调整为时，const rotX = deg2rad(photo.pitch * -.5);您会看到中点交点与源图像的中点交点（更接近）：
不知何故，我认为解决方案是可以实现的，并且最终会归结为一些被忽视的非常小的细节。我真的很期待看到回复。如果仍有不清楚的地方，请说出来，如果需要，我会提供必要的详细信息^^
感谢您到目前为止阅读这篇文章！
乍一看，我看到了三种可能性：
如果不显示您如何使用该方法，则很难看出问题出在哪里
createCamera()。你可以用pitch类似heading的东西交换。在 Three.js 中，航向是绕 Y 轴旋转、绕 X 轴俯仰和绕 Z 轴滚动。其次，您知道
heading, pitch, roll传感器按什么顺序进行测量吗？这将影响您启动的方式THREE.Euler(xRad, yRad, zRad, 'XYZ')，因为应用轮换的顺序也可能不同'YZX', 'ZXY', 'XZY', 'YXZ' or 'ZYX'。最后，您必须思考“这
heading: 0对传感器意味着什么？” 现实世界坐标系和 Three.js 坐标系之间的含义可能不同。Three.js 中没有旋转的相机会直接向下看向-Z轴，但您的传感器可能会将其指向+Z、 或+X等。
编辑：
我在下面添加了一个演示，我认为这就是您从屏幕截图中需要的。请注意，我乘以pitch * -1相机“向下看”，并添加+180到标题，以便它们指向右侧......标题。
const DATA = [
{name: "DJI_0174.JPG", x: 3.116820957, y: -44.25690188, alt: 14.05258109, heading: -26.86297007, pitch: 66.43104338, roll: 1.912026354},
{name: "DJI_0175.JPG", x: -5.22E-02, y: -46.97266554, alt: 14.18056658, heading: -16.2033133, pitch: 66.11532302, roll: 3.552072396},
{name: "DJI_0176.JPG", x: -3.056586953, y: -49.00754998, alt: 14.3474763, heading: 4.270483155, pitch: 65.35247679, roll: 5.816970677},
{name: "DJI_0177.JPG", x: -6.909437337, y: -50.15910066, alt: 14.38391206, heading: 19.4459053, pitch: 64.26828897, roll: 6.685020944},
{name: "DJI_0178.JPG", x: -11.23696688, y: -50.36025313, alt: 14.56924433, heading: 19.19192622, pitch: 64.40188316, roll: 6.265995184},
{name: "DJI_0179.JPG", x: -16.04060554, y: -49.92320365, alt: 14.69721478, heading: 19.39979452, pitch: 64.85507307, roll: 6.224929846},
{name: "DJI_0180.JPG", x: -20.95614556, y: -49.22915437, alt: 14.92273203, heading: 20.39327092, pitch: 65.02028543, roll: 6.164031482},
{name: "DJI_0181.JPG", x: -25.9335097, y: -48.45330177, alt: 15.37330388, heading: 34.24388008, pitch: 64.82707628, roll: 6.979877709},
{name: "DJI_0182.JPG", x: -30.40507957, y: -47.21269946, alt: 15.67804925, heading: 49.98858409, pitch: 64.29238807, roll: 7.449650513},
{name: "DJI_0183.JPG", x: -34.64277285, y: -44.84034207, alt: 15.89229254, heading: 65.84203906, pitch: 62.9109777, roll: 7.065942792},
{name: "DJI_0184.JPG", x: -39.17179024, y: -40.22577764, alt: 16.28164396, heading: 65.53938063, pitch: 63.2592604, roll: 6.676581293},
{name: "DJI_0185.JPG", x: -43.549378, y: -33.09364534, alt: 16.64130671, heading: 68.61427166, pitch: 63.15205908, roll: 6.258411625},
{name: "DJI_0186.JPG", x: -46.5381556, y: -24.2992233, alt: 17.2286956, heading: 74.42382577, pitch: 63.75110346, roll: 6.279208736},
{name: "DJI_0187.JPG", x: -48.18737751, y: -14.67333218, alt: 17.85446854, heading: 79.54477952, pitch: 63.0503902, roll: 5.980759013},
{name: "DJI_0188.JPG", x: -48.48581505, y: -13.79840485, alt: 17.84756621, heading: 93.43316271, pitch: 61.87561678, roll: 5.110113503},
{name: "DJI_0189.JPG", x: -48.32815991, y: -13.88055437, alt: 17.77818573, heading: 106.3277582, pitch: 60.87171036, roll: 4.039469869},
];
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(
45,
window.innerWidth / window.innerHeight,
1,
1000
);
camera.position.z = 100;
const renderer = new THREE.WebGLRenderer({
antialias: true,
canvas: document.querySelector("#canvas")
});
renderer.setSize(window.innerWidth, window.innerHeight);
const controls = new THREE.OrbitControls( camera, renderer.domElement );
// Helpers
const axesHelper = new THREE.AxesHelper( 20 );
scene.add(axesHelper);
const plane = new THREE.Plane( new THREE.Vector3( 0, 1, 0 ), 0 );
const planeHelper = new THREE.PlaneHelper( plane, 50, 0xffff00 );
scene.add(planeHelper);
let deg2rad = THREE.MathUtils.degToRad;
function createCam(photo) {
let tempCam = new THREE.PerspectiveCamera(10, 2.0, 1, 30);
// Altitude is actually y-axis,
// "y" is actually z-axis
tempCam.position.set(photo.x, photo.alt, photo.y);
// Modify pitch & heading so it matches Three.js coordinates
const rotX = deg2rad(photo.pitch * -1);
const rotY = deg2rad(photo.heading + 180);
const rotZ = deg2rad(photo.roll);
tempCam.rotation.set(rotX, rotY, rotZ, "YXZ");
let helper = new THREE.CameraHelper(tempCam);
scene.add(tempCam);
scene.add(helper);
}
for(let i = 0; i < DATA.length; i++) {
createCam(DATA[i]);
}
function animate() {
renderer.render(scene, camera);
requestAnimationFrame(animate);
}
animate();html, body { margin:0; padding:0;}<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script><script src="https://rawgit.com/mrdoob/three.js/dev/examples/js/controls/OrbitControls.js"></script>
<canvas id="canvas"></canvas>- @DiabitiMark 好吧，任何人都可以提出自己对“前进”的定义。在虚构的 3D 世界中，0° 航向可能意味着北、南、东或西。因此添加 +180 只是确保您的传感器和 Threejs 都对齐。请参阅上面我原来的答案中的第三个要点。 (2认同)
| 归档时间： |
|
| 查看次数： |
831 次 |
| 最近记录： |