iOS ARView 和 MTKView - 如何恢复相机位置？

Question

我有一个 iOS 点云应用程序，并试图允许用户在多个会话中不断向现有点云添加点。

我有一个 Metal MTKView，覆盖并与ARView下面对齐。当我的.showSceneUnderstanding启用，我可以验证金属视图的点云的网格相匹配ARView。

我可以通过归档和取消归档ARWorldMap并将其分配给ARView 来恢复 ARView 的世界状态initialWorldMap。经过一些设备旋转，场景将匹配来自 ARWorldMap 的锚点，并显示适当的场景理解。我还有一个由 XYZ 定义的点云。

如何让 MTKView 中的相机与 ARView 中的当前相机方向相对应？

换句话说，我可以恢复现有的世界地图并显示它，但让 ARView 从不同的、非原始视口和相机位置显示场景。当我恢复点云时，原点设置为当前设备位置/方向，点云与现实世界失去对齐。

如何获取 ARView/ARWorldMap/ARFrame 相机位置并定位我的点云，以便我在 MTKView 中的视口将正确显示点云的一部分，该部分对应于 的这个视口ARView？

arView.debugOptions.insert(.showSceneUnderstanding)

func preloadWorld(for configuration: ARWorldTrackingConfiguration) {
    
    guard let restoredMap: ARWorldMap = GeometryManager<ARWorldMap>().value else {
        return
    }
    configuration.initialWorldMap = restoredMap
    
    // 
}

这是捕获相机位置的代码：

static func create(with frame: ARFrame, viewportSize: CGSize) -> PointCloudUniforms {
var uniforms = PointCloudUniforms.defaults

uniforms.viewportWidth  = Float(viewportSize.width)
uniforms.viewportHeight = Float(viewportSize.height)

let screenOrientation = UIInterfaceOrientation.portrait

let camera = frame.camera
let cameraIntrinsicsInversed = camera.intrinsics.inverse
let viewMatrix = camera.viewMatrix(for: screenOrientation) //TODO: pass in orientation

let viewMatrixInversed = viewMatrix.inverse
let projectionMatrix = camera.projectionMatrix(for: screenOrientation,
                                               viewportSize: viewportSize,
                                               zNear: 0.01,
                                               zFar: 100.0)

uniforms.viewProjectionMatrix = projectionMatrix * viewMatrix
uniforms.localToWorld = viewMatrixInversed * screenOrientation.rotationToARCameraMatrix
uniforms.cameraIntrinsicsInversed = cameraIntrinsicsInversed

uniforms.cameraResolution = Float2(Float(frame.camera.imageResolution.width), Float(frame.camera.imageResolution.height))

return uniforms
}

这是根据当前相机位置旋转每个点云点的着色器。

/// This is what is displaying an individual point in the point cloud
vertex ParticleVertexOut particleVertex(uint vertexID [[vertex_id]],
                                        constant PointCloudUniforms &uniforms [[buffer(kPointCloudUniforms)]],
                                        constant ParticleUniforms *particleUniforms [[buffer(kParticleUniforms)]]) {
    // get point data
    const auto particleData = particleUniforms[vertexID];
    const auto position = particleData.position;
    
    // animate and project the point
    float4 projectedPosition = uniforms.viewProjectionMatrix * float4(position, 1.0);
    
    projectedPosition /= projectedPosition.w;
    
    ParticleVertexOut out;
    out.position = projectedPosition;
    
    out.color = float4(particleData.color, 1);
    return out;
}

请参阅下面的示例将茶壶与其网格从 ARView 对齐。