从街景上的像素获取标题和音高

Question

我认为这是这个问题的最佳位置.

我正在尝试获取嵌入式Google街景中任何点击点的标题和音高.

我知道并且可以获得的唯一信息是:

• 视野(度)
• 中心点的航向和俯仰(以度为单位)以及x和y像素位置
• 鼠标单击的x和y像素位置

我在这里包含了一个简化测量的屏幕截图作为示例:

我最初只是认为你可以用像素宽度划分视野以获得每个像素的度数,但它更复杂,我认为它与投射到球体内部有关,其中相机位于球体的中心.球？

如果你能告诉我如何反过来也可以获得奖金......

澄清: 目标不是将视图移动到单击的点,而是提供有关单击点的信息.每像素度数方法不起作用,因为视口不是线性的.

我在这里的值只是示例,但视野可以更大或更小(从[0.something,180],中心不固定,它可以是[0,360]范围内的任何值和垂直[-90,90].点[0,0]只是摄影时拍摄照片的标题(水平度)和音高(垂直度),并不代表任何东西.

Answer 1

TL; DR:本答案末尾包含的概念证明的JavaScript代码.

标题和俯仰参数h0以及p0全景图像对应于方向.通过使用f相机的焦距来缩放此方向矢量,可以在以下位置获取(x0, y0, z0)视口中心的3D坐标(u0, v0):

x0 = f * cos( p0 ) * sin( h0 )
y0 = f * cos( p0 ) * cos( h0 )
z0 = f * sin( p0 ) 

现在的目标是找到(u, v)图像中某些给定像素坐标处的点的3D坐标.首先,将这些像素坐标映射到(du, dv)视口中心的像素偏移(向右和向上):

du = u - u0 = u - w / 2
dv = v0 - v = h / 2 - v

然后必须找到3D中视口的局部正交2D基础.单位矢量(ux, uy, uz)支持x轴(沿着增加标题的方向向右)并且矢量(vx, vy, vz)支持图像的y轴(沿着增加的间距的方向的顶部).确定这两个矢量后,视口上与视口中的(du, dv)像素偏移匹配的点的3D坐标就是:

x = x0 + du * ux + dv * vx
y = y0 + du * uy + dv * vy
z = z0 + du * uz + dv * vz

然后标题和俯仰参数h以及p这一点:

R = sqrt( x * x + y * y + z * z )
h = atan2( x, y )
p = asin( z / R )

最后得到两个单位向量,(ux, uy, uz)并(vx, vy, vz)通过航向和俯仰参数计算球坐标的导数(p0, h0),并得到:

vx = -sin( p0 ) * sin ( h0 )
vy = -sin( p0 ) * cos ( h0 )
vz =  cos( p0 ) 

ux =  sgn( cos ( p0 ) ) * cos( h0 )
uy = -sgn( cos ( p0 ) ) * sin( h0 )
uz = 0

在哪里sgn( a ),+1如果a >= 0否则-1.

补充:

• 焦距来自水平视野和图像宽度:

  f = (w / 2) / Math.tan(fov / 2)
  

• 从航向和俯仰参数到像素坐标的反向映射可以类似地完成:

  1. 找到与(x, y, z)指定的航向和俯仰参数对应的射线方向的3D坐标,
  2. 找到3D坐标(x0, y0, z0)对应于视口的中心光线的方向的(相关联的图像平面位于(x0, y0, z0)与(x0, y0, z0)正常),
  3. 将指定航向和俯仰参数的光线与图像平面相交,这样就可以从视口中心获得3D偏移,
  4. 在本地基础上投影此3D偏移,获得2D偏移du和dv
  5. 地图du和dv绝对像素坐标.

• 实际上,这种方法似乎在方形和矩形视口上都能很好地工作.

概念证明代码(onLoad()在包含带有"全景"id的大小的画布元素的网页上调用该函数)

'use strict';

var viewer;

function onClick(e) {
  viewer.click(e);
}

function onLoad() {
  var element = document.getElementById("panorama");
  viewer = new PanoramaViewer(element);
  viewer.update();
}

function PanoramaViewer(element) {
  this.element = element;
  this.width = element.width;
  this.height = element.height;
  this.pitch = 0;
  this.heading = 0;

  element.addEventListener("click", onClick, false);
}

PanoramaViewer.FOV = 90;

PanoramaViewer.prototype.makeUrl = function() {
  var fov = PanoramaViewer.FOV;

  return "https://maps.googleapis.com/maps/api/streetview?location=40.457375,-80.009353&size=" + this.width + "x" + this.height + "&fov=" + fov + "&heading=" + this.heading + "&pitch=" + this.pitch;
}

PanoramaViewer.prototype.update = function() {
  var element = this.element;

  element.style.backgroundImage = "url(" + this.makeUrl() + ")";

  var width = this.width;
  var height = this.height;

  var context = element.getContext('2d');

  context.strokeStyle = '#FFFF00';

  context.beginPath();
  context.moveTo(0, height / 2);
  context.lineTo(width, height / 2);
  context.stroke();

  context.beginPath();
  context.moveTo(width / 2, 0);
  context.lineTo(width / 2, height);
  context.stroke();
}

function sgn(x) {
  return x >= 0 ? 1 : -1;
}

PanoramaViewer.prototype.unmap = function(heading, pitch) {
  var PI = Math.PI
  var cos = Math.cos;
  var sin = Math.sin;
  var tan = Math.tan;

  var fov = PanoramaViewer.FOV * PI / 180.0;
  var width = this.width;
  var height = this.height;

  var f = 0.5 * width / tan(0.5 * fov);

  var h = heading * PI / 180.0;
  var p = pitch * PI / 180.0;

  var x = f * cos(p) * sin(h);
  var y = f * cos(p) * cos(h);
  var z = f * sin(p);

  var h0 = this.heading * PI / 180.0;
  var p0 = this.pitch * PI / 180.0;

  var x0 = f * cos(p0) * sin(h0);
  var y0 = f * cos(p0) * cos(h0);
  var z0 = f * sin(p0);

  //
  // Intersect the ray O, v = (x, y, z)
  // with the plane at M0 of normal n = (x0, y0, z0)
  //
  //   n . (O + t v - M0) = 0
  //   t n . v = n . M0 = f^2
  //
  var t = f * f / (x0 * x + y0 * y + z0 * z);

  var ux = sgn(cos(p0)) * cos(h0);
  var uy = -sgn(cos(p0)) * sin(h0);
  var uz = 0;

  var vx = -sin(p0) * sin(h0);
  var vy = -sin(p0) * cos(h0);
  var vz = cos(p0);

  var x1 = t * x;
  var y1 = t * y;
  var z1 = t * z;

  var dx10 = x1 - x0;
  var dy10 = y1 - y0;
  var dz10 = z1 - z0;

  // Project on the local basis (u, v) at M0
  var du = ux * dx10 + uy * dy10 + uz * dz10;
  var dv = vx * dx10 + vy * dy10 + vz * dz10;

  return {
    u: du + width / 2,
    v: height / 2 - dv,
  };
}

PanoramaViewer.prototype.map = function(u, v) {
  var PI = Math.PI;
  var cos = Math.cos;
  var sin = Math.sin;
  var tan = Math.tan;
  var sqrt = Math.sqrt;
  var atan2 = Math.atan2;
  var asin = Math.asin;

  var fov = PanoramaViewer.FOV * PI / 180.0;
  var width = this.width;
  var height = this.height;

  var h0 = this.heading * PI / 180.0;
  var p0 = this.pitch * PI / 180.0;

  var f = 0.5 * width / tan(0.5 * fov);

  var x0 = f * cos(p0) * sin(h0);
  var y0 = f * cos(p0) * cos(h0);
  var z0 = f * sin(p0);

  var du = u - width / 2;
  var dv = height / 2 - v;

  var ux = sgn(cos(p0)) * cos(h0);
  var uy = -sgn(cos(p0)) * sin(h0);
  var uz = 0;

  var vx = -sin(p0) * sin(h0);
  var vy = -sin(p0) * cos(h0);
  var vz = cos(p0);

  var x = x0 + du * ux + dv * vx;
  var y = y0 + du * uy + dv * vy;
  var z = z0 + du * uz + dv * vz;

  var R = sqrt(x * x + y * y + z * z);
  var h = atan2(x, y);
  var p = asin(z / R);

  return {
    heading: h * 180.0 / PI,
    pitch: p * 180.0 / PI
  };
}

PanoramaViewer.prototype.click = function(e) {
  var rect = e.target.getBoundingClientRect();
  var u = e.clientX - rect.left;
  var v = e.clientY - rect.top;

  var uvCoords = this.unmap(this.heading, this.pitch);

  console.log("current viewport center");
  console.log("  heading: " + this.heading);
  console.log("  pitch: " + this.pitch);
  console.log("  u: " + uvCoords.u)
  console.log("  v: " + uvCoords.v);

  var hpCoords = this.map(u, v);
  uvCoords = this.unmap(hpCoords.heading, hpCoords.pitch);

  console.log("click at (" + u + "," + v + ")");
  console.log("  heading: " + hpCoords.heading);
  console.log("  pitch: " + hpCoords.pitch);
  console.log("  u: " + uvCoords.u);
  console.log("  v: " + uvCoords.v);

  this.heading = hpCoords.heading;
  this.pitch = hpCoords.pitch;
  this.update();
}