Kibana 使用 nginx 作为反向代理只会导致 404 错误

Question

我正在用来docker-compose运行一个项目的基础设施，其中一部分包括 ELK 堆栈。我使用nginx作为反向代理kibana（除其他外，使用 LetsEncrypt 作为 CA 来处理 SSL）。

当我尝试通过访问 kibana 实例时，https://my.host.com/kibana我得到了一个JSON编码的 404 页面。我还在 kibana 日志中收到大量错误（见下文）。我尝试过修改server.host、server.basePath、server.rewriteBasePath、 nginx 的proxy_redirect（将其设置为off）。没有什么真正有帮助。任何帮助将非常感激。

这是我的kibana.yml：

elasticsearch.hosts: ["http://elasticsearch:9200"]
logging.dest: stdout
server.host: kibana
server.basePath: /kibana
server.rewriteBasePath: true

这是我的相关部分nginx.conf：

  location /kibana {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_set_header        Connection "Keep-Alive";
    proxy_set_header        Proxy-Connection "Keep-Alive";

    proxy_pass              http://kibana:5601/;
    proxy_read_timeout      90;
    proxy_redirect          http://kibana:5106 https://my.host.com/kibana;
    proxy_buffering         off;
  }

这是相关部分docker-compose.yml：

  nginx:
    build: ./nginx
    depends_on:
      - logstash
      - kibana
    networks:
      - backend
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/www:/var/www
      - /etc/letsencrypt:/var/www-secure

  logstash:
    build: ./logstash
    depends_on:
      - elasticsearch
    networks:
      - backend

  elasticsearch:
    image: elasticsearch:7.6.1
    environment:
      discovery.type: single-node
    networks:
      - backend

  kibana:
    build: ./kibana
    networks:
      - backend

我在kibana容器的日志中收到的错误是这样的：

{"type":"response","@timestamp":"2020-03-29T22:36:12Z","tags":[],"pid":6,"method":"get","statusCode":404,"req":{"url":"/","method":"get","headers":{"host":"my.host.com","x-real-ip":"185.200.118.103","x-forwarded-for":"185.200.118.103","x-forwarded-proto":"https","connection":"Keep-Alive","proxy-connection":"Keep-Alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:74.0) Gecko/20100101 Firefox/74.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-GB,en;q=0.5","accept-encoding":"gzip, deflate, br","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"172.28.0.10","userAgent":"172.28.0.10"},"res":{"statusCode":404,"responseTime":188,"contentLength":9},"message":"GET / 404 188ms - 9.0B"}
{"type":"error","@timestamp":"2020-03-29T22:36:12Z","tags":["warning","process"],"pid":6,"level":"error","error":{"message":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)","name":"UnhandledPromiseRejectionWarning","stack":"UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)\n    at emitWarning (internal/process/promises.js:97:15)\n    at emitPromiseRejectionWarnings (internal/process/promises.js:143:7)\n    at process._tickCallback (internal/process/next_tick.js:69:34)"},"message":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"}
{"type":"error","@timestamp":"2020-03-29T22:36:12Z","tags":["warning","process"],"pid":6,"level":"error","error":{"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 7)","name":"UnhandledPromiseRejectionWarning","stack":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"},"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 7)"}
{"type":"response","@timestamp":"2020-03-29T22:36:13Z","tags":[],"pid":6,"method":"get","statusCode":404,"req":{"url":"/","method":"get","headers":{"host":"my.host.com","x-real-ip":"185.200.118.103","x-forwarded-for":"185.200.118.103","x-forwarded-proto":"https","connection":"Keep-Alive","proxy-connection":"Keep-Alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:74.0) Gecko/20100101 Firefox/74.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-GB,en;q=0.5","accept-encoding":"gzip, deflate, br","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"172.28.0.10","userAgent":"172.28.0.10"},"res":{"statusCode":404,"responseTime":160,"contentLength":9},"message":"GET / 404 160ms - 9.0B"}
{"type":"error","@timestamp":"2020-03-29T22:36:13Z","tags":["warning","process"],"pid":6,"level":"error","error":{"message":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)","name":"UnhandledPromiseRejectionWarning","stack":"UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)\n    at emitWarning (internal/process/promises.js:97:15)\n    at emitPromiseRejectionWarnings (internal/process/promises.js:143:7)\n    at process._tickCallback (internal/process/next_tick.js:69:34)"},"message":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"}
{"type":"error","@timestamp":"2020-03-29T22:36:13Z","tags":["warning","process"],"pid":6,"level":"error","error":{"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 12)","name":"UnhandledPromiseRejectionWarning","stack":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"},"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 12)"}

Answer 1

看来 kibanaserver.rewriteBasePath选项没有按预期工作。我通过更新 nginx 配置来进行显式重写并设置rewriteBasePath为false.

新的 nginx.conf：

  location /kibana {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_set_header        Connection "Keep-Alive";
    proxy_set_header        Proxy-Connection "Keep-Alive";

    proxy_pass              http://kibana:5601/;
    proxy_read_timeout      90;
    proxy_redirect          http://kibana:5106 https://my.host.com/kibana;
    rewrite                 /kibana/(.*)$ /$1 break;
    proxy_buffering         off;
  }

新的 kibana.yml：

elasticsearch.hosts: ["http://elasticsearch:9200"]
logging.dest: stdout
server.host: kibana
server.basePath: /kibana
server.rewriteBasePath: false

（我已将其报告为kibana 问题跟踪器中的错误。）