docker stack/swarm 中的弹性

Question

我有两个节点的群

[ra@speechanalytics-test ~]$ docker node ls
ID                            HOSTNAME                  STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
mlwwmkdlzbv0zlapqe1veq3uq     speechanalytics-preprod   Ready               Active                                  18.09.3
se717p88485s22s715rdir9x2 *   speechanalytics-test      Ready               Active              Leader              18.09.3

我正在尝试在堆栈中使用弹性运行容器。这是我的docker-compose.yml文件

version: '3.4'
services:
  elastic:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
    environment:
      - cluster.name=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata:/usr/share/elasticsearch/data
    deploy:
      placement:
        constraints:
          - node.hostname==speechanalytics-preprod

volumes:
  esdata:
    driver: local

使用 docker stack 开始后

docker stack deploy preprod -c docker-compose.yml

容器在 20 秒内崩溃

docker service logs preprod_elastic 
...
   | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   | OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=0
   | [2019-04-03T16:41:30,044][WARN ][o.e.b.JNANatives         ] [unknown] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
   | [2019-04-03T16:41:30,049][WARN ][o.e.b.JNANatives         ] [unknown] This can result in part of the JVM being swapped out.
   | [2019-04-03T16:41:30,049][WARN ][o.e.b.JNANatives         ] [unknown] Increase RLIMIT_MEMLOCK, soft limit: 16777216, hard limit: 16777216
   | [2019-04-03T16:41:30,050][WARN ][o.e.b.JNANatives         ] [unknown] These can be adjusted by modifying /etc/security/limits.conf, for example:
   | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   |     # allow user 'elasticsearch' mlockall
   | OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=0
   |     elasticsearch soft memlock unlimited
   | [2019-04-03T16:41:02,949][WARN ][o.e.b.JNANatives         ] [unknown] Unable to lock JVM Memory: error=12, reason=Cannot allocate memory
   |     elasticsearch hard memlock unlimited
   | [2019-04-03T16:41:02,954][WARN ][o.e.b.JNANatives         ] [unknown] This can result in part of the JVM being swapped out.
   | [2019-04-03T16:41:30,050][WARN ][o.e.b.JNANatives         ] [unknown] If you are logged in interactively, you will have to re-login for the new limits to take effect.
   | [2019-04-03T16:41:02,954][WARN ][o.e.b.JNANatives         ] [unknown] Increase RLIMIT_MEMLOCK, soft limit: 16777216, hard limit: 16777216
preprod

在我有的两个节点上

ra@speechanalytics-preprod:~$ sysctl vm.max_map_count
vm.max_map_count = 262144

任何想法如何解决？

Answer 1

您在 Elasticsearch 中看到的 memlock 错误是一个常见问题，并非使用 Docker 所独有，而是在 Elasticsearch 被告知锁定其内存但无法这样做时发生。您可以通过从文件中删除以下环境变量来规避错误docker-compose.yml：

- bootstrap.memory_lock=true

Memlock可以与 Docker Swarm 模式一起使用，但有一些警告。

并非所有适用于docker-compose(Docker Compose) 的选项都适用于docker stack deploy(Docker Swarm Mode)，反之亦然，尽管两者都共享 docker-compose YAML 语法。一个这样的选项是ulimits:，当与 docker stack deploy 一起使用时，将被忽略并显示警告消息，如下所示：

Ignoring unsupported options: ulimits

我的猜测是与你的docker-compose.yml文件，Elasticsearch运行正常使用docker-compose up，但不与docker stack deploy。

使用 Docker Swarm 模式，默认情况下，您定义的 Elasticsearch 实例会遇到 memlock 问题。目前，尚未正式支持为 docker swarm 服务设置 ulimit 。不过，有一些方法可以解决这个问题。

如果主机是Ubuntu，则可以在 docker 服务中启用无限制的 memlock（请参阅此处和此处）。这可以通过以下命令实现：

- bootstrap.memory_lock=true

然而，将 memlock 设置为无穷大并非没有缺点，正如 Elastic 自己在此处阐明的那样。

根据我的测试，该解决方案适用于 Docker 18.06，但不适用于 18.09。鉴于不一致和 Elasticsearch 无法启动的可能性，更好的选择是在 Swarm 上部署时不要将 memlock 与 Elasticsearch一起使用。相反，您可以选择Elasticsearch Docs 中提到的任何其他方法来获得类似的结果。