如何在 docker-compose 中为 tensorflow-serving 指定“model_config_file”变量？

Question

我先说我对 docker 和 docker-compose 缺乏经验。我正在尝试将我的docker run ...命令转换为 docker-compose.yml文件，但是，我无法找到models.config文件。

我能够使用以下docker run ...命令正确运行 tensorflow 服务 docker 容器：

docker run -t --rm \
  tensorflow/serving \
  -p 8501:8501 \
  -v "$(pwd)/models/:/models/" \
  --model_config_file=/models/models.config \
  --model_config_file_poll_wait_seconds=60

这按预期工作，models.config文件位于容器中/models/models.config按预期。

tensorflow-serving 页面没有提到任何关于 docker-compose 的内容，但是，我更愿意使用它而不是docker run ...命令。我对 docker-compose 文件的尝试是：

version: '3.3'
services:
  server:
    image: tensorflow/serving
    ports:
      - '8501:8501'
    volumes:
      - './models:/models'
    environment:
      - 'model_config_file=/models/models.config'
      - 'model_config_file_poll_wait_seconds=60'

使用这个 docker-compose 文件，容器运行，但是，环境变量似乎被完全忽略了，所以我不确定我是否应该这样设置它们。容器映像在models.config文件的默认位置查找，但它不存在，因此它不会加载在models.config 中定义的配置。

那么，如何使用 docker-compose 定义这些值，或运行 tensorflow 服务容器？

我很感激任何帮助。

谢谢

Answer 1

所以我在其他地方遇到了一个解决方案，我在任何讨论张量流/服务的线程/帖子/等上都没有找到，所以我会在这里发布我的答案。

在一个command部分下面添加这些选项，如下所示。

version: '3.3'
services:
  server:
    image: tensorflow/serving
    ports:
      - '8501:8501'
    volumes:
      - './models:/models'
    command:
      - '--model_config_file=/models/models.config'
      - '--model_config_file_poll_wait_seconds=60'

我对 docker 了解不多，所以我不知道这是否是一个明显的答案，但即使经过大量谷歌搜索，我也没有找到解决方案。

Answer 2

The problem is that the option --model_config_file is not an environment variable. It is an argument that you can pass to the command that is set as an entrypoint in the image tensorflow/serving. If we look at the Dockerfile that was used to build the image, we can see :

# Create a script that runs the model server so we can use environment variables
# while also passing in arguments from the docker command line
RUN echo '#!/bin/bash \n\n\
tensorflow_model_server --port=8500 --rest_api_port=8501 \
--model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME} \
"$@"' > /usr/bin/tf_serving_entrypoint.sh \
&& chmod +x /usr/bin/tf_serving_entrypoint.sh

ENTRYPOINT ["/usr/bin/tf_serving_entrypoint.sh"]

This script accept various arguments (you can see them by running docker run -t --rm tensorflow/serving --help)

As far as I know, the only environment variables used by TF serving are MODEL_VERSION and MODEL_NAME. model_config_file and model_config_file_poll_wait_seconds are just arguments to the CLI serving executable.

To achieve the result you are after, you could override the entrypoint of the docker image by setting your own entrypoint in the docker-compose.yml :

version: '3.3'
services:
  server:
    image: tensorflow/serving
    ports:
      - '8501:8501'
    volumes:
      - './models:/models'
    entrypoint:
      - /usr/bin/tf_serving_entrypoint.sh
      - --model_config_file=/models/models.config
      - --model_config_file_poll_wait_seconds=60

(Note : I did not test that docker-compose.yml file)