避免在弹性beanstalk中重建node_modules

Question

我们有一个相当简单的node.js应用程序,但由于AWS Elastic Beanstalk部署机制,git aws.push即使在单个文件提交后,也需要大约5分钟来推出新版本(通过).

即提交本身(和上传)很快(只有1个文件要推送),但随后Elastic Beanstalk从S3获取整个包,解压缩并运行npm install,这会导致node-gyp编译一些模块.安装/构建完成后,Elastic Beanstalk会擦除/var/app/current并替换为新的应用程序版本.

毋庸置疑,不需要进行常规node_modules重建,并且在我的旧Macbook Air上重建需要30秒,在ec2.micro实例上花费大约5分钟,并不好玩.

我在这里看到两种方法:

1. 调整/opt/containerfiles/ebnode.py并使用node_modules位置以避免在部署时删除和重建.
2. 在Elastic Beanstalk EC2实例上设置一个git repo并基本上自己重写部署过程,所以/ var/app/current npm install只在必要时接收推送和运行(这使得Elastic Beanstalk看起来像OpsWorks ..)

当Amazon更新其Elastic Beanstalk挂钩和架构时,这两个选项都缺乏优势并且容易出现问题.

也许有人有更好的想法如何避免不断重建已存在于app dir中的node_modules？谢谢.

Answer 1

谢谢基里尔,这真的很有帮助!

我只是为那些只看简单解决方案的人分享我的配置文件npm install.此文件需要放在.ebextensions项目的文件夹中,因为它不包含节点安装的最后版本,并且可以使用,所以它更轻.

它还会动态检查安装的节点版本,因此不需要将其包含在env.vars文件中.

.ebextensions/00_deploy_npm.config

files:
  "/opt/elasticbeanstalk/env.vars" :
    mode: "000775"
    owner: root
    group: users
    content: |
      export NPM_CONFIG_LOGLEVEL=error
      export NODE_PATH=`ls -td /opt/elasticbeanstalk/node-install/node-* | head -1`/bin
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: users
    content: |
      #!/bin/bash
      . /opt/elasticbeanstalk/env.vars
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      #install not-installed yet app node_modules
      if [ ! -d "/var/node_modules" ]; then
        mkdir /var/node_modules ;
      fi
      if [ -d /tmp/deployment/application ]; then
        ln -s /var/node_modules /tmp/deployment/application/
      fi

      OUT=$([ -d "/tmp/deployment/application" ] && cd /tmp/deployment/application && $NODE_PATH/npm install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT
  "/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh" :
    mode: "000666"
    owner: root
    group: users
    content: |
       #no need to run npm install during configdeploy

Answer 2

25/01/13注意:更新脚本运行NPM -g版本升级(只有一次,在最初的情况下推出或重建)和EB配置更改(当应用程序目录不存在,以避免错误和期间避免NPM操作加快配置更新).

好吧,Elastic Beanstalk与最近的node.js版本(包括大概支持的v.0.10.10)表现得很狡猾,所以我决定继续调整EB以执行以下操作:

1. 根据您的env.config安装任何node.js版本(包括AWS EB尚不支持的最新版本)
2. 避免重建现有节点模块,包括应用程序内node_modules目录
3. 全局安装node.js(以及任何所需的模块).

基本上,我使用env.config来替换自定义的钩子和配置钩子(见下文).此外,在默认的EB容器设置中,一些env变量丢失($HOME例如),node-gyp有时在重建期间因为它而失败(花了我2个小时的谷歌搜索并重新安装libxmljs来解决这个问题).

以下是与您的构建一起包含的文件.您可以通过env.config注入它们作为内联代码或通过source: URL(如本示例中所示)

env.vars (所需的节点版本和拱包含在这里和env.config中,见下文)

export HOME=/root
export NPM_CONFIG_LOGLEVEL=error
export NODE_VER=0.10.24
export ARCH=x86
export PATH="$PATH:/opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/:/root/.npm"

40install_node.sh (fetch和ungzip所需的node.js版本,制作全局符号链接,更新全局npm版本)

#!/bin/bash
#source env variables including node version
. /opt/elasticbeanstalk/env.vars

function error_exit
{
  eventHelper.py --msg "$1" --severity ERROR
  exit $2
}

#UNCOMMENT to update npm, otherwise will be updated on instance init or rebuild
#rm -f /opt/elasticbeanstalk/node-install/npm_updated

#download and extract desired node.js version
OUT=$( [ ! -d "/opt/elasticbeanstalk/node-install" ] && mkdir /opt/elasticbeanstalk/node-install ; cd /opt/elasticbeanstalk/node-install/ && wget -nc http://nodejs.org/dist/v$NODE_VER/node-v$NODE_VER-linux-$ARCH.tar.gz && tar --skip-old-files -xzpf node-v$NODE_VER-linux-$ARCH.tar.gz) || error_exit "Failed to UPDATE node version. $OUT" $?.
echo $OUT

#make sure node binaries can be found globally
if [ ! -L /usr/bin/node ]; then
  ln -s /opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/node /usr/bin/node
fi

if [ ! -L /usr/bin/npm ]; then
ln -s /opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/npm /usr/bin/npm
fi

if [ ! -f "/opt/elasticbeanstalk/node-install/npm_updated" ]; then
/opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/ && /opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/npm update npm -g
touch /opt/elasticbeanstalk/node-install/npm_updated
echo "YAY! Updated global NPM version to `npm -v`"
else
  echo "Skipping NPM -g version update. To update, please uncomment 40install_node.sh:12"
fi

50npm.sh (创建/ var/node_modules,将它符号链接到app dir并运行npm install.你可以从这里全局安装任何模块,它们将落在/root/.npm)

#!/bin/bash
. /opt/elasticbeanstalk/env.vars
function error_exit
{
  eventHelper.py --msg "$1" --severity ERROR
  exit $2
}

#install not-installed yet app node_modules
if [ ! -d "/var/node_modules" ]; then
  mkdir /var/node_modules ;
fi
if [ -d /tmp/deployment/application ]; then
  ln -s /var/node_modules /tmp/deployment/application/
fi

OUT=$([ -d "/tmp/deployment/application" ] && cd /tmp/deployment/application && /opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/npm install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
echo $OUT

env.config (此处也注意节点版本,为了安全起见,也将所需的节点版本放在AWS控制台的env配置中.我不确定哪些设置优先.)

packages:
  yum:
    git: []
    gcc: []
    make: []
    openssl-devel: []

option_settings:
  - option_name: NODE_ENV
    value: production
  - option_name: RDS_HOSTNAME
    value: fill_me_in
  - option_name: RDS_PASSWORD
    value: fill_me_in
  - option_name: RDS_USERNAME
    value: fill_me_in
  - namespace: aws:elasticbeanstalk:container:nodejs
    option_name: NodeVersion
    value: 0.10.24

files:
  "/opt/elasticbeanstalk/env.vars" :
    mode: "000775"
    owner: root
    group: users
    source: https://dl.dropbox.com/....
  "/opt/elasticbeanstalk/hooks/configdeploy/pre/40install_node.sh" :
    mode: "000775"
    owner: root
    group: users
    source: https://raw.github.com/....
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: users
    source: https://raw.github.com/....
  "/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh" :
    mode: "000666"
    owner: root
    group: users
    content: |
       #no need to run npm install during configdeploy
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/40install_node.sh" :
    mode: "000775"
    owner: root
    group: users
    source: https://raw.github.com/....

你有它:在t1.micro实例部署现在需要20-30秒而不是10-15分钟!如果您每天部署10次,这个调整将为您节省一年中的3(3)周.希望它有助于特别感谢AWS EB员工为我失去的周末:)

Answer 3

有npm包npm install通过截断以下文件来覆盖命令的默认EB行为:

• /opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh
• /opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh

https://www.npmjs.com/package/eb-disable-npm

可能比仅仅从SO复制脚本更好,因为这个包被维护,并且可能在EB行为改变时更新.