在AWS Ubuntu实例中为NodeJS部署设置GitLab CI
dev*_*evo 8 ubuntu amazon-web-services node.js gitlab gitlab-ci
我有一个配置了GitLab CE的AWS Ubuntu实例.现在我想配置GitLab CI以在每次提交后部署我的NodeJS应用程序.我没有任何适当的逐步解决方案.
我的应用程序的NodeJS在运行/var/www/mean/my-app上http://myapp.mydomain.com与主机是由Apache代理处理,
<VirtualHost *:80>
ServerAdmin anshad@mydomain.com
ServerName gitlab.mydomain.com
ServerAlias www.gitlab.mydomain.com
ServerSignature Off
ProxyPreserveHost On
AllowEncodedSlashes NoDecode
<Location />
Require all granted
ProxyPassReverse http://localhost:8080
ProxyPassReverse http://gitlab.mydomain.com/
</Location>
RewriteEngine on
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f [OR]
RewriteCond %{REQUEST_URI} ^/uploads/.*
RewriteRule .* http://127.0.0.1:8080%{REQUEST_URI} [P,QSA,NE]
DocumentRoot /home/git/gitlab/public
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
ErrorLog /var/log/apache2/gitlab_error.log
CustomLog /var/log/apache2/gitlab_forwarded.log common_forwarded
CustomLog /var/log/apache2/gitlab_access.log combined env=!dontlog
CustomLog /var/log/apache2/gitlab.log combined
</VirtualHost>
该应用程序使用forever模块进行自举
forever start app.js
gitlab配置检查sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production给出,
Checking GitLab Shell ...
GitLab Shell version >= 4.0.0 ? ... OK (4.0.0)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
dev / my-app ... ok
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /home/git/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
Reply by email is disabled in config/gitlab.yml
Checking Reply by email ... Finished
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
projects have namespace: ...
dev / my-app ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.3.1)
Your git bin path is "/usr/bin/git"
Git version >= 2.7.3 ? ... yes (2.7.4)
Active users: 1
Checking GitLab ... Finished
我以前使用SSH从我的系统登录到实例,
ssh -i API-Key.pem ubuntu@ec2-XX-XX-XXX-XXX.ap-south-1.compute.amazonaws.com
使用命令创建密钥
ssh-keygen -t rsa
Runner配置 /etc/gitlab-runner/config.toml
concurrent = 1
check_interval = 0
[[runners]]
name = "Production Runner"
url = "http://gitlab.mydomain.com/ci"
token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
executor = "ssh"
[runners.ssh]
user = "ubuntu"
host = "ip-XXX-XX-XX-XXX"
identity_file = "/home/ubuntu/.ssh/id_rsa"
[runners.cache]
代码 .gitlab-ci.yml
test_async:
script:
- npm install
由于配置错误,跑步者会出错,
Running with gitlab-ci-multi-runner 1.7.1 (f896af7)
Using SSH executor...
ERROR: Preparation failed: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Will be retried in 3s ...
我的困惑是:
.gitlab-ci.yml将已提交的代码部署到应用程序位置的文件内容应该是什么？
以及如何为此配置跑步者？如果我必须使用ssh跑步者,那里的配置应该是什么？
更新:
提供.pem文件后identity_file,收到以下错误
Running with gitlab-ci-multi-runner 1.7.1 (f896af7)
Using SSH executor...
Running on ip-xxx-xx-xx-xxx via ip-xxx-xx-xx-xxx...
Cloning repository...
Cloning into 'builds/a92f1b91/0/dev/my-app'...
fatal: unable to access 'http://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab.mydomain.com/dev/my-app.git/': The requested URL returned error: 500
ERROR: Build failed: Process exited with: 1. Reason was: ()
现在有一个问题,git克隆的http不能正常工作,但ssh克隆工作.
注意: gitlab和构建环境都是相同的主机(相同的aws实例)
GitLab中也报告了Bug(http克隆问题).
在 /etc/gitlab-runner/config.toml 中
concurrent = 1
check_interval = 0
[[runners]]
name = "Production Runner"
url = "http://gitlab.mydomain.com/ci"
token = "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
executor = "ssh"
[runners.ssh]
user = "ubuntu"
host = "ip-XXX-XX-XX-XXX"
identity_file = "/home/ubuntu/.ssh/id_rsa"
[runners.cache]
你定义
- 主持人
- 用户
- 和身份文件
host 应该是您的 Build 主机 IP（换句话说，您要执行构建的位置）
user 应该是Build host上的用户。不在 gitlab 主机上。
您可以通过以下方式测试无密码 ssh 的工作原理
- 以 root 身份登录 gitlab 主机
- ssh -i /home/ubuntu/.ssh/id_rsa ubuntu@ip-XXX-XX-XX-XXX
如果这有效并且不要求您输入密码 - 一切都很好。
如果出现问题 - 意味着您没有正确设置无密码身份验证。
设置基于无密码公钥的身份验证的最简单方法是使用名为
ssh-copy-id
例如，我想在 gitlab 和构建主机之间设置无密码 ssh 身份验证。
我的构建主机 IP 是 192.168.0.42，主机名是 build.home
我已经在 gitlab 主机上的 /home/ubuntu/.ssh 下生成了 id_rsa 和 id_rsa.pub 。
现在让我们将公钥从 gitlab 主机推送到我们的构建主机。第一次它会要求您输入密码。
[root@gitlab ~]# ssh-copy-id -i /home/ubuntu/.ssh/id_rsa.pub ubuntu@build.home
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
ubuntu@build.home's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'ubuntu@build.home'"
and check to make sure that only the key(s) you wanted were added.
请注意，在上面的示例中，我将公钥推送到远程主机。
但是当我现在连接到这个远程主机时 - 我将指定我的私钥。
[root@gitlab ~]# ssh -i /home/ubuntu/.ssh/id_rsa ubuntu@build.home
[ubuntu@build ~]$ hostname
build.home
尝试测试 gitlab 主机和远程主机之间的公钥身份验证并更新您的问题。
资源：
https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/executors/ssh.md
PS：今天晚些时候我回家后会发布我的工作环境配置。
编辑1：
这是我的配置。
我的 gitlab 主机名为 gitlab.home 192.168.0.41
我还有另一个名为 sshbuild.home 192.168.0.43 的虚拟机
下面是我添加 ssh 运行程序的方法
步骤 1. 在我的 gitlab.home 上安装
yum install gitlab-ci-multi-runner并将我的远程 sshbuild.home VM 注册为 ssh 运行程序
我还需要确保 gitlab.home 和 sshbuild.home 之间的无密码身份验证有效，所以
[root@gitlab gitlab-runner]# ssh-copy-id 192.168.0.43
The authenticity of host '192.168.0.43 (192.168.0.43)' can't be established.
ECDSA key fingerprint is b4:6a:1b:72:d1:7d:1f:34:f7:bb:ef:ad:69:42:11:13.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.0.43's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '192.168.0.43'"
and check to make sure that only the key(s) you wanted were added.
[root@gitlab gitlab-runner]# ssh 192.168.0.43
Last login: Fri Nov 18 17:05:06 2016 from 192.168.0.101
[root@sshbuild ~]# exit
然后我禁用了其他运行程序（shell 运行程序）并专门创建了新的 SSH 运行程序项目，以确保当我提交时 - 它将在 ssh 运行程序上执行
我确实提交了，瞧 - 我们已经在 sshbuild.home 主机上运行了成功的测试
这里有几个链接可能有助于更好地理解这个主题
- https://about.gitlab.com/2016/07/29/the-basics-of-gitlab-ci/
- https://docs.gitlab.com/ce/ci/runners/README.html
- http://docs.gitlab.com/runner/commands/README.html
- https://docs.gitlab.com/ce/ci/yaml/README.html
PS：这是我的 /etc/gitlab-runner/config.toml 文件
[root@gitlab gitlab-runner]# cat /etc/gitlab-runner/config.toml
concurrent = 1
check_interval = 0
[[runners]]
name = "sshbuild"
url = "http://gitlab.home/"
token = "2bc1825d8fbde09fd632637c60e9e7"
executor = "ssh"
[runners.ssh]
user = "root"
host = "192.168.0.43"
port = "22"
identity_file = "/root/.ssh/id_rsa"
[runners.cache]
PS：如果我在 Web 界面的“设置”下为我的存储库禁用 HTTP，我会遇到与您类似的错误。但是错误不是500而是403。
编辑2：
现在我将基于一个简单的 HelloWorld 项目介绍 .gitlab-ci.yml
在我的 HelloWorld 中，我有一个名为 server.js 的文件，当从节点运行时，该文件将简单地创建在端口 3000 上运行的 Web 服务器，并在上回复“Hello World”获取请求。
1 const http = require('http');
2
3 const hostname = '0.0.0.0';
4 const port = 3000;
5
6 const server = http.createServer((req, res) => {
7 res.statusCode = 200;
8 res.setHeader('Content-Type', 'text/plain');
9 res.end('Hello World!\n');
10 });
11
12 server.listen(port, hostname, () => {
13 console.log(`Server running at http://${hostname}:${port}/`);
14 });
我的目标是能够针对它运行测试用例。在这种情况下，我将运行简单
curl localhost:3000 | grep "Hello World"
但我需要将其放入一个单独的脚本中，成功时退出状态为 0，失败时退出状态为非零
cat -n simpletest.sh
1 #!/bin/bash
2
3 cleanup ()
4 {
5 count=`netstat -anp|grep ":3000"|grep LISTEN|awk '{print $NF}'|cut -d\/ -f1|wc -l`
6 if [ $count -ne 0 ]
7 then
8 pid=`netstat -anp|grep ":3000"|grep LISTEN|awk '{print $NF}'|cut -d\/ -f1`;
9 echo "Need to kill PID $pid";
10 kill $pid
11 fi
12 }
13
14 echo "Running simple test"
15 curl localhost:3000|grep "Hello World"
16 if [ $? -eq 0 ]
17 then
18 echo "Test was successfull"
19 echo "Clean up node.js process"
20 cleanup
21 exit 0
22 else
23 echo "Test failed"
24 echo "Clean up node.js process"
25 cleanup
26 exit 1
27 fi
现在让我们介绍我的 .gitlab-ci.yml
cat -n .gitlab-ci.yml
1 test:
2
3 before_script:
4 - echo "Before script"
5 - hostname
6 - /bin/bash cleanup.sh
7
8 script:
9 - echo "Main Script"
10 - node server.js &
11 - sleep 3
12 - /bin/bash simpletest.sh
我有一个名为测试的工作。
在 before_script 中，它运行 cleanup.sh 脚本，该脚本会在发现此类情况时简单地终止端口 3000 上的 PID 侦听。
cat -n cleanup.sh
1 #!/bin/bash
2 count=`netstat -anp|grep ":3000"|grep LISTEN|awk '{print $NF}'|cut -d\/ -f1|wc -l`
3 if [ $count -ne 0 ]
4 then
5 pid=`netstat -anp|grep ":3000"|grep LISTEN|awk '{print $NF}'|cut -d\/ -f1`;
6 echo "Need to kill PID $pid";
7 kill $pid
8 fi
9 exit 0
在脚本下：它使用我的 server.js 运行节点，给它 3 秒的启动时间，然后对其运行测试。
该测试还将在测试完成后杀死节点 PID。
因此，让我们提交并检查构建的状态
现在让我们更改 server.js，使其输出的不是“Hello World”而是“HelloWorld”，因此中间没有空格。我预计我的测试用例会失败，因为它实际上期望“Hello World”。它确实失败了。
这是我能想到的最简单的 CI 用例。
现在，如果根据测试用例的状态，您想将代码部署到另一个环境 - 您必须开始使用
- 阶段和
- 环境
所以你的 .gitlab-ci.yml 会变成这样的东西（真实的工作示例）
cat -n .gitlab-ci.yml
1 stages:
2 - test
3 - deploy
4
5 run_test_case:
6 stage: test
7 before_script:
8 - echo "Before script"
9 - hostname
10 - /bin/bash cleanup.sh
11
12 script:
13 - echo "Main Script"
14 - node server.js &
15 - sleep 3
16 - /bin/bash simpletest.sh
17
18 deploy_to_production:
19 stage: deploy
20 script:
21 - echo "Run code here to do production deployment"
22 environment:
23 name: production
git Push 会成功。
在第 21 行，我只是运行了 echo，但这可以替换为一个脚本，该脚本将推送到您的远程暂存或生产环境。
| 归档时间： |
|
| 查看次数： |
1379 次 |
| 最近记录： |