mVC*_*Chr 22 python deployment threads parallel-processing jenkins
我们有一个在多个主机上并行运行的 Python 结构命令,如下所示:
$ fab --hosts=prod1.server,prod2.server,prod3.server --parallel copy_cache
Run Code Online (Sandbox Code Playgroud)
这会将缓存复制到并行列出的生产服务器。由于 XXgig 缓存目录可能需要数小时,因此在整个过程中会发生各种日志记录以指示我们的进展情况。由于复制是并发进行的,因此在命令行上运行时的输出会实时交错返回,如下所示:
[prod1.server] Executing task 'nginx_cache_copy'
[prod2.server] Executing task 'nginx_cache_copy'
[prod3.server] Executing task 'nginx_cache_copy'
2014-09-16 10:02:29.688243
[prod1.server] INFO: rsyncing cache dir
[prod1.server] run: rsync -a -q cache.server:"repo/cache/some.site.com" \
"repo/cache/."
2014-09-16 10:02:29.716345
[prod2.server] INFO: rsyncing cache dir
[prod2.server] run: rsync -a -q cache.server:"repo/cache/some.site.com" \
"repo/cache/."
2014-09-16 10:02:29.853275
[prod3.server] INFO: rsyncing cache dir
[prod3.server] run: rsync -a -q cache.server:"repo/cache/some.site.com" \
"repo/cache/."
2014-09-16 10:02:29.984154
[prod1.server] INFO: Reloading nginx config
[prod1.server] run: sbin/nginx -s reload -c "repo/nginx.conf"
2014-09-16 10:02:30.025155
[prod2.server] INFO: Reloading nginx config
[prod2.server] run: sbin/nginx -s reload -c "repo/nginx.conf"
2014-09-16 10:02:30.100169
[prod1.server] SUCCESS: CACHE COPY COMPLETE
2014-09-16 10:02:30.181938
[prod2.server] SUCCESS: CACHE COPY COMPLETE
2014-09-16 10:02:30.331402
[prod3.server] INFO: Reloading nginx config
[prod3.server] run: sbin/nginx -s reload -c "repo/nginx.conf"
2014-09-16 10:02:30.559271
[prod3.server] SUCCESS: CACHE COPY COMPLETE
Done.
Run Code Online (Sandbox Code Playgroud)
但是,当任务通过 Jenkins 运行时,在所有任务完成之前,控制台输出不会显示,因为 Jenkins 在所有线程完成后加入线程后对输出进行分组。因此,一旦所有命令都完成,输出如下所示:
[prod1.server] Executing task 'nginx_cache_copy'
2014-09-16 10:02:29.688243
[prod1.server] INFO: rsyncing cache dir
[prod1.server] run: rsync -a -q cache.server:"repo/cache/some.site.com" \
"repo/cache/."
2014-09-16 10:02:29.984154
[prod1.server] INFO: Reloading nginx config
[prod1.server] run: sbin/nginx -s reload -c "repo/nginx.conf"
2014-09-16 10:02:30.100169
[prod1.server] SUCCESS: CACHE COPY COMPLETE
[prod2.server] Executing task 'nginx_cache_copy'
2014-09-16 10:02:29.716345
[prod2.server] INFO: rsyncing cache dir
[prod2.server] run: rsync -a -q cache.server:"repo/cache/some.site.com" \
"repo/cache/."
2014-09-16 10:02:30.025155
[prod2.server] INFO: Reloading nginx config
[prod2.server] run: sbin/nginx -s reload -c "repo/nginx.conf"
2014-09-16 10:02:30.181938
[prod2.server] SUCCESS: CACHE COPY COMPLETE
[prod3.server] Executing task 'nginx_cache_copy'
2014-09-16 10:02:29.853275
[prod3.server] INFO: rsyncing cache dir
[prod3.server] run: rsync -a -q cache.server:"repo/cache/some.site.com" \
"repo/cache/."
2014-09-16 10:02:30.331402
[prod3.server] INFO: Reloading nginx config
[prod3.server] run: sbin/nginx -s reload -c "repo/nginx.conf"
2014-09-16 10:02:30.559271
[prod3.server] SUCCESS: CACHE COPY COMPLETE
Done.
Run Code Online (Sandbox Code Playgroud)
虽然这更具可读性,但并不理想,因为我们希望通过实时读取控制台输出来跟踪进程的状态。需要注意的是,当这种织物运行命令没有的--parallel选项控制台输出确实发生在实时,但显然这是因为连续的过程需要更长的时间来运行是不可行的。
我无法在 Jenkins 中找到禁用此线程分组的设置。有没有人有任何想法?
| 归档时间: |
|
| 查看次数: |
2941 次 |
| 最近记录: |