Par*_*iva 5 postgresql jenkins flyway docker jenkins-pipeline
我正在尝试运行一个适用于数据库迁移脚本的 CI 系统。因此，这可能会阻止我们运行由于语法错误而在迁移时不起作用的 SQL 脚本。对于我们的本地设置，我们使用 docker-compose 并一次运行多个容器。不幸的是，这不是我们在 Jenkins 管道中的选择
我使用以下策略创建了这个 Jenkinsfile。
我的实现基于此处的文档（运行 docker sidecar）。然而，这不起作用，第一个容器（步骤 1）似乎正在停止。我添加了一些额外的调试（try catch）来查看该容器的日志
我的Jenkinsfile的内容
def docker_repository = '<CUSTOM-REGISTRY>'
def docker_user_credentialsId = '<DOCKER-USER>'
pipeline {
agent { label 'docker && linux && nonprod' }
options {
buildDiscarder(logRotator(daysToKeepStr: '90', numToKeepStr: '20', artifactDaysToKeepStr: '90', artifactNumToKeepStr: '20'))
timeout(time: 20, unit: 'MINUTES')
}
stages {
stage('build & test') {
environment {
POSTGRES_DB = 'mydb'
POSTGRES_USER = 'postgres'
POSTGRES_PASSWORD = 'postgres'
FLYWAY_URL = 'jdbc:postgresql://localhost:5432/mydb'
// FLYWAY_URL = 'jdbc:postgresql://db-container:5432/mydb'
FLYWAY_USER = 'postgres'
FLYWAY_PASSWORD = 'postgres'
}
steps {
checkout scm
withCredentials([ usernamePassword(credentialsId: docker_user_credentialsId, passwordVariable: 'ARTIFACTORY_API_TOKEN', usernameVariable: 'ARTIFACTORY_API_USER'),]) {}
script {
docker.withRegistry(docker_repository, docker_user_credentialsId) {
docker.image('<REGISTRY>/postgres').withRun("--name=db-container -e POSTGRES_PASSWORD=postgres") { c ->
try {
docker.image('<REGISTRY>/postgres').inside("--link ${c.id}:db") {
sh '''
while ! pg_isready -h db -p 5432
do
echo $
echo "$(date) - waiting for database to start"
sleep 10
done
'''
}
docker.image('<REGISTRY>/flyway/flyway').inside("--link ${c.id}:db") {
sh 'info'
// sh 'migrate'
}
} catch (exc) {
sh "docker logs ${c.id}"
throw exc
}
}
}
}
}
}
}
post {
always {
cleanWs()
}
}
}
暂时删除了步骤 2 以获取此日志，因为步骤 2 在循环中等待并且docker 日志
+ docker logs 9d9e8699b57430e288520c485c8333a0261f9283f749aec2832cfb0e5f19ef9e
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2021-03-31 15:21:29.923 UTC [48] LOG: starting PostgreSQL 13.2
(Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-03-31 15:21:29.929 UTC [48] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-03-31 15:21:29.946 UTC [49] LOG: database system was shut down at 2021-03-31 15:21:29 UTC
2021-03-31 15:21:29.951 UTC [48] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
2021-03-31 15:21:30.062 UTC [48] LOG: received fast shutdown request
waiting for server to shut down....2021-03-31 15:21:30.064 UTC [48] LOG: aborting any active transactions
2021-03-31 15:21:30.065 UTC [48] LOG: background worker "logical replication launcher" (PID 55)
exited with exit code 1
2021-03-31 15:21:30.071 UTC [50] LOG: shutting down
2021-03-31 15:21:30.099 UTC [48] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2021-03-31 15:21:30.188 UTC [1] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-03-31 15:21:30.190 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-03-31 15:21:30.190 UTC [1] LOG: listening on IPv6 address "::", port 5432
2021-03-31 15:21:30.196 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-03-31 15:21:30.203 UTC [67] LOG: database system was shut down at 2021-03-31 15:21:30 UTC
2021-03-31 15:21:30.208 UTC [1] LOG: database system is ready to accept connections
postgres 和 Flyway 版本在这里不应发挥重要作用，但正如在日志中看到的那样，Postgres 版本是默认标记提供的 13.2。
这里有什么指点吗？
经过一些尝试和错误后，这对我们来说是有效的。尽管它与初始设置非常相似，但关键的变化在于使用inside和的选择withRun。最终的 Jenkinsfile 如下所示。
// credentials
def docker_user_credentialsId = '<DOCKER-USER>'
def docker_repository = '<CUSTOM-REGISTRY>'
// docker images
def pg = docker.image('postgres:12-alpine')
def flyway = docker.image('flyway/flyway:7-alpine')
pipeline {
agent none
options {
buildDiscarder(logRotator(daysToKeepStr: '90', numToKeepStr: '20', artifactDaysToKeepStr: '90', artifactNumToKeepStr: '20'))
timeout(time: 120, unit: 'MINUTES')
}
stages {
stage('db migration test') {
agent {
label 'docker && linux'
}
environment {
PGHOST = 'db'
PGPORT = '5432'
PGDATABASE = 'postgres'
PGUSER = 'postgres'
PGPASSWORD = 'postgres'
}
steps {
script {
docker.withRegistry(docker_repository, docker_user_credentialsId) {
def db = pg.withRun(
"-v ${WORKSPACE}/docker/init-user-db.sh:/docker-entrypoint-initdb.d/init-user-db.sh -e POSTGRES_USER=${env.PGUSER} -e POSTGRES_PASSWORD=${env.PGPASSWORD} -e POSTGRES_DB=${env.PGDATABASE}"
) { db ->
pg.inside("--link ${db.id}:${env.PGHOST}") {
sh '''
echo "$(date) - waiting for database to start"
while ! pg_isready
do
sleep 10
done
'''
}
flyway.withRun("-e FLYWAY_LOCATIONS=filesystem:/tmp/database/server,filesystem:/tmp/database/local -e FLYWAY_URL=jdbc:postgresql://${PGHOST}:${env.PGPORT}/${env.PGDATABASE} -e FLYWAY_USER=${env.PGUSER} -e FLYWAY_PASSWORD=${env.PGPASSWORD} -v ${WORKSPACE}/database/local:/tmp/database/local -v ${WORKSPACE}/database/server:/tmp/database/server --link ${db.id}:db", "migrate") { f ->
sh "docker logs -f ${f.id}"
def output = sh script: "docker inspect ${f.id} --format='{{.State.ExitCode}}'", returnStdout: true
sh "exit ${output}"
}
}
}
}
}
}
}
}
| 归档时间： |
|
| 查看次数： |
2175 次 |
| 最近记录： |