小编Kal*_*lly的帖子

我正在使用 Terraform 代码在 AWS 中创建 Secret。我的 Jenkins 管道将每 2 小时创建一次基础设施并销毁它。2 小时后重新创建基础设施后，AWS Secrets 不允许我再次重新创建并抛出以下错误。请建议。

Error: error creating Secrets Manager Secret: InvalidRequestException: You can't create this secret because a secret with this name is already scheduled for deletion.
    status code: 400, request id: e4f8cc85-29a4-46ff-911d-c5115716adc5

TF代码：-

resource "aws_secretsmanager_secret" "secret" {
  description         = "${var.environment}"
  kms_key_id          = "${data.aws_kms_key.sm.arn}"
  name                = "${var.environment}-airflow-secret"
}
resource "random_string" "rds_password" {
  length = 16
  special = true
}


resource "aws_secretsmanager_secret_version" "secret" {
  secret_id     = "${aws_secretsmanager_secret.secret.id}"
  secret_string = <<EOF
{
  "rds_password": …

我看到我的所有执行程序经常在我的Jenkins slave机器(Windows 2008 R2 SP2)中更改为Dead状态.

詹金斯诉.1.651.3

我重新启动了Jenkins服务器以及服务.

错误记录 -

Unexpected executor death
java.io.IOException: Failed to create a temporary file in /var/lib/jenkins/jobs/ABCD/jobs/EFGH/jobs/Build
    at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:68)
    at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:55)
    at hudson.util.TextFile.write(TextFile.java:118)
    at hudson.model.Job.saveNextBuildNumber(Job.java:293)
    at hudson.model.Job.assignBuildNumber(Job.java:351)
    at hudson.model.Run.<init>(Run.java:284)
    at hudson.model.AbstractBuild.<init>(AbstractBuild.java:167)
    at hudson.model.Build.<init>(Build.java:92)
    at hudson.model.FreeStyleBuild.<init>(FreeStyleBuild.java:34)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:175)
    at hudson.model.AbstractProject.newBuild(AbstractProject.java:1018)
    at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1209)
    at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
    at hudson.model.Executor$1.call(Executor.java:364)
    at hudson.model.Executor$1.call(Executor.java:346)
    at hudson.model.Queue._withLock(Queue.java:1365)
    at hudson.model.Queue.withLock(Queue.java:1230)
    at hudson.model.Executor.run(Executor.java:346)
Caused by: java.io.IOException: Permission denied
    at java.io.UnixFileSystem.createFileExclusively(Native Method)
    at java.io.File.createNewFile(File.java:1006)
    at java.io.File.createTempFile(File.java:1989)
    at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:66) …

我想删除Jenkins下的Views而不影响视图下的Jobs.我问这个为什么,因为即使在以管理员身份删除之后我也无法输入相同的View名称.

我检查了Jenkins文件夹下的Config文件,并尝试编辑View名称,但这不起作用.

我需要确认以下脚本是仅删除视图名称还是删除视图下的作业.

Jenkins.instance.getView("MyView").items.each { item ->
    println "deleting $item.name"
    item.delete()
}

尝试在多模块Java项目中执行以下maven命令时，观察到以下错误。

-DargLine="-DDB_SERVER=localhost -DDB_PORT=1234 -DDB_USER=Project -DDB_PASSWORD=password -DDB_NAME=account -DDB_MAX_POOL=10" clean install

错误日志

除最后一个错误以下的模块外，所有其他模块均已成功运行。无法追踪问题的确切位置。

[INFO] --- maven-failsafe-plugin:2.20:verify (integration-tests) @ module ---
New build name is '22-08-2017-2396'
channel stopped
[ERROR] There are test failures.
Variable with name 'BUILD_DISPLAY_NAME' already exists, current value: '22-08-2017-2396', new value: '22-08-2017-2396'

Please refer to /Users/sonata-project/.jenkins/jobs/project/jobs/account/jobs/account-Build/workspace/module/target/failsafe-reports for the individual test results.
Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Build was aborted …

terraform init升级到0.12.2版本后执行命令时出现以下错误。早期的相同代码在 terraform0.11.10版本中运行良好，没有问题。

alb.tf

tags {
    Name             = "${var.name}"
    Environment      = "${lookup(var.environment, terraform.workspace)}"
  }

lc.tf：-

  vars {
EFS_ENDPOINT = "${aws_efs_file_system.jenkins.dns_name}"

}

remote_state.tf 第 4 行

config {
    bucket      =   "s3-terraform-state"
    key         =   "env:/${lookup(var.environment, terraform.workspace)}/cicd-security-groups/terraform.tfstate"
    region      =   "${var.region}"
  }

remote_state.tf 第 14 行

config {
    bucket      =   "ttgsl-s3-terraform-state"
    key         =   "env:/${lookup(var.environment, terraform.workspace)}/cicd-kms-ebs/terraform.tfstate"
    region      =   "${var.region}"
  }

地形验证-无颜色

Error: Unsupported block type

  on alb.tf line 40, in resource "aws_lb_target_group" "jenkins_master":
  40:   tags {

Blocks of type "tags" …

在执行时terraform plan我没有错误，但是当我执行时terraform apply我收到以下错误。

地形计划输出：-

+ aws_route53_record.alm_route_record1
      id:                                        <computed>
      allow_overwrite:                           <computed>
      fqdn:                                      <computed>
      name:                                      "${aws_acm_certificate.acm.domain_validation_options.0.resource_record_name}"
      records.#:                                 <computed>
      ttl:                                       "300"
      type:                                      "${aws_acm_certificate.acm.domain_validation_options.0.resource_record_type}"
      zone_id:                                   "Z2J2U5QM63SJ94"

文件

resource "aws_acm_certificate" "acm" {
  domain_name       = “example.com”
  validation_method = "DNS"

  tags = {
    Environment   = "${lookup(var.environment, terraform.workspace)}"
    ManagedBy     = "terraform"
  }
}

data "aws_route53_zone" "route_zone" {
  name         = "example.org."
  private_zone = false
}

resource "aws_route53_record" "alm_route_record1" {
  name    = "${aws_acm_certificate.acm.domain_validation_options.0.resource_record_name}"
  type    = "${aws_acm_certificate.acm.domain_validation_options.0.resource_record_type}"
  zone_id = "${data.aws_route53_zone.route_zone.id}"
  records = ["${aws_acm_certificate.acm.domain_validation_options.0.resource_record_value}"]
  ttl     = 300
} …

如何在EMR主群集（由Terraform创建）和Airflow之间建立连接。我在具有相同SG，VPC和子网的AWS EC2服务器下设置了Airflow。

我需要解决方案，以便Airflow可以与EMR对话并执行Spark提交。

https://aws.amazon.com/blogs/big-data/build-a-concurrent-data-orchestration-pipeline-using-amazon-emr-and-apache-livy/

这些博客对建立连接后的执行情况有一定的了解。

在气流中，我已经使用用于AWS和EMR的UI建立了连接：

下面的代码将列出处于活动状态和已终止的EMR群集，我也可以进行微调以获得活动群集：-

from airflow.contrib.hooks.aws_hook import AwsHook
import boto3
hook = AwsHook(aws_conn_id=‘aws_default’)
    client = hook.get_client_type(‘emr’, ‘eu-central-1’)
    for x in a:
        print(x[‘Status’][‘State’],x[‘Name’])

我的问题是-如何更新我的上述代码可以执行Spark提交操作

我有一个如下所示的 Jenkinsfile-A，在这个文件中，我需要在执行所有阶段后调用另一个 Jenkinsfile-B。

Jenkinsfile-A

#!groovy\xe2\x80\x8b\npipeline {\n    agent { label "" }\n    triggers { pollSCM(\'*/5 * * * *\') }\n     stages {\n        stage(\'Build Artifact\') {\n            steps {\n            }\n        }\n        stage(\'Publish Artifact\') {\n            steps {\n            }\n        }\n     }//stages\n     post {\n        always {\n            deleteDir() /* clean up our workspace */\n        }\n    }\n}//pipeline\ndef jenkinsFile\nstage(\'Loading Jenkinsfile\'){\n\n  jenkinsFile = fileLoader.fromGit(\'Jenkinsfile\', \'git@bitbucket.org:org-us-dev/org-cicd-pipelines.git\', \'master\', \'org-service-user\', \'ORG\')\n}\njenkinsFile.start()\n

Jenkinsfile-B

#!groovy\nvoid Download_Repositories(){\n}\nvoid SonarQube_Analysis(){\n}\nvoid Deploy_And_Test(){\n}\nvoid Prepare_Sonar_Scanner(){\n}\nvoid Publish_Artifact(){\n}\nvoid Host_Verification(){\n}\npipeline {\n    agent { label \'\' }\n//    triggers { pollSCM(\'*/5 * …

我有一个名为 ***.zip 的 zip 文件。我使用下面的命令来解压缩它。一旦我解压，里面的文件也是“Zip”文件（超过3个zip文件）。您能否让我知道如何解压这些文件？

unzip zipFile: "$project_version",dir:"D:\\jenkins\\DEV\\extract\\project", quiet: true

尝试做-

unzip dir: 'D:\\jenkins\\DEV\\extract\\project', glob: '', zipFile: 'D:\\jenkins\\DEV\\extract\\project\\project_*.zip'

错误日志

java.io.IOException: D:\jenkins\DEV\extract\project\project_*.zip does not exist.
    at org.jenkinsci.plugins.pipeline.utility.steps.zip.UnZipStepExecution.run(UnZipStepExecution.java:77)
    at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
    at hudson.security.ACL.impersonate(ACL.java:260)
    at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Finished: FAILURE

解压主 zip 文件后我的目录下的文件。

05/16/2018  04:31 PM    <DIR>          .
05/16/2018  04:31 PM    <DIR>          ..
05/15/2018  12:51 PM           265,637 project-project1_1.0.0.24_bdd86e0c.zip
05/15/2018  12:51 PM         7,924,188 project-project2_1.4.0.130_43dce5e4.zip
05/15/2018  12:51 PM         6,862,842 project-project3_1.0.0.207_c7d5d471.zip
               3 File(s)     15,052,667 bytes
               2 Dir(s)  432,451,330,048 bytes …

我有以下PowerShell脚本

Function Publish
{
    Param(
        [parameter(Mandatory=$true)]
        [String]
        $RELEASEDIR,

        [parameter(Mandatory=$true)]
        [String]
        $SERVICENAME,

        [parameter(Mandatory=$true)]
        [String]
        $SERVER
    )

    Get-ChildItem "$RELEASEDIR\*"
    $service = Get-Service -Name $SERVICENAME -Computername $SERVER -ErrorAction SilentlyContinue
    $service.Status
}
Publish

我是如何执行这个:

PS C:\Release\RPCPS> .\RPCPublish.ps1 -RELEASEDIR "C:\Location" -SERVICENAME "value" -SERVER "server"
cmdlet Publish at command pipeline position 1
Supply values for the following parameters:
RELEASEDIR:

即使在执行时传递参数,脚本也会再次期待它.我在这做错了什么？

使用terraform，我能够创建启用KMS加密的S3存储桶。但是，当我尝试从启用了S3 KMS的存储桶中下载任何文件时，它无法下载Access Denied

错误日志：

download failed: s3://services-1234567890-cicd-storage/jars/jdbc-0.211.jar to utilities/jdbc-0.211.jar An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

主文件

resource "aws_s3_bucket" "s3_bucket_two" {
  bucket = "dev-analytics-data"
#  bucket = "services-${lookup(var.aws_account_id, terraform.workspace)}-cicd-storage"
  acl    = "${var.acl}"
  versioning {
    enabled = "${var.enable_versioning}"
  }
  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        kms_master_key_id = "${data.terraform_remote_state.kms_s3.key_arn}"
        sse_algorithm     = "aws:kms"
      }
    }
  }
}

使用的IAM政策：-

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::12345678910:role/iam_role_devops_engineer",
                    "arn:aws:iam:: 12345678910:role/EMR_AutoScaling_DefaultRole",
                    "arn:aws:iam:: 12345678910:role/EMR_DefaultRole",
                    "arn:aws:iam:: 12345678910:user/iam_user_cng_jenkins",
                    "arn:aws:iam:: …

任何人都可以为我提供比下面更好的命令

rsync -e "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -l $USER `cat commandResult` "ansible-playbook --connection=local /home/ec2-user/ansible/plays/install.yml"