EventBridge 触发器：Sagemaker 处理作业已完成

Question

我目前正在使用 AWS 为我的 ML 模型开发一些 ETL。问题是我想在某些 Sagemaker 处理作业完成时触发Lambda。传递给 Lambda的事件应该是 Sagemaker 处理作业的配置信息（作业名称、参数等）。

Q1：如何在处理作业完成时触发事件？

问题 2：如何将处理作业配置作为Lambda 的事件传递？

Answer 1

您可以使用以下 EventBridge 规则模式：

{
  "source": ["aws.sagemaker"],
  "detail-type": ["SageMaker Processing Job State Change"],
  "detail": {
    "ProcessingJobStatus": ["Failed", "Completed", "Stopped"]
  }
}

可以根据您要处理的状态修改ProcessingJobStatus 列表。

您可以将 Lambda 函数设置为 EventBridge 规则的目标。

以下是一个从 AWS 控制台获取的示例事件，该事件将传递给您的 Lambda：

{
  "version": "0",
  "id": "0a15f67d-aa23-0123-0123-01a23w89r01t",
  "detail-type": "SageMaker Processing Job State Change",
  "source": "aws.sagemaker",
  "account": "123456789012",
  "time": "2019-05-31T21:49:54Z",
  "region": "us-east-1",
  "resources": ["arn:aws:sagemaker:us-west-2:012345678987:processing-job/integ-test-analytics-algo-54ee3282-5899-4aa3-afc2-7ce1d02"],
  "detail": {
    "ProcessingInputs": [{
      "InputName": "InputName",
      "S3Input": {
        "S3Uri": "s3://input/s3/uri",
        "LocalPath": "/opt/ml/processing/input/local/path",
        "S3DataType": "MANIFEST_FILE",
        "S3InputMode": "PIPE",
        "S3DataDistributionType": "FULLYREPLICATED"
      }
    }],
    "ProcessingOutputConfig": {
      "Outputs": [{
        "OutputName": "OutputName",
        "S3Output": {
          "S3Uri": "s3://output/s3/uri",
          "LocalPath": "/opt/ml/processing/output/local/path",
          "S3UploadMode": "CONTINUOUS"
        }
      }],
      "KmsKeyId": "KmsKeyId"
    },
    "ProcessingJobName": "integ-test-analytics-algo-54ee3282-5899-4aa3-afc2-7ce1d02",
    "ProcessingResources": {
      "ClusterConfig": {
        "InstanceCount": 3,
        "InstanceType": "ml.c5.xlarge",
        "VolumeSizeInGB": 5,
        "VolumeKmsKeyId": "VolumeKmsKeyId"
      }
    },
    "StoppingCondition": {
      "MaxRuntimeInSeconds": 2000
    },
    "AppSpecification": {
      "ImageUri": "012345678901.dkr.ecr.us-west-2.amazonaws.com/processing-uri:latest"
    },
    "NetworkConfig": {
      "EnableInterContainerTrafficEncryption": true,
      "EnableNetworkIsolation": false,
      "VpcConfig": {
        "SecurityGroupIds": ["SecurityGroupId1", "SecurityGroupId2", "SecurityGroupId3"],
        "Subnets": ["Subnet1", "Subnet2"]
      }
    },
    "RoleArn": "arn:aws:iam::012345678987:role/SageMakerPowerUser",
    "ExperimentConfig": {},
    "ProcessingJobArn": "arn:aws:sagemaker:us-west-2:012345678987:processing-job/integ-test-analytics-algo-54ee3282-5899-4aa3-afc2-7ce1d02",
    "ProcessingJobStatus": "Completed",
    "LastModifiedTime": 1589879735000,
    "CreationTime": 1589879735000
  }
}

编辑：

如果你想匹配带有特定前缀的ProcessingJobName：

{
  "source": ["aws.sagemaker"],
  "detail-type": ["SageMaker Processing Job State Change"],
  "detail": {
    "ProcessingJobStatus": ["Failed", "Completed", "Stopped"],
    "ProcessingJobName": [{
      "prefix": "standarize-data"
    }]
  }
}