如何使用 Terraform 创建连接到 S3 的 AWS Kinesis Firehose？

Question

我正在尝试使用该kinesis_firehose_delivery_stream资源创建一个具有Direct PUT源、无数据转换和extended_s3目标的 Kinesis Firehose。

\n

我修改了此示例中的代码（以删除 lambda 函数），所以现在它看起来像这样：

\n

terraform {\n  required_providers {\n    aws = {\n      source  = "hashicorp/aws"\n      version = "~> 3.0"\n    }\n  }\n}\n\nprovider "aws" {\n  region = "us-east-1"\n}\n\nresource "aws_s3_bucket" "bucket" {\n  bucket = "test-kinesis-destination-bucket"\n  acl    = "private"\n}\n\nresource "aws_kinesis_firehose_delivery_stream" "kinesis_event_stream" {\n  name        = "kinesis-test-stream"\n  destination = "extended_s3"\n\n  extended_s3_configuration {\n    role_arn   = aws_iam_role.firehose_role.arn\n    bucket_arn = aws_s3_bucket.bucket.arn\n    buffer_size = 1\n    buffer_interval = 60\n  }\n}\n\nresource "aws_iam_role" "firehose_role" {\n  name = "firehose_test_role"\n\n  assume_role_policy = <<EOF\n{\n  "Version": "2012-10-17",\n  "Statement": [\n    {\n      "Action": "sts:AssumeRole",\n      "Principal": {\n        "Service": "firehose.amazonaws.com"\n      },\n      "Effect": "Allow",\n      "Sid": ""\n    }\n  ]\n}\nEOF\n}\n

\n

Terraform 能够成功应用所有内容，但 Firehose 似乎无法写入 S3。

\n

我的 IAM 角色是否遗漏了某些内容？如果是这样我该如何解决它？

\n

编辑

\n

我已根据 @Marcin 的回答更新了我的 terraform 文件以更新 IAM 策略，以授予 Firehose 写入 S3 的权限。

\n

terraform {\n  required_providers {\n    aws = {\n      source  = "hashicorp/aws"\n      version = "~> 3.0"\n    }\n  }\n}\n\nprovider "aws" {\n  region = "us-west-2"\n}\n\nresource "aws_s3_bucket" "bucket" {\n  bucket = "test-kinesis-destination-bucket"\n  acl    = "private"\n}\n\nresource "aws_kinesis_firehose_delivery_stream" "kinesis_event_stream" {\n  name        = "kinesis-test-stream"\n  destination = "extended_s3"\n\n  extended_s3_configuration {\n    role_arn   = aws_iam_role.firehose_role.arn\n    bucket_arn = aws_s3_bucket.bucket.arn\n    buffer_size = 1\n    buffer_interval = 60\n  }\n}\n\nresource "aws_iam_role" "firehose_role" {\n  name = "firehose_test_role"\n\n  assume_role_policy = <<EOF\n{\n  "Version": "2012-10-17",  \n  "Statement":\n  [\n    {\n      "Action": "sts:AssumeRole",\n      "Principal": {\n        "Service": "firehose.amazonaws.com"\n      },\n      "Effect": "Allow",\n      "Sid": ""\n    }\n  ]\n}\nEOF\n\n  inline_policy {\n    name = "kinesis-s3-inline-policy"\n    policy = jsonencode({\n      Version = "2012-10-17"\n      Statement = [\n        {      \n          Effect = "Allow",      \n          Action = [\n            "s3:AbortMultipartUpload",\n            "s3:GetBucketLocation",\n            "s3:GetObject",\n            "s3:ListBucket",\n            "s3:ListBucketMultipartUploads",\n            "s3:PutObject"\n          ]      \n          Resource = [        \n            "arn:aws:s3:::test-kinesis-destination-bucket",\n            "arn:aws:s3:::test-kinesis-destination-bucket/*"            \n          ]    \n        },\n        {\n          Effect = "Allow"\n          Action = [\n            "kinesis:DescribeStream",\n            "kinesis:GetShardIterator",\n            "kinesis:GetRecords",\n            "kinesis:ListShards"\n          ]\n          Resource = aws_kinesis_firehose_delivery_stream.kinesis_event_stream.arn\n        }\n      ]\n    })\n  }\n}\n

\n

但是当我运行时，terraform plan我收到以下错误：

\n

terraform {\n  required_providers {\n    aws = {\n      source  = "hashicorp/aws"\n      version = "~> 3.0"\n    }\n  }\n}\n\nprovider "aws" {\n  region = "us-east-1"\n}\n\nresource "aws_s3_bucket" "bucket" {\n  bucket = "test-kinesis-destination-bucket"\n  acl    = "private"\n}\n\nresource "aws_kinesis_firehose_delivery_stream" "kinesis_event_stream" {\n  name        = "kinesis-test-stream"\n  destination = "extended_s3"\n\n  extended_s3_configuration {\n    role_arn   = aws_iam_role.firehose_role.arn\n    bucket_arn = aws_s3_bucket.bucket.arn\n    buffer_size = 1\n    buffer_interval = 60\n  }\n}\n\nresource "aws_iam_role" "firehose_role" {\n  name = "firehose_test_role"\n\n  assume_role_policy = <<EOF\n{\n  "Version": "2012-10-17",\n  "Statement": [\n    {\n      "Action": "sts:AssumeRole",\n      "Principal": {\n        "Service": "firehose.amazonaws.com"\n      },\n      "Effect": "Allow",\n      "Sid": ""\n    }\n  ]\n}\nEOF\n}\n

\n

如何在其 IAM 策略中引用 Firehose 的 ARN？

\n

Answer 1

您创建的角色firehose_role仅具有信任关系，但没有实际的 S3 权限。您的角色应具有文档中所述的以下权限（如果您不将 lambda 与 kinesis、kms 或 firehose 可以使用的其他服务一起使用，则可以缩减权限）：

{
    "Version": "2012-10-17",  
    "Statement":
    [    
        {      
            "Effect": "Allow",      
            "Action": [
                "s3:AbortMultipartUpload",
                "s3:GetBucketLocation",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:PutObject"
            ],      
            "Resource": [        
                "arn:aws:s3:::bucket-name",
                "arn:aws:s3:::bucket-name/*"            
            ]    
        },        
        {
            "Effect": "Allow",
            "Action": [
                "kinesis:DescribeStream",
                "kinesis:GetShardIterator",
                "kinesis:GetRecords",
                "kinesis:ListShards"
            ],
            "Resource": "arn:aws:kinesis:region:account-id:stream/stream-name"
        },
        {
           "Effect": "Allow",
           "Action": [
               "kms:Decrypt",
               "kms:GenerateDataKey"
           ],
           "Resource": [
               "arn:aws:kms:region:account-id:key/key-id"           
           ],
           "Condition": {
               "StringEquals": {
                   "kms:ViaService": "s3.region.amazonaws.com"
               },
               "StringLike": {
                   "kms:EncryptionContext:aws:s3:arn": "arn:aws:s3:::bucket-name/prefix*"
               }
           }
        },
        {
           "Effect": "Allow",
           "Action": [
               "logs:PutLogEvents"
           ],
           "Resource": [
               "arn:aws:logs:region:account-id:log-group:log-group-name:log-stream:log-stream-name"
           ]
        },
        {
           "Effect": "Allow", 
           "Action": [
               "lambda:InvokeFunction", 
               "lambda:GetFunctionConfiguration" 
           ],
           "Resource": [
               "arn:aws:lambda:region:account-id:function:function-name:function-version"
           ]
        }
    ]
}