Spring Cloud AWS SQS 无法在本地连接到服务端点

Question

我正在尝试在我正在处理的项目中使用 Spring Cloud 的 AWS SQS。目前，我仅在我的开发机器上本地运行该应用程序。因此，我想要的是连接到 AWS 上的 SQS，而不必将我的应用程序部署到 EC2 实例。

但是，似乎 Spring Cloud 的 AWS 包中使用的 AWS SDK 将尝试通过元数据进行身份验证并希望解析 169.254.169.254/latest/meta-data/instance-id. 由于我仍在本地运行应用程序，因此无法解析端点并引发错误：

2019-12-29 16:38:27.420  WARN 22462 --- [  restartedMain] com.amazonaws.util.EC2MetadataUtils      : Unable to retrieve the requested metadata (/latest/meta-data/instance-id). Failed to connect to service endpoint: 

com.amazonaws.SdkClientException: Failed to connect to service endpoint: 
    at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100) ~[aws-java-sdk-core-1.11.699.jar:na]
    at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:70) ~[aws-java-sdk-core-1.11.699.jar:na]
    at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:75) ~[aws-java-sdk-core-1.11.699.jar:na]
    at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:62) ~[aws-java-sdk-core-1.11.699.jar:na]
    at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:400) ~[aws-java-sdk-core-1.11.699.jar:na]
    at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:369) ~[aws-java-sdk-core-1.11.699.jar:na]
    at org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils.isRunningOnCloudEnvironment(AwsCloudEnvironmentCheckUtils.java:38) ~[spring-cloud-aws-context-2.2.1.RELEASE.jar:2.2.1.RELEASE]
    at org.springframework.cloud.aws.context.annotation.OnAwsCloudEnvironmentCondition.matches(OnAwsCloudEnvironmentCondition.java:37) ~[spring-cloud-aws-context-2.2.1.RELEASE.jar:2.2.1.RELEASE]
    at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108) ~[spring-context-5.2.2.RELEASE.jar:5.2.2.RELEASE]
    at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:221) ~[spring-context-5.2.2.RELEASE.jar:5.2.2.RELEASE]
    at org.springframework.context.annotation.ConfigurationClassParser.processImports(ConfigurationClassParser.java:587) ~[spring-context-5.2.2.RELEASE.jar:5.2.2.RELEASE]
...

我试图在我的 bean 中显式提供一个 SQS 端点，但它仍然尝试连接到169.254.169.254导致上述错误：

public AmazonSQSAsync sqsClient() {
   EndpointConfiguration endpointConfig = new AwsClientBuilder.EndpointConfiguration(
        "sqs.us-east-1.amazonaws.com",
        "us-east-1"
   );

   return AmazonSQSAsyncClientBuilder
                .standard()
                .withEndpointConfiguration(endpointConfig)
                .withCredentials(new AWSStaticCredentialsProvider(new DefaultAWSCredentialsProviderChain().getCredentials()))
                .build();
}

虽然我不确定这一点，但我怀疑发生错误是因为我在我的开发机器上本地运行应用程序，所以它无法解析到端点。但我对此也不太确定，因为我正在使用 AWS 开发工具包在同一应用程序中运行其他 AWS 服务，并且我没有出现此错误。

我的 pom.xml 中有以下依赖项，似乎拥有其中任何一项都会导致错误发生。也就是说，我什至不必创建 bean 就可以出现该错误。由于某种原因，添加这些依赖项将立即导致 SDK 尝试解析该端点并因该错误而失败。

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-aws</artifactId>
            <version>2.2.1.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-aws-messaging</artifactId>
            <version>2.2.1.RELEASE</version>
        </dependency>

我还应该做些什么来修复它无法连接到服务端点的错误？

Answer 1

我知道很多人已经解释了问题所在以及如何跳过此错误。但是，以上所有方法对我来说都不起作用。最后，经过大量搜索，我设置了以下 aws 配置属性，它对我有用。

cloud.aws.region.use-default-aws-region-chain=true
cloud.aws.stack.auto=false
cloud.aws.credentials.access-key=
cloud.aws.credentials.secret-key=
cloud.aws.region.auto=false
cloud.aws.region.static=
cloud.aws.stack=false

logging.level.com.amazonaws.util.EC2MetadataUtils=error
logging.level.com.amazonaws.internal.InstanceMetadataServiceResourceFetcher=error

Answer 2

以下配置对我有用：

cloud:
  aws:
    region:
      auto: false
      static: eu-west-1
      use-default-aws-region-chain: true
    stack: false
    stack.auto: false
logging:
  level:
    com:
      amazonaws:
        internal:
          InstanceMetadataServiceResourceFetcher: error
        util:
          EC2MetadataUtils: error

Answer 3

修复在 application.yaml 文件中添加的问题：

logging:
  level:
    com:
      amazonaws:
        util:
          EC2MetadataUtils: error

Answer 4

为 AWS SNS 运行 Spring Boot 项目时收到以下错误

i.InstanceMetadataServiceResourceFetcher : Fail to retrieve token
com.amazonaws.util.EC2MetadataUtils      : Unable to retrieve the requested metadata (/latest/dynamic/instance-identity/document). Failed to connect to service endpoint:


Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'stackResourceRegistryFactoryBean' defined in class path resource [org/springframework/cloud/aws/autoconfigure/context/ContextStackAutoConfiguration.class]: Unsatisfied dependency expressed through method 'stackResourceRegistryFactoryBean' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amazonCloudFormation' defined in class path resource [org/springframework/cloud/aws/autoconfigure/context/ContextStackAutoConfiguration.class]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: There is no EC2 meta data available, because the application is not running in the EC2 environment. Region detection is only possible if the application is running on a EC2 instance

Error creating bean with name 'org.springframework.cloud.aws.core.env.ResourceIdResolver.BEAN_NAME': Invocation of init method failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'stackResourceRegistryFactoryBean' defined in class path resource [org/springframework/cloud/aws/autoconfigure/context/ContextStackAutoConfiguration.class]: Unsatisfied dependency expressed through method 'stackResourceRegistryFactoryBean' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amazonCloudFormation' defined in class path resource [org/springframework/cloud/aws/autoconfigure/context/ContextStackAutoConfiguration.class]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: There is no EC2 meta data available, because the application is not running in the EC2 environment. Region detection is only possible if the application is running on a EC2 instance

应用的解决方案：将以下代码片段添加到 SpringBootApplication 并单击 IDE 中的运行按钮

i.InstanceMetadataServiceResourceFetcher : Fail to retrieve token
com.amazonaws.util.EC2MetadataUtils      : Unable to retrieve the requested metadata (/latest/dynamic/instance-identity/document). Failed to connect to service endpoint:


Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'stackResourceRegistryFactoryBean' defined in class path resource [org/springframework/cloud/aws/autoconfigure/context/ContextStackAutoConfiguration.class]: Unsatisfied dependency expressed through method 'stackResourceRegistryFactoryBean' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amazonCloudFormation' defined in class path resource [org/springframework/cloud/aws/autoconfigure/context/ContextStackAutoConfiguration.class]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: There is no EC2 meta data available, because the application is not running in the EC2 environment. Region detection is only possible if the application is running on a EC2 instance

Error creating bean with name 'org.springframework.cloud.aws.core.env.ResourceIdResolver.BEAN_NAME': Invocation of init method failed; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'stackResourceRegistryFactoryBean' defined in class path resource [org/springframework/cloud/aws/autoconfigure/context/ContextStackAutoConfiguration.class]: Unsatisfied dependency expressed through method 'stackResourceRegistryFactoryBean' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amazonCloudFormation' defined in class path resource [org/springframework/cloud/aws/autoconfigure/context/ContextStackAutoConfiguration.class]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: There is no EC2 meta data available, because the application is not running in the EC2 environment. Region detection is only possible if the application is running on a EC2 instance

Answer 5

在 EC2 实例之外使用 AWS Java SDK 时，您可以使用System property或Environment variable禁用从 EC2 元数据实例服务加载凭证或区域。

• 系统属性： com.amazonaws.sdk.disableEc2Metadata
• 环境变量： AWS_EC2_METADATA_DISABLED

但是，您仍然会在应用程序启动时收到警告，它告诉您 EC2Metadata 已被配置禁用：“EC2 Instance Metadata Service is disabled”

2020-03-27 18:10:42.483  WARN 71123 --- [main] com.amazonaws.util.EC2MetadataUtils      : Unable to retrieve the requested metadata (/latest/meta-data/instance-id). EC2 Instance Metadata Service is disabled

com.amazonaws.AmazonClientException: EC2 Instance Metadata Service is disabled
    at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:65)
    at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)
    at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:402)
    at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:371)
    at org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils.isRunningOnCloudEnvironment(AwsCloudEnvironmentCheckUtils.java:38)
    ...

Answer 6

据我了解，它试图检查您的应用程序是否在云环境上运行，从最后 5行开始stacktrace

org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils.isRunningOnCloudEnvironment(AwsCloudEnvironmentCheckUtils.java:38)

我检查了repo，这是代码片段 -

public static boolean isRunningOnCloudEnvironment() {
    if (isCloudEnvironment == null) {
        try {
            isCloudEnvironment = EC2MetadataUtils
                    .getData(EC2_METADATA_ROOT + "/instance-id", 1) != null;
        }
        catch (AmazonClientException e) {
            isCloudEnvironment = false;
        }
    }
    return isCloudEnvironment;
}

根据代码片段，它尝试获取实例metadata，如果失败return false，则意味着您不在Cloud Environment. 然而，这段代码预计会捕获AmazonClientException，但您共享的堆栈跟踪正在抛出SdkClientException异常，这部分代码片段无法捕获异常catch，因此无法返回false您不在云环境中的信息。

似乎这个库的构建存在一些问题，根据您正在使用的堆栈跟踪，该堆栈跟踪仅在9 天2.2.1.RELEASE1前发布。您能否尝试使用旧版本之一，然后报告您是否仍然面临同样的问题。

或者作为临时修复，您可以尝试通过隧道传输到您的VPC's entry server. 如果需要，您可以使用sshuttle进行隧道传输。这将主要使该 IP 也可从本地计算机访问。