使用 Apache Camel 测试 Localstack AWS

Question

我正在尝试利用localstack进行本地 AWS 模拟测试。具体来说，我想使用 Apache Camel S3 路由从 localstack S3 存储桶下载文件。但是，我遇到了错误。以下是我采取的步骤...

我通过输入创建了一个测试 localstack S3 存储桶...

aws --endpoint-url=http://localhost:4572 s3 mb s3://mytestbucket

然后，我将测试文件上传到此存储桶...

aws --endpoint-url=http://localhost:4572 s3 cp docker-compose.yml s3://mytestbucket

两个操作都成功完成。在我的 Java 代码中，我创建了一个 CDI Producer 来生成一个 com.amazonaws.services.s3.AmazonS3 客户端对象......

AWSCredentials credentials = new BasicAWSCredentials("XXXXX", "XXXXX");
AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(credentials);
AmazonS3ClientBuilder clientBuilder = AmazonS3ClientBuilder.standard()
                .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("http://localhost:4572", "us-east-1"))
                .withCredentials(credentialsProvider);

AmazonS3 s3Client = clientBuilder.build();  

最后，我创建了一个 Camel 路由，旨在将 mytestbucket 的内容下载到本地目录中......

<route id="s3test">
        <from
            uri="aws-s3://arn:aws:s3:::mytestbucket?amazonS3Client=#s3Client&amp;deleteAfterRead=false&amp;maxMessagesPerPoll=25&amp;delay=5000" />
        <log message="S3 consumer fired" loggingLevel="DEBUG" />
        <log message="Sending S3 file to ${in.header.CamelAwsS3Key}..." />
        <to
            uri="file://tmp/camel?fileName=${in.header.CamelAwsS3Key}" />
    </route>

但是，当我的应用程序启动并尝试运行该路由时，出现以下异常：

引起：org.jboss.weld.exceptions.DeploymentException：无法创建路由s3test：Route(s3test)[[From[aws-s3://arn:aws:s3:::mytestbucket?amazo...因为无法执行 HTTP 请求：mytestbucket.localhost at org.jboss.weld.bootstrap.events.AbstractDeploymentContainerEvent.fire(AbstractDeploymentContainerEvent.java:38) at org.jboss.weld.bootstrap.events.AfterDeploymentValidationImpl.fire(AfterDeploymentValidationImpl.java:28)在 org.jboss.weld.bootstrap.WeldStartup.validateBeans(WeldStartup.java:499) 在 org.jboss.weld.bootstrap.WeldBootstrap.validateBeans(WeldBootstrap.java:93) 在 org.jboss.as.weld.WeldStartService.start (WeldStartService.java:97) 在 org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1736) 在 org.jboss.msc.service。ServiceControllerImpl$StartTask.execute(ServiceControllerImpl.java:1698) ... 6 导致：org.apache.camel.FailedToCreateRouteException：无法创建路由 s3test：Route(s3test)[[From[aws-s3://arn: aws:s3:::mytestbucket?amazo... 因为无法执行 HTTP 请求：mytestbucket.localhost at org.apache.camel.impl.RouteService.warmUp(RouteService.java:147) at org.apache.camel.impl .DefaultCamelContext.doWarmUpRoutes(DefaultCamelContext.java:3949) at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:3856) at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:3642) org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:3494) 在 org.apache.camel.impl.DefaultCamelContext。access$000(DefaultCamelContext.java:209) at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:3253) at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:3249) at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:3272) at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:3249) at org.apache.camel.support.ServiceSupport.start( ServiceSupport.java:61) at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:3165) at org.apache.camel.impl.DefaultCamelContext$Proxy$_$$_WeldClientProxy.start(Unknown Source) at org .apache.camel.cdi.CdiCamelExtension.afterDeploymentValidation(CdiCamelExtension.java:425) 在 sun.reflect.NativeMethodAccessorImpl。invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java: 498) 在 org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:95) 在 org.jboss.weld.injection.MethodInvocationStrategy$SpecialParamPlusBeanManagerStrategy.invoke(MethodInvocationStrategy.java:144) 在 org.weld.events .ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:330) 在 org.jboss.weld.event.ExtensionObserverMethodImpl.sendEvent(ExtensionObserverMethodImpl.java:123) 在 org.jboss.weld.event.ObserverMethodImpl.sendImplEvent(ObserverMethodImpl.sendImpl8)at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:286) 在 org.jboss.weld.util.Observers.notify(Observers.java:172) 在 org.jboss.weld.event.ObserverNotifier.notifySyncObservers(ObserverNotifier.java:285) 在 org.jboss.weld.event .ObserverNotifier.notify(ObserverNotifier.java:273) at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:177) at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:171) at org.jboss.weld.bootstrap.events.AbstractContainerEvent.fire(AbstractContainerEvent.java:53) at org.jboss.weld.bootstrap.events.AbstractDeploymentContainerEvent.fire(AbstractDeploymentContainerEvent.java:35) ... 12 更多 引起：com .amazonaws.SdkClientException：无法执行 HTTP 请求：在 com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1114) 在 com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1064) 在 com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) 在 com.amazonaws.http.AmazonHttpClient $RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)在 com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) 在 com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) 在 com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client) .java:4330) 在 com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4277) 在 com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4271) 在 com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:835) 在 org.apache.camel.component.aws .s3.S3Endpoint.doStart(S3Endpoint.java:112) at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61) at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:75) ) 在 org.apache.camel.impl.RouteService.doWarmUp(RouteService.java:157) 在 org.apache.camel.impl.RouteService.warmUp(RouteService.java:145) ... 42 更多 引起：java.net .UnknownHostException: mytestbucket.localhost at java.net.InetAddress.getAllByName0(InetAddress.java:1280) at java.net.InetAddress.getAllByName(InetAddress.java:1192) at java.net.InetAddress.getAllByName(InetAddress.java:1126) ) 在 com。amazonaws.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:27) at com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator).connect(DefaultHttpClientConnectionOperator)在 org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun .reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) ) 在 com.amazonaws。http.conn.$Proxy246.connect(Unknown Source) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec. java:236) 在 org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) 在 org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) 在 org.apache .http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) 在 org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) 在 com.amazonaws.http.apache.client.impl .SdkHttpClient.execute(SdkHttpClient.java:72) 在 com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1236) 在 com.amazonaws.http.AmazonHttpClient$RequestExecutor。executeHelper(AmazonHttpClient.java:1056) ... 57 更多

我认为，由于我的AmazonS3客户端明确指出这http://localhost:4572是我的端点，因此它不应该联系实际的 S3 服务。情况似乎是这样，但出于某种原因，它试图mytestbucket.localhost像访问网络主机一样进行访问。知道我在这里可能做错了什么，或者为什么这条路线/配置不起作用？

Answer 1

对于遇到此问题的其他任何人，答案在于创建这样的客户端：

AWSCredentials credentials = new BasicAWSCredentials("XXXXX", "YYYYY");
AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(credentials);
AmazonS3ClientBuilder clientBuilder = AmazonS3ClientBuilder.standard().withPathStyleAccessEnabled(true)
                    .disableChunkedEncoding()
                    .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(S3_ENDPOINT, "us-east-1"))
                    .withCredentials(credentialsProvider);

关键是.withPathStyledAccessEnabled(true)。添加这将正确生成端点 URL。