将响应流从 gRPC 服务返回到 RESTful 客户端时出现问题

use*_*500 5 java rest spring-boot grpc-java

我正在尝试编写一组服务,为外部客户端提供 REST 接口,同时使用 gRPC/protobuf 进行内部服务间通信。Spring Boot 用于帮助提供后台基础设施支持。我大约一周前才开始学习和使用 Google protobuf 和 gRPC。

我已经能够成功地获得更简单的端到端工作处理,但这些返回单个对象(即没有流)。我在进行旨在流式传输响应对象集合的调用时遇到问题。另一件需要注意的事情是,到目前为止我尝试过的所有事情都是 GET 请求。

下面的示例仅显示了我遇到问题的相关部分。

细节:

Protobuf 定义:

syntax = "proto3";

package experiment;

message BOM {
    PartSpec part_spec = 1;
    string description = 2;
    string category = 3;
}

message PartSpec {
    string part_number = 1;
    string cage_code = 2;
}

service BOMService {
    rpc getBOMList(google.protobuf.Empty) returns (stream BOM) {}
}
Run Code Online (Sandbox Code Playgroud)

“域名”服务:

实现 .proto 文件定义的服务。响应来自“网关”服务的请求。

@GrpcService
public class BOMDomainService extends BOMServiceImplBase {

  private final Logger logger = LogManager.getLogger();

  private Random       rand   = new Random(1);

  @Override
  public void getBOMList(Empty request, StreamObserver<BOM> responseObserver) {
    for (int i = 1; i <= rand.nextInt(40); i++) {
      PartSpec part = PartSpec.newBuilder().setPartNumber("part-1").setCageCode("cage-1").build();
      BOM bom = BOM.newBuilder().setPartSpec(part).setDescription("some part").build();
      // Stream BOM response.
      responseObserver.onNext(bom);
    }

    // Complete server response.
    responseObserver.onCompleted();
  }
}
Run Code Online (Sandbox Code Playgroud)

“网关”

网关服务:

通过 gRPC 调用(阻塞存根)与“域”服务交互。

@Service
public class BOMGatewayService {

  private final Logger           logger = LogManager.getLogger();

  @GrpcClient("bom-service")
  private BOMServiceBlockingStub bomSvcStub;

  public List<BOM> getBOMList() {
    List<BOM> result = new ArrayList<>();

    try {
      Iterator<BOM> iter = bomSvcStub.getBOMList(Empty.getDefaultInstance());

      while (iter.hasNext()) {
        result.add(iter.next());
      }
    } catch (StatusRuntimeException e) {
      logger.warn("RPC failed: {}", e.getStatus());
    }

    return result;
  }
}
Run Code Online (Sandbox Code Playgroud)

网关控制器:

@RestController
@RequestMapping("/alisn/bom")
public class BOMController {

  @Autowired
  private BOMGatewayService service;

  @GetMapping(value = "/getBOMList", produces = { MediaType.APPLICATION_JSON_VALUE })
  public DeferredResult<ResponseEntity<List<BOM>>> getBOMList() {
    DeferredResult<ResponseEntity<List<BOM>>> result = new DeferredResult<>();
    List<BOM> response = service.getBOMList();
    result.setResult(ResponseEntity.ok(response));
    return result;
  }
}
Run Code Online (Sandbox Code Playgroud)

外部“客户”服务:

通过 REST 调用与“网关”服务交互。不使用 gRPC,但使用从 .proto 文件生成的消息类。

注意:我并不是想在这里获取“BOM”列表。如果我尝试使用ResponseEntity<List<BOM>>,该getForEntity()方法会引发编译器错误。我不确定我应该在这里做什么,也许这就是问题的一部分。

@Service
public class ClientService {

  private final Logger logger = LogManager.getLogger();

  @Autowired
  private RestTemplate restTemplate;

  @Scheduled(fixedDelay = 5000)
  public void getBOMList() {
    ResponseEntity<BOM> response = restTemplate
        .getForEntity("http://localhost:7090/alisn/bom/getBOMList", BOM.class);
    logger.info("Received:\n{}", response.getBody());
  }
}
Run Code Online (Sandbox Code Playgroud)

我尝试通过两种方式来练习:

  1. 来自使用 aRestTemplate 进行 REST 调用的客户端 Java Spring Boot 服务(上面显示的“客户端”)。
  2. 从 Postman 发出 HTTP GET 请求。

两种方法的响应似乎或多或少相同。

Java客户端方法:

客户端控制台:

[2019-09-27 10:01:16.373] scheduling-1 ERROR: support.TaskUtils$LoggingErrorHandler:96 - Unexpected error occurred in scheduled task.
org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 null
    at org.springframework.web.client.HttpServerErrorException.create(HttpServerErrorException.java:79) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:124) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:102) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:778) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:736) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:670) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.client.RestTemplate.getForEntity(RestTemplate.java:338) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at edu.mit.ll.alisn.exp.client.service.ClientService.getBOMList(ClientService.java:26) ~[classes/:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_201]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_201]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_201]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
Run Code Online (Sandbox Code Playgroud)

“网关”控制台:

[2019-09-27 10:01:16.356] http-nio-7090-exec-3 ERROR: [/].[dispatcherServlet]:175 - Servlet.service() for servlet [dispatcherServlet] threw exception
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["defaultInstanceForType"])
    at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContentsUsing(CollectionSerializer.java:171) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:116) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1392) ~[jackson-databind-2.9.9.jar:2.9.9]
    at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[jackson-databind-2.9.9.jar:2.9.9]
    at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:287) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:103) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters(AbstractMessageConverterMethodProcessor.java:290) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.HttpEntityMethodProcessor.handleReturnValue(HttpEntityMethodProcessor.java:225) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:82) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:122) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882) ~[spring-webmvc-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114) ~[spring-boot-actuator-2.1.6.RELEASE.jar:2.1.6.RELEASE]
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104) ~[spring-boot-actuator-2.1.6.RELEASE.jar:2.1.6.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:109) ~[spring-web-5.1.8.RELEASE.jar:5.1.8.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.doDispatch(ApplicationDispatcher.java:633) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:601) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.AsyncContextImpl$AsyncRunnable.run(AsyncContextImpl.java:566) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.AsyncContextImpl.doInternalDispatch(AsyncContextImpl.java:355) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.catalina.connector.CoyoteAdapter.asyncDispatch(CoyoteAdapter.java:235) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProcessor.dispatch(AbstractProcessor.java:241) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:53) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:853) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1587) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.21.jar:9.0.21]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
Run Code Online (Sandbox Code Playgroud)

邮差:

{
    "timestamp": "2019-09-27T14:03:38.530+0000",
    "status": 500,
    "error": "Internal Server Error",
    "message": "Type definition error: [simple type, class com.google.protobuf.UnknownFieldSet]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM[\"unknownFields\"]->com.google.protobuf.UnknownFieldSet[\"defaultInstanceForType\"])",
    "path": "/alisn/bom/getBOMList"
}
Run Code Online (Sandbox Code Playgroud)

任何关于我做错了什么的建议,或者关于更好的方法来完成此类过程的建议将不胜感激。

更新

我还包括客户端和网关服务的 bean 配置。

客户:

@Configuration
@EnableScheduling
@ComponentScan(XXXXXX)
public class Configurer {
  @Bean
  ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufJsonFormatHttpMessageConverter();
  }

  @Bean
  RestTemplate restTemplate(ProtobufHttpMessageConverter hmc) {
    return new RestTemplate(Arrays.asList(hmc));
  }
}
Run Code Online (Sandbox Code Playgroud)

网关:

@Configuration
@ComponentScan(XXXXXX)
public class Configurer {
  @Bean
  ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufJsonFormatHttpMessageConverter();
  }
}
Run Code Online (Sandbox Code Playgroud)

更新

我做了一个更简单的实验,创建了一项仅返回一个BOM对象的新服务。这很好用。

我不明白为什么这有效,而List<BOM>案例却无效。抛出的异常中的信息似乎指向生成的BOM类中的内部结构。

网关控制台:

[2019-10-02 08:37:25.960] http-nio-7090-exec-3 ERROR: [/].[dispatcherServlet]:175 - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class com.google.protobuf.UnknownFieldSet$Parser]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])] with root cause
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.google.protobuf.UnknownFieldSet$Parser and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"])
Run Code Online (Sandbox Code Playgroud)

对我的引用java.util.ArrayList[0]->edu.mit.ll.alisn.exp.proto.BOM["unknownFields"]->com.google.protobuf.UnknownFieldSet["parserForType"]似乎表明类的内部结构存在问题BOM,但我可以BOM毫无问题地返回一个。

cre*_*oup 0

根据 Gateway 日志,似乎jackson在将 proto 转换为 json 时遇到问题。

你能试一下com.googlecode.protobuf.format.JsonFormat吗 ?我不是spring用户,但是应该有用于自定义序列化的API。或者,您可以使用 jackson 添加自定义序列化JsonFormat,但这听起来有点奇怪。