Spring Feign 不压缩响应

Question

我正在使用 spring feign 压缩请求和响应

在服务器端：

server:
  servlet:
    context-path: /api/v1/
  compression:
    enabled: true
    min-response-size: 1024

当我从 chrome 中点击 api 时，我看到它添加了 'Accept-Encoding': "gzip, deflate, br"

在客户端：

    server:
      port: 8192
      servlet:
        context-path: /api/demo



feign.compression.response.enabled: true

feign.client.config.default.loggerLevel: HEADERS

logging.level.com.example.feigndemo.ManagementApiService: DEBUG

eureka:
  client:
    enabled: false

management-api:
  ribbon:
    listOfServers: localhost:8080

当我看到传递的请求标头时，feign 正在传递两个标头。

Accept-Encoding: deflate
Accept-Encoding: gzip

gradle 文件

plugins {
        id 'org.springframework.boot' version '2.1.8.RELEASE'
        id 'io.spring.dependency-management' version '1.0.8.RELEASE'
        id 'java'
    }

    group = 'com.example'
    version = '0.0.1-SNAPSHOT'
    sourceCompatibility = '1.8'

    configurations {
        compileOnly {
            extendsFrom annotationProcessor
        }
    }

    repositories {
        mavenCentral()
    }

    ext {
        set('springCloudVersion', "Greenwich.SR2")
    }

    dependencies {
        implementation 'org.springframework.boot:spring-boot-starter-web'
        compile ('org.springframework.cloud:spring-cloud-starter-netflix-ribbon')
        compile('org.springframework.cloud:spring-cloud-starter-openfeign')
    // https://mvnrepository.com/artifact/io.github.openfeign/feign-httpclient
    // https://mvnrepository.com/artifact/io.github.openfeign/feign-httpclient
        //compile group: 'io.github.openfeign', name: 'feign-httpclient', version: '9.5.0'

        compileOnly 'org.projectlombok:lombok'
        annotationProcessor 'org.projectlombok:lombok'
        testImplementation 'org.springframework.boot:spring-boot-starter-test'
    }

    dependencyManagement {
        imports {
            mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
        }
    }

响应未压缩。我所看到的是 Spring feign 将“接受编码”作为两个不同的值发送

让我知道这里是否有问题

Answer 1

几周前我遇到了同样的问题，我开始知道没有富有成效/直接的方法来做到这一点。我也才知道，当@patan报道与弹簧社会问题@patan报道issue1和@patan报道issue2出现了为Tomcat端创建以尝试解决该问题（一票的问题链接）。Jetty 端也有一张与此相关的票（票链接）。一开始打算用github推荐的方法，后来才知道这个库已经合并到spring-cloud-openfeign-corejar下了org.springframework.cloud.openfeign.encoding包裹。然而，我们无法按预期实现压缩，并面临以下两个挑战：

1. 当我们能够通过设置佯压缩org.springframework.cloud.openfeign.encoding.FeignAcceptGzipEncodingInterceptor（代码链接）类增加了Accept-Encoding标头值gzip和deflate，但由于问题（票）Tomcat服务器不能把它解释为压缩信号的符号。作为解决方案，我们必须添加手动 Feign 解释器来覆盖
   FeignAcceptGzipEncodingInterceptor功能并连接标头。
2. Feign 的默认压缩设置在最简单的场景中完美地工作，但是当出现 feignClient calling microservice and that microservice calling another microservice through feign无法处理压缩响应的情况时，因为 Spring cloud open feign 解码器默认不解压响应（默认 spring open feign 解码器）最终结束与问题（问题链接）。所以我们必须自己编写解码器来实现解压。

我终于找到了基于各种可用资源的解决方案，因此只需按照 spring feign 压缩的步骤操作即可：

应用程序.yml

spring:
  http:
    encoding:
      enabled: true

#to enable server side compression
server:
  compression:
    enabled: true
    mime-types:
      - application/json
    min-response-size: 2048

#to enable feign side request/response compression
feign:
  httpclient:
    enabled: true
  compression:
    request:
      enabled: true
      mime-types:
        - application/json
      min-request-size: 2048
    response:
      enabled: true

注意：上述 feign 配置我默认启用对所有 feign 客户端的压缩。

自定义FeignDecoder

import feign.Response;
import feign.Util;
import feign.codec.Decoder;
import org.springframework.cloud.openfeign.encoding.HttpEncoding;

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.lang.reflect.Type;
import java.nio.charset.StandardCharsets;
import java.util.Collection;
import java.util.Objects;
import java.util.zip.GZIPInputStream;

public class CustomGZIPResponseDecoder implements Decoder {

    final Decoder delegate;

    public CustomGZIPResponseDecoder(Decoder delegate) {
        Objects.requireNonNull(delegate, "Decoder must not be null. ");
        this.delegate = delegate;
    }

    @Override
    public Object decode(Response response, Type type) throws IOException {
        Collection<String> values = response.headers().get(HttpEncoding.CONTENT_ENCODING_HEADER);
        if(Objects.nonNull(values) && !values.isEmpty() && values.contains(HttpEncoding.GZIP_ENCODING)){
            byte[] compressed = Util.toByteArray(response.body().asInputStream());
            if ((compressed == null) || (compressed.length == 0)) {
               return delegate.decode(response, type);
            }
            //decompression part
            //after decompress we are delegating the decompressed response to default 
            //decoder
            if (isCompressed(compressed)) {
                final StringBuilder output = new StringBuilder();
                final GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(compressed));
                final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8));
                String line;
                while ((line = bufferedReader.readLine()) != null) {
                    output.append(line);
                }
                Response uncompressedResponse = response.toBuilder().body(output.toString().getBytes()).build();
                return delegate.decode(uncompressedResponse, type);
            }else{
                return delegate.decode(response, type);
            }
        }else{
            return delegate.decode(response, type);
        }
    }

    private static boolean isCompressed(final byte[] compressed) {
        return (compressed[0] == (byte) (GZIPInputStream.GZIP_MAGIC)) && (compressed[1] == (byte) (GZIPInputStream.GZIP_MAGIC >> 8));
    }
}

Feign自定义配置

import feign.RequestInterceptor;
import feign.RequestTemplate;
import feign.optionals.OptionalDecoder;
import org.springframework.beans.factory.ObjectFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.http.HttpMessageConverters;
import org.springframework.cloud.openfeign.support.ResponseEntityDecoder;
import org.springframework.cloud.openfeign.support.SpringDecoder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class CustomFeignConfiguration {


    @Autowired
    private ObjectFactory<HttpMessageConverters> messageConverters;

    //concatenating headers because of https://github.com/spring-projects/spring-boot/issues/18176
    @Bean
    public RequestInterceptor gzipInterceptor() {
        return new RequestInterceptor() {
            @Override
            public void apply(RequestTemplate template) {
                template.header("Accept-Encoding", "gzip, deflate");
            }
        };
    }

    @Bean
    public CustomGZIPResponseDecoder customGZIPResponseDecoder() {
        OptionalDecoder feignDecoder = new OptionalDecoder(new ResponseEntityDecoder(new SpringDecoder(this.messageConverters)));
        return new CustomGZIPResponseDecoder(feignDecoder);
    }
}

其他提示

如果您打算仅使用feign-core库构建 CustomDecoder

import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.type.TypeFactory;
import feign.Response;
import feign.Util;
import feign.codec.DecodeException;
import feign.codec.Decoder;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.util.StringUtils;
import org.springframework.web.client.HttpMessageConverterExtractor;

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;
import java.lang.reflect.WildcardType;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedList;
import java.util.Map;
import java.util.Objects;
import java.util.zip.GZIPInputStream;

import static java.util.zip.GZIPInputStream.GZIP_MAGIC;

public class CustomGZIPResponseDecoder implements Decoder {

    private final Decoder delegate;

    public CustomGZIPResponseDecoder(Decoder delegate) {
        Objects.requireNonNull(delegate, "Decoder must not be null. ");
        this.delegate = delegate;
    }

    @Override
    public Object decode(Response response, Type type) throws IOException {
        Collection<String> values = response.headers().get("Content-Encoding");
        if (Objects.nonNull(values) && !values.isEmpty() && values.contains("gzip")) {
            byte[] compressed = Util.toByteArray(response.body().asInputStream());
            if ((compressed == null) || (compressed.length == 0)) {
                return delegate.decode(response, type);
            }
            if (isCompressed(compressed)) {
                Response uncompressedResponse = getDecompressedResponse(response, compressed);
                return getObject(type, uncompressedResponse);
            } else {
                return getObject(type, response);
            }
        } else {
            return getObject(type, response);
        }
    }

    private Object getObject(Type type, Response response) throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        if (response.status() == 404 || response.status() == 204)
            return Util.emptyValueOf(type);
        if (Objects.isNull(response.body()))
            return null;
        if (byte[].class.equals(type))
            return Util.toByteArray(response.body().asInputStream());
        if (isParameterizeHttpEntity(type)) {
            type = ((ParameterizedType) type).getActualTypeArguments()[0];
            if (type instanceof Class || type instanceof ParameterizedType
                    || type instanceof WildcardType) {
                @SuppressWarnings({"unchecked", "rawtypes"})
                HttpMessageConverterExtractor<?> extractor = new HttpMessageConverterExtractor(
                        type, Collections.singletonList(new MappingJackson2HttpMessageConverter(mapper)));
                Object decodedObject = extractor.extractData(new FeignResponseAdapter(response));
                return createResponse(decodedObject, response);
            }
            throw new DecodeException(HttpStatus.INTERNAL_SERVER_ERROR.value(),
                    "type is not an instance of Class or ParameterizedType: " + type);
        } else if (isHttpEntity(type)) {
            return delegate.decode(response, type);
        } else if (String.class.equals(type)) {
            String responseValue = Util.toString(response.body().asReader());
            return StringUtils.isEmpty(responseValue) ? Util.emptyValueOf(type) : responseValue;
        } else {
            String s = Util.toString(response.body().asReader());
            JavaType javaType = TypeFactory.defaultInstance().constructType(type);
            return !StringUtils.isEmpty(s) ? mapper.readValue(s, javaType) : Util.emptyValueOf(type);
        }
    }

    public static boolean isCompressed(final byte[] compressed) {
        return (compressed[0] == (byte) (GZIP_MAGIC)) && (compressed[1] == (byte) (GZIP_MAGIC >> 8));
    }

    public static Response getDecompressedResponse(Response response, byte[] compressed) throws IOException {
        final StringBuilder output = new StringBuilder();
        final GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(compressed));
        final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8));
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            output.append(line);
        }
        return response.toBuilder().body(output.toString().getBytes()).build();
    }

    public static String getDecompressedResponseAsString(byte[] compressed) throws IOException {
        final StringBuilder output = new StringBuilder();
        final GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(compressed));
        final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8));
        String line;
        while ((line = bufferedReader.readLine()) != null) {
            output.append(line);
        }
        return output.toString();
    }

    private boolean isParameterizeHttpEntity(Type type) {
        if (type instanceof ParameterizedType) {
            return isHttpEntity(((ParameterizedType) type).getRawType());
        }
        return false;
    }

    private boolean isHttpEntity(Type type) {
        if (type instanceof Class) {
            Class c = (Class) type;
            return HttpEntity.class.isAssignableFrom(c);
        }
        return false;
    }

    private <T> ResponseEntity<T> createResponse(Object instance, Response response) {

        MultiValueMap<String, String> headers = new LinkedMultiValueMap<>();
        for (String key : response.headers().keySet()) {
            headers.put(key, new LinkedList<>(response.headers().get(key)));
        }

        return new ResponseEntity<>((T) instance, headers, HttpStatus.valueOf(response
                .status()));
    }

    private class FeignResponseAdapter implements ClientHttpResponse {

        private final Response response;

        private FeignResponseAdapter(Response response) {
            this.response = response;
        }

        @Override
        public HttpStatus getStatusCode() throws IOException {
            return HttpStatus.valueOf(this.response.status());
        }

        @Override
        public int getRawStatusCode() throws IOException {
            return this.response.status();
        }

        @Override
        public String getStatusText() throws IOException {
            return this.response.reason();
        }

        @Override
        public void close() {
            try {
                this.response.body().close();
            } catch (IOException ex) {
                // Ignore exception on close...
            }
        }

        @Override
        public InputStream getBody() throws IOException {
            return this.response.body().asInputStream();
        }

        @Override
        public HttpHeaders getHeaders() {
            return getHttpHeaders(this.response.headers());
        }

        private HttpHeaders getHttpHeaders(Map<String, Collection<String>> headers) {
            HttpHeaders httpHeaders = new HttpHeaders();
            for (Map.Entry<String, Collection<String>> entry : headers.entrySet()) {
                httpHeaders.put(entry.getKey(), new ArrayList<>(entry.getValue()));
            }
            return httpHeaders;
        }
    }

}

如果您打算构建自己的 Feign 构建器，那么您可以像下面这样配置

 Feign.builder().decoder(new CustomGZIPResponseDecoder(new feign.optionals.OptionalDecoder(new feign.codec.StringDecoder())))
                 .target(SomeFeignClient.class, "someurl");

更新上述答案： 如果您打算更新 to 的依赖项版本，spring-cloud-openfeign-core请'org.springframework.cloud:spring-cloud-openfeign-core:2.2.5.RELEASE'注意FeignContentGzipEncodingAutoConfiguration 类中的以下更改。在FeignContentGzipEncodingAutoConfiguration类中，ConditionalOnProperty注解的签名从 更改 @ConditionalOnProperty("feign.compression.request.enabled", matchIfMissing = false)为@ConditionalOnProperty(value = "feign.compression.request.enabled")，因此默认情况下，FeignContentGzipEncodingInterceptor如果您feign.request.compression=true的环境中有应用程序属性，bean 将被注入到 spring 容器中，如果超出默认/配置的大小限制，则压缩请求正文。如果您的服务器没有处理压缩请求的机制，这会导致问题，在这种情况下，将属性添加/修改为feign.request.compression=false