Spring boot Java堆空间用于下载大文件

Question

我正在尝试编写一个 REST api 来允许用户在 Spring boot 上下载大文件（即 > 2GB）。我遇到了“Java Heap outOfMemoryException”。我尝试对问题进行分类，我发现 HttpServetResponse 对象的类型为：ContentCachingResponseWrapper。此类缓存写入输出流的所有内容，当缓存的数据大小变为 258MB 左右时，我会收到 OutOfMemoryException。为什么是 248 MB，因为 JVM 有 256 MB 堆内存。

ContentCachingResponseWrapper 中默认的flushBuffer()方法是空的。如果我尝试调用 copyBodyToResponse()（用于将数据从缓存复制到流），它工作正常，但它也会关闭流。这导致仅将第一块数据发送到客户端。

有什么建议 ？

public void myDownloader(HttpServletRequest request, HttpServletResponse response) {
             //response.getClass() is: ContentCachingResponseWrapper  
             byte[] buffer = new byte[1048576];  // 1 MB Chunks
             FileInputStream inputStream = new FileInputStream(PATH_TO_SOME_VALID_FILE);
             int bytesRead= 0;
             ServletOutputStream outputStream = response.getOutputStream();

             while ((bytesRead = inputStream.read(buffer)) != -1) {              
                 outputStream.write(buffer, 0, bytesRead);
                 outputStream.flush();
                 response.flushBuffer();
               } 
}

我收到以下错误：

Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.springframework.util.FastByteArrayOutputStream.addBuffer(FastByteArrayOutputStream.java:303) ~[spring-core-5.2.8.RELEASE.jar!/:5.2.8.RELEASE]
    at org.springframework.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:118) ~[spring-core-5.2.8.RELEASE.jar!/:5.2.8.RELEASE]
    at org.springframework.web.util.ContentCachingResponseWrapper$ResponseServletOutputStream.write(ContentCachingResponseWrapper.java:239) ~[spring-web-5.2.8.RELEASE.jar!/:5.2.8.RELEASE]

Answer 1

这是下载文件的一种非常基本的方法，但是如果您从浏览器调用它，浏览器会将其显示在屏幕上，并且可能会永远旋转（如果您问我，则为浏览器问题）：

@RequestMapping(path = "/downloadLargeFile", method = RequestMethod.GET)
public ResponseEntity<Resource> downloadLargeFile() {
    final File file = new File("c:/large.bin");
    final FileSystemResource resource = new FileSystemResource(file);
    return ResponseEntity.ok().body(resource);
}

因此，您可以包含一些包含文件信息的标头，浏览器会将其下载到下载目录中的文件中，并且不会旋转：

@RequestMapping(path = "/downloadLargeFile2", method = RequestMethod.GET)
public ResponseEntity<Resource> downloadLargeFile2() {
    final HttpHeaders httpHeaders = new HttpHeaders();
    final File file = new File("c:/large.bin");
    final FileSystemResource resource = new FileSystemResource(file);
    httpHeaders.set(HttpHeaders.LAST_MODIFIED, String.valueOf(file.lastModified()));
    httpHeaders.set(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + file.getName() + "\"");
    httpHeaders.set(HttpHeaders.CONTENT_LENGTH, String.valueOf(file.length()));
    return ResponseEntity.ok()
        .headers(httpHeaders)
        .contentLength(file.length())
        .contentType(MediaType.APPLICATION_OCTET_STREAM)
        .body(resource);
}

要对响应进行分块，请使用InputStreamResource，并且显然Spring关闭了InputStream：

@RequestMapping(path = "/pub/storage/downloadLargeFile4", method = RequestMethod.GET)
public ResponseEntity<InputStreamResource> downloadLargeFile4()
    throws Exception {
    final HttpHeaders httpHeaders = new HttpHeaders();
    final File file = new File("c:/large.bin");
    final InputStream inputStream = new FileInputStream(file);
    final InputStreamResource resource = new InputStreamResource(inputStream);
    httpHeaders.set(HttpHeaders.LAST_MODIFIED, String.valueOf(file.lastModified()));
    httpHeaders.set(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + file.getName() + "\"");
    httpHeaders.set(HttpHeaders.CONTENT_LENGTH, String.valueOf(file.length()));
    return ResponseEntity.ok()
        .headers(httpHeaders)
        .contentLength(file.length())
        .contentType(MediaType.APPLICATION_OCTET_STREAM)
        .body(resource);
}

进口：

import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

我还发现这个线程很有用： download a file from Spring boot Rest service