HttpServer上的java.lang.OutOfMemoryError下载大数据

Question

我有java 6嵌入式HttpServer.它有一个句柄,允许客户下载一个大文本文件.问题是,当服务器有10个以上的同时客户端时,我会遇到内存异常.我很清楚问题出在Http Server周围.

   HttpServer m_server = HttpServer.create(new InetSocketAddress(8080), 0);
   m_server.createContext("/DownloadFile", new DownloadFileHandler() );

   public class DownloadFileHandler implements HttpHandler {

         private static byte[] myFile = new String("....................").getBytes(); //string about 8M

         @Override
         public void handle(HttpExchange exchange) throws IOException {
                exchange.sendResponseHeaders(HTTP_OK, myFile .length);                 OutputStream responseBody = exchange.getResponseBody();
                responseBody.write(myFile );
                responseBody.close();
         } 
   }

现在我得到的例外是:

java.lang.OutOfMemoryError: Java heap space 
at java.nio.HeapByteBuffer.<init>(Unknown Source)
at java.nio.ByteBuffer.allocate(Unknown Source)
at sun.net.httpserver.Request$WriteStream.write(Unknown Source)
at sun.net.httpserver.FixedLengthOutputStream.write(Unknown Source) 
at java.io.FilterOutputStream.write(Unknown Source) 
at sun.net.httpserver.PlaceholderOutputStream.write(Unknown Source) 
at com.shunra.javadestination.webservices.DownloadFileHandler.handle(Unknown Source) 
at com.sun.net.httpserver.Filter$Chain.doFilter(Unknown Source) 
at sun.net.httpserver.AuthFilter.doFilter(Unknown Source) 
at com.sun.net.httpserver.Filter$Chain.doFilter(Unknown Source) 
at sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(Unknown Source) 
at com.sun.net.httpserver.Filter$Chain.doFilter(Unknown Source)
at sun.net.httpserver.ServerImpl$Exchange.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Exception in thread "pool-1-thread-24" java.lang.OutOfMemoryError: 

关于getBytes()的建议不会更改异常.我试图保持对byte []的静态引用,而不是每次都创建它.我仍然得到同样的例外.

Answer 1

不要为大文件这样做:

byte[] bytesToSend = myFile.getBytes();

这是低效的,您需要堆空间来存储整个文件数据.当你第一次完全读取文件然后完整地写它时,你浪费了大量的堆空间.

而是将文件数据以特定大小的块直接从文件读取/写入响应.您可以自己编写代码,也可以只使用IOUtilsApache Commons IO中的实用程序类.

在编写之前不要先读取整个文件是很重要的.而是用较小的块来做.在这里使用流并避免处理byte []的任何事情,除了缓冲和小块.

编辑:这是Apache IO的一些代码...

public static void main(String[] args) {
    HttpExchange exchange = ...;
    OutputStream responseBody = null;

    try {
        File file = new File("big-file.txt");
        long bytesToSkip = 4711; //detemine how many bytes to skip

        exchange.sendResponseHeaders(200, file.length() - bytesToSkip);
        responseBody = exchange.getResponseBody();
        skipAndCopy(file, responseBody, bytesToSkip);           
    }
    catch (IOException e) {
        // handle it
    }
    finally {
        IOUtils.closeQuietly(responseBody);
    }
}


private static void skipAndCopy(File src, @WillNotClose OutputStream dest, long bytesToSkip) throws IOException {
    InputStream in = null;

    try {
        in = FileUtils.openInputStream(src);

        IOUtils.skip(in, bytesToSkip);
        IOUtils.copyLarge(in, dest);
    }
    finally {
        IOUtils.closeQuietly(in);
    }
}

Answer 2

如果一次检索文件的所有字节,它必须将所有字节读入内存,然后将它们写入文件系统.尝试类似的东西:

FileReader reader = new FileReader(myFile);
try{
    char buffer[] = new char[4096];
    int numberOfBytes=0;
    while ((numberOfBytes=reader.read(buffer)) != -1){
        responseBody.write(buffer);
    }
}catch(Exception e){
    //TODO do something with the exception.
}finally{
    reader.close();
}