Java TCP套接字:数据传输速度很慢

Question

我设置了一个带有ServerSocket的服务器,用客户机连接到它.它们通过交换机直接联网,ping时间<1ms.

现在,我尝试通过套接字的输出流将"大量"数据从客户端推送到服务器.转移0.6Gb需要23分钟.我可以通过scp在几秒钟内推送一个更大的文件.

知道我可能做错了什么吗？我基本上只是循环并在套接字上调用writeInt.速度问题与数据来源无关,即使我只是发送一个常量整数而不是从磁盘读取.

我尝试将两侧的发送和接收缓冲区设置为4Mb,没有骰子.我为读写器使用缓冲流,没有骰子.

我错过了什么吗？

编辑:代码

这是我制作套接字的地方

System.out.println("Connecting to " + hostname);

    serverAddr = InetAddress.getByName(hostname);

    // connect and wait for port assignment
    Socket initialSock = new Socket();
    initialSock.connect(new InetSocketAddress(serverAddr, LDAMaster.LDA_MASTER_PORT));
    int newPort = LDAHelper.readConnectionForwardPacket(new DataInputStream(initialSock.getInputStream()));
    initialSock.close();
    initialSock = null;

    System.out.println("Forwarded to " + newPort);

    // got my new port, connect to it
    sock = new Socket();
    sock.setReceiveBufferSize(RECEIVE_BUFFER_SIZE);
    sock.setSendBufferSize(SEND_BUFFER_SIZE);
    sock.connect(new InetSocketAddress(serverAddr, newPort));

    System.out.println("Connected to " + hostname + ":" + newPort + " with buffers snd=" + sock.getSendBufferSize() + " rcv=" + sock.getReceiveBufferSize());

    // get the MD5s
    try {
        byte[] dataMd5 = LDAHelper.md5File(dataFile),
               indexMd5 = LDAHelper.md5File(indexFile);

        long freeSpace = 90210; // ** TODO: actually set this **

        output = new DataOutputStream(new BufferedOutputStream(sock.getOutputStream()));
        input  = new DataInputStream(new BufferedInputStream(sock.getInputStream()));

这是我在服务器端连接的地方:

    ServerSocket servSock = new ServerSocket();
    servSock.setSoTimeout(SO_TIMEOUT);
    servSock.setReuseAddress(true);
    servSock.bind(new InetSocketAddress(LDA_MASTER_PORT));

    int currPort = LDA_START_PORT;

    while (true) {
        try {
            Socket conn = servSock.accept();
            System.out.println("Got a connection.  Sending them to port " + currPort);
            clients.add(new MasterClientCommunicator(this, currPort));
            clients.get(clients.size()-1).start();

            Thread.sleep(500);

            LDAHelper.sendConnectionForwardPacket(new DataOutputStream(conn.getOutputStream()), currPort);

            currPort++;
        } catch (SocketTimeoutException e) {
            System.out.println("Done listening.  Dispatching instructions.");
            break;
        }
        catch (IOException e) {
            e.printStackTrace();
        }
        catch (Exception e) {
            e.printStackTrace();
        }
    }

好吧,这是我发送~0.6Gb数据的地方.

public static void sendTermDeltaPacket(DataOutputStream out, TIntIntHashMap[] termDelta) throws IOException {
    long bytesTransferred = 0, numZeros = 0;

    long start = System.currentTimeMillis();

    out.write(PACKET_TERM_DELTA); // header     
    out.flush();
    for (int z=0; z < termDelta.length; z++) {
        out.writeInt(termDelta[z].size()); // # of elements for each term
        bytesTransferred += 4;
    }

    for (int z=0; z < termDelta.length; z++) {
        for (int i=0; i < termDelta[z].size(); i++) {
            out.writeInt(1);
            out.writeInt(1);
        }
    }

到目前为止看起来非常简单......

Answer 1

你不希望在您传输大量数据写入单个字节.

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;

public class Transfer {

    public static void main(String[] args) {
        final String largeFile = "/home/dr/test.dat"; // REPLACE
        final int BUFFER_SIZE = 65536;
        new Thread(new Runnable() {
            public void run() {
                try {
                    ServerSocket serverSocket = new ServerSocket(12345);
                    Socket clientSocket = serverSocket.accept();
                    long startTime = System.currentTimeMillis();
                    byte[] buffer = new byte[BUFFER_SIZE];
                    int read;
                    int totalRead = 0;
                    InputStream clientInputStream = clientSocket.getInputStream();
                    while ((read = clientInputStream.read(buffer)) != -1) {
                        totalRead += read;
                    }
                    long endTime = System.currentTimeMillis();
                    System.out.println(totalRead + " bytes read in " + (endTime - startTime) + " ms.");
                } catch (IOException e) {
                }
            }
        }).start();
        new Thread(new Runnable() {
            public void run() {
                try {
                    Thread.sleep(1000);
                    Socket socket = new Socket("localhost", 12345);
                    FileInputStream fileInputStream = new FileInputStream(largeFile);
                    OutputStream socketOutputStream = socket.getOutputStream();
                    long startTime = System.currentTimeMillis();
                    byte[] buffer = new byte[BUFFER_SIZE];
                    int read;
                    int readTotal = 0;
                    while ((read = fileInputStream.read(buffer)) != -1) {
                        socketOutputStream.write(buffer, 0, read);
                        readTotal += read;
                    }
                    socketOutputStream.close();
                    fileInputStream.close();
                    socket.close();
                    long endTime = System.currentTimeMillis();
                    System.out.println(readTotal + " bytes written in " + (endTime - startTime) + " ms.");
                } catch (Exception e) {
                }
            }
        }).start();
    }
}

这会在我的机器上短时间内复制1 GiB数据.这里的关键是使用InputStream.read和OutputStream.write方法接受字节数组作为参数.缓冲区的大小并不重要,它应该比比如说大一点.5.尝试使用上面的BUFFER_SIZE来查看它如何影响速度但是请记住,对于你运行的每台机器来说它可能都不同这个节目.64 KiB似乎是一个很好的妥协.

Answer 2

嘿,我想我会跟进任何有兴趣的人.

这是故事中奇怪的道德:

永远不要使用DataInputStream/DataOutputStream和套接字!!

如果我将套接字包装在BufferedOutputStream/BufferedInputStream中,那么生活就很棒.写它原始就好了.

但是将套接字包装在DataInputStream/DataOutputStream中,或者甚至让DataOutputStream(BufferedOutputStream(sock.getOutputStream())极其缓慢.

对此的解释对我来说非常有趣.但在将所有内容交换进去之后,这就是最新情况.如果你不相信我自己尝试一下.

不过,谢谢你们所有的快速帮助.

Answer 3

也许你应该尝试以块(帧)发送ur数据,而不是单独写每个字节.并将帧与TCP数据包大小对齐以获得最佳性能.

Answer 4

您应该下载一个好的数据包嗅探器。我个人是WireShark的忠实粉丝，每次进行套接字编程时我都会使用它。请记住，您必须让客户端和服务器在不同的系统上运行才能接收任何数据包。