使用commons-net重试上传到FTP的正确方法是什么？

Question

虽然我用ftpClient.setControlKeepAliveTimeout(CONTROL_CONNECTION_KEEP_ALIVE_INTERVAL_SECONDS)发送保持活动请求,但我在处理服务器问题时会关闭我的FTP控制连接.有时它只是死于SocketException而不是正常的FTPConnectionClosedException.总而言之,FTP是一个非常狡猾的协议我正在使用很多,我连接的每个服务器都需要一些调整,但是这个很难.

我知道有一百万件事我可能做错了,我的问题是,是否有一些解决方案,如果你失去了控制连接,已经在FTP上实现了重试,因为这不是令人震惊的事情(代理/防火墙有时只会随机丢失你的联系).或者是否有一些更优雅的方法来解决这个问题.

我有类似的东西.

public void store(File fileToUpload) throws IOException, InterruptedException {
    String filename = fileToUpload.getName();
    int retries = 0;
    while (true) {
        try {
            ftpClient.storeFile(filename, inputStreamFactory.getInputStream(fileToUpload));
        } catch (FTPConnectionClosedException | SocketException exception) {
            LOGGER.debug("Control connection lost uploading {}, continuing.", filename);
        }

        // This sleep is because there's an anti-malware in the servers which makes the file not to appear
        // available immediately after an upload
        LOGGER.debug("Waiting {} milliseconds for anti-malware protection to process file", WAIT_AFTER_UPLOAD_MILLISECONDS);
        threadWrapper.sleep(WAIT_AFTER_UPLOAD_MILLISECONDS);
        if (!ftpClient.isConnected()) {
            connect();
        }

        LOGGER.debug("Checking if {} is already uploaded", filename);
        if (ftpFileChecker.isFileCompleted(listFiles(null), filename, fileToUpload.length())) {
            // Note this is likely to happen every time since their server will close the control
            // connection quite fast and FTPClient uses it at the end of storeFile
            LOGGER.debug("File {} was uploaded correctly", filename);
            break;
        } else {
            if (++retries > MAX_RETRIES) {
                throw new RemoteTimeoutException("Could not upload file, max retries exceeded");
            } else {
                LOGGER.info("File {} was not uploaded, retrying", filename);
            }
        }
    }
}

public void connect() throws IOException, InterruptedException {
    int retries = 0;
    while (true) {
        try {
            ftpClient = ftpClientFactory.createFtpClient();
            ftpClient.connect(server, FTP_PORT);
            if (!ftpClient.login(username, password)) {
                LOGGER.error("Login to FTP failed");
                throw new ConfigurationException("Login to FTP failed");
            }
            ftpClient.enterLocalPassiveMode();
            ftpClient.setControlKeepAliveTimeout(CONTROL_CONNECTION_KEEP_ALIVE_INTERVAL_SECONDS);
            ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
            ftpClient.changeWorkingDirectory(uploadDir);
            break;
        } catch (FTPConnectionClosedException | SocketException exception) {
            if (++retries > MAX_RETRIES) {
                throw new RemoteTimeoutException("Could not upload file, max retries exceeded", exception);
            } else {
                LOGGER.info("Could not login, retrying");
            }
        }
        LOGGER.debug("Sleeping {} milliseconds before trying to reconnect", WAIT_BETWEEN_CONNECT_RETRIES_MILLISECONDS);
        threadWrapper.sleep(WAIT_BETWEEN_CONNECT_RETRIES_MILLISECONDS);
    }
}

public FTPFile[] listFiles(String directory) throws IOException, InterruptedException {
    int retries = 0;
    while (true) {
        try {
            return ftpClient.listFiles(directory);
        } catch (FTPConnectionClosedException exception) {
            LOGGER.debug("Control connection lost when listing files, continuing");
        } catch (SocketException exception) {
            LOGGER.debug("Socket exception when listing files, continuing");
        }
        if (!ftpClient.isConnected()) {
            connect();
        }
        if (++retries > MAX_RETRIES) {
            throw new RemoteTimeoutException("Could not list files, max retries exceeded");
        } else {
            LOGGER.info("Could not list files, retrying");
        }
    }
}

现在我在listFiles上得到了SocketException Broken Pipe,我无法弄明白,因为在我的本地服务器中它工作得很完美,但在这个特定的一个(提示它在Windows NT上运行 - :( - 它运行一些恶意软件保护)这可以防止文件立即出现在服务器上,显然是在一些非常奇怪的防火墙后面,并且在大约5秒后它会丢弃空闲连接,并且它们不会改变配置,因为它们是一家大公司并声称它适用于其他所有人).

我尝试了VFS并调查了其他FTP客户端,但我发现似乎没有解决问题,甚至更无益,他们中的大多数(如ftp4j)不在maven中心,这真的让我不去尝试它们,除非有保证它会解决我的问题问题.

欢迎任何帮助.

编辑:给出的代码反映了这个的起始复杂性,我目前的解决方案更加稳定,复杂程度要高得多,但它根本不优雅,所以我保留问题,以防有人关心提供一个很好的解决方案.

Answer 1

如果使用 Spring，则考虑 Spring Retry。我相信最新的 Maven 版本是：

    <dependency>
        <groupId>org.springframework.retry</groupId>
        <artifactId>spring-retry</artifactId>
        <version>1.1.2.RELEASE</version>
    </dependency>