我有6台服务器,每台都包含很多日志.我想通过rsync将这些日志放到hadoop fs上.现在我正在使用fuse和rsync直接写入熔丝安装的fs/mnt/hdfs.但是有一个大问题.大约一天后,保险丝守护程序占用5 GB的RAM,并且无法对安装的fs做任何事情.所以我必须重新安装保险丝,一切都很好,但只是一段时间.Rsync命令是
rsync --port=3360 -az --timeout=10 --contimeout=30 server_name::ap-rsync/archive /mnt/hdfs/logs
一段时间后,Rsync会生成错误消息:
rsync error: timeout in data send/receive (code 30) at io.c(137) [sender=3.0.7]
rsync: connection unexpectedly closed (498784 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(601) [receiver=3.0.7]
rsync: connection unexpectedly closed (498658 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(601) [generator=3.0.7]