在本地更改zookeeper C客户端的日志级别

Question

我正在使用spark-shell带有mesos和zookeeper的Apache Spark ,这似乎运行良好,除了我从动物园管理员那里得到的记录比我想要的多,这有点让人分心:

Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.5.0
      /_/

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60)
Type in expressions to have them evaluated.
Type :help for more information.
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.6
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@716: Client environment:host.name=myhost
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@724: Client environment:os.arch=3.2.0-34-generic
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@725: Client environment:os.version=#53-Ubuntu SMP Thu Nov 15 10:48:16 UTC 2012
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@733: Client environment:user.name=myname
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=localhost:9999 sessionTimeout=10000 watcher=0xffffffffff sessionId=0 sessionPasswd=<null> context=0x777777777777 flags=0
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@check_events@1703: initiated connection to server [127.0.0.1:9999]
2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@check_events@1750: session establishment complete on server [127.0.0.1:9999], sessionId=0x11111111111111, negotiated timeout=10000
Spark context available as sc.
SQL context available as sqlContext.

scala>

我试过改变 $SPARK_CONF_DIR/log4j.properties

log4j.logger.org.apache.zookeeper=WARN

哪个不起作用,但我并不感到惊讶,因为mesos文档提到zookeeper记录直接转到stderr:

注意:第三方日志消息(例如ZooKeeper)仅写入stderr!

添加一个jar到spark的类路径有一个带有外部函数接口的静态块调用zookeeper C api函数来改变日志级别似乎可能有用,但我不确定,我没有任何经验从JVM调用C代码.

Answer 1

此外，

log4j.logger.org.apache.zookeeper=WARN

还配置 stderr 日志级别以仅打印优先级ERROR（及以上）的日志。

# Send WARN or higher to stderr
log4j.appender.stderr = org.apache.log4j.ConsoleAppender
log4j.appender.stderr.Threshold = ERROR
log4j.appender.stderr.Target = System.err

另外，aLevelRangeFilter也可能被证明很有用，如此处所示。