无法在Hadoop的本机库中加载bzip2

Question

我的环境是CentOS 7; Spark 1.6.1; Hadoop 2.6.4; 我在集群模式下有两个从节点.

当我尝试hadoop命令时,我得到了 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

我检查hadoop checknative -a,我得到了所有错误的回应.当我添加时,部分问题得以解决

export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export HADOOP_OPTS="-Djava.library.path=${HADOOP_HOME}/lib/native/"

在hadoop-env.sh并重新安装OpenSSL的-devel的.但是,当我输入时,我仍然收到警告hadoop checknative -a:

[hadoop@host-10-174-101-17 ~]$ hadoop checknative -a
16/07/13 14:36:24 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
16/07/13 14:36:24 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop:  true /usr/local/hadoop/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
snappy:  true /lib64/libsnappy.so.1
lz4:     true revision:99
bzip2:   false 
openssl: true /lib64/libcrypto.so
16/07/13 14:36:24 INFO util.ExitUtil: Exiting with status 1

我重新安装了bizp2并检查bzip2 --version:

[hadoop@host-10-174-101-17 ~]$ bzip2 --version
bzip2, a block-sorting file compressor.  Version 1.0.6, 6-Sept-2010.

   Copyright (C) 1996-2010 by Julian Seward.

   This program is free software; you can redistribute it and/or modify
   it under the terms set out in the LICENSE file, which is included
   in the bzip2-1.0.6 source distribution.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   LICENSE file for more details.

bzip2: I won't write compressed data to a terminal.
bzip2: For help, type: `bzip2 --help'.

更重要的是,我查词典〜/ lib64目录/,都libbz2.so.1和libbz2.so存在,它说,如果bzip2的良好加载,这应该是路径.似乎bzip2安装得很好,但是hadoop无法加载它.我还尝试重新编译为https://issues.apache.org/jira/browse/HADOOP-10409.如果我完全按照它,它不起作用:

[hadoop@host-10-174-101-17 ~]$ strings /export/apps/hadoop/latest/lib/native/libhadoop.so | grep initIDs
strings: '/export/apps/hadoop/latest/lib/native/libhadoop.so': No such file

如果我改变了我的hadoop路径的路径:/usr/local/hadoop/lib/native/libhadoop.so,这就是它得到的:

[hadoop@host-10-174-101-17 ~]$ strings /usr/local/hadoop/lib/native/libhadoop.so | grep initIDs
Java_org_apache_hadoop_io_compress_lz4_Lz4Compressor_initIDs
Java_org_apache_hadoop_io_compress_lz4_Lz4Decompressor_initIDs
Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs
Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_initIDs
Java_org_apache_hadoop_crypto_OpensslCipher_initIDs
Java_org_apache_hadoop_io_compress_zlib_ZlibCompressor_initIDs
Java_org_apache_hadoop_io_compress_zlib_ZlibDecompressor_initIDs
Java_org_apache_hadoop_io_compress_lz4_Lz4Compressor_initIDs
Java_org_apache_hadoop_io_compress_lz4_Lz4Decompressor_initIDs
Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs
Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_initIDs
Java_org_apache_hadoop_crypto_OpensslCipher_initIDs
Java_org_apache_hadoop_io_compress_zlib_ZlibCompressor_initIDs
Java_org_apache_hadoop_io_compress_zlib_ZlibDecompressor_initIDs
Java_org_apache_hadoop_io_compress_lz4_Lz4Decompressor_initIDs
Java_org_apache_hadoop_io_compress_lz4_Lz4Compressor_initIDs
Java_org_apache_hadoop_io_compress_zlib_ZlibDecompressor_initIDs
Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs
Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_initIDs
Java_org_apache_hadoop_io_compress_zlib_ZlibCompressor_initIDs
Java_org_apache_hadoop_crypto_OpensslCipher_initIDs

虽然,我检查hadoop checknative -a,但它仍然无法正常工作.在这种情况下我该怎么办？非常感谢你.