小编Ska*_*der的帖子

在PySpark 1.4.1中将SparseVector转换为DenseVector时出现意外错误:

from pyspark.mllib.linalg import SparseVector, DenseVector

DenseVector(SparseVector(5, {4: 1.}))

这在Ubuntu上正常运行,运行pyspark,返回:

DenseVector([0.0,0.0,0.0,0.0,1.0])

这导致RedHat出错,运行pyspark,返回:

回溯(最近一次调用最后一次):文件"",第1行,在文件"/usr/lib/spark/python/pyspark/mllib/linalg.py",第206行,在init ar = np.array(ar,dtype) = np.float64)文件"/usr/lib/spark/python/pyspark/mllib/linalg.py",第673行,在 getitem中 引发ValueError("索引%d超出范围."%index)ValueError:Index 5 out边界

此外,在这两个平台上,评估以下内容也会导致错误:

DenseVector(SparseVector(5, {0: 1.}))

我希望:

DenseVector([1.0,0.0,0.0,0.0,0.0])

但得到:

• Ubuntu的:

回溯(最近一次调用最后一次):文件"",第1行,在文件"/home/skander/spark-1.4.1-bin-hadoop2.6/python/pyspark/mllib/linalg.py",第206行,in INIT AR = np.array(AR,D型细胞= np.float64)文件"/home/skander/spark-1.4.1-bin-hadoop2.6/python/pyspark/mllib/linalg.py",线路676,在的GetItem row_ind = inds [insert_index] IndexError:索引超出范围

注意:此错误消息与前一个错误消息不同,尽管错误发生在同一个函数中(代码位于https://spark.apache.org/docs/latest/api/python/_modules/pyspark/mllib/linalg.html)

• RedHat:相同的命令导致分段错误,这会导致Spark崩溃.