我可以使用什么库来计算Java或Scala中的大型稀疏矩阵？

Question

当我使用大型稀疏矩阵时,最好使用CCS,CRS等压缩矩阵.

我尝试使用ScalaNLP,la4j,colc来计算100,000*100,000稀疏矩阵.有一些问题.

1. 微风(ScalaNLP/Scalala)

   • 它给我的CSCMatrix类型可以有100,000*100,000大小.
   • 但问题是它正在开发中.
   • 所以我们不能的钙元素之积CSCMatrix与CSCMatrix一样csc1 :* csc2.
   • 而且你也不能互相添加CSCMatrix.

2. la4j

   • 它有CCSMatrix和CRSMatrix.
   • 但是在创建(new CCSMatrixFactory).createMatrix(100000,100000)时,它会出现OutOfMemoryError.
   • 矩阵应为零,因此不应使用大的内存空间.

3. COLC

   • 它有SparseDoubleMatrix2D.
   • 但是当创建像新的SparseDoubleMatrix2d(100000,100000)这样的矩阵时,它会说IllegalArgumentException:矩阵太大了.

要计算大型稀疏矩阵,我可以使用哪个库？你能告诉我这个例子吗？

Answer 1

我对 Breeze 很好奇，所以我调查了来源。这有点混乱，因为运算符都是从一些 println 样式代码生成中发出的（！）...但我想出了这个：

\n\n

import breeze.linalg.operators.{BinaryOp, OpMulScalar}\n\nobject CSCMatrixExtraOps {\n  abstract class CSCMatrixCanMulM_M[@specialized (Int, Float, Long, Double) A]\n    extends BinaryOp[CSCMatrix[A], CSCMatrix[A], OpMulScalar, CSCMatrix[A]] {\n\n    protected def times(a: A, b: A): A\n\n    protected def zeros  (rows: Int, cols: Int): CSCMatrix[A]\n    protected def builder(rows: Int, cols: Int, sz: Int): CSCMatrix.Builder[A]\n\n    final def apply(a: CSCMatrix[A], b: CSCMatrix[A]): CSCMatrix[A] = {\n      val rows  = a.rows\n      val cols  = a.cols\n      require(rows == b.rows, "Matrices must have same number of rows!")\n      require(cols == b.cols, "Matrices must have same number of cols!")\n\n      if (cols == 0) return zeros(rows, cols)\n

\n\n

\xc2\xa0

\n\n

      val res     = builder(rows, cols, math.min(a.activeSize, b.activeSize))\n      var ci      = 0\n      var acpStop = a.colPtrs(0)\n      var bcpStop = b.colPtrs(0)\n      while (ci < cols) {\n        val ci1 = ci + 1\n        var acp = acpStop\n        var bcp = bcpStop\n        acpStop = a.colPtrs(ci1)\n        bcpStop = b.colPtrs(ci1)\n        while (acp < acpStop && bcp < bcpStop) {\n          val ari = a.rowIndices(acp)\n          val bri = b.rowIndices(bcp)\n          if (ari == bri) {\n            val v = times(a.data(acp), b.data(bcp))\n            res.add(ari, ci, v)\n            acp += 1\n            bcp += 1\n          } else if (ari < bri) {\n            acp += 1\n          } else /* ari > bri */ {\n            bcp += 1\n          }\n        }\n        ci = ci1\n      }\n\n      res.result()\n    }\n  }\n

\n\n

\xc2\xa0

\n\n

  implicit object CSCMatrixCanMulM_M_Int extends CSCMatrixCanMulM_M[Int] {\n    protected def times(a: Int, b: Int) = a * b\n    protected def zeros(rows: Int, cols: Int) = CSCMatrix.zeros(rows, cols)\n    protected def builder(rows: Int, cols: Int, sz: Int) = \n      new CSCMatrix.Builder(rows, cols, sz)\n  }\n\n  implicit object CSCMatrixCanMulM_M_Double extends CSCMatrixCanMulM_M[Double] {\n    protected def times(a: Double, b: Double) = a * b\n    protected def zeros(rows: Int, cols: Int) = CSCMatrix.zeros(rows, cols)\n    protected def builder(rows: Int, cols: Int, sz: Int) = \n      new CSCMatrix.Builder(rows, cols, sz)\n  }\n}\n

\n\n\n\n

例子：

\n\n

import breeze.linalg._\nimport CSCMatrixExtraOps._\n\nval m1 = CSCMatrix((0, 0, 0), (0, 5, 0), (0, 0, 10), (0, 13, 0))\nval m2 = CSCMatrix((0, 0, 0), (0, 5, 0), (0, 0, 10), (13, 0, 0))\n(m1 :* m2).toDenseMatrix\n

\n\n

结果：

\n\n

0  0   0    \n0  25  0    \n0  0   100  \n0  0   0    \n

\n